The Whitney Museum of American Art has since 1966 occupied one of New York’s iconic Modernist buildings, designed by Marcel Breuer. In the 1980s, the Whitney wanted to expand its building with a large “post-Modernist” addition by Michael Graves. For various reasons, that project went unrealized, as did a later expansion plan by Rem Koolhaas. Now the Whitney has retained Renzo Piano to undertake an expansion. Mr. Piano is very hot in New York right now. He is the architect of the Morgan Library’s massive expansion and also of the new New York Times headquarters, to be built on Eighth Avenue.

All these Whitney expansion plans touched off storms of controversy. The museum is located in a dense neighborhood in the Upper East Side Historic District. Any expansion will require permission to destroy buildings that have legal protection to remain standing in perpetuity. Traditionally, the Landmarks Preservation Commission grants such permission grudgingly, if at all. The commission has, however, yielded in recent times in cases involving fashionable architects designing “statement” buildings, examples being Mr. Piano’s Morgan Library addition, James Stewart Polshek’s new front on the Brooklyn Museum, and Norman Foster’s tower above the Hearst Magazine Building.

But the Whitney’s putative need to expand does beg a question. The museum has occupied three main buildings in its brief history. It also pioneered the “satellite” outpost that other museums have now taken to global dimensions. Why is the time now for the Whitney no longer to consider relocation?

Gertrude Vanderbilt Whitney founded the museum. The daughter of Cornelius Vanderbilt II, Whitney grew up in the city in a stupendous mansion on the west side of Fifth Avenue between 57th and 58th Streets, where Bergdorf-Goodman now stands. Though the mansion is gone, we may infer its scale and grandeur by looking at its huge fireplace, designed by Augustus Saint-Gaudens, which graces the Engelhard Court of the Metropolitan Museum of Art’s American Wing. Many readers will also know Whitney’s family’s summer “cottage”: the Breakers, in Newport, R.I.

Whitney typified the rich men’s sons and daughters drawn by the romantic allure of Greenwich Village in its bohemian heyday in the 1910s. Bohemians colonized the Village in part for its low rents. The diaspora of uptown plutocrats’ children made “gentrification” a feature of the New York scene. Whitney herself was a sculptor. We see her work in the statue of Peter Stuyvesant in Stuyvesant Square, and in the Washington Heights World War I memorial at Broadway and 168th Street. We remember her, however, for her efforts to promote American artists, efforts that culminated in the Whitney Museum of American Art.

The museum grew out of the provisional institutions (the Whitney Studio, the Whitney Studio Club, the Whitney Studio Galleries) created by Whitney and her friend and adviser Juliana Force. Force became the first director of the final Whitney Museum of American Art, which opened in 1931 at 8-12 West 8th Street in the buildings now occupied by the New York Studio School of Drawing, Painting, and Sculpture. (These three 1830s houses stood behind Whitney’s own studio in MacDougal Alley, the charming cul-de-sac of stables extending west from MacDougal Street between 8th Street and Washington Square North.)

The quintessential institution of Greenwich Village moved in 1954, eight years after Whitney’s death, to 22 West 54th Street. The architects Miller & Noel designed both the 8th Street and 54th Street Whitneys. (That firm’s Augustus Noel happened to be Whitney’s son-in-law.) The 54th Street Whitney, which stood on land owned by the Museum of Modern Art, faced north onto 54th and, to the west, onto MoMA’s Sculpture Garden. To ensure architectural harmony between the Whitney and MoMA, Philip Johnson consulted with Miller & Noel on the Whitney’s design. The Whitney’s construction also coincided with Johnson’s radical redesign of MoMA’s Sculpture Garden. In general, critics found few words of praise for the Whitney’s new building.

In 1963 the Whitney, under director Lloyd Goodrich, announced its intention to move to Madison Avenue and 75th Street. The museum felt its space on 54th Street to be inadequate, and disliked being overshadowed by MoMA. The Whitney hired Marcel Breuer to design its new building. (The museum had considered, then reject ed, Louis I. Kahn.) Though today we are inured to the aggressive Modernist geometries of Breuer’s “moated,” dark-granite-clad “ziggurat,” it was once one of the most talked about buildings in New York. Oddly for a building of such aggressive stance, perhaps its most admired element was, and remains, its elegant and rather intimate teak-hand railed, granite-stepped interior stairways.

Breuer said, “I didn’t try to fit the building to its neighbors because the neighboring buildings aren’t any good.” It does not surprise us that Breuer served as Penn Central’s architect when, in the 1970s, it proposed a gigantic Modernist tower to be built atop Grand Central Terminal. (Alas, nowadays, the Breuer tower would be called a “parabuilding” and would probably be approved.)

Now the Whitney feels it has outgrown its Breuer building. Roger Kimball of the New Criterion has written that these museum expansions have got to stop, and that the Whitney could lead the way by simply not enlarging itself. I agree. But if the Whitney does feel it needs to expand, why might it not move for a third time, and let the Upper East Side Historic District be?

I think the Whitney could serve both an urban and economic purpose by relocating to Brooklyn, to the neighborhood of the Brooklyn Academy of Music, where the museum could take its place in the new cultural nucleus that includes Frank Gehry’s Theater for a New Audience, Enrique Norten’s Brooklyn Public Library Visual and Performing Arts Library, the Mark Morris Dance Center, and BAM itself.

In fact, Bruce Ratner could be the link. This developer has shown a notable interest in that part of Brooklyn. He also knows from Renzo Piano, as Mr. Ratner is the developer of the New York Times Building. Having brought Target to the neighborhood, surely he could bring the Whitney. Maybe Target could even start a new line of Whitney branded merchandise.