For the real estate moguls of New York’s Milstein family, patience is a virtue, apparently.

After two decades of sitting on a vacant 20,000-square-foot lot near Union Square, Milstein Properties says it’s finally planning tear down the razor wire fences and erect a residential tower. The pledge by the company mirrors the life of a lot on 42nd Street and Eighth Avenue, which it recently sold for more than $300 million after holding onto it since 1983.

The group’s seemingly longheld reluctance to develop the Union Square property, which is strewn with loose bricks, has mystified neighborhood businesses and residents. The site now bears a banner advertisement for EmigrantDirect.com, a banking entity controlled by the Milsteins. “Make money for your money,” the ad says.

After the body of a homeless woman was found inside a trunk on the lot in 2004, a spokesman for owner Howard Milstein, George Arzt, told The New York Times that development was planned within a year.

This week, Mr. Arzt renewed Mr. Milstein’s vow to build soon on the long-dormant site the company has held since 1986, saying it is being cleaned up and that plans are in the works for a residential building of up to 100,000 square feet and eight stories.

“The market was off sometimes, and then there were other priorities for the company,” Mr. Arzt said, adding that there was no set timetable for development.

Real estate analysts say the Milsteins have a reputation for proceeding at a snail’s pace on projects of these sorts — the family held a faux-groundbreaking in 2002 at their 42nd Street site for their own 35-story tower before selling the lot to SJP properties two years later. SJP will construct a 42-story office building.

“I’m not going to start holding my breath,” an investment analyst with Real Capital Analytics, Daniel Fasulo, said. He said it is now an ideal time to capitalize on a vacant lot, as pricing is considered to be at an all-time high.

In recent years, Union Square has become the Midtown for 20-somethings — a hot spot for both retail and residential development. NYU and New School students flog the sidewalks, vendors and panhandlers are seemingly permanently set up on the street and in the subway station, and the always-packed Whole Foods and Trader Joe’s employ multiple workers to stand at the end of the checkout lines.

The square’s vibrancy has failed to significantly penetrate, until now, many of the immediately surrounding streets, particularly east of Third Avenue on 14th Street, where small cell phone shops and souvenir vendors compose much of the retail.

Whatever happens with the Milstein lot, the owner of a smaller vacant lot adjacent to the property, Slavko Bernic, said he would attempt to erect an eight-story residential building, perhaps within a year. Another building at the southeast corner of Third Avenue and 14th Street may also follow suit, as sources familiar with the deal said the owner was in talks about developing a bigger residential building.

These potential projects come on top of the near completion of a Toll Brothers luxury condo tower on the northwest corner of Third Avenue and 13th Street, for which 90% of its units have already been snatched up with another three months until the closing date. “We’re way ahead of our sales projections — this project is much more successful than anticipated,” a project manager at Toll Brothers, Marc Bassi, said.

The executive director of the Union Square Partnership, Jennifer Falk, said interest is increasing every day east and west of the epicenter of Union Square, along the 14th Street corridor.

“The Partnership wants to see those improvements continue both to the east and west all the way to the rivers, and we are actively engaged in improving the quality of the retail environment along 14th Street and encouraging growth that benefits our community,” she said in an e–mail.

As to whether a potential development on the Milstein lot would spruce up the area, a real estate broker who works across the street, Jack Bick, said development would accelerate on the block regardless. “This neighborhood is hot,” Mr. Bick said. “Whether that’s empty or that goes up, the neighborhood is on fire.”