Bostonians complained about the Big Dig for years. New Yorkers may soon understand why.

Like a long-awaited arrival of the cavalry, cranes and trucks finally showed up at ground zero yesterday. Soon, Lower Manhattan from the Battery up to Canal Street will be overflowing with hard hats, bright orange traffic cones, trucks, and noise. When the dust settles and about 15,000 construction workers head home – in roughly five or six years – the financial district will be entirely different.

Private developers and government agencies are preparing to pour more than $20 billion into new office and apartment buildings. But amid widespread enthusiasm that rebuilding is finally on track, nearly five years after the September 11, 2001, attacks, there is growing concern among those who live and work downtown about what the construction process will mean for them.

“If you do everything too fast, you could potentially make Lower Manhattan virtually impossible to work in and live in,” the district manager of down town’s Community Board 1, Paul Goldstein, said.

Mayor Bloomberg and Governor Pataki in 2004 created the Lower Manhattan Construction Command Center to coordinate rebuilding among various public and private developers.

A command center director, Dan Mc-Cormick, acknowledged that his two dozen employees will have a tough task. He pointed out that Boston’s Big Dig, a $14.6 billion highway project, took about 15 years to complete.

“The Big Dig was a huge project, but it was a much bigger space,” he said. “In the consolidated space we are talking about, and in the third-largest business district in the country, this is a huge task.”

Mr. McCormick said the peak of construction will not arrive for about two more years, in 2008.

Rep. Jerrold Nadler, a Democrat who represents Lower Manhattan, said he wants construction done in a way that would minimize potential negative health effects to residents.

“There is a lot of contamination left over from 9/11. Demolition and construction could put a lot of that in the air,” Mr. Nadler said.

“A nuisance is a nuisance, that’s bad enough. But environmental carelessness can kill people,” he continued.

Mr. Nadler said that initial environmental planning, including the 2,000-page environmental report prepared by the city and state in 2004, was not adequate. But he said environmental controls are “moving in the right direction.”

The amount of planned construction is staggering. This week, a framework agreement was reached between the Port Authority and developer Larry Silverstein that will allow construction to begin on the commercial development at ground zero.

Within a two-block radius of the former World Trade Center site, $9.9 billion of construction and infrastructure projects will take place. That includes the Freedom Tower, the memorial, the Calatrava PATH station, three giant commercial towers along Church Street, a retail center, a residential building on the site of the badly damaged Deutsche Bank building, and the Fulton Street transit hub.

Nearby, Goldman Sachs will build its $2 billion headquarters. Big residential towers, mostly slated to be luxury apartments, are planned north and south of ground zero and to the west in Battery Park City. To the east, Forest City Ratner will build a 75-story residential tower that contains a school.

The city’s Department of Transportation is planning on improving both Chambers Street and lower Broadway, and Route 9A/West Street will be rebuilt from the Battery north to Chambers Street. All over the area, workers will be carrying out the bland but important work of relocating utilities.

Planners are comparing the Lower Manhattan projects to the planned $15 billion expansion of Chicago’s O’Hare Airport and the rebuilding of New Orleans after Hurricane Katrina.

The president of the Alliance for Downtown New York, Eric Deutsch, said his organization is seeking to coordinate communication between the public and private sectors to keep car and pedestrian traffic moving, and to make sure the construction site looks clean and neat.

When the work is done, he said, “We are going to see a Lower Manhattan that will rival any dynamic central business district anywhere.”