The World Trade Center site developer, Larry Silverstein, said yesterday he is “confused” by Mayor Bloomberg’s new public questions about the future course of the downtown rebuilding project – and about whether Mr. Silverstein should have a role in it.

Mr. Bloomberg has approved the site plan for the World Trade Center site and has signed off on the design for the Freedom Tower, which was designed by Mr. Silverstein’s architect. In the wake of Governor Pataki’s decision last month to expel the planned International Freedom Center from the site, however, Mr. Bloomberg has asserted publicly that it is time to rethink both the plans and the pace for rebuilding at the trade center site.

Yesterday, the mayor was more aggressive than ever, suggesting that New Yorkers might be better off if Mr. Silverstein were completely removed from the rebuilding process.

He was quoted in the Daily News as saying, “It would be in the city’s best interest to get Silverstein out.” Yesterday morning at a campaign event in Brooklyn, he did not back down.

“I would like to see everybody come out a winner here, but my concern is to make sure that what’s right for the city is the way it turns out,” the mayor said. “We’ve worked well with Larry Silverstein so far, but I just don’t think we’re making as much progress as we should, particularly things that might not be high on his economic agenda but are high on our economic agenda from a city planning point of view.”

Mr. Bloomberg has said he wants to create more residential and retail space in Lower Manhattan. Currently, 550,000 square feet of retail space is planned at the World Trade Center site. Mr. Bloomberg also has said he does not know if the city’s commercial real estate market can absorb the entire 10 million square feet of office space planned for the trade center site, and he has suggested that the planned uses for Buildings 3 and 4, on the eastern edge of the 16-acre site, should be reconsidered.

Mr. Bloomberg pointed out yesterday that changing the plan is possible, as he controls some votes on the board of the Lower Manhattan Development Corporation, which was established to oversee the rebuilding process. But at the World Trade Center site, changes are rarely simple.

Steve Coleman, a spokesman for the Port Authority of New York and New Jersey, which owns the site, said the bistate transportation agency is forbidden by the “covenant” that was established at its creation in 1921 from owning residential properties. The governors of both New York and New Jersey would have to approve any change to the covenant, he said.

Mr. Coleman added that if it were possible to throw Mr. Silverstein off the site, planners would “have to start from scratch, developing another site plan that included residential.”

Another complication could be Mr. Silverstein’s insurance commitments, which require him to rebuild the office space that was lost in the terrorist attacks of September 11, 2001.

The executive director of the American Institute of Architects, Frederic Bell, said the rebuilding plans should have been reconsidered when the court ruled earlier this year that Mr. Silverstein would receive about $3.5 billion instead of $7 billion in insurance payments.

Mr. Bell said now is “absolutely” a good time to take another look at the site plan.

“Pataki’s decision about the Freedom Center, in my opinion, creates a vacuum of leadership that the mayor is filling by talking about what can re main in the site plan and what can be revisited,” he said.

Despite some calls for a re-evaluation, the LMDC, the Port Authority, and Mr. Silverstein seem dedicated to pushing forward with the progress they say they have begun.

The LMDC’s president, Stefan Pryor, said in a statement yesterday that everything from the transportation hub to the Deutsche Bank building’s deconstruction to the creation of the Freedom Tower and the memorial are on schedule and “advancing vigorously.”

Mr. Silverstein said: “Together with the governor, the mayor has frequently urged us to proceed as quickly as possible. I believe that New Yorkers want to see rapid rebuilding and not yet another exercise in planning and re-planning. I am confident in downtown’s future as a great business center, and I remain committed to getting it done.”

With the election over two weeks away, the Ferrer campaign attributed the mayor’s recent statements to politics.

“For four years Mike Bloomberg was too busy obsessing over the West Side Stadium and chasing after the Olympics to worry about building anything at ground zero,” Fernando Ferrer’s communications director, Jen Bluestein, said. “Now, at election time, he’s talking tough and blaming others for a problem he helped to create – a gaping hole at ground zero still.”