A developer that roiled many Upper East Side residents during the construction of an East 79th Street condominium is moving forward with a plan to tear down four adjacent buildings in the area, supplanting them with an upscale apartment house.

Some neighbors are opposing the project, citing the record of safety and noise violations on Robert Chou’s earlier projects, as well as Chou’s criminal background.

Chou is the husband of the four buildings’ owner, Katherine Chou. The couple’s daughter, Rita Chou, said at a recent community board meeting that her family plans to erect a 12-story condominium on First Avenue at 78th Street. An application has been filed with the state’s Division of Housing and Community Renewal to evict five rent-stabilized tenants living in three of the buildings, she said, though at the meeting a lawyer for some of those tenants, David Rozenholz, said the Chous intend to put up a building that is “much larger” than their plans suggest.

Asked if the developer had any intention of erecting a larger-scale building, given that the city zoning code could permit such a structure to go up along First Avenue, Rita Chou’s attorney, Sherwin Belkin, said he would have to consult with his client, but did not immediately return phone calls on the matter.

The chairman of Community Board 8, which serves residents of the Upper East Side, David Liston, said he was “cautiously optimistic” about the Chous’ project. He noted, however, that the board office has been fielding calls from neighbors expressing concern about the welfare of the tenants who could be displaced, and from those worried about the environmental impact of the project.

Some participants at a Community Board 8 meeting last week discussed Robert Chou’s criminal record — he has been convicted of bribery and perjury — and said they feared a repeat of what they said were dangerous conditions during the construction of the East 79th Street development three years ago.

The city cited the project for more than 20 environmental violations, including operating a crane in an unsafe manner and failure to safeguard the public from falling debris — and issued fines in excess of $10,000. During construction, two pedestrians sustained minor injuries after a piece of plywood fell from a sidewalk bridge erected at the site.

In a letter to the deputy commissioner of the New York City Department of Housing Preservation and Development after the incident, state Senator Liz Krueger and the then speaker of the City Council, Gifford Miller, said the site had “emerged as a public safety threat owned by a company that has routinely engaged in duplicity.”

Ms. Krueger and Mr. Miller also asked the commission to consider denying the developer’s ultimately successful application for a tax abatement, which would require the construction or renovation of on- or off-site “affordable” housing units. A “willful criminal might be rewarded with a financial incentive that would shift the tax burden for a luxury apartment building to the public,” the letter read.

At the Community Board 8 meeting last week, Rita Chou said those units were not installed in the East 79th Street building and said she did not know where they are situated.

In recent weeks, the office of Ms. Krueger, whose district comprises the Upper East Side, has received calls from residents who expressed fears that the Chous’ proposed First Avenue project would place local residents in danger, a spokeswoman for the senator said.

Mr. Liston said the Chous have assured him that they want to be good neighbors and would be willing to engage in an ongoing dialogue with the community about the project.

“I believe every project should be judged on its own merits,” he said, noting that a community board task force about the East 79th Street project was ultimately successful in addressing the grievances about the construction. “If we get the sense that things are going awry, we will not hesitate to create a task force, and involve elected officials. We’re not wishy-washy about expressing our opinions.”

Upper East Side residents have high standards for developers who build in the neighborhood, Mr. Liston said. “Noise, traffic, litter, safety issues — on the Upper East Side there’s zero tolerance.”

The president of the East 79th Street Neighborhood Association, which has members in buildings in the East 70s, East 80s, and East 90s, said it would welcome responsible construction but is apprehensive about the development team behind the proposal. “When we had so many problems with just one building, we have reason to be concerned when they’re doing almost a whole block,” the president of the association, Betty Cooper Wallerstein, said. “We’re starting with that concern and hoping they’ll do a better job next time.”