At 9 o’clock on a recent Friday morning, a big guy in a scruffy blue coat stood dealing drugs in front of the main gate to the Graham Court, an Italian Renaissance palazzo called the most luxurious apartment house in Harlem by the AIA guide book. In its 1984 designation, the Landmarks Commission said the Graham Court reflected a “conscious effort to evoke an image of luxury,” and is “one of the premier reminders of the urban development of Harlem at the turn of the century.” Developed by William Waldorf Astor and designed by architects Charles Clinton and Hamilton Russell in 1898-1901, the Graham Court spans the block between 116th and 117th streets on the east side of Adam Clayton Powell Boulevard.

That same morning, an elderly man who looked like he could be an oldtime jazz musician walked frantically in a series of half-circles down 117th Street, slashing the air with a knife he held in his right hand. Spotting the open service entrance to the Graham Court, which was untended, he peered in cautiously then disappeared underground, a part of the building that had been one of the engineering marvels of its day, built to hold the carriages and the horses of the wealthy.

Harlem is in the midst of one of the most renowned comebacks in urban history, but this jewel of a building seems mired in earlier times. The elaborate but over painted iron fence surrounding the eight-story brick and limestone building is missing many finials. The guardhouse that once stood in the two-story Palladium entrance has been removed.The garden is forlorn, with scant signs of the fountain that once stood at its center. There are broken windows on the upper floors. The wooden doors at each entryway have been replaced with metal ones. Most of the ornate cast-iron lampposts have been lost, replaced with rubberized replicas, which don’t work. The Italian mosaic tile floors are cracked in huge X’s. Garbage sits in front of the residential elevators.

“It should be the finest building in Harlem,” says a Harlem real estate broker, Willie Kathryn Suggs, who points out that many prominent black New Yorkers live there, including actor Danny Glover, poet Quincy Troupe, and interior designer Sheila Bridges. “But it hasn’t backed up its promise. Everybody’s just hoping and praying something good happens soon.”

That was precisely the wish of a 42-year-old architect, Kyle Taylor, when he rented a three-bedroom, two-bath, 1,500-square-foot apartment in 2004. “I’d studied the Graham Court in school at Columbia,” says Mr. Taylor, in a course called “Reading the Built Environment,” and had moved in with the promise that the building would go coop. Now he finds himself as the “youngest and newest tenant,” paying a market rent of $2,200, with no sign of a co-op in sight and lamentable services. The building is overseen by a superintendent and two porters. (“You see the super when you see him,” Mr. Taylor says. “You can’t call him.”) Mr. Taylor complains about mice, as does his downstairs neighbor, Leslie Powell, who says, “The super told me I should be happy I have mice because that means I won’t have rats. They don’t coexist together. Well, that’s a bit of useful information I didn’t have!”

The building has no doorman or intercom system. While a security guard on duty from 4 p.m. to midnight will admit visitors, the rest of the time tenants must go down to the gate to greet guests. Residents say there is a direct connection between the lack of security and the regular presence of drug dealers, some of whom live in the building.

Some tenants say the drug dealing has been going on for a long time, citing one woman on the ground floor who sold crack through her window, while others say that drugs and violence are getting worse by the day. Indeed, the crime data from the 28th Precinct as a whole are ominous. While robbery, for example, has increased a tiny 0.8% citywide since last year at this time, it has jumped 38.5% in the 28th. And while rape is down 5.4% citywide and grand larceny is down 1.8%, rape has increased 41.6% in the 28th, and grand larceny 39.9%. As Ms. Suggs says, “People are worried, and some homeowners in Harlem are thinking about selling.”

The story of the Graham Court tracks the story of Harlem. The first of Astor’s courtyard buildings, it was located “farther uptown than fashion would tolerate for very long,”according to Elizabeth Hawes in “New York, New York: How the Apartment House Transformed the Life of the City (1869-1930).” Built between 1898 and 1901, when many fine buildings were being financed by a speculative boom, the Graham Court then got caught in the market collapse of 1904-05, which hit Harlem particularly hard. Moving up from the West Side, blacks turned Harlem into a “community where Negroes as a whole are better housed than in any other part of the country,” according to historian Gilbert Osofsky. The Graham Court, however, did not rent to any black tenants until 1933, says historian Michael Henry Adams, when new management’s decision to “accept colored was accompanied by a reduction in services and an increase in the reduced, Depression-era rents.” Many eminent blacks moved in, including Dr. Cyril Dolly, a physician who organized the Consolidated Tenants League to protect tenants.

Bad as the Depression was, the 1970s were probably even worse, when arson and housing abandonment resulted in the city of New York becoming Harlem’s largest landlord. The Graham Court fell into city hands in 1978, when its owner failed to pay taxes. The current manager, Sam Becker, emphasizes that the building was in bad shape when the Graham Court Owners Corporation bought it in 1987, long after many of the apartments had been cut up into smaller units and architectural objects removed. A pharmacist, Mohammed Siddiqui, who owned a clinic on the ground floor, bought the Graham Court for $55,000 plus a back tax payment of $150,000 in 1979. He fell behind in tax payments and let the building deteriorate so that the city took over its management. In 1986, the city moved to foreclose the building for nonpayment of $933,000 in taxes. Instead, Mr. Siddiqui sold it for $2 million to Leon Scharf, a West Side real estate investor, who soon sold it to the current owners.

“Although we’ve had owners, like Leon Scharf, who wanted to be a good landlord, we’ve never had owners and managers who really took care of the building,”says Laconia Smedley, who is 70 years old and has lived in the Graham Court since 1960. “Today’s management is aloof and indifferent. One of them was quoted as saying about us, ‘Oh, they think they live on Fifth Avenue.’ Where we live is what should be the best building in Harlem.”

None of the tenants knows what the future holds, and management isn’t saying. Of the 100 apartments, 12 are rent-controlled and 42 rent-stabilized, according to the state housing agency, leaving a large number available for market rents. But the landlord is embroiled in disputes with the new market-rate tenants as well as ongoing suits from older tenants. “Their lawyers just camp out down at housing court,” Mr.Taylor says.

High above the Graham Court’s decrepit grounds, the monogram of William Waldorf Astor, once known as “New York’s landlord,” still stands out in relief.