Brooke Astor, the 104-year-old philanthropist known as a fashion icon and a savior of the New York Public Library, is emerging yet again at the center of New York high society — this time, as the subject of a heated debate over whether she is the victim of elder abuse.

The first skirmishes of an ugly and protracted legal fight began yesterday when the Daily News reported that Brooke Astor’s grandson is petitioning to have his own father removed as her guardian.

In the court papers, which were sealed by the court yesterday, Phillip Marshall is claiming that his father, Anthony Marshall, has reduced his mother’s caregiving staff and denied money for medicines, medical equipment, and quality of life amenities such as hair coloring and flowers for her Park Avenue apartment.

The case sheds light on the legal system that is supposed to protect incapacitated people and also puts a spotlight on a growing and underreported problem in America: elder abuse. The New York City Department of Aging estimates that 50,000 seniors in New York City are victims of elder abuse, although the instances reported to the department annually numbered 1,600.

“We don’t know what really happened. I can expect a vigorous defense from Mr. Marshall. If these things are not true it’s a public humiliation, a libel,” the managing attorney of Bellin Associates LLC, who handles contested guardian matters, Aytan Bellin, said.

“The guardianship statues are set up to protect incapacitated individuals. Toward that end there is yearly court monitoring of the conduct of the guardian, however, absent a report by anyone else that the guardian is doing something inappropriate, there’s no way for the court to know,” Mr. Bellin said.

Told of the allegations, 95-year-old Kitty Carlisle Hart, who attended parties at Mrs. Astor’s home, responded in disbelief. “But he was very good to her. He was around a lot, he was at the parties,” she recalled. “And he was charming, and she loved him, he seemed to have loved her.”

In advance of the celebration of his mother’s 100th birthday, Mr. Marshall, who was 77 at the time, told a reporter, “I think I’ll talk about how I have loved and admired my mother for more than three-quarters of a century.”

Mr. Marshall is Mrs. Astor’s only son, by her first marriage to J. Dryden Kuser, when she was 17 years old. She is said to have described the marriage as “the worst years of her life.” Her husband was physically abusive. She was happy in her third marriage to Vincent Astor, which lasted six years and left her a fortune which she spent 30 years directing to improve New York’s cultural institutions and bring services to the poor.

“If it’s true, I feel terrible, just terrible. It’s so sad because she was so attractive and she was so kind to everybody and she was wonderful and that this has happened to her is really awful, awful. It’s unheard of,” Mrs. Hart said.

A woman with experience in the high-flying social scene Mrs. Astor presided over, the fund-raising consultant Toni Goodale, said she’d heard there were problems with the way Mr. Marshall was taking care of his mother.

“He came to dinner at my house once and he seemed like a very nice man. I have heard he’s kept people away and he’s been very controlling about it,” Ms. Goodale said.

The court papers request that a good friend of Mrs. Astor’s, Annette de la Renta, be appointed temporary guardian. Mrs. de la Renta’s fitness as a guardian is addressed in an affidavit by Henry Kissinger. But Mr. Bellin said that the primary issue the court will have to address is whether Mr. Marshall should remain guardian. Only then can it address Mrs. de la Renta’s fitness as one.

Experts said there could be plausible explanations for some of the claims made in the papers. But one doctor interviewed could not think of a reason to withhold the medication Procrit, which the court papers claim Mrs. Astor was taking for anemia at a cost of $1,000 a month, until Mr. Marshall directed to take her off it.

“It should help her because hemoglobin is important for well being and energy. Anemia gives you chest pain, fatigue, and dizziness, and would interfere with her normal function, especially her muscle capacity. If she doesn’t get it her body will deteriorate even further. It’s not reasonable to withdraw it, and if there are no issues with money, then it’s just unreasonable behavior,” Dr. Polina Fegin, who works in Queens, said.

“Shame on him. This sounds like frustration and greed,” the author of “Elder Rage” Jacqueline Marcell, said. “It’s heartbreaking, however this goes on all the time with people who have a lot less money,” Ms. Marcell said.

“I’m sure the son is exasperated with caring for her. But I don’t want to cut him any slack, because she has plenty of money to be cared for properly.This guy sounds like he’s just trying to hasten it, hasten the inevitable,” Ms. Marcell said.