ALBANY — A First Amendment lawyer, Floyd Abrams, said he would consider providing legal counsel to a celebrity divorce lawyer, Raoul Felder, whose satirical new book has ignited an effort to oust him from his position as chairman of the New York State Commission on Judicial Conduct.

The dispute between Mr. Felder and commission members has the makings of a high-profile freedom of speech case. On Friday, Mr. Felder, a Pataki appointee who became chairman last year, received a unanimous vote of no confidence from the other commission members, who argued that passages in Mr. Felder’s book concerning race, ethnicity, and religion compromise the dignity of the state body and undermine his ability to appear impartial when reviewing allegations against judges.

Mr. Abrams, a Manhattan lawyer who has defended clients in some of the most important First Amendment cases of the past 40 years, said he hasn’t spoken with Mr. Felder or finished reading his book co-written with comedian Jackie Mason, “Schmucks!”

Mr. Abrams said Mr. Felder’s allegation that his First Amendment rights are being violated is a “close” call.

“He can’t be dismissed simply because he wrote a book that has disagreeable ideas,” Mr. Abrams said. “At some point the totality of views expressed based upon considerations of race and national origin could so compromise his real or reasonably expected conduct that it could be appropriate to dismiss him.”

As chairman, Mr. Felder is head of the disciplinary agency that reviews complaints of judicial misconduct in New York. The commission said in a statement announcing its no confidence vote that Mr. Felder’s book is vulgar and “invokes racial, ethnic, and religious invective.” Mr. Felder said the claim is an “offensive lie” and that he would go to court to keep his position.

Governor Spitzer said he supports the commission’s vote and insists that Mr. Felder resign from his position. Commission members are weighing their options and could vote to dismiss Mr. Felder.

In an interview, Mr. Felder said he had been planning to step down as chairman by September, but that he changed his mind and would fight it out. “I can’t surrender on this issue,” he said.

Mr. Felder, who is writing an autobiography and is also working on a Broadway play about the life of his late brother, songwriter Doc Pomus, previously said he did not have enough time for the job and was searching for a replacement. He is also writing a second book with Mr. Mason scheduled to be completed in September.