So it turns out that western governments were wrong about Lebanon. The Lebanese really are like the rest of us, just as the exiles and regime critics always said. They seem to want democracy and to be willing to run huge risks to get it. What is more, Syria seems to be responding to American and French pressure – even though we were long told it never would.

But misjudgments should come as no surprise. Government experts are often wrong, especially about dissidents and regime change. Intelligence experts were wrong about the Iraqi elections. (They mocked a former exile, Ahmad Chalabi, as having no political base. Then he came in second in the race for prime minister.) They underestimated the potential of the Orange movement in Ukraine. They were wrong a decade ago, with disastrous results, when they refused to take coup plans in northern Iraq seriously. And they were wrong a generation back when they told us that Soviet communism enjoyed plenty of democratic support. The reasons for these outstanding failures are worth getting into – especially one that we can call the Rum Tum Tugger problem.

The errors start not with the intelligence officers themselves, but rather with their bosses. George H.W. Bush wanted a short Iraq war so badly that he did not listen to those who said longer engagement might be necessary. Bill “Containment” Clinton was president in March 1995. He did not want to take responsibility for change in Iraq. His intelligence staff therefore shut out or played down a piece of important news: that an Iraqi general with a large following was planning a coup. And that Mr. Chalabi and Jalal Talabani of the Patriotic Union of Kurdistan would probably support the general.

Robert Baer, the Central Intelligence Agency’s man in northern Iraq, stuck his neck out to make these facts known to his superiors. And he even uttered a statement that will go down in a spy thriller: “Jalal, I assure you that Washington wants Saddam gone.” But, at the last minute, Tony Lake, national security adviser, warned the rebels to halt – sending an email negative enough to kill the entire project.

The second problem with intelligence officers is philosophical bias. At American colleges, the dominant philosophy is that containment is best – the fewer regime changes, the better. Change abroad – as Michael Ledeen, an Iran expert, says – “ruins their cocktail hour.” A third problem is the modern bias against fieldwork. After the humiliations of Vietnam, the CIA retreated to its Langley headquarters. Satellite work replaced fieldwork. But the truth is that there are things only staff on the ground can tell you.*

Worst of all is the Rum Tum Tugger problem – from one of T.S. Eliot’s “Practical Cats”: “If you offer him cream then he sniffs and sneers/ For he only likes what he finds for himself.”

Intelligence bureaucrats, too, only like what they find for themselves – information that is exclusive to them. After all, they are most valuable when they can provide what no one else can. These Rum Tum Tuggers therefore tend to favor regime insiders whom they select personally and whom they can control.

Their ideal relationship is government-to-government, as James Woolsey, former director of central intelligence, points out.

Dissidents, especially dissident exiles, resemble government sources in some ways. Dissidents, too, are friendly with dubious people; they, too, sometimes do bad things. But, in other ways, they differ from the classic source. They chatter indiscriminately. They meet with congressmen. They go on Fox, and they publish their views in newspapers. Back in the early 1990s, Amin Gemayel, the former president of Lebanon, wanted to fight the Syrian occupation. He made it his business to be on first-name terms with every op-ed and comment page editor from Paris to Los Angeles. Such chatterers obviate the need for a professional intelligence gatherer. And that professional therefore eschews them like a finicky parlor cat.

But how to change the arrogant Rum Tum Tugger? One way is to reward alternative policy analysis with public respect, so that the Tuggers see there is value in trying out good ideas.

After all, even if democracy never does take root in the Middle East, it is clear now to all that it has a better chance than the Tuggers once thought. (Mr. Baer, Mr. Ledeen, Mr. Woolsey, Douglas Feith, undersecretary of defense, and former defense adviser Richard Perle – all spent lonely years in the 1990s insisting the Middle East might be less cynical if America mixed some nation-building in with its realpolitik.)

It is especially important to take note of the dissidents and exiles, who made sacrifices on a much greater scale to argue democracy’s possibilities: Elena Bonner, widow of the physicist Andrei Sakharov; Ojars Kalnins, an exiled Latvian who swore his country wanted to be part of Europe; Kanan Makiya of Iraq, and Ziad Abdelnour of the U.S. Committee for a Free Lebanon.

A larger step would be to foster competition among intelligence agencies by rewarding the better forecaster in cash or staff. This may become more difficult after the recent establishment of a centralized National Intelligence entity. John Negroponte, the national intelligence director, has plenty of battles ahead. But the largest will be his wrestle with the cat.

* “The War Against the Terror Masters” (American Enterprise Institute)