Intelligence Analysis versus Manipulation

So, now theWashington Post has reported that the Pentagon Inspector General found that former Undersecretary of Defense Doug Feith's briefing to the White House in 2002 "undercuts the Intelligence Community" and "did draw conclusions that were not fully supported by the available intelligence." This will presumably be the opening salvo in a great investigation and debate as to whether the Administration manipulated intelligence to justify the invasion of Iraq. To which most reasonable people would say, "Well, duh!" It has long been known that the Vice President's office, distrusting the usual intelligence agencies, set up its Office of Special Plans (OSP) in the Pentagon to sort through raw data to look for any items that would support the case for war. Is this not, by itself, proof that intelligence was manipulated to support the case for war?

According to Bush supporters, it is not. Rather, they say, it was an attempt to correct systematic biases and errors by the intelligence community. Pro-Bush blogger Captain Ed believes that Feith was simply doing what Congress and the 9/11 Commission recommended, expressing dissenting views. "Back then, dissenters got celebrated as visionaries who had the courage to try to wake up the decisionmakers. Now Congress wants to punish someone who essentially did what Congress demanded during those reviews." Feith made similar arguments in an interview with NPR:

The criticism that is being directed now at my former office is because my office was trying to prevent an intelligence failure. We were - we had people in the Pentagon who thought that the CIA's speculative assessments were not of top quality. They were not raising all the questions they should raise and considering all the information they should consider.

What Feith and his defenders are ignoring is that he is not being criticized for presenting dissenting viewpoints, but for how he reached his conclusions. Feith and his colleagues were not just raising shaking up the intelligence community with new and unorthodox ideas, they abandoning sound methods of intelligence analysis and instead cherry picking material to support his own preconceived notions. Feith's defenders might reply that his critics are defining "sound analysis" to mean reaching conclusions they like and "cherry picking" to mean reaching conclusions they do not like. But there are objective standards of what does and does not constitute sound analysis.

There is a reason intelligence agencies filter raw data before passing it on to policy makers. The material they gather, along with valuable information, includes an abundance of unreliable witness accounts, gossip, hearsay, fourth-hand rumors, lies, enemy disinformation, and a few outright forgeries. One of the jobs of intelligence analysts is to determine what is genuine. Even genuine material is not all equally valuable. Like a good journalist or historian, a good analyst should beware of single sources, should give due weight to conflicting evidence, and should search for overall trends and patterns rather than rare anomalies. Trees can be important, but don't overlook the forest! Of course, unlike a journalist or historian, an intelligence analyst is looking for threats that can get us killed, it does seem a reasonable rule that the more alarming a report, the more investigation it merits and the higher the more disproof should be required before it can be dismissed. But that is not the same as automatically believing every alarming rumor.

Granted, all this is easier said than done. Analysts can be fooled into believing inaccurate information or dismissing genuine material as false. They can dismiss findings as anomalies only to find later that they are important, or they can miss the forest for the trees. There is plenty of room for even professionals to disagree on what constitutes good analysis. But there are some things any amateur should be able to recognize as bad analysis. Examples would include: Relying on documents widely believed to be forgeries and contradicted by investigation on the ground. Overriding the judgment of nuclear scientists that aluminum tubes going to Iraq were not suitable for enriching uranium. Believing accounts of mobile bioweapons labs from a single source, widely viewed as a fabricator. Ignoring the professional judgment of the Air Force that Iraqi unmanned drones were unsuited for delivering chemical or biological weapons. Underlying all these individual errors was a basis error in approach. Instead of examining the evidence to see what conclusions it would support, Feith and the OSP began with a conclusion and then looked for supporting evidence, regardless of source, that would support it.

Significantly, Feith and his defenders do not focus on on the OSP's search for any and all evidence of Iraq's weapons of mass destruction. The well-publicized cases of bad data they accepted, and the lack of physical evidence of such an arsenal are too embarrassing. Instead, they discuss the OSP's attempts to prove links between Saddam Hussein and Al-Qaeda, a claim much more difficult to disprove. In the murky world of international intelligence and terrorism, everyone has contacts with everyone else. Indeed, some conspiracy theorists have even found evidence (of dubious quality) of contacts between Bin Laden and the US. Searching through the murk for evidence of contacts between the Iraqi government and al-Qaeda, Feith found evidence that the two organizations had some contacts during the 1990's, put out feelers toward each other, and considered a collaberation. But there was nothing concrete to indicate that Saddam supplied al-Qaeda with money, weapon, training, havens or other meaningful assistance. Feith's defender's argue that the OSP was simply trying to overcome CIA preconceptions that groups as ideologically opposed as Islamicists and Baathist would not cooperate, But, in fact, the CIA appears to have based its estimate, not just on this assumption, but on a high-level "source" that assured them there was no collaberative relationship. Furthermore, despite claims that CIA had a bias against believing in such a relationship, Paul Pillar, the former CIA senior analyst for the Middle East points out that too exhaustive study of such links can create a bias in the opposite direction:

On any given subject, the intelligence community faces what is in effect a field of rocks, and it lacks the resources to turn over every one to see what threats to national security may lurk underneath. In an unpoliticized environment, intelligence officers decide which rocks to turn over based on past patterns and their own judgments. But when policymakers repeatedly urge the intelligence community to turn over only certain rocks, the process becomes biased. The community responds by concentrating its resources on those rocks, eventually producing a body of reporting and analysis that, thanks to quantity and emphasis, leaves the impression that what lies under those same rocks is a bigger part of the problem than it really is.

Granted, intelligence officers' judgments can be faulty, and following past pattern creates a bias in favor of looking at past threats that may miss a new one. But Pillar is, again, pointing out the dangers of beginning with a conclusion and working back to find evidence for it.

The important point to keep in mind about Feith and his defenders is that they are not conceding that the OSP was wrong, about Saddam's ties to Al-Qaeda or even (potentially) about WMD. In fact, they believe they have an ace up their sleeve proving the OSP was right, the Iraqi Freedom Documents. These documents are a collection of some 48,000 boxes of documents captured in Iraq. Although the CIA and DIA (Defense Intelligence Agency) found nothing in these documents to change their viewpoints, at the insistence of Congressman Pete Hoekstra, they were made public, and numerous Bush supporters are pouring over them, looking for evidence to support the OSP's judgment. They do, indeed, appear to have found at least one significant document from 1995 about contacts between Saddam's government and Bin Laden, and others that are suggestive.

Nothing in these archives, however, has changed the minds of intelligence professionals. They are subject to the same caveats as any other intelligence source -- they contain an abundance of contradictory information, anomalies, unreliable sources, fourth-hand rumors and the like. Historian Fritz Umbach, examining the documents, found that they contained some 40 files relating to jihadists, but having nothing to do with Iraq. (He speculates that either Iraqi intelligence departments downloaded them from jihadi websites, or U.S. forces could have "captured" them by downloading them from locations inside Iraq). And administration supporters are showing every inclination to duplicate the errors of Douglas Feith and the OSP on a larger scale -- starting with a conclusion and cherry picking any evidence that will support it.

The OSP and its defenders today are operating under a series of assumptions. They assume that any amateur can analyze raw data as well as a professional, that intelligence professionals who disagree with them are inherently biased, and that their own views are not biased. Above all, they fail to understand what is wrong with beginning with a conclusion and working backward to support it. Indeed, convinced from the start that their conclusion is right, they presuppose that the facts must support it and that anyone who disagrees can only be speaking from bias and preconception. This is what critics call faith-based (or even Feith-based) intelligence.