Doubts About Daubert: Psychiatric Anecdata as a Case Study

Introduction In Daubert v. Merrell Dow Pharmaceuticals, Inc.,1 the Supreme Court sensibly held that testimony purporting to be scientific is admissible only if it possesses sufficient indicia of scientific validity.2 In Kumho Tire Co. v. Carmichael,3 the Court more questionably held that opinion evidence based on "technical" and "specialized" knowledge must meet the same admissibility threshold as scientific testimony.4 This Article addresses the implications of these two decisions for opinion evidence presented by mental health professionals in criminal trials. At first glance, those implications appear to be significant. Daubert interpreted Rule 702's requirement that opinion testimony "assist" the factfinder5 to mean that testimony offered by experts must be based on "reliable" methodology and theory.6 Whether reliability is defined in its scientific sense to mean consistency of result or, in the sense the Court appeared to use it, to mean a measure of accuracy or validity,7 much behavioral science testimony does not fare well under this standard. Consider first studies of inter-rater reliability using the relatively refined criteria found in recent editions of the official Diagnostic and Statistical Manual (DSM), published by the American Psychiatric Association.8 Although initial research, conducted in the "laboratory," provided encouraging results,9 field research indicates that mental health professionals involved in everyday practice may disagree more than half the time even on major diagnostic categories such as schizophrenia and organic brain syndrome.10 Diagnostic opinions concerning people who are not psychotic are even less reliable in the scientific sense.11 The reliability of determinations about symptoms, such as whether a person is engaging in bizarre thought processes, may be low as well.12 Agreement among professionals also is very poor when they try to explain behavior, both because there are so many competing theories Freudian, behavioral, social, and so on13 - and because even within a given theoretical framework there is considerable dispute.14 If the question is simply whether a person is grossly impaired (without reference to a specific diagnosis), agreement is much better,15 but for reasons to be discussed below16 that type of assessment may not be particularly helpful to the law. If reliability is low, validity is suspect as well, because a lack of agreement between two raters means that at least one of them is wrong. Even if reliability is high,17 however, validity may be poor; unanimity of opinion may hide the fact that all raters are wrong. Unfortunately, the validity of psychiatric opinion is hard to gauge. A diagnosis is merely a hypothetical construct; it lacks clear objective referents.18 Even many symptoms - such as whether a person is "depressed," "anxious," or suffering from "low self-esteem" - are unverifiable in the same way a physical fact is because the terms themselves are so amorphous and subjective.19 Attempts to explain the causes of behavior (e.g., unconscious conflicts, chemical imbalances, abuse as a child, relationship with parents) are even more speculative. …