Criminal and Municipal Law

How objective are forensic experts when they are retained by opposing sides in adversarial legal proceedings? Despite longstanding concerns from the legal system, we know little about whether experts can provide opinions unbiased by the side that retained them. In this experiment, we paid 108 forensic psychologists and psychiatrists to review the same offender case files, but deceived some to believe they were consulting for the defense, and some to believe they were consulting for the prosecution. Experts scored each offender on two commonly-used, well-researched risk assessment instruments. Experts who believed they were working for the prosecution tended to assign higher risk scores to offenders, whereas those who believed they were working for the defense tended to assign lower risk scores to the same offenders, with effect sizes up to d = . 85. Results provide strong evidence of an allegiance effect among some forensic experts in adversarial legal proceedings. 3 ARE FORENSIC EXPERTS BIASED? Are forensic experts biased by the side that retained them? Recently, the National Research Council (NRC, 2009) warned that the accuracy and reliability of many popular forensic science techniques are unknown, that error rates are rarely acknowledged, and that forensic scientists are prone to bias because they “lack independence” from those requesting their services. Emerging research has clearly documented subjectivity and bias even in the forensic science procedures that courts have tended to consider most reliable, such as analyses of DNA (Dror & Hampikian, 2011) and fingerprints (Dror & Cole, 2010). Thus the NRC urged further research on the cognitive and contextual biases that influence forensic experts. The NRC report did not specifically address mental health experts or forensic psychological evaluations. But psychological evaluations—like other forensic science procedures—are often admitted as evidence or presented via expert testimony in adversarial legal proceedings. Indeed, evaluations by mental health experts influence decisions as grave as death sentences (Barefoot v. Estelle, 1983) and indefinite civil confinement (Kansas v. Hendricks, 1997). Therefore, recent concerns regarding forensic science raise questions about whether forensic psychological evaluations might suffer similar problems of unreliability and bias. So how reliable are forensic psychologists and psychiatrists when they are retained as experts in adversarial legal proceedings? For more than a century, courts and legal scholars have lamented apparent bias among medical experts (Bernstein, 2008; Hand, 1901; Mnookin, 2008; Wigmore, 1923). Likewise, practicing judges and attorneys complain that experts sacrifice objectivity for advocacy (e.g., Krafka et al., 2002). But little psychological research has investigated what we call “adversarial allegiance” (Murrie et al., 2009), the presumed tendency for experts to reach conclusions that support the party who retained them. Psychology’s delay in 4 ARE FORENSIC EXPERTS BIASED? investigating adversarial allegiance is disappointing, because psychologists are uniquely suited to explore reliability and bias in decision-making. Field Studies of Risk Instruments Suggest, but Do Not Prove, Adversarial Allegiance Recently, we investigated adversarial allegiance by examining sex offender civil commitment proceedings, also known as Sexually Violent Predator (SVP) trials. SVP trials provide an ideal context for studying the possibility of adversarial allegiance, because court decisions depend largely on weighing testimony from opposing experts. Twenty states and the federal system have SVP laws, which allow them to identify sexual offenders whom they consider likely to re-offend and confine them indefinitely after their incarceration (Kansas v. Hendricks, 1997). SVP proceedings routinely involve forensic psychologists and psychiatrists retained by opposing sides who conduct risk assessments of the same offender and consider the same data, often using the same instruments. So we could study adversarial allegiance in SVP proceedings by comparing the scores that defense-retained and prosecution-retained evaluators assigned on popular risk assessment instruments (Murrie, Boccaccini, Johnson, & Janke, 2008; Murrie et al., 2009). Scores on risk instruments are an ideal metric to measure expert opinions, because (a) experts routinely administer these instruments to inform legal proceedings, and (b) dozens of studies document strong rater agreement when clinicians score these instruments in research and practice contexts that are not adversarial. For example, Hare’s (2003) Psychopathy ChecklistRevised (PCL-R), an instrument that relies on clinical interview and review of records, is widely used in forensic assessments of risk for violence or sexual violence (Skeem et al, 2011). The PCL-R manual reports strong rater agreement statistics (intraclass correlation [ICC] = .87; Hare, 2003). Indeed, most (92%) pairs of scores from trained raters who score the same offender differ 5 ARE FORENSIC EXPERTS BIASED? by fewer than two points (Gacono & Hutton, 2004), even though PCL-R scores can range from 0-40. However, in a small sample of SVP proceedings that featured PCL-R scores from defense-retained and prosecution-retained evaluators, the ICC for opposing evaluators was . 42, indicating that less than half of the variance in PCL-R scores could be attributed to the offenders’ true standing on the PCL-R (Murrie et al., 2009). Moreover, the average PCL-R score from prosecution experts was 24, whereas the average score from defense experts was only 18 (Cohen’s d = .78). The PCL-R may be especially vulnerable to this allegiance effect because it requires clinicians to make inferences about an offender’s personality and emotions (e.g., lack of guilt or remorse, superficial charm). The adversarial allegiance effect was smaller (d = .34) for the Static-99 (Hanson & Thornton, 2000), a highly structured measure scored from file information about criminal history that requires less subjective judgment. These field studies (Murrie et al., 2008; 2009) strongly suggest adversarial allegiance, in that prosecution-retained evaluators assigned higher scores and defense-retained evaluators assigned lower scores to the same offenders. But we cannot draw firm conclusions from these field studies alone, because they investigated scores from experts selected by attorneys. Conceivably, attorneys could have chosen specific experts because they perceived the experts already held attitudes or scoring tendencies conducive to their case. Or perhaps attorneys consulted many experts, but arranged testimony only from those whose opinions were most supportive of their case. For example, a defense attorney might retain several evaluators to examine a client, but request testimony only from the evaluator who assigned the lowest risk scores. Thus, the apparent allegiance in field studies might reflect selection effects, whether in 6 ARE FORENSIC EXPERTS BIASED? terms of which expert an attorney selected to perform an evaluation or which findings an attorney selected to present at trial. Understanding Adversarial Allegiance Requires a True Experiment Field studies raise an important question that can only be answered with a true experiment. Is apparent allegiance due simply to attorneys choosing evaluators who have preexisting attitudes that favor their side, or attorneys calling only experts with the most favorable findings to testify in court (selection effects)? Or do evaluators, once retained and promised payment by one side, tend to form opinions that favor that side (allegiance effects)? If an experiment using random assignment failed to find allegiance effects, it would suggest that the apparent allegiance in the field is due primarily to one or both of these selection effects. But if an experiment using random assignment did find allegiance effects, it would suggest that being retained and paid by one side in an adversarial system may compromise objectivity among experts. To answer this question, we recruited over 100 experienced forensic psychologists and psychiatrists, provided two days of in-person training on risk instruments from established experts, had them meet with an attorney, and then paid them to score risk instruments for up to four offenders. We deceived participants to believe they were performing a large-scale, paid forensic consultation. But unbeknownst to participants, they all received exactly the same four offender files, and each participant was randomly assigned to believe that they were working for the prosecution or the defense.