Popper, Meehl, and Progress: The Evolving Concept of Risky Test in the Science of Psychopathology

The science of psychopathology is in an unusual state of disarray—even for itself. Researchers at the National Institute of Mental Health have called into question the validity of the constructs classified by the Diagnostic and Statistical Manual of Mental Disorders (DSM) and hope to replace them with the Research Domain Criteria (RDoC; Casey et al., 2013; Cuthbert & Insel, 2010; Cuthbert & Kozak, 2013; Sanislow et al., 2010). The long-planned for dimensional model revolution in the diagnosis of personality disorder hit a roadblock during the development of the DSM-5 (American Psychiatric Association, 2013) due to internal dissension among dimensional model advocates and a rejection of dimensional approaches to personality disorder by many practicing psychiatrists (Gunderson, 2013; Krueger, 2013; Miller & Lynam, 2013; Skodol, Morey, Bender, & Oldham, 2013; Zimmerman, 2013). With the recent advances in statistical modeling, brain imaging, and molecular genetics, our understanding of psychopathology is clearly different than it was 40 years ago, but not unambiguously better. Whether one reads Galileo and Descartes, or Bacon, Locke, and Hume, a recurring tradition among modern philosophers and beyond is that current epistemic conventions have become a barrier to progress. Although the early modern philosophers’ target was scholasticism, the scholastic tradition was not inherently problematic. The problem was that scholasticism had evolved to the point where its methodology was being applied dogmatically. Furthermore, the topics worked on were highly specialized and rather than pushing lines of inquiry into new, important territory, the scholastics were seen as running around in circles and exploring dead-end streets. In the 17th century as now, many academics were career oriented and adept at producing work that had a chance of bringing them prestige and accolades according to the conventions of their particular discipline. With such rewards and their economic concomitants available, output continues unabated despite lack of substantial progress. To use more contemporary terms, the early moderns were attuned to a negative feature of Kuhnian paradigms. Kuhn (1962) argued that scientific progress is produced by communities of researchers who agree on what counts as good problems, good solutions, and relevant facts. He called this normal science. In emphasizing that learning to become a researcher is to be inducted into a community, Kuhn highlighted the conservative side of academia. The negative feature of learning to fit in and go with the program is it encourages conformity rather than innovation. The target article by Uma Vaidyanathan, Scott Vrieze, and William Iacono (this issue) ably embodies this modern tradition of diagnosing barriers to progress and applies it to the discipline of psychopathology. It is an important tradition, because the problematic consequences of running on methodological autopilot are endemic to academic professions but not universally intractable. This commentary describes and expands upon the historical and philosophical background of their proposal. I discuss Karl Popper’s notion of how scientific progress is propelled by empirical tests of theory and Paul Meehl’s application of Popper to psychology under the auspices of the risky test. I conclude with some reflections on the skill set that supports scientific success, why the empiricist virtue of the self-correction of error is difficult to exercise in scientific psychology, and the epistemic value of seeking convergence in addition to correction of error.