Repeated diagnoses of lifetime alcohol use disorders in a prospective study: insights into the extent and nature of the reliability and validity problem.

Lifetime prevalence of alcohol use disorders (AUDs; alcohol abuse and/or alcohol dependence [AD]) is derived from cross-sectional studies (e.g., Epidemiological Catchment Area study [ECA; Robins and Regier, 1991], the National Comorbidity Survey [NCS; Kessler et al., 1994], the National Comorbidity Survey—Replication [NCS-R; Kessler et al., 2005], the National Longitudinal Alcohol Epidemiological Survey [Grant et al., 1994a, b, 2004], and the National Epidemiological Survey on Alcohol Related Conditions [NESARC], Grant et al., 2003). Despite that estimates derived from these studies have research and policy implications, there is limited but compelling evidence that cross-sectional, retrospective assessments result in gross underestimates of the prevalence of several disorders, including AUDs. Moffitt and colleagues (2010) compared lifetime prevalence of several internalizing and externalizing disorders from a prospective, birth cohort study (The Dunedin Study; Moffitt et al., 2001) to 3 national cross-sectional studies (New Zealand Mental Health Survey [NZMHS; Degenhardt et al., 2008; Wells et al., 2006], the NCS, and the NCS-R). For all disorders, the prospective estimates were markedly higher compared with estimates derived cross-sectionally. These results indicate that lifetime prevalence of disorders is likely underestimated when based on a single assessment of lifetime diagnosis. These huge underestimates of AUDs are alarming considering that researchers rely on prevalence rates from large cross-sectional studies to understand the etiology of disorders. For example, single, retrospective assessments have been used to define phenotypes of lifetime AUDs in genetic studies (including linkage and association studies; e.g., Gizer et al., 2011; Wall et al., 2005), deduce developmental subtypes of conduct disorder (Nock et al., 2006), and derive developmental subtypes of AUD (using both past-year and lifetime retrospective data) to understand the developmental course with implications for treatment and prevention of these disorders (Moss et al., 2007, 2010). Further, age of onset of psychiatric disorders is typically understood using lifetime retrospective data (e.g., Falk et al., 2008; Hesselbrock et al., 1985; Kessler et al., 2005). If lifetime prevalence rates from cross-sectional data are fundamentally flawed, then subsequent research relying on these types of data may be misleading. Therefore, it is imperative that researchers further characterize the reliability and validity problem with lifetime diagnosis to estimate the extent of the problem. Although data from the Moffitt and colleagues (2010) study were provocative, there were a number of methodological limitations. (i) Cumulative lifetime prevalence for The Dunedin Study was calculated based on past-year assessments of AD at ages 18, 21, 26, and 32. Thus, considering the gaps in assessment between waves and that past-year, not lifetime, diagnoses were assessed, even the cumulative prevalence for AD derived from this study may be an underestimate. (ii) Multiple DSM criteria were used to calculate cumulative prevalence from ages 18 to 32. Strong evidence, shown in table 2 of Moffitt and colleagues (2010), indicates that DSM version impacts lifetime estimates of AD (i.e., DSM-III-R lifetime estimate from NCS was 16.9%, whereas DSM-IV lifetime estimates from NZMHS and NCS-R ranged from 6.3 to 6.4%). Thus, the cumulative lifetime estimates using consistent DSM criteria are currently unknown. (iii) Multiple data sets were used to compare the difference in prevalence from cross-sectional versus prospective data. As noted by Susser and Shrout (2010), inferences from this approach are limited by the methodological differences between the samples. Previous research indicates that poor reliability in reporting may present in terms of individuals meeting criteria for a lifetime AUD at 1 time period, but failing to do so at a subsequent time period (i.e., negative prevalence; Jackson et al., 2006; Robins, 1985; Shrout et al., 2011; Vandiver and Sher, 1991). Evidence for negative prevalence comes from longitudinal studies varying in both assessment instruments and follow- up intervals (e.g., Copeland et al., 2011; Cottler et al., 1989; Culverhouse et al., 2005; DeMallie et al., 1995; Vandiver and Sher, 1991), indicating that the age-related decline in lifetime estimates observed in cross-sectional studies (e.g., ECA, NCS-R, and NESARC) may be due, in part, to individuals reporting less lifetime AUD symptoms as they age (Verges et al., 2011). Negative prevalence suggests issues with sensitivity (i.e., false negatives) in that each assessment does not consistently resolve the true prevalence rate of lifetime AUDs. Therefore, negative prevalence is indicative of problems with reliability (and, consequentially, validity) given the underestimation of lifetime prevalence rate of AUDs. A critical issue is “how badly?” the prevalence rate is underestimated.