Criterion validity of the general factor of psychopathology in a prospective study of girls

Prevalent dimensions of psychopathology are substantially correlated with one another from childhood through adulthood (Angold & Costello, 2009; Angold, Costello, & Erkanli, 1999; Krueger & Markon, 2006a). Furthermore, the magnitudes of those correlations vary in patterned ways that reliably give rise to second-order ‘externalizing’ and ‘internalizing’ factors (Achenbach, Conners, Quay, Verhulst, & Howell, 1989; Krueger & Markon, 2006a). It is less often noted, however, that the internalizing and externalizing factors are themselves substantially correlated with one another (Angold & Costello, 2009; Angold, et al., 1999; Krueger & Markon, 2006b). This is important because it suggests the testable hypothesis that internalizing and externalizing factors are correlated because they share etiologies and psychobiological mechanisms, which are distinguishable from the shared etiologic factors and mechanisms that cause the first-order dimensions of psychopathology to correlate together on the internalizing and externalizing factors. That is, there may be a general factor of psychopathology that reflects etiologies and mechanisms shared to varying degrees by all prevalent dimensions of common forms of psychopathology (Lahey et al., 2012). Related arguments have been made regarding a general factor of personality (Block, 1965; Musek, 2007; Pettersson, Turkheimer, Horn, & Menatti, 2012) We initially tested the general factor of psychopathology hypothesis using phenotypic data from structured diagnostic interviews on psychopathology of 43,093 18–64 year old adults from the representative National Epidemiologic Study of Alcohol and Related Conditions (NESARC) (Grant et al., 2004). A hierarchical confirmatory factor analysis (CFA) model of 11 prevalent mental disorders that included two second-order internalizing factors (fears and distress) and an externalizing factor fit the data well. Like previous studies (Krueger & Markon, 2006b), however, the correlations among these second-order factors were substantial. Consistent with our hypothesis, an alternative model that also included a general factor fit the data significantly better (Lahey, et al., 2012). In addition, data from a study of 9–17 year old twins provided support for the general factor of psychopathology at the level of shared etiologic influences (Lahey, Van Hulle, Singh, Waldman, & Rathouz, 2011). A model of genetic covariances among 11 dimensions of psychopathology specifying orthogonal internalizing, externalizing, and general factors fit significantly better than a correlated two-factor (internalizing/externalizing) model, suggesting that the general factor of psychopathology partly reflects highly pleiotropic genetic influences that are shared to varying degrees by all of the first-order dimensions of psychopathology. In the same sample, associations of factors from a general factor model based on phenotypic covariances with dispositional constructs were described, but a formal test of the comparative fit of the general factor and correlated two-factor models was not conducted (Tackett et al., 2013). In addition, the general factor hypothesis was examined recently in young adults (Caspi et al., 2014). A formal test of improvement in fit was not conducted, however, and fit statistics suggested that a model specifying three correlated dimensions of psychopathology may fit as well as the general factor model. Thus, there is need for additional formal tests of the comparative fit of the general factor model across the lifespan. If replicated, the hypothesized general factor of psychopathology could be important to our understanding of psychopathology. Indeed, it may force a hierarchical conceptualization of the nature of psychopathology in which the etiology and psychobiological mechanisms of varying dimensions are seen as both widely shared and dimension-specific. Before the field can seriously consider such an interpretation of the general factor of psychopathology, however, alternative explanations must be evaluated empirically. One viable alternative is the possibility that the general factor partly or wholly reflects systematic measurement biases rather than correlations among veridically measured dimensions of psychopathology. For example, the general factor could reflect specious correlations arising from common method variance—because the same informants and methods are used to assess each dimension of psychopathology (Campbell & Fiske, 1959; Podsakoff, MacKenzie, Lee, & Podsakoff, 2003). Similarly, the general factor could arise from respondents’ general tendency to rate all negatively worded items similarly (Pettersson et al., 2014) or from implicit theories that certain emotions and behaviors tend to occur together (Schneider, 1973). Systematic measurement biases such as these could well result in reporting symptoms in correlated ways even when the symptoms are not actually experienced or observed. Indeed, it is difficult to imagine that the observed correlations among symptoms and dimensions of psychopathology do not reflect such forms of biased measurement to some degree. The essential question addressed in this paper is whether the general psychopathology factor represents only correlations among dimensions of psychopathology arising from biased measurement. We approach this issue from the perspective of measurement theory, in which the rating of each symptomatic behavior is assumed to be composed of a ‘true’ score plus measurement error (Nunnally, 1978; Spearman, 1904). Random measurement error is only a problem in studies of the correlational structure of psychopathology to the extent that it attenuates correlations among true scores on psychopathology dimensions (Spearman, 1904; Thorndike, 1920). In contrast, systematic sources of measurement error could give rise to a specious general factor that reflects only systematic error. In measurement theory, this is a question of criterion validity (Cronbach & Meehl, 1955). If the general psychopathology factor reflects substantive correlations among dimensions of psychopathology, it will be correlated with important variables that are independent to the measurement of psychopathology, such as measures of daily functioning obtained from different informants. If the general factor reflects only systematically biased measurement, it will not be correlated with such external criteria. In our earlier paper on the general factor of psychopathology in adults (Lahey, et al., 2012), we conducted limited tests of the criterion validity of the general factor using data from NESARC. Controlling for the internalizing and externalizing factors, the general factor of psychopathology was found to account for unique variance in associations with putative risk factors, measures of adaptive functioning, and future psychopathology three years later. Unfortunately, because each of these variables was reported by the same informant, these correlations could simply be due shared measurement error. In the current paper, we report the results of much stronger tests of the criterion validity of the general factor of psychopathology by using independent measures of adaptive functioning obtained from a different informant. In the present analyses, we use CFA to compare the fit of alternative models of the phenotypic structure of psychopathology when the general factor is, and is not, included in a representative sample of girls whose psychopathology has been assessed repeatedly at approximately one-year intervals (Keenan et al., 2010). We model the structure of psychopathology using parent reports of symptoms across 5–11 years of age. Because the general factor model has not been tested in this young age range, these analyses will test the generality of this model. We then test the criterion validity of the general factor of psychopathology by evaluating its unique correlations with independent measures of functioning obtained both concurrently (at 5–11 years) and prospectively (at 12–16 years).