# Validity of Adult Psychopathology Model Using Psychiatric Patient Sample from a Developing Country: Confirmatory Factor Analysis

Authors: Naser Abdulhafeeth Alareqe; Siti Aishah Hassan; Engku Mardiah Engku Kamarudin; Musheer A. Aljaberi; Mohamad Sahari Nordin; Nadeem Mohamed Ashureay; Lubna Ali Mohammed
Journal: Mental Illness
 (2022)

---

## Abstract

Objectives. This study is aimed at testing and validating the two-factor measurement model of the Millon Clinical Multiaxial Inventory (MCMI). Specifically, this paper reported construct validity, particularly focusing on convergent and discriminant validities of the internalizing-externalizing MCMI model of adult psychopathology using a psychiatric sample from a developing country, the Republic of Yemen. Methods. MCMI was distributed among 232 outpatients from the Hospital of Taiz City and two private psychiatry clinics in Yemen; data were collected using structured interviews over four months. We used exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) to explore and confirm the latent structure MCMI and verify the evidence of convergent and discriminant validity. Results. The CFA results indicated that MCMI was a good fit for the internalizing-externalizing two-factor model of adult psychopathology, comparative fit index CFI=0.95, and RMSEA=0.07. The results of the CFA provide evidence of convergent and discriminant validity characterized by MCMI with the internalizing-externalizing model. Conclusion. The adult psychopathology of internalizing-externalizing is a valid measurement model of MCMI with ten personality disorders and eight clinical syndromes.


---

## Body

## 1. Introduction

Children’s behavioral issues are now conceptualized differently by researchers, clinicians, and educators as a result of Achenbach’s original findings on childhood psychopathology assessment [1, 2]. One of the fundamental aspects of the analyses is the hierarchical conceptualization of mental disorders in terms of two broad behavioral spectra, which Achenbach termed internalizing and externalizing. The spectra accounted for the systematic covariation among more narrowly defined behavioral problems [3]. Internalizing problems in childhood include anxiety, depression, and somatization-based disorders while externalizing problems include behavioral misconduct, anger, and attentional difficulties [2].Another contribution of Achenbach’s work is its impact on approaches to conceptualizing adult psychopathology [4]. As a potential theoretical framework for organizing the structure of adult mental disorders, this internalizing-externalizing model has received a significant amount of attention [4–6]. Internalization is the propensity to express distress inwards; common internalizing disorders include major depression, dysthymia, generalized anxiety disorder, agoraphobia, social phobia, simple phobia, and obsessive-compulsive disorder. Conversely, externalization describes the propensity to express distress outwards; commonly recognized externalizing disorders include conduct disorder, antisocial personality disorder, marijuana dependence, and alcohol dependence [7–20].The internalizing-externalizing spectra have been proposed as an organizational system applied in DSM-5 [5]. Regarding this change, several advantages have been highlighted, including the use of a more parsimonious and meaningful organizational model/schema, the ability to simplify problems caused by excessive comorbidity, aiding intervention and treatment development, and creating an important area for future research [6, 8, 21–23]. This internalizing-externalizing framework has been demonstrated across multiple disorders [17, 24], genders [25, 26], ethnicities [26], cultures [10], and over time [7]. It has also been highly successful in accounting for the relationships between psychopathology and other constructs [27] in explaining the etiology of psychopathology in twins’ studies [28, 29] and in studying adults with attention-deficit/hyperactivity disorders (ADHD) [30]. Although there is growing recognition that comorbidity among individual mental disorders is better understood by the broad psychiatric dimensions of internalization and externalization, expanding the scope of this framework is still needed [12, 24].Previous studies indicated that the internalizing and externalizing spectra capture much of the genetic vulnerability to common mental disorders [31, 32]. In a severely extremely sick cohort, Kotov et al. [33] study also supported the essential internalizing-externalizing concept. These spectra seem to be comparable across cultures, ages, and inpatient samples. This knowledge can be useful because the new classification promises to improve our understanding of psychopathology in a variety of ways. However, assessments of internalizing-externalizing difficulties were based on ad hoc questions, scales, or scale combinations that were unique to each study. It is recommended to recognize that internalizing and externalizing problems are not mutually exclusive or completely independent of one another but are moderately correlated in many samples. Assessments of broadband problem groups, narrow-band syndromes, and individual problems can all be valuable for early clinical evaluation progress, outcome evaluations, and research on etiologies, treatments, outcomes, and epidemiology [34].The Hierarchical Taxonomy of Psychopathology (HiTOP) is a scientific effort to address shortcomings of traditional mental disorder diagnoses, which suffer from arbitrary boundaries between psychopathology and normality, frequent disorder cooccurrence, heterogeneity within disorders, and diagnostic instability [33, 35–38]. The Hierarchical Taxonomy of Psychopathology (HiTOP) is a new classification of mental illness. It aims to address several major shortcomings of traditional taxonomies and provide a better framework for researchers and clinicians.The HiTOP model included six spectra: internalizing (or negative affectivity), thought disorder (or psychoticism), disinhibited externalizing, antagonistic externalizing, detachment, and somatoform. Given the direct correspondence between internalizing and negative affectivity as well as between thought disorder and psychoticism, each of these pairs is represented by one dimension. Externalizing behavior has two personality counterparts: disinhibition and antagonism. Disinhibition is particularly prominent in substance-related disorders, and antagonism is especially significant in narcissistic, histrionic, paranoid, and borderline PDs. Both disinhibition and antagonism contribute to antisocial behavior, aggression, ODD, ADHD, and IED [33, 35–43].


### 1.1. Theoretical Gap

Millon et al. [44] theorized that psychopathology reflected by MCMI-III can be classified under four latent structures or the four-factor model of Millon: the 11 clinical personality disorders, the three severe personality disorders, the seven clinical syndromes, and the three severe clinical syndromes. However, both cluster analysis and exploratory/confirmatory factor analysis studies of mental disorders have found evidence that internalizing and externalizing dimensions have underlying common psychopathology in adults [4, 9, 27]. There is also evidence that a limited number of mental disorders represent the internalizing-externalizing spectrum of psychopathology. In particular, the internalizing dimension in previous studies of adult samples was composed of a few mental disorders. For example, Krueger and Markon [24] performed a meta-analysis of relevant literature, applied a CFA to pool data on 11 common mental disorders, and replicated the two general dimensions. The externalizing cluster was composed of substance use disorders, conduct disorders, and adult antisocial behavior. However, the internalizing conditions were only all depressive and anxiety disorders. Although adult psychopathology’s internalizing-externalizing forms are now reasonably established, this 2-spectrum framework is still far from a comprehensive structure, including several mental disorders [7, 33]. Krueger and Markon [24] concluded that expanding the scope of the internalizing-externalizing model by covering a greater diversity of psychopathological symptoms and personality constructs is required and would be a novel step for this model. For example, Krueger and Markon [24] carried out a meta-analysis of relevant literature, applied a CFA to pool data on 11 common mental disorders, and replicated the two general dimensions. The externalizing cluster was composed of substance use disorders, conduct disorders, and adult antisocial behavior. However, the internalizing conditions were only for depressive and anxiety disorders. Therefore, the current study is aimed at expanding the internalizing dimension by covering several more mental disorders, including personality disorders and clinical syndromes, totaling up to 18 internalizing psychopathologies.


### 1.2. Methodological Gap

In addition, the validity of the internalizing-externalizing model for psychopathology in adults has been replicated in population-based samples from several Western, industrialized nations [4, 9, 19, 27, 45]. However, the applicability of this model to mental disorders in non-Western countries remains unknown [10]. Hence, applying the internalizing-externalizing model of psychopathology in a non-Western country is in order. Methodologically, robust evidence for the internalizing-externalizing model for psychopathology has been supported by investigations with diagnostic interviews and multidimensional self-report measures [4, 9, 10, 45, 46]. Krueger and Markon [24] indicated that advanced statistical methods (e.g., structural equation models, item-response models, and growth-curve models) have allowed a tremendous increase in sophistication in our confidence regarding psychological theory conclusions tested using those methods. These methods also hold great promise for understanding psychopathology because they allow empirical comparison of different classification paradigms. Such paradigms can be represented by different quantitative models and can be rigorously compared by comparing the fit of those models to psychological data (p.114, 2006).However, the Millon Clinical Multiaxial Inventory [47] is a widely used measure of personality disorders and psychopathology; it is evaluated as a third international scale in the clinical population [48–50]. The scale has provided good evidence of convergent and discriminant validity [44, 51] with several versions. It has been used in limited studies to examine (i) Millon’s four factors of psychopathology (11 clinical personality disorders, three severe personality disorders, seven clinical syndromes, and three severe clinical syndromes) and (ii) adequate fit of the internalizing-externalizing model using the maximum likelihood approach of CFA. One possibility of this advanced analysis is to provide powerful evidence of construct validity for the instrument [52].Many researchers have employed exploratory techniques for MCMI [53–58], while few recent studies have used confirmatory methods with limited scales of MCMI. For example, Cuevas et al. [59] analyzed 14 personality disorders of MCMI using the maximum likelihood of confirmatory factor modeling among undergraduate students as the study sample. Another pioneering study by Rossi et al. [60] used the confirmatory factor method to analyze 14 disorders of MCMI, obtaining a four-factor solution including the internalizing-externalizing model in clinical patients’ forensic settings. Only Ruiz and Edens [61] used a CFA to analyze the 24 MCMI clinical scales. However, their model used a correlation matrix of MCMI, estimating the internalizing and externalizing dimensions and the general factor of personality disorders. Although their model demonstrated adequate fit, it did not provide MCMI’s construct validity evidence in convergent and discriminant validity. In summary, evidence of convergent and discriminant validity using the maximum likelihood of confirmatory factor modeling for MCMI has been limited; therefore, this research addressed this concern. Mainly, the current study is aimed at examining the Millon Clinical Multiaxial Inventory-III (MCMI) fit with the internalizing-externalizing model of psychopathology and, secondly, evaluating evidence of convergent and discriminant validity MCMI.


## 1.1. Theoretical Gap

Millon et al. [44] theorized that psychopathology reflected by MCMI-III can be classified under four latent structures or the four-factor model of Millon: the 11 clinical personality disorders, the three severe personality disorders, the seven clinical syndromes, and the three severe clinical syndromes. However, both cluster analysis and exploratory/confirmatory factor analysis studies of mental disorders have found evidence that internalizing and externalizing dimensions have underlying common psychopathology in adults [4, 9, 27]. There is also evidence that a limited number of mental disorders represent the internalizing-externalizing spectrum of psychopathology. In particular, the internalizing dimension in previous studies of adult samples was composed of a few mental disorders. For example, Krueger and Markon [24] performed a meta-analysis of relevant literature, applied a CFA to pool data on 11 common mental disorders, and replicated the two general dimensions. The externalizing cluster was composed of substance use disorders, conduct disorders, and adult antisocial behavior. However, the internalizing conditions were only all depressive and anxiety disorders. Although adult psychopathology’s internalizing-externalizing forms are now reasonably established, this 2-spectrum framework is still far from a comprehensive structure, including several mental disorders [7, 33]. Krueger and Markon [24] concluded that expanding the scope of the internalizing-externalizing model by covering a greater diversity of psychopathological symptoms and personality constructs is required and would be a novel step for this model. For example, Krueger and Markon [24] carried out a meta-analysis of relevant literature, applied a CFA to pool data on 11 common mental disorders, and replicated the two general dimensions. The externalizing cluster was composed of substance use disorders, conduct disorders, and adult antisocial behavior. However, the internalizing conditions were only for depressive and anxiety disorders. Therefore, the current study is aimed at expanding the internalizing dimension by covering several more mental disorders, including personality disorders and clinical syndromes, totaling up to 18 internalizing psychopathologies.


## 1.2. Methodological Gap

In addition, the validity of the internalizing-externalizing model for psychopathology in adults has been replicated in population-based samples from several Western, industrialized nations [4, 9, 19, 27, 45]. However, the applicability of this model to mental disorders in non-Western countries remains unknown [10]. Hence, applying the internalizing-externalizing model of psychopathology in a non-Western country is in order. Methodologically, robust evidence for the internalizing-externalizing model for psychopathology has been supported by investigations with diagnostic interviews and multidimensional self-report measures [4, 9, 10, 45, 46]. Krueger and Markon [24] indicated that advanced statistical methods (e.g., structural equation models, item-response models, and growth-curve models) have allowed a tremendous increase in sophistication in our confidence regarding psychological theory conclusions tested using those methods. These methods also hold great promise for understanding psychopathology because they allow empirical comparison of different classification paradigms. Such paradigms can be represented by different quantitative models and can be rigorously compared by comparing the fit of those models to psychological data (p.114, 2006).However, the Millon Clinical Multiaxial Inventory [47] is a widely used measure of personality disorders and psychopathology; it is evaluated as a third international scale in the clinical population [48–50]. The scale has provided good evidence of convergent and discriminant validity [44, 51] with several versions. It has been used in limited studies to examine (i) Millon’s four factors of psychopathology (11 clinical personality disorders, three severe personality disorders, seven clinical syndromes, and three severe clinical syndromes) and (ii) adequate fit of the internalizing-externalizing model using the maximum likelihood approach of CFA. One possibility of this advanced analysis is to provide powerful evidence of construct validity for the instrument [52].Many researchers have employed exploratory techniques for MCMI [53–58], while few recent studies have used confirmatory methods with limited scales of MCMI. For example, Cuevas et al. [59] analyzed 14 personality disorders of MCMI using the maximum likelihood of confirmatory factor modeling among undergraduate students as the study sample. Another pioneering study by Rossi et al. [60] used the confirmatory factor method to analyze 14 disorders of MCMI, obtaining a four-factor solution including the internalizing-externalizing model in clinical patients’ forensic settings. Only Ruiz and Edens [61] used a CFA to analyze the 24 MCMI clinical scales. However, their model used a correlation matrix of MCMI, estimating the internalizing and externalizing dimensions and the general factor of personality disorders. Although their model demonstrated adequate fit, it did not provide MCMI’s construct validity evidence in convergent and discriminant validity. In summary, evidence of convergent and discriminant validity using the maximum likelihood of confirmatory factor modeling for MCMI has been limited; therefore, this research addressed this concern. Mainly, the current study is aimed at examining the Millon Clinical Multiaxial Inventory-III (MCMI) fit with the internalizing-externalizing model of psychopathology and, secondly, evaluating evidence of convergent and discriminant validity MCMI.


## 2. Methodology

### 2.1. Research Design

The research design of the current study was based on correlational structural equation modeling that was dependent on the covariance matrix analysis [52, 62]. The list of observable variables of personality disorders and psychopathology in MCMI was endogenous/dependent variables. In contrast, latent constructs, namely, internalizing and externalizing dimensions of psychopathology, were evaluated as exogenous variables. Subsequently, the structural model in the current study was under confirmatory factor modeling [52, 62–64].


### 2.2. Sample Size Determination

Hair et al. suggested that for covariance based on SEM (e.g., AMOS), the analysis needs a sample size greater than 100 [63]. In this study, 212 sample size was considered adequate for employing structural equation modeling (SEM) to address the research objectives. Concisely, the determined sample size was satisfactory for conducting confirmatory factor analysis (CFA) [64].


### 2.3. Participants and Procedures

The data was collected from 232 outpatients who were mainly diagnosed with mood disorders (71.9%), followed by substance abuse disorders (28.0%) in the Hospital of Taiz City and two private psychiatry clinics in Yemen. The inclusion criteria for selecting the outpatients were (i) Yemeni nationality, (ii) receiving a medical treatment in a psychiatric clinic for more than one year, and (iii) being more than eighteen years old and above. For the second criterion, the report of 12 months of treatment was used to clearly classify the patient into psychiatric and nonpsychiatric cases [65-67]. The final eligible sample size of 212 was used, considered the optimal sample size for CFA [68]. The data were collected over four months using a simple random sampling technique of the eligible registered patients. The response included males (86.3%). The age of the outpatients ranged from 20 to >50 years, with an average age of 30.81 years. Most of the participants were married (56.63%), whereas 34% were single and 9.4% were divorced. About 28.3% of the sample had primary education (writing and reading skills), 23.1% had primary education, 35.4% held secondary school certificates, and 13.2% were degree holders.The study received ethical approval from the academic committees of the Department of Educational Psychology and Counseling at International Islamic University Malaysia (IIUM), Malaysia. In addition, every patient gave informed consent to participate in this research.


### 2.4. Patient and Public Involvement

In this study, the data was collected using Millon Clinical Multiaxial Inventory-III after consent had been taken from the outpatient to participate by answering the survey. However, no patient was involved in setting the research question or the outcome measures. Furthermore, the participants were also not involved in the design or conduct of the study. No patients were requested to provide advice on interpreting or reporting the results. The study results were not planned to be disseminated to the participants.


### 2.5. Millon Clinical Multiaxial Inventory (MCMI)

Psychopathology was measured using the Millon Clinical Multiaxial Inventory-III (MCMI-III) [47]. This self-report personality and diagnostic inventory consisted of 175 true-false items addressing axis I (10 clinical syndromes) and axis II (14 personality disorders) based on the Diagnostic and Statistical Manual of Mental Disorders (4th ed. (DSM–IV)). The MCMI had an adequate internal consistency (alphas ranged from 0.66 for the compulsive scale to 0.90 for the major depression scale) [44]. The MCMI-III was translated into Arabic by the first author of this study and two psychology professors who were proficient in the language with the help of the English Center of the International Islamic University Malaysia because there was no Arabic version of the test at the time the current study was conducted. The differences between the two versions were resolved when the first and back translations were done [69–72]. Moreover, the test-retest reliability of the MCMI scales was obtained by readministering the scale between 5 and 14 days after the initial administration to 0.87 subjects. The validity ranged from 0.84 for the anxiety scale to 0.96 for the somatoform scale. The median stability coefficient was 0.91, thus indicating that the MCMI results were highly stable over a short period [44, 53].


### 2.6. Statistical Analysis

According to Forbush and Watson [39], data analysis was conducted in three steps. First, in Alareqe et al. [53], the items were subjected to exploratory factor analysis (EFA) to create scales that would be used in the subsequent structural studies. Second, a baseline confirmatory factor analysis (CFA) model was developed using the results of EFAs. Third, we used a Monte Carlo-based simulation that created 100 random datasets with the same number of participants and variables as the actual dataset. This analysis allowed for deciding when factors may be considered insignificant and when additional factor extraction was no longer necessary.


#### 2.6.1. Exploratory Factor Analysis (EFA)

Using the orthogonal rotation method, EFA was conducted to determine the number of underlying factors in MCMI. The extracted factors should meet several criteria, including theoretical justification, scree plot, Kaiser’seigenvalue≥1, and total variance explained ≥0.60 [72–76]. In addition, the direction and proportion of factor loading should be ≥0.55, and the corrected item-total correlation must be at least 0.30 and positive [77].


#### 2.6.2. Confirmatory Factor Analysis

While EFA is used to explore the underlying dimensions of the measured variable, CFA is conducted to confirm the hypothesized measurement model. CFA is meant for measurement models within structural equation modeling (SEM) analysis, whereas full-fledge SEM consists of a measurement model and structural model. In other words, CFA deals with the relationships between observed measures or indicators (e.g., test items, test scores, and behavioral observation ratings) and latent variables or factors, but not the relationship between exogenous latent variables to endogenous latent variables [52, 69, 78–81]. Therefore, multiple tests were used in this study to evaluate the fit of the hypothesized model for the data. Kline [64] recommended a minimal set of four fit indexes, including the chi-square test (χ2), the comparative fit index (CFI) [82], standardized root mean residuals (SRMR), and root mean square errors of approximation (RMSEA). Chi-square model included chi-square test, degree of freedom (DF), and nonsignificant p value. However, it is well known that significant p testing is greatly influenced by certain factors such as sample size, high correlations [64], and model complexity [83]. Both SRMR and RMSEA must be ≤0.08 [64, 84]. Additionally, normed chi-square (χ2/df) and incremental fit index (IFI) were also used to evaluate the hypothesized model fit [85]. The normed chi-square (χ2/df) should be ≤3. Both CFI and IFI must be ≥0.90 [64, 74, 78, 82, 86, 87].


#### 2.6.3. Convergent Validity and Discriminant Validity

Convergent and discriminant validity of MCMI as components of construct validity was tested based on suggestions provided by several statisticians [52, 64, 77, 84, 85, 88]. This study refers to how well each indicator (disorder) represents or relates to its construct (internalizing or externalizing. For convergent validity, the threshold value for the factor loading is ≥0.4. The threshold value for average variance extraction (AVE) is ≥0.5 by using the following formula [89, 90]:
(1)AVE=∑i=1kσxi2pi∑i=1kσxi2.For construct reliability (CR), the threshold value is ≥0.7 by using the following formula [90]:
(2)CR=∑i=1kλi2∑i=1kλi2+∑i=1kσei2.From the point of the construct level, the discriminant validity refers to the strength of the correlation coefficient among the latent constructs; that is, when two latent constructs are moderately correlated (not too high, e.g., 0.85, <0.90), the value of correlation coefficient is not too small (<0.20) [91]; and at the item level, it demonstrates the absence of cross-loading [90]. Discriminant validity is assessed by comparing the shared variance (squared correlation) between each pair of constructs against the average of the AVEs for these two constructs [90, 92]. In addition, Henseler et al. [93] suggested the heterotrait-monotrait ratio of correlations (HTMT) as the new approach to assessing discriminant validity; “If the indicators of two constructs, ξi, and ξj, exhibit an HTMT value that is clearly smaller than one, the true correlation between the two constructs is most likely different from one, and they should differ”[89].


## 2.1. Research Design

The research design of the current study was based on correlational structural equation modeling that was dependent on the covariance matrix analysis [52, 62]. The list of observable variables of personality disorders and psychopathology in MCMI was endogenous/dependent variables. In contrast, latent constructs, namely, internalizing and externalizing dimensions of psychopathology, were evaluated as exogenous variables. Subsequently, the structural model in the current study was under confirmatory factor modeling [52, 62–64].


## 2.2. Sample Size Determination

Hair et al. suggested that for covariance based on SEM (e.g., AMOS), the analysis needs a sample size greater than 100 [63]. In this study, 212 sample size was considered adequate for employing structural equation modeling (SEM) to address the research objectives. Concisely, the determined sample size was satisfactory for conducting confirmatory factor analysis (CFA) [64].


## 2.3. Participants and Procedures

The data was collected from 232 outpatients who were mainly diagnosed with mood disorders (71.9%), followed by substance abuse disorders (28.0%) in the Hospital of Taiz City and two private psychiatry clinics in Yemen. The inclusion criteria for selecting the outpatients were (i) Yemeni nationality, (ii) receiving a medical treatment in a psychiatric clinic for more than one year, and (iii) being more than eighteen years old and above. For the second criterion, the report of 12 months of treatment was used to clearly classify the patient into psychiatric and nonpsychiatric cases [65-67]. The final eligible sample size of 212 was used, considered the optimal sample size for CFA [68]. The data were collected over four months using a simple random sampling technique of the eligible registered patients. The response included males (86.3%). The age of the outpatients ranged from 20 to >50 years, with an average age of 30.81 years. Most of the participants were married (56.63%), whereas 34% were single and 9.4% were divorced. About 28.3% of the sample had primary education (writing and reading skills), 23.1% had primary education, 35.4% held secondary school certificates, and 13.2% were degree holders.The study received ethical approval from the academic committees of the Department of Educational Psychology and Counseling at International Islamic University Malaysia (IIUM), Malaysia. In addition, every patient gave informed consent to participate in this research.


## 2.4. Patient and Public Involvement

In this study, the data was collected using Millon Clinical Multiaxial Inventory-III after consent had been taken from the outpatient to participate by answering the survey. However, no patient was involved in setting the research question or the outcome measures. Furthermore, the participants were also not involved in the design or conduct of the study. No patients were requested to provide advice on interpreting or reporting the results. The study results were not planned to be disseminated to the participants.


## 2.5. Millon Clinical Multiaxial Inventory (MCMI)

Psychopathology was measured using the Millon Clinical Multiaxial Inventory-III (MCMI-III) [47]. This self-report personality and diagnostic inventory consisted of 175 true-false items addressing axis I (10 clinical syndromes) and axis II (14 personality disorders) based on the Diagnostic and Statistical Manual of Mental Disorders (4th ed. (DSM–IV)). The MCMI had an adequate internal consistency (alphas ranged from 0.66 for the compulsive scale to 0.90 for the major depression scale) [44]. The MCMI-III was translated into Arabic by the first author of this study and two psychology professors who were proficient in the language with the help of the English Center of the International Islamic University Malaysia because there was no Arabic version of the test at the time the current study was conducted. The differences between the two versions were resolved when the first and back translations were done [69–72]. Moreover, the test-retest reliability of the MCMI scales was obtained by readministering the scale between 5 and 14 days after the initial administration to 0.87 subjects. The validity ranged from 0.84 for the anxiety scale to 0.96 for the somatoform scale. The median stability coefficient was 0.91, thus indicating that the MCMI results were highly stable over a short period [44, 53].


## 2.6. Statistical Analysis

According to Forbush and Watson [39], data analysis was conducted in three steps. First, in Alareqe et al. [53], the items were subjected to exploratory factor analysis (EFA) to create scales that would be used in the subsequent structural studies. Second, a baseline confirmatory factor analysis (CFA) model was developed using the results of EFAs. Third, we used a Monte Carlo-based simulation that created 100 random datasets with the same number of participants and variables as the actual dataset. This analysis allowed for deciding when factors may be considered insignificant and when additional factor extraction was no longer necessary.


### 2.6.1. Exploratory Factor Analysis (EFA)

Using the orthogonal rotation method, EFA was conducted to determine the number of underlying factors in MCMI. The extracted factors should meet several criteria, including theoretical justification, scree plot, Kaiser’seigenvalue≥1, and total variance explained ≥0.60 [72–76]. In addition, the direction and proportion of factor loading should be ≥0.55, and the corrected item-total correlation must be at least 0.30 and positive [77].


### 2.6.2. Confirmatory Factor Analysis

While EFA is used to explore the underlying dimensions of the measured variable, CFA is conducted to confirm the hypothesized measurement model. CFA is meant for measurement models within structural equation modeling (SEM) analysis, whereas full-fledge SEM consists of a measurement model and structural model. In other words, CFA deals with the relationships between observed measures or indicators (e.g., test items, test scores, and behavioral observation ratings) and latent variables or factors, but not the relationship between exogenous latent variables to endogenous latent variables [52, 69, 78–81]. Therefore, multiple tests were used in this study to evaluate the fit of the hypothesized model for the data. Kline [64] recommended a minimal set of four fit indexes, including the chi-square test (χ2), the comparative fit index (CFI) [82], standardized root mean residuals (SRMR), and root mean square errors of approximation (RMSEA). Chi-square model included chi-square test, degree of freedom (DF), and nonsignificant p value. However, it is well known that significant p testing is greatly influenced by certain factors such as sample size, high correlations [64], and model complexity [83]. Both SRMR and RMSEA must be ≤0.08 [64, 84]. Additionally, normed chi-square (χ2/df) and incremental fit index (IFI) were also used to evaluate the hypothesized model fit [85]. The normed chi-square (χ2/df) should be ≤3. Both CFI and IFI must be ≥0.90 [64, 74, 78, 82, 86, 87].


### 2.6.3. Convergent Validity and Discriminant Validity

Convergent and discriminant validity of MCMI as components of construct validity was tested based on suggestions provided by several statisticians [52, 64, 77, 84, 85, 88]. This study refers to how well each indicator (disorder) represents or relates to its construct (internalizing or externalizing. For convergent validity, the threshold value for the factor loading is ≥0.4. The threshold value for average variance extraction (AVE) is ≥0.5 by using the following formula [89, 90]:
(1)AVE=∑i=1kσxi2pi∑i=1kσxi2.For construct reliability (CR), the threshold value is ≥0.7 by using the following formula [90]:
(2)CR=∑i=1kλi2∑i=1kλi2+∑i=1kσei2.From the point of the construct level, the discriminant validity refers to the strength of the correlation coefficient among the latent constructs; that is, when two latent constructs are moderately correlated (not too high, e.g., 0.85, <0.90), the value of correlation coefficient is not too small (<0.20) [91]; and at the item level, it demonstrates the absence of cross-loading [90]. Discriminant validity is assessed by comparing the shared variance (squared correlation) between each pair of constructs against the average of the AVEs for these two constructs [90, 92]. In addition, Henseler et al. [93] suggested the heterotrait-monotrait ratio of correlations (HTMT) as the new approach to assessing discriminant validity; “If the indicators of two constructs, ξi, and ξj, exhibit an HTMT value that is clearly smaller than one, the true correlation between the two constructs is most likely different from one, and they should differ”[89].


## 2.6.1. Exploratory Factor Analysis (EFA)

Using the orthogonal rotation method, EFA was conducted to determine the number of underlying factors in MCMI. The extracted factors should meet several criteria, including theoretical justification, scree plot, Kaiser’seigenvalue≥1, and total variance explained ≥0.60 [72–76]. In addition, the direction and proportion of factor loading should be ≥0.55, and the corrected item-total correlation must be at least 0.30 and positive [77].


## 2.6.2. Confirmatory Factor Analysis

While EFA is used to explore the underlying dimensions of the measured variable, CFA is conducted to confirm the hypothesized measurement model. CFA is meant for measurement models within structural equation modeling (SEM) analysis, whereas full-fledge SEM consists of a measurement model and structural model. In other words, CFA deals with the relationships between observed measures or indicators (e.g., test items, test scores, and behavioral observation ratings) and latent variables or factors, but not the relationship between exogenous latent variables to endogenous latent variables [52, 69, 78–81]. Therefore, multiple tests were used in this study to evaluate the fit of the hypothesized model for the data. Kline [64] recommended a minimal set of four fit indexes, including the chi-square test (χ2), the comparative fit index (CFI) [82], standardized root mean residuals (SRMR), and root mean square errors of approximation (RMSEA). Chi-square model included chi-square test, degree of freedom (DF), and nonsignificant p value. However, it is well known that significant p testing is greatly influenced by certain factors such as sample size, high correlations [64], and model complexity [83]. Both SRMR and RMSEA must be ≤0.08 [64, 84]. Additionally, normed chi-square (χ2/df) and incremental fit index (IFI) were also used to evaluate the hypothesized model fit [85]. The normed chi-square (χ2/df) should be ≤3. Both CFI and IFI must be ≥0.90 [64, 74, 78, 82, 86, 87].


## 2.6.3. Convergent Validity and Discriminant Validity

Convergent and discriminant validity of MCMI as components of construct validity was tested based on suggestions provided by several statisticians [52, 64, 77, 84, 85, 88]. This study refers to how well each indicator (disorder) represents or relates to its construct (internalizing or externalizing. For convergent validity, the threshold value for the factor loading is ≥0.4. The threshold value for average variance extraction (AVE) is ≥0.5 by using the following formula [89, 90]:
(1)AVE=∑i=1kσxi2pi∑i=1kσxi2.For construct reliability (CR), the threshold value is ≥0.7 by using the following formula [90]:
(2)CR=∑i=1kλi2∑i=1kλi2+∑i=1kσei2.From the point of the construct level, the discriminant validity refers to the strength of the correlation coefficient among the latent constructs; that is, when two latent constructs are moderately correlated (not too high, e.g., 0.85, <0.90), the value of correlation coefficient is not too small (<0.20) [91]; and at the item level, it demonstrates the absence of cross-loading [90]. Discriminant validity is assessed by comparing the shared variance (squared correlation) between each pair of constructs against the average of the AVEs for these two constructs [90, 92]. In addition, Henseler et al. [93] suggested the heterotrait-monotrait ratio of correlations (HTMT) as the new approach to assessing discriminant validity; “If the indicators of two constructs, ξi, and ξj, exhibit an HTMT value that is clearly smaller than one, the true correlation between the two constructs is most likely different from one, and they should differ”[89].


## 3. Results

### 3.1. CFA and Millon’s Four Factors of Psychopathology

First, we accessed Millon’s hypothesized four psychopathology factors underlying the MCMI. The results of EFA provide evidence that the four-factor structure did not converge to an admissible solution, and the results of the covariance matrix were not definitely positive. Moreover, the four intercorrelation factors from the six relationships yielded offending estimates (r>1). Subsequent attempts to solve this improper problem by eliminating the intercorrelations and fixing them at 1 also led to an inadmissible solution. Similarly, using the different methods of estimation such as generalized least squares, unweighted least squares, scale-free least squares, and asymptotically distribution-free produced an improper solution. Millon’s classification of psychopathology in the four-factor model did not fit the present data. Accordingly, to search for a more appropriate solution, we explored the number of factor structures underlying MCMI using EFA to obtain an initial validation, shown in Figure 1. The scree plot provides evidence for the three-factor solution of MCMI.Figure 1
Scree plot of MCMI.


### 3.2. Exploratory Factor Analysis (EFA)

The eigenvalue of EFA produced an initial three-factor solution, which accounted for about 72.10% of the total variance, confirmed by the scree plot (Figure1). The first factor, which explained 57.10% of the total variance, is named “internalizing psychopathology” and its eigenvalue═13.70. This factor consisted of the following eighteen factors with an excellent ratio of loading (>0.55): paranoid (0.83), anxiety (0.82), posttraumatic stress (0.80), depressive (0.80), thought disorder (0.81), schizotypal (0.80), dysthymia (0.80), avoidant (0.79), major depression (0.79), borderline (0.75), somatoform (0.73), delusional disorder (0.73), bipolar manic (0.70), masochistic (0.70), negativistic (0.70), dependent (0.66), sadistic (0.67), and schizoid (0.59).The second factor, which modeled 7.49% of the total variance, is “externalizing psychopathology” with aneigenvalue=1.90. This term involved four factors with an excellent ratio of loading (>0.55) as follows: antisocial personality disorder (0.87), drug dependence (0.81), alcohol dependence (0.74), and compulsive personality disorder (-0.61) (Table 1). It was noted that compulsive personality disorder was related in the opposite direction across the orthogonal rotation with the other three disorders in the same factor. The third factor, which accounted for 4.40% of the total variance, is “emotional disturbances” and eigenvalue═1.06. This factor contained two factors with small loading (<0.55), which are narcissistic personality disorder (0.51) and histrionic personality disorder (0.43) as identified by orthogonal rotation. Therefore, the third factor was dropped in the subsequent analysis (CFA).Table 1
Result of EFA of MCMI-III’s.
Items for disorderInternalizing psychopathologyLoadings on factor 1Paranoid0.83Anxiety0.82Posttraumatic stress0.80Depressive0.80Thought disorder0.81Schizotypal0.80Dysthymia0.80Avoidant0.79Major depression0.79Borderline0.75Somatoform0.73Delusional disorder0.73Bipolar manic0.70Masochistic0.70Negativistic0.70Dependent0.66Sadistic0.67Schizoid0.59Externalizing psychopathologyLoadings on factor 2Antisocial0.87Drug dependence0.81Alcohol dependence0.73Compulsive-0.60Emotional disturbancesLoadings on factor 3Narcissistic0.51Histrionic0.43


### 3.3. Confirmatory Factor Analysis (CFA)

The skewness and kurtosis indices demonstrated no evidence of nonnormality for the MCMI’s 21 factors (largest skewnessvalue=−0.57 for paranoid; largest kurtosis value=−0.88 for posttraumatic stress disorder; a value of +/-1.0 is generally considered the cut-off for data nonnormality). The value of Mardia’s coefficient was 1.21, thus demonstrating evidence for multivariate normality. An estimate of ≤2.0 is usually reflected as the cut-off for multivariate data normality. Given that every variable was normally distributed, the ML estimation was used to analyze the confirmatory factor modelling.CFA revealed that the two-factor model of the MCMI (internalizing and externalizing forms of psychopathology) was free from any improper solution of CFA. There were no offending estimates, loading ≥1.0, and no negative error variance value. The model converged to an admissible solution with all parameter estimations reaching a statistically significant level. However, its fit statistics were less satisfactory, thus suggesting the need for revising it. For instance,CFI=0.89, IFI=0.89, and RMSEA=0.11 failed to be reasonably adequate. After allowing the errors to be correlated between depressive and major depression and major depression and thought disorder (not shown in the figure for neat and clarity purposes), CFA results of the revised measurement model of the two-factor model of MCMI indicated that its fit statistics seemed reasonably good and plausible (e.g., χ2=383.65, DF =175, χ2/df=2.19, p≤0.01, CFI=0.95, IFI=0.95, SRMR=0.05, and RMSEA=0.07) (Figure 2).Figure 2
Hypothesized measurement two-factor model of psychopathology using MCMI.The critical values indicate that the unstandardized factor loadings for all disorders were statistically significant (≥1.96), ranging from 10.16 for paranoid personality disorder to 20.02 for drug dependence. More specifically, the standardized factor loadings for each disorder in the internalizing psychopathology construct varied from good (0.68) for paranoid to excellent (0.90) for thought disorder. Similarly, the standardized factor loading for each indicator in the externalizing psychopathology construct was profound. Hence, the convergent validity for the two-factor model of MCMI seemed substantially credible.The squared multiple correlations (SMC) represent the proportion of variance in each disorder, accounted for by its respective factor. Indicators with small SMC values represent potentially weak evidence of construct validity, while the squared multiple correlations (SMC) are 0.50 and above, which is an ideal value. Except for the paranoid (R2=0.46 as a good value), the squared multiple correlations of all disorders had an ideal value of SMC≥0.50, showing that both internalizing and externalizing psychopathology constructs explained a high proportion of variance in their respective disorders as shown in Figure 2.


### 3.4. Convergent Validity

The average variance extraction (AVE) indicated that the latent construct of internalizing psychopathology obtained a coefficient of 0.65, which fell above the recommended amount of 0.50. Likewise, the average variance extraction of externalizing psychopathology was 0.80, which reached the recommended threshold. This means that the large amounts of variance extracted for internalizing and externalizing constructs indicated an adequate percentage or greater than half of the variance for each set of the specified disorders modeled by each representative construct.The construct reliability for internalizing and externalizing psychopathology constructs was 0.97 and 0.92. The values exceeded 0.70 as an acceptable level. Furthermore, the construct reliability based on standardized loading was 0.97 and 0.92 for internalizing and externalizing psychopathology constructs, exceeding the suggested threshold of 0.70. In conclusion, the construct reliability methods excellently indicated that the observed measures of disorders all adequately and consistently represented the same latent construct (internalizing and externalizing psychopathology).


### 3.5. Discriminant Validity

In this study, the finding showed that the interfactor correlation between internalizing and externalizing constructs of psychopathology was statistically significant:r=0.73, b=10.39, S.E=1.39, CR=7.27 (10.39/1.39), and p≤0.01 (Figure 2). This correlation value was considered good evidence of the discriminant validity [90, 92, 93]. Next, the discriminant validity was estimated by comparing the AVE to squared correlation (r2) or shared variance (SV) between internalizing psychopathology and externalizing psychopathology. The variance extracted was expected to be greater than the squared correlation value. Variance extractions of internalizing psychopathology (0.65) and externalizing psychopathology (0.80) were more than the squared correlation (0.53) between these two constructs, thus providing good evidence of the discriminant validity (Table 2).Table 2
Tests of discriminant validity for the two-factor model of MCMI.
Internalizing psychopathologyExternalizing psychopathologyAVEr2AVEr20.650.530.800.53At the item level, discriminant validity means that individual measured variables should represent only one latent construct. The absence of cross-loading for two constructs indicates discriminant validity. Factor score weights showed that antisocial, alcoholic, and drug disorders correlate highly with externalizing psychopathology. In contrast, all remaining disorders correlated highly with internalizing psychopathology, except the paranoid (Figure2). At the construct level, discriminant validity can also be tested using chi-square difference tests. The unconstrained CFA model (with all factors freely correlated) was compared with a constrained model, and the correlation of two factors was set equal to one.A significantχ2 change supports the discriminant validity. Table 3 shows that the chi-square difference and the goodness of fit were due to the added constraint, all getting significant (CFI and RMSEA) power; e.g., the constrained model fit is poorer than the unconstrained one. Thus, the two internalizing and externalizing psychopathology factors exhibited a discriminant validity. Next, the discriminant validity was determined by comparing the fit of the original model against the one-factor model. Table 3 shows that internalizing-externalizing psychopathology, as a two-factor model of MCMI, was more significantly plausible than the one-factor model of MCMI. Hence, the two-factor model of MCMI was sufficiently characterized by good evidence of the discriminant validity.Table 3
Fit statistics for discriminant validity.
Fit statisticsComparison 1Comparison 2Unconstrained modelConstrained modelTwo-factor modelOne-factor modelχ2383. 65499.42383.65655.35DF175176175176χ2/df2.19∗2.84∗2.19∗3.72CFI0.950.930.950.89IFI0.950.930.950.89SRMR0.0460.220.050.06RMSEA0.070.090.070.11∗p<0.001 and ∗p<0.05.


## 3.1. CFA and Millon’s Four Factors of Psychopathology

First, we accessed Millon’s hypothesized four psychopathology factors underlying the MCMI. The results of EFA provide evidence that the four-factor structure did not converge to an admissible solution, and the results of the covariance matrix were not definitely positive. Moreover, the four intercorrelation factors from the six relationships yielded offending estimates (r>1). Subsequent attempts to solve this improper problem by eliminating the intercorrelations and fixing them at 1 also led to an inadmissible solution. Similarly, using the different methods of estimation such as generalized least squares, unweighted least squares, scale-free least squares, and asymptotically distribution-free produced an improper solution. Millon’s classification of psychopathology in the four-factor model did not fit the present data. Accordingly, to search for a more appropriate solution, we explored the number of factor structures underlying MCMI using EFA to obtain an initial validation, shown in Figure 1. The scree plot provides evidence for the three-factor solution of MCMI.Figure 1
Scree plot of MCMI.


## 3.2. Exploratory Factor Analysis (EFA)

The eigenvalue of EFA produced an initial three-factor solution, which accounted for about 72.10% of the total variance, confirmed by the scree plot (Figure1). The first factor, which explained 57.10% of the total variance, is named “internalizing psychopathology” and its eigenvalue═13.70. This factor consisted of the following eighteen factors with an excellent ratio of loading (>0.55): paranoid (0.83), anxiety (0.82), posttraumatic stress (0.80), depressive (0.80), thought disorder (0.81), schizotypal (0.80), dysthymia (0.80), avoidant (0.79), major depression (0.79), borderline (0.75), somatoform (0.73), delusional disorder (0.73), bipolar manic (0.70), masochistic (0.70), negativistic (0.70), dependent (0.66), sadistic (0.67), and schizoid (0.59).The second factor, which modeled 7.49% of the total variance, is “externalizing psychopathology” with aneigenvalue=1.90. This term involved four factors with an excellent ratio of loading (>0.55) as follows: antisocial personality disorder (0.87), drug dependence (0.81), alcohol dependence (0.74), and compulsive personality disorder (-0.61) (Table 1). It was noted that compulsive personality disorder was related in the opposite direction across the orthogonal rotation with the other three disorders in the same factor. The third factor, which accounted for 4.40% of the total variance, is “emotional disturbances” and eigenvalue═1.06. This factor contained two factors with small loading (<0.55), which are narcissistic personality disorder (0.51) and histrionic personality disorder (0.43) as identified by orthogonal rotation. Therefore, the third factor was dropped in the subsequent analysis (CFA).Table 1
Result of EFA of MCMI-III’s.
Items for disorderInternalizing psychopathologyLoadings on factor 1Paranoid0.83Anxiety0.82Posttraumatic stress0.80Depressive0.80Thought disorder0.81Schizotypal0.80Dysthymia0.80Avoidant0.79Major depression0.79Borderline0.75Somatoform0.73Delusional disorder0.73Bipolar manic0.70Masochistic0.70Negativistic0.70Dependent0.66Sadistic0.67Schizoid0.59Externalizing psychopathologyLoadings on factor 2Antisocial0.87Drug dependence0.81Alcohol dependence0.73Compulsive-0.60Emotional disturbancesLoadings on factor 3Narcissistic0.51Histrionic0.43


## 3.3. Confirmatory Factor Analysis (CFA)

The skewness and kurtosis indices demonstrated no evidence of nonnormality for the MCMI’s 21 factors (largest skewnessvalue=−0.57 for paranoid; largest kurtosis value=−0.88 for posttraumatic stress disorder; a value of +/-1.0 is generally considered the cut-off for data nonnormality). The value of Mardia’s coefficient was 1.21, thus demonstrating evidence for multivariate normality. An estimate of ≤2.0 is usually reflected as the cut-off for multivariate data normality. Given that every variable was normally distributed, the ML estimation was used to analyze the confirmatory factor modelling.CFA revealed that the two-factor model of the MCMI (internalizing and externalizing forms of psychopathology) was free from any improper solution of CFA. There were no offending estimates, loading ≥1.0, and no negative error variance value. The model converged to an admissible solution with all parameter estimations reaching a statistically significant level. However, its fit statistics were less satisfactory, thus suggesting the need for revising it. For instance,CFI=0.89, IFI=0.89, and RMSEA=0.11 failed to be reasonably adequate. After allowing the errors to be correlated between depressive and major depression and major depression and thought disorder (not shown in the figure for neat and clarity purposes), CFA results of the revised measurement model of the two-factor model of MCMI indicated that its fit statistics seemed reasonably good and plausible (e.g., χ2=383.65, DF =175, χ2/df=2.19, p≤0.01, CFI=0.95, IFI=0.95, SRMR=0.05, and RMSEA=0.07) (Figure 2).Figure 2
Hypothesized measurement two-factor model of psychopathology using MCMI.The critical values indicate that the unstandardized factor loadings for all disorders were statistically significant (≥1.96), ranging from 10.16 for paranoid personality disorder to 20.02 for drug dependence. More specifically, the standardized factor loadings for each disorder in the internalizing psychopathology construct varied from good (0.68) for paranoid to excellent (0.90) for thought disorder. Similarly, the standardized factor loading for each indicator in the externalizing psychopathology construct was profound. Hence, the convergent validity for the two-factor model of MCMI seemed substantially credible.The squared multiple correlations (SMC) represent the proportion of variance in each disorder, accounted for by its respective factor. Indicators with small SMC values represent potentially weak evidence of construct validity, while the squared multiple correlations (SMC) are 0.50 and above, which is an ideal value. Except for the paranoid (R2=0.46 as a good value), the squared multiple correlations of all disorders had an ideal value of SMC≥0.50, showing that both internalizing and externalizing psychopathology constructs explained a high proportion of variance in their respective disorders as shown in Figure 2.


## 3.4. Convergent Validity

The average variance extraction (AVE) indicated that the latent construct of internalizing psychopathology obtained a coefficient of 0.65, which fell above the recommended amount of 0.50. Likewise, the average variance extraction of externalizing psychopathology was 0.80, which reached the recommended threshold. This means that the large amounts of variance extracted for internalizing and externalizing constructs indicated an adequate percentage or greater than half of the variance for each set of the specified disorders modeled by each representative construct.The construct reliability for internalizing and externalizing psychopathology constructs was 0.97 and 0.92. The values exceeded 0.70 as an acceptable level. Furthermore, the construct reliability based on standardized loading was 0.97 and 0.92 for internalizing and externalizing psychopathology constructs, exceeding the suggested threshold of 0.70. In conclusion, the construct reliability methods excellently indicated that the observed measures of disorders all adequately and consistently represented the same latent construct (internalizing and externalizing psychopathology).


## 3.5. Discriminant Validity

In this study, the finding showed that the interfactor correlation between internalizing and externalizing constructs of psychopathology was statistically significant:r=0.73, b=10.39, S.E=1.39, CR=7.27 (10.39/1.39), and p≤0.01 (Figure 2). This correlation value was considered good evidence of the discriminant validity [90, 92, 93]. Next, the discriminant validity was estimated by comparing the AVE to squared correlation (r2) or shared variance (SV) between internalizing psychopathology and externalizing psychopathology. The variance extracted was expected to be greater than the squared correlation value. Variance extractions of internalizing psychopathology (0.65) and externalizing psychopathology (0.80) were more than the squared correlation (0.53) between these two constructs, thus providing good evidence of the discriminant validity (Table 2).Table 2
Tests of discriminant validity for the two-factor model of MCMI.
Internalizing psychopathologyExternalizing psychopathologyAVEr2AVEr20.650.530.800.53At the item level, discriminant validity means that individual measured variables should represent only one latent construct. The absence of cross-loading for two constructs indicates discriminant validity. Factor score weights showed that antisocial, alcoholic, and drug disorders correlate highly with externalizing psychopathology. In contrast, all remaining disorders correlated highly with internalizing psychopathology, except the paranoid (Figure2). At the construct level, discriminant validity can also be tested using chi-square difference tests. The unconstrained CFA model (with all factors freely correlated) was compared with a constrained model, and the correlation of two factors was set equal to one.A significantχ2 change supports the discriminant validity. Table 3 shows that the chi-square difference and the goodness of fit were due to the added constraint, all getting significant (CFI and RMSEA) power; e.g., the constrained model fit is poorer than the unconstrained one. Thus, the two internalizing and externalizing psychopathology factors exhibited a discriminant validity. Next, the discriminant validity was determined by comparing the fit of the original model against the one-factor model. Table 3 shows that internalizing-externalizing psychopathology, as a two-factor model of MCMI, was more significantly plausible than the one-factor model of MCMI. Hence, the two-factor model of MCMI was sufficiently characterized by good evidence of the discriminant validity.Table 3
Fit statistics for discriminant validity.
Fit statisticsComparison 1Comparison 2Unconstrained modelConstrained modelTwo-factor modelOne-factor modelχ2383. 65499.42383.65655.35DF175176175176χ2/df2.19∗2.84∗2.19∗3.72CFI0.950.930.950.89IFI0.950.930.950.89SRMR0.0460.220.050.06RMSEA0.070.090.070.11∗p<0.001 and ∗p<0.05.


## 4. Discussion

The research objective of the present study was to examine the construct validity of MCMI with the two-factor model of psychopathology using the collected data from a psychiatric sample from a developing country, the Republic of Yemen. The result of CFA indicated that the MCMI was an adequate scale with a two-factor model, i.e., the internalizing-externalizing model of psychopathology. Precisely, fit statistics of CFA (CFI, IFI, and RMSEA) provided evidence for model fit data on MCMI with the internalizing and externalizing psychopathology from the psychiatric samples. Thus, it suggested that psychopathology is ideally classified into internalizing and externalizing. The eighteen disorders, schizoid, avoidant, depressive, dependent, sadistic, negativistic, masochistic, schizotypal, borderline, paranoid, anxiety, somatoform, bipolar manic, dysthymia, posttraumatic stress disorder, thought disorder, major depression, and delusional disorder, were ideally loaded on the internalizing form of psychopathology. The three disorders, antisocial, alcohol, and drug, were excellently loaded on the externalizing form of psychopathology. These findings on internalizing and externalizing are consistent with the results of previous studies [4, 7–19, 22, 23, 25, 30, 51, 60, 94].However, it should be noted that the previous studies had found a small number of disorders (four to six disorders) representing and manifesting the internalizing model of psychopathology. The structure of internalizing psychopathology in this research was extended to ten personality disorders and eight clinical syndromes (a total of eighteen disorders). It captured the DSM-IV model of psychopathology: axis II (personality disorders) and axis I (clinical syndromes). Uniquely, the internalizing model in the current study not only validated mental disorders used in previous research on psychopathology but also included a wider range of mental disorders. This is congruent with Millon’s theory, which considers the importance and dominance of the broad general maladjustment factor with loadings from many disorders [44]. One plausible interpretation is that the expanded scope of internalizing disorders is justifiable since they are attributable to a shared set of genetic factors [4, 9]. Røysamb et al. [95] found that the internalizing factor contained six indicators from axis I (clinical disorders): anxiety disorder, major depressive disorder, dysthymia, anorexia nervosa, posttraumatic stress, and pain disorder. In addition, the two indicators from axis II (personality disorders), depressive and borderline, were loaded partly on this factor.Theoretically, the expanded conceptualization of internalizing psychopathology is consistent with the ideas of Krueger and Markon [24], suggesting that the internalizing model can be tested with many mental disorders. Empirically, the result of the study is similar to the results of a previous study by Rushton and Irwing [16] who analyzed the correlation matrix of MCMI. Although the current study expands the model of internalizing-externalizing psychopathology, there are notable differences between the current model and previous models. For example, previous studies delineated internalizing psychopathology with two underlying factors: fear and distress [4, 10, 17, 22, 25], whereas the internalizing model of EFA [53] and the current study did not bifurcate into distinct distress and fear components of internalizing disorders. The bifurcation of two underlying factors was specified: fear and distress did not provide an admissible solution.The externalizing psychopathology in this research included antisocial, alcoholic, and drug disorders. The present study results are consistent with the results of previous studies [4, 9, 12, 16, 17, 94, 95], which demonstrated that antisocial, alcoholic, and drug disorders all reflect an externalizing dimension. Substance abuse (e.g., alcohol and drug) and antisocial behavior are genetically linked to an impulsive personality [96]. In addition, these are both externalizing disorder indicators as the emotional release is directed outward without restriction [97], likely carried from a childhood of social values.Additionally, the present study provides evidence of convergent and discriminant validities of MCMI with the internalizing-externalizing model of psychopathology. The result of CFA indicated the MCMI with the two-factor model (internalizing-externalizing model of psychopathology) was evidently characterized by convergent and discriminant validity. Given the ideal factor loadings of MCMI and the large proportion of variance explained, excellent construct reliability, and high average variance extraction, it is evident that the disorders within each construct of psychopathology measure consistently and sufficiently their respective construct: internalizing-externalizing of psychopathology. It can be concluded that a convergent validity adequately characterized MCMI with the internalizing-externalizing model of psychopathology. These findings are consistent with the results of the previous studies [16, 54, 58, 59]. Similarly, findings across five discriminant validity evidence confirmed that MCMI with the internalizing-externalizing model of psychopathology were characterized by discriminant validity.The value of the latent correlation was 0.73 (p≤0.01), which was consistent with previous studies [4, 19]. The squared correlation between the two psychopathology constructs was less than their average variance extraction. These techniques of discriminant validity predict that each construct of psychopathology assesses distinct concepts and meanings in the hypothesized model. This finding supports previous studies that verified the distinction between internalizing and externalizing psychopathologies [24]. Therefore, it can be concluded that the two-factor model of MCMI was differently designed to be assessed, indicating that the internalizing psychopathology construct was distinct from the externalizing psychopathology construct; this was good evidence of discriminant validity. Next, the two factors for MCMI exhibited discriminant validity, which strongly confirmed that the two-factor model of psychopathology for MCMI appeared to be the most favorable. Psychopathology with more than two factors or single factor had unacceptable fit indices. These findings of the discriminant validity were consistent with a previous study that aimed to test the discriminant validity of MCMI regardless of the different methods of analysis [51].


### 4.1. Strengths/Originality and Limitations

The current research is the first to verify psychometric evaluations of construct validity comprised of convergent and discriminant validity, evidence of MCMI in a developing country using a sample of patients from outside Western culture, particularly in Yemen. This study is a novel attempt to offer evidence of the fit of the adult psychopathology model in terms of internalizing-externalizing forms using a psychiatric sample from a developing society. The study provides original information on MCMI in the clinical population using exploratory factor analysis (EFA) and confirmatory factor analysis (CFA).Theoretically, this broadened the conceptualization of internalizing psychopathology, which is in line with the theories put out by Krueger et al. [32], who believe that the internalizing model will likely be tested with a variety of mental diseases. The research expanded the structure of internalizing psychopathology to capture simultaneously ten personality disorders and eight clinical syndromes. It is consistent with the ideal vision that new research would expand the scope of the structure of internalizing-externalizing psychopathology. Practically, the evaluation of psychopathology according to the internalizing-externalizing model in the developing country is considered a modern conceptual framework for adult psychopathology, which is more consistent with the recent DSM-5. There is no more demarcation between axis I (clinical syndromes) and axis II (personality disorders) as in DSM-IV. Psychotherapy as an add-on treatment to pharmacotherapy [98] is more likely to benefit from this new nosology.Although the results of this study are significant, certain limitations should be addressed for future research. The first limitation is that the important findings from this study were based on correlational analyses, relying on participants’ self-report of psychopathology measures. Therefore, developmental longitudinal studies are needed to verify the findings reported here. Moreover, although the MCMI with the two-factor model, internalizing and externalizing psychopathology, was evident by its construct validation, replication of this study is needed, using different methodologies such as multitrait-multimethod [82, 83, 99] (e.g., self-reports, clinical observations, and interviews) and different samples (e.g., prisoners, patients, and community). Another cautionary note is that the sample was only from outpatients from the Hospital of Taiz City and two private psychiatry clinics in Yemen; hence, there are limitations on generalization and clinical implications. Future study is suggested using a bigger sample size and comparing the fit indices for the evidence of validity between the four-factor model (if possible after EFA) and the two-factor model to better explain the culture-sensitive item.


### 4.2. Implications and Recommendations

The MCMI is defined theoretically by internalizing-externalizing forms of psychopathology in clinical populations, which was significantly characterized by construct validity. Therefore, clinicians, clinical psychologists, and researchers can assess psychopathology/mental disorders and personality disorders using MCMI in clinical only; for nonclinical populations, an external validity study is needed in future research. In addition to the Mooney Problem Checklist of Measurement which is usually used in developing country educational settings [100], MCMI may be considered in the future. Moreover, MCMI is a clinically accurate assessment for diagnosing mental disorders. It can be used in treatment to track changes over time by looking at patients’ scores on MCMI-scale elevations.


## 4.1. Strengths/Originality and Limitations

The current research is the first to verify psychometric evaluations of construct validity comprised of convergent and discriminant validity, evidence of MCMI in a developing country using a sample of patients from outside Western culture, particularly in Yemen. This study is a novel attempt to offer evidence of the fit of the adult psychopathology model in terms of internalizing-externalizing forms using a psychiatric sample from a developing society. The study provides original information on MCMI in the clinical population using exploratory factor analysis (EFA) and confirmatory factor analysis (CFA).Theoretically, this broadened the conceptualization of internalizing psychopathology, which is in line with the theories put out by Krueger et al. [32], who believe that the internalizing model will likely be tested with a variety of mental diseases. The research expanded the structure of internalizing psychopathology to capture simultaneously ten personality disorders and eight clinical syndromes. It is consistent with the ideal vision that new research would expand the scope of the structure of internalizing-externalizing psychopathology. Practically, the evaluation of psychopathology according to the internalizing-externalizing model in the developing country is considered a modern conceptual framework for adult psychopathology, which is more consistent with the recent DSM-5. There is no more demarcation between axis I (clinical syndromes) and axis II (personality disorders) as in DSM-IV. Psychotherapy as an add-on treatment to pharmacotherapy [98] is more likely to benefit from this new nosology.Although the results of this study are significant, certain limitations should be addressed for future research. The first limitation is that the important findings from this study were based on correlational analyses, relying on participants’ self-report of psychopathology measures. Therefore, developmental longitudinal studies are needed to verify the findings reported here. Moreover, although the MCMI with the two-factor model, internalizing and externalizing psychopathology, was evident by its construct validation, replication of this study is needed, using different methodologies such as multitrait-multimethod [82, 83, 99] (e.g., self-reports, clinical observations, and interviews) and different samples (e.g., prisoners, patients, and community). Another cautionary note is that the sample was only from outpatients from the Hospital of Taiz City and two private psychiatry clinics in Yemen; hence, there are limitations on generalization and clinical implications. Future study is suggested using a bigger sample size and comparing the fit indices for the evidence of validity between the four-factor model (if possible after EFA) and the two-factor model to better explain the culture-sensitive item.


## 4.2. Implications and Recommendations

The MCMI is defined theoretically by internalizing-externalizing forms of psychopathology in clinical populations, which was significantly characterized by construct validity. Therefore, clinicians, clinical psychologists, and researchers can assess psychopathology/mental disorders and personality disorders using MCMI in clinical only; for nonclinical populations, an external validity study is needed in future research. In addition to the Mooney Problem Checklist of Measurement which is usually used in developing country educational settings [100], MCMI may be considered in the future. Moreover, MCMI is a clinically accurate assessment for diagnosing mental disorders. It can be used in treatment to track changes over time by looking at patients’ scores on MCMI-scale elevations.


## 5. Conclusion

This study provides evidence of model fit data of the adult psychopathology two-factor internalizing and externalizing using psychiatric samples from a developing country. More significantly, this study expanded the structure of internalizing psychopathology to identify ten personality disorders and eight clinical syndromes. It is consistent with the ideal vision that new research would expand the scope of the structure of internalizing and externalizing psychopathology. Concisely, it is a novel conceptual framework for adult psychopathology regarding internalizing-externalizing forms.



---

*Source: 9594914-2022-12-21.xml*