# Validation of the Mental Health Literacy Scale in French University Students

Ilaria Montagni, Juan Luis Gonzalez Caballero

► To cite this version:

Ilaria Montagni, Juan Luis Gonzalez Caballero. Validation of the Mental Health Literacy Scale in French University Students. Behavioral Sciences, 2022, 12 (8), pp.259. 10.3390/bs12080259. hal-03819671

HAL Id: hal-03819671

https://hal.science/hal-03819671

Submitted on 18 Oct 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Article

# Validation of the Mental Health Literacy Scale in French University Students

Ilaria Montagni <fnref n="1" />,* and Juan Luis González Caballero <fnref n="2" />,†

**Abstract:** Background: Mental health literacy is a determinant of mental health, which can facilitate early detection of psychological problems and endorse timely access to care. Instruments to measure mental health literacy exist, but not in French. Assessment of mental health literacy in young adults is essential to tailor appropriate educational interventions promoting psychological wellbeing and preventing mental health problems in this vulnerable population. The aim of this study was to validate the French version of the Mental Health Literacy Scale (MHLS-FR) in university students. Methods: A total of 482 students from the University of Bordeaux, France, completed the translated version of the scale. Collected data were used to validate the MHLS-FR through psychometric analyses: descriptive statistics, item distribution, test-retest reliability, exploratory structural equation model, confirmatory factor analysis, Cronbach's alpha and McDonald's omega coefficients, and hypothesis testing. Results: The final scale included 26 items covering 6 dimensions. Cronbach's alpha and McDonald's omega coefficients were 0.744 and 0.961, respectively. With test-retest, about 50% of items had intraclass correlation coefficients superior to 0.5. Conclusions: The MHLS-FR can be considered as a valid and reliable instrument for measuring mental health literacy in French students.

**Keywords:** measure; knowledge; psychometric analyses; validation; mental health literacy; students

<figure><img src="image_2.png" /></figure>

check for
updates

**Citation:** Montagni, I.; González Caballero, J.L. Validation of the

Mental Health Literacy Scale in French University Students. *Behav. Sci.* 2022, 12, 259. https://doi.org/

10.3390/bs12080259

Academic Editor: Scott D. Lane

Received: 30 June 2022

Accepted: 25 July 2022

Published: 28 July 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

<figure><img src="image_3.png" /></figure>

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## 1. Introduction

Mental health literacy (MHL) was first defined as “knowledge and beliefs about mental disorders which aid their recognition, management or prevention” [1] and consisted of six domains: “ability to recognise specific disorders or different types of psychological distress”; “knowledge and beliefs about risk factors and causes”; “knowledge and beliefs about self-help interventions”; “knowledge and beliefs about professional help available”; “attitudes which facilitate recognition and appropriate help-seeking”; and “knowledge of how to seek mental health information” [2]. MHL is a determinant of mental health that can facilitate the early detection of psychological problems and promote timely access to care [3]. The underpinning assumption is that mental health knowledge can improve mental health per se. Improving MHL empowers people by helping them develop skills they need for their psychological wellbeing such as positive attitudes and good mental health care decisions. Therefore, it is crucial to increase MHL levels across populations.

### 1.1. *Measuring Mental Health Literacy*

Measuring instruments in public health have three objectives: provide baseline information for monitoring patterns of health; provide information for planning, developing, and evaluating an intervention; and raise awareness and improve the agenda-setting for health promotion [4]. The unique instrument that captures all dimensions of MHL is the Mental Health Literacy Scale (MHLS) [5]. It consists of 35 items, which are grouped so as to cover the six dimensions identified by Jorm et al. [1]. The MHLS is a methodologically and

<footnote n="1">¹ Bordeaux Population Health Research Center UMRS1219, University of Bordeaux, Inserm, F-33000 Bordeaux, France</footnote>

<footnote n="2">² Department of Statistics and Operational Research, University of Cádiz, 11003 Cádiz, Spain; juanluis.gonzalez@uca.es</footnote>

<footnote>* Correspondence: ilaria.montagni@u-bordeaux.fr</footnote>
psychometrically reliable measurement [6,7]. It has been conceptualized and validated in Australia in a large population (n = 617) including university students. Performed analyses have demonstrated its construct validity, internal consistency, and test-retest reliability. The MHLS has been translated and validated in several languages. A translated scale should measure the same construct (MHL) in exactly the same way across languages and countries through functional equivalence [8]. However, when interpreting the data, the specific social, cultural, and environmental context must be taken into account. Indeed, translating and validating scales is an optimal way to obtain data that is comparable across countries, considering that the populations under study are similar. A robust comparison can suggest potential universal interventions that could be standardized. All things considered, the French version of the MHLS has not yet been developed.

## 1.2. *Students' Mental Health*

Although university students report mental health conditions similar to their non-university peers [9], recent studies suggest an increase and severity of mental health problems in this population around the world in the last few years [10–12]. A systematic review has showed that students are particularly vulnerable to depression, stress, and anxiety problems [13]. Studying at the university represents a heavy stressor [14]; students face new challenges such as leaving their homes and being distant from their support networks, making independent decisions about their lives, and suffering academic pressure [15]. Among all academic disciplines, healthcare studies are particularly demanding. Extensive time and emotional commitment are necessary for students to devote to their training and this might cause distress [16]. Another risk factor is sex: being a female student increases the chances of having a mental health problem [17]. Environmental and genetic influences are at the origin of this difference [18].

The COVID-19 crisis has exacerbated students' distress with containment measures increasing the prevalence of depressive and anxiety disorders [19]. As in other countries, French students have experienced and are still suffering from severe mental health problems due to the pandemic [20].

## 1.3. *Students' Mental Health Literacy: Data and Interventions*

Improving students' MHL may help them recognize signs of a mental health problem, seek for timely help, and fight against stigmatisation. Globally, untreated mental disorders are highly prevalent in students [11], and MHL is meant to enhance recovery. The few surveys using the MHLS have shown low levels of MHL in students. Two surveys have been conducted in the UK among medical students [21] and students from all disciplines [22]. In the first study, medical students reported a mean score of 127.69/160. In the second study, the mean MHLS score was slightly lower, 122.88/160. These studies used the complete 35-item MHLS as an already-validated tool in the English language. The mean MHLS score in Persian students was 69.59/106, using a 23-items MHLS version [23]. In Saudi Arabia, students scored 112.53/160 using the full MHLS [24]. In Malaysia, students' mean score of the 35-item MHLS was 111.42/160. Overall, these data show a homogenous low level of MHL in students (around 40 points below the maximum score). No data exist on student's MHL in France.

Research on MHL reported that males' scores are lower than females' [21], probably due to the prevailing gender role for males, i.e., the traditional masculine stereotype, rejecting any type of help and avoiding to disclose a personal mental health problem [25]. With the advancement of age, individuals tend to gain more knowledge and exposure to mental health issues, which might increase their level of MHL. In line with these results, students from late years of undergraduate programs report higher MHL scores [26]. The field of study also plays a pivotal role in MHL scoring. Studying psychology or medicine, for instance, is associated with high MHL scores since students of these disciplines have heard of and could define mental health problems based on their studies [27]. Previous research has shown that international students had a lower MHL score because of a
different conceptualisation of MHL relating to cultural diversity [28]. Finally, low revenues correspond to limited MHL. This might be explained by a poor parental education level with negative attitudes toward the mentally ill [25].

In order to increase students' MHL, several interventions have been implemented worldwide [29,30]. These included multi-channel communication (posters, social network posts, email, etc.) [31], digital videos [32], mental health first-aid programs [33], etc. There are several categories of MHL interventions: whole-of-community campaigns; community campaigns addressed exclusively to young people; university-based interventions teaching MHL; and programs training individuals in a mental health crisis.

To assess their effectiveness, some interventions underwent evaluation through different designs (ex. mixed-methods, randomized controlled trials, pre-post studies) using different outcome measures. The majority of the questionnaires employed for the evaluation were ad hoc ones [34], and validated scales were used only to a lesser extent [30]. The use of the MHLS to assess the improvement of MHL is recommended [5,35].

## 1.4. *The Present Study*

The objective of this study was to validate the French version of the MHLS, the MHSL-FR, in a group of French university students. The psychometric qualities of the translated and adapted scale were explored. The MHLS-FR is meant to be used to measure MHL in French students so as to inform the development of educational programs addressed to them. The scale should also be used to evaluate the effectiveness of these programs aimed to increase French students' MHL.

# 2. Materials and Methods

## 2.1. *Translation of the MHLS and Item Adaptation*

The original MHLS was translated in French and back-translated in English following the standard methodologies for questionnaire translation [36]. Like the original scale, translated version initially contained 35 items. During a test phase, wording of the items was revised by 5 university students in order to make reading and comprehension accessible. The final version was read and approved by a panel of 6 public health researchers and mental health professionals.

## 2.2. *Participants and Data Collection*

Data collection took place between October and December 2019 at the University of Bordeaux, France. Self-administered questionnaires were distributed by peers to university students. A total of 550 participants were approached at the entrance of the university libraries and canteens. All participants had to be ≥18 years and French speaking. Before starting the questionnaire, respondents were informed about the content of the survey and asked for their consent. In total, 482 students out of 550 (87.6% response rate) completed the questionnaire (test). Their mean age was 20.46 (standard deviation, SD 2.46), and the majority were female (68.5%) and French students (89.4%). They were mostly healthcare students (37.6%) and attended the 1st year (33.2%). Their main source of income was family (63.7%) and they are for the most part living with less than 500 euros per month (46.9%). Among these students from the test phase, participants who completed the questionnaire twice (retest) were those who had voluntarily left their email address to be contacted again (n = 72). Finally, only 51 students out of 72 (70.8% response rate) agreed to participate in the retest with a final retention rate of 10.6% (51/482).

The study was conducted in compliance with the Declaration of Helsinki and with the approval of French Research Ethics Committee and the University of Bordeaux. Consent forms were obtained from the participants before data collection. The anonymity of the participants was maintained throughout the study with anonymized dataset.
## 2.3. Measurement and Missing Values

The full translated French version of the scale was identical to the original MHLS. Items were scored according to a Likert scoring system. The first dimension "ability to recognize specific disorders or different types of psychological distress" was composed of 8 items; the second dimension "knowledge and beliefs about risk factors and causes" of 2 items; the third dimension "knowledge and beliefs about self-help interventions" of 5 items; the fourth dimension "knowledge of where to seek information" of 4 items; the fifth dimension "wrong beliefs about mental health" of 9 items; and the sixth dimension "attitudes towards people with a mental health problem" of 7 items.

Items with a 4-point scale were rated 1—very unlikely/unhelpful, 4—very likely/helpful and for 5-point scale 1—strongly disagree/definitely unwilling, 5—strongly agree/definitely willing. The total score ranged from 35 to 160. Higher values corresponded to higher levels of MHL.

Only the item "A mental illness is not a real medical illness" (MAL_MED), due to an informatic error in the coding, presented 70.8% (341/482) missing values. We preferred reporting the descriptive and reliability analyses of this item with the 141 valid values by using pairwise case exclusion when introduced in the factorial models.

## 2.4. Data Analyses

We followed the procedure suggested by the Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) [37,38].

First, we evaluated the facet validity of the French version of the MHLS through descriptive statistics and the judgement of a panel of experts (professional psychologists, epidemiologists, health professionals, researchers, and lecturers). An online 8-item checklist was sent to the experts to rate the MHLS-FR according to the criteria of acceptability, practicality, and relevance.

Second, we analysed the distribution of the items per dimension through percentages. For sake of clarity, we use the term "factor" when referring to our scale and "dimension" when referring to the original MHLS. The two terms are actually synonyms and explain to the same concept. Response options were reported to assess ceiling effect. We used descriptive methods to evaluate the relevance and the comprehensiveness of the items in the population under study. We used sex and field of study as the reference variables to distinguish the 35 items. Sex was used as a well-known variable influencing mental health status and, consequently, also familiarity with mental health topics. Field of study was used as a variable that characterizes students; we wanted to observe differences between students with high knowledge of health and psychology subjects versus students from other disciplines. We recoded the items so that values from 0 to 2 or from 0 to 3 were considered as low MHL (0) and >2 or >3 as higher MHL. When present, the answer "I don't know" was scored as 0.

Third, we performed the test-retest reliability using the intraclass correlation coefficient (ICC) [39] calculated in the sub-sample of 51 students of the retest. Values < 0.40, 0.41–0.75, and >0.75 were indicative of bad, good, and very good reliability, respectively [39]. Calculations were made for each item and for each factor before and after the validation process.

Fourth, we assessed dimensionality and structural validity through Exploratory Structural Equation Model (ESEM) [40] in order to allow items to present cross-loadings [41] between obtained factors so as to provide a better fit compared to the Confirmatory Factor Analysis (CFA). We aimed to identify the dimensions proposed by O'Connor and Casey [5] so as to confirm them by using CFA models. In both ESEM and CFA models we used the polychoric correlation matrix [42] between the items of the scale and the Weighted Least-Square Mean and Variance (WLSMV) method. For evaluating both models, we used two types of criteria: related to the items and related to the global model. For acceptable items, we used the factor loadings superior to 0.3 and the communalities (R-square) superior to 0.25. For the global model, we used the standard criteria: Chi-square test ($\chi^2$); the Comparative Fit Index (CFI), and Tucker–Lewis Index (TLI) with values >0.90 considered
adequate (preferably >0.95); Root Mean Square Error of Approximation (RMSEA) with cut-off values of <0.8 (preferably <0.5); and Standardized Root Mean Square Residual (SRMR) where values <0.10 were considered as acceptable. The correlations between factors were evaluated using Pearson's r.

Fifth, concerning the structural validity, for the analysis of the Measurement Invariance (MI) of the obtained factors [43] across sex groups, we used CFA multigroups models for the ordinal categorical variables [44], by using a top-down approach [45] to compare the two added models [46]. We started with the most restrictive or complete invariance model where all factor loadings and thresholds remain invariant within groups (H0 or scale invariance). Than we further compared results with the model where we released the equality restrictions of the previous parameters in models with the same factorial structure for each group (H1 or configural invariance). In order to evaluate the MI, we used the change criterion of the χ² [47] together with other alternative adjustment indicators [48,49] that suggested to accept MI for changes minor to 0.01 in CFI, minor to 0.015 in RMSEA, and minor to 0.030 in SRMR.

Sixth, for the reliability analysis and the internal consistency of the scale and the factors, we used the Cronbach's alpha coefficient [50] and the McDonald's omega [51]. Well-accepted values for both coefficients are between 0.70 and 0.90, respectively [52]. We also calculated the standard error of measurement (SEM) via the (SD) [53].

Finally, we used hypothesis tests using Student's t-test or ANOVA and Pearson's correlation coefficients to assess the convergent and discriminant validity of the total scale score and of the obtained factors with the sociodemographic variables sex, age, year, and field of study, being an international student, main source of income, and monthly all-inclusive resources. All *p* values were two-tailed, and we considered a *p* < 0.05 to be statistically significant. Analyses were conducted using SPSS v24 (SPSS, Chicago, IL, USA) and MPlus v8.4.

## 3. Results

### 3.1. Face Validity

Experts rated very high acceptability at 48.9%, very high practicality at 68.9%, and very high relevance at 46.6%. Remaining percentages corresponded to high or medium values for each criterion.

### 3.2. Distribution of the Items

We analysed the distribution of the items and the psychometric proprieties of the original MHLS with the 35 translated items. Acronyms and short definitions are available in Table S1 in the Supplementary Material. Mean score was 90.52 (SD = 8.95, Minimum = 60.00, Maximum = 115.00, 95% CI = 89.72–91.72).

Table S2 in the Supplementary Material shows that all the answer options of the items were used (e.g., from “very unlikely” to “very likely”). This suggests that some items were more difficult and had lower capacity of discrimination.

We analysed the differences across the answers according to sex and field of study chosen a priori among covariates (Table S3 in the Supplementary Material). We observed that PHO_SOC, DYS, BIPOL, TCC, MAL_MED, PAS_FORT, and POLI_MAL were significantly different (*p* values from 0.002 to 0.036) between females and males. PHO_SOC, ANX_GEN, DYS, FEM_RIS_MAL, TCC, DIF_ANX_R, DANGER_R, TRAIT_PAS_EFF_R, and MARIA_MAL were significantly different across fields of study (*p* values from 0.001 to 0.047). Overall, the majority of the 35 items (between 75 to 80%) were not different across these two sociodemographic variables, i.e., items were independent.
### 3.3. Test Retest

Table S2 in the Supplementary Material shows the ICCs for each item. The majority of the items presented acceptable values. There was about 63% of items with good ICCs and 37% of bad items, although only three items had ICCs below 0.2: TROU_PERSO, TCC, and DIF_ANX.

### 3.4. Dimensionality

We used ESEM models from six factors, and we obtained models whose factors had factor loadings superior to 0.3 and items with communalities (R-square) superior to 0.25. For this, we eliminated items or factors so as to obtain good fit indicators, until we obtained the possible dimension or sub-dimensions of the original MHLS in our study population.

Table 1 shows the different models we obtained until the proposed criteria were met. Within them, we had to eliminate the dimension of “knowledge and beliefs about self-help interventions”, explained by the items SOMM and DIF_ANX, in addition to other 5 items that did not reach a communality higher than 0.25.

<table><caption>**Table 1.** ESEM models and indicators.</caption><thead><tr><th></th><th></th><th>Chi-Squ.<br>Base</th><th>Chi-Squ</th><th>df</th><th>RMSEA</th><th>90 CI</th><th>CFI</th><th>TLI</th><th>SMRM</th></tr></thead><tbody><tr><td>MODEL 1</td><td>ESEM 35 items; 6 factors</td><td>5,339,633</td><td>630,037</td><td>400</td><td>0.035</td><td>0.029 0.040</td><td>0.952</td><td>0.928</td><td>0.041</td></tr><tr><td>MODEL 2</td><td>ESEM 35 items; 7 factors</td><td>5,339,633</td><td>554,691</td><td>371</td><td>0.032</td><td>0.026 0.037</td><td>0.961</td><td>0.938</td><td>0.038</td></tr><tr><td>MODEL 3</td><td>ESEM 35 items; 8 factors<br>ESEM 32 items; 8 factors</td><td>5,339,633</td><td>490,922</td><td>343</td><td>0.03</td><td>0.024 0.036</td><td>0.969</td><td>0.946</td><td>0.033</td></tr><tr><td>MODEL 4</td><td>Elim: DEPRESS, DIF_ANX,<br>DANGER<br>ESEM 28 items; 8 factors</td><td>5,205,246</td><td>371,601</td><td>343</td><td>0.028</td><td>0.021 0.035</td><td>0.978</td><td>0.959</td><td>0.031</td></tr><tr><td>MODEL 5</td><td>Elim: CONFI_PROB ENT,<br>SOMM, VOU_SOR,<br>DIR_PER</td><td>4,850,961</td><td>233,366</td><td>297</td><td>0.024</td><td>0.014 0.033</td><td>0.989</td><td>0.976</td><td>0.026</td></tr></tbody></table>

Table 2 presents the ESEM model 5 whose RMSEA, CFI, TLI, and SMRM fit indicators were excellent. Through the ESEM, we meant to explore whether the factors found in our study corresponded to the dimensions of the original MHLS.

In model 5, the three dimensions obtained by O'Connor and Casey on “knowledge of risk factors and causes”, “knowledge and beliefs about self-help intervention”, and “knowledge of where to seek information” were present in our population, and sensibly with the same items than the original scale (except for the item CONFI_PROB ENT). Regarding the other two dimensions, the factor “ability to recognize disorders” was divided into two sub-factors that appeared highly significantly correlated (r = 0.401), and the factor “stigmatisation” was divided into four sub-factors, two related to “wrong beliefs about mental health” and the other two related to “attitudes towards people with a mental health problem”. It is worth noting that the correlation (r = 0.407) between the two sub-factors corresponding to “stigmatisation” was significant. Factor 2 included one sub-factor of the “ability to recognize disorders” and of “knowledge and beliefs about self-help interventions”. Table 2 presents the factors identified in our study crossed with the dimensions of MHLS. Factors with high correlations (ex. F1 and F2) allows to think that they both define the same dimension corresponding to the MHLS related dimension. Correlations between factors inform the following CFA.
<table><caption>**Table 2.** MHLS-FR dimensions based on model 5.</caption><thead><tr><th rowspan="2"></th><th colspan="8">Factors of the MHLS-FR</th><th rowspan="2">R Square</th><th rowspan="2">O'Connor and Casey's<br>MHLS Dimensions</th></tr><tr><th>F1</th><th>F2</th><th>F3</th><th>F4</th><th>F5</th><th>F6</th><th>F7</th><th>F8</th></tr></thead><tbody><tr><td>PHO_SOC_1</td><td><strong>0.625</strong></td><td>0.031</td><td>0.117</td><td>-0.006</td><td>0.012</td><td>-0.007</td><td>-0.020</td><td>-0.022</td><td>0.451</td><td rowspan="8">Ability to recognize<br>disorders</td></tr><tr><td>ANX_GEN_2</td><td><strong>0.575</strong></td><td>0.094</td><td>-0.007</td><td>0.046</td><td>-0.028</td><td>0.007</td><td>-0.049</td><td>-0.009</td><td>0.394</td></tr><tr><td>TROU_PERSO_4</td><td>0.083</td><td><strong>0.407</strong></td><td>0.126</td><td>0.165</td><td>-0.073</td><td>0.021</td><td>-0.050</td><td>-0.045</td><td>0.309</td></tr><tr><td>DYS_5</td><td>0.005</td><td><strong>0.538</strong></td><td>-0.024</td><td>0.166</td><td>-0.040</td><td>-0.203</td><td>0.047</td><td>-0.066</td><td>0.295</td></tr><tr><td>AGORA_6</td><td>0.083</td><td><strong>0.635</strong></td><td>-0.047</td><td>-0.103</td><td>0.024</td><td>0.039</td><td>0.040</td><td>-0.001</td><td>0.449</td></tr><tr><td>BIPOL_7</td><td>0.091</td><td><strong>0.447</strong></td><td>0.041</td><td>0.017</td><td>0.039</td><td>0.052</td><td>-0.011</td><td>-0.123</td><td>0.281</td></tr><tr><td>DRUG_8</td><td>0.018</td><td><strong>0.660</strong></td><td>0.020</td><td>0.001</td><td>0.088</td><td>0.010</td><td>0.002</td><td>0.057</td><td>0.474</td></tr><tr><td>FEM_RIS_MAL_9</td><td>-0.040</td><td>0.224</td><td>-0.011</td><td>0.024</td><td><strong>-0.562</strong></td><td>-0.017</td><td>-0.029</td><td>-0.022</td><td>0.368</td><td rowspan="4">Knowledge of risk factors<br>and causes<br>Knowledge and beliefs about<br>self-help interventions</td></tr><tr><td>HOM_RIS_ANX_10R</td><td>-0.066</td><td>0.101</td><td>-0.030</td><td>0.036</td><td><strong>0.645</strong></td><td>0.018</td><td>-0.005</td><td>-0.039</td><td>0.433</td></tr><tr><td>TCC_13</td><td>0.073</td><td><strong>0.653</strong></td><td>0.003</td><td>0.022</td><td>-0.008</td><td>-0.272</td><td>0.058</td><td>0.049</td><td>0.429</td></tr><tr><td>CONFI_DANG_IMM_14</td><td>-0.169</td><td><strong>0.516</strong></td><td>0.120</td><td>-0.019</td><td>-0.018</td><td>0.169</td><td>-0.093</td><td>0.135</td><td>0.377</td></tr><tr><td>INFO_MAL_16</td><td>-0.091</td><td>0.097</td><td><strong>0.763</strong></td><td>-0.066</td><td>0.025</td><td>-0.069</td><td>0.000</td><td>-0.021</td><td>0.581</td><td rowspan="5">Knowledge of where<br>to seek information</td></tr><tr><td>ORDI_INFO_17</td><td>0.012</td><td>0.000</td><td><strong>0.683</strong></td><td>-0.015</td><td>0.025</td><td>0.053</td><td>-0.025</td><td>0.110</td><td>0.504</td></tr><tr><td>QUES_MAL_18</td><td>0.080</td><td>-0.138</td><td><strong>0.377</strong></td><td>0.065</td><td>-0.020</td><td>-0.065</td><td>0.398</td><td>0.041</td><td>0.373</td></tr><tr><td>RESS_INFO_19</td><td>0.036</td><td>0.000</td><td><strong>0.648</strong></td><td>0.069</td><td>-0.036</td><td>0.154</td><td>0.086</td><td>-0.051</td><td>0.516</td></tr><tr><td>FRAG_PERS_21R</td><td>-0.018</td><td>-0.005</td><td>0.030</td><td>0.106</td><td>0.027</td><td><strong>0.460</strong></td><td>0.040</td><td>-0.040</td><td>0.251</td></tr><tr><td>MAL_MED_22R</td><td>0.491</td><td>0.043</td><td>0.001</td><td>-0.197</td><td>0.001</td><td><strong>0.395</strong></td><td>0.035</td><td>0.070</td><td>0.413</td><td rowspan="10">Stigmatisation</td></tr><tr><td>FREQ_DEV_24R</td><td>0.000</td><td>-0.019</td><td>-0.005</td><td>0.298</td><td>0.068</td><td><strong>0.566</strong></td><td>-0.012</td><td>0.036</td><td>0.498</td></tr><tr><td>PAS_FORT_26R</td><td>0.097</td><td>0.020</td><td>-0.024</td><td>0.118</td><td>-0.021</td><td><strong>0.523</strong></td><td>0.436</td><td>-0.043</td><td>0.597</td></tr><tr><td>NO_AID_27R</td><td>-0.076</td><td>0.008</td><td>0.027</td><td>-0.051</td><td>0.056</td><td>0.023</td><td><strong>0.920</strong></td><td>0.016</td><td>0.865</td></tr><tr><td>TRAIT_PAS_EFF_28R</td><td>-0.045</td><td>0.057</td><td>0.009</td><td>-0.033</td><td>-0.017</td><td>0.357</td><td><strong>0.390</strong></td><td>0.013</td><td>0.336</td></tr><tr><td>HAB_MAL_29</td><td>-0.034</td><td>0.052</td><td>0.013</td><td><strong>0.532</strong></td><td>0.083</td><td>0.079</td><td>-0.010</td><td>0.134</td><td>0.410</td></tr><tr><td>DISC_MAL_30</td><td>-0.031</td><td>0.164</td><td>-0.016</td><td><strong>0.806</strong></td><td>0.029</td><td>-0.008</td><td>0.031</td><td>-0.010</td><td>0.705</td></tr><tr><td>AMI_MAL_31</td><td>0.011</td><td>0.021</td><td>0.006</td><td><strong>0.821</strong></td><td>-0.051</td><td>0.025</td><td>-0.022</td><td>0.085</td><td>0.752</td></tr><tr><td>TRAV_MAL_32</td><td>0.027</td><td>-0.040</td><td>0.009</td><td><strong>0.608</strong></td><td>-0.019</td><td>0.019</td><td>0.039</td><td>0.313</td><td>0.639</td></tr><tr><td>MARIA_MAL_33</td><td>0.128</td><td>-0.059</td><td>0.037</td><td>0.288</td><td>0.195</td><td>-0.071</td><td>-0.011</td><td><strong>0.526</strong></td><td>0.561</td></tr><tr><td>POLI_MAL_34</td><td>-0.017</td><td>0.074</td><td>-0.010</td><td>-0.008</td><td>0.003</td><td>0.035</td><td>-0.013</td><td><strong>0.806</strong></td><td>0.661</td></tr><tr><td>EMB_MAL_35</td><td>-0.028</td><td>0.025</td><td>-0.034</td><td>0.112</td><td>-0.047</td><td>0.012</td><td>0.063</td><td><strong>0.691</strong></td><td>0.561</td></tr><tr><td colspan="11">Correlations between factors</td></tr><tr><td>F1</td><td>1.000</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>F2</td><td>0.401*</td><td>1.000</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>F3</td><td>0.235*</td><td>0.267*</td><td>1.000</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>F4</td><td>0.192*</td><td>0.164*</td><td>0.145*</td><td>1.000</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>F5</td><td>-0.073</td><td>-0.022</td><td>0.013</td><td>0.050</td><td>1.000</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>F6</td><td>0.010</td><td>0.372*</td><td>0.087</td><td>0.196</td><td>0.184</td><td>1.000</td><td></td><td></td><td></td><td></td></tr><tr><td>F7</td><td>0.084</td><td>0.105</td><td>0.169*</td><td>0.154*</td><td>0.106</td><td>0.158*</td><td>1.000</td><td></td><td></td><td></td></tr><tr><td>F8</td><td>0.044</td><td>0.060</td><td>0.126*</td><td>0.407*</td><td>0.102</td><td>0.141</td><td>0.123*</td><td>1.000</td><td></td><td></td></tr></tbody></table>

### 3.5. *Structural Validity*

First, dimensional analyses were performed through the ESEM models. Then, we used different CFA models in order to confirm the factors or sub-factors obtained by the original MHLS. In the search for the best CFA model, we continued to use the criteria described for factor loadings and communalities, and eliminated the items FRAG_PERS_21 and MAL_MED_22 for having low communalities. Table 3 shows the model with the 26 items and the six factors, where there were still three items (PHO_SOC_1, ANX_GEN_2, and HOM_RIS_ANX_10) with a communality inferior to 0.25 but with a global acceptable fit (RMSEA = 0.054, CFI = 0.909, TLI = 0.895 and SMRM = 0.061) and a sound interpretation of the obtained factors (i.e., coherence of the items and comparability with the dimensions of the original scale). The Table crosses again the factors we obtained with the dimensions of the original scale. Significant correlations confirmed the division of the original dimensions.

The factors obtained in Table 4 (visual representation in Figure S1) show the presence of 5 of the 6 dimensions obtained by O'Connor and Casey, and a strong correlation between them except for dimension 2 "knowledge of risk factors and causes".

<footnote>* significant (*p* < 0.05). Factor 1: Ability to recognize disorders. Factor 2: Ability to recognize disorders + Knowledge and beliefs about self-help interventions. Factor 3: Knowledge of where to seek information. Factor 4: Stigmatisation. Factor 5: Knowledge of risk factors and causes. Factor 6–8: Stigmatisation. Values in bold correspond to factor loadings >0.3 which are necessary to group items in a factor.</footnote>
<table><caption>**Table 3.** Final MHLS-FR version with 26 items and 6 factors.</caption><thead><tr><th rowspan="2"></th><th colspan="6">Factors of the MHLS-FR</th><th rowspan="2">R Square</th><th rowspan="2">O'Connor and Casey's<br>MHLS Dimensions</th></tr><tr><th>F1</th><th>F2</th><th>F3</th><th>F4</th><th>F5</th><th>F6</th></tr></thead><tbody><tr><td>PHO_SOC_1</td><td>0.434</td><td></td><td></td><td></td><td></td><td></td><td>0.188</td><td></td></tr><tr><td>ANX_GEN_2</td><td>0.427</td><td></td><td></td><td></td><td></td><td></td><td>0.183</td><td></td></tr><tr><td>TROU_PERSO_4</td><td>0.589</td><td></td><td></td><td></td><td></td><td></td><td>0.347</td><td rowspan="6">Ability to recognize<br>disorders</td></tr><tr><td>DYS_5</td><td>0.486</td><td></td><td></td><td></td><td></td><td></td><td>0.236</td></tr><tr><td>AGORA_6</td><td>0.585</td><td></td><td></td><td></td><td></td><td></td><td>0.342</td></tr><tr><td>BIPOL_7</td><td>0.490</td><td></td><td></td><td></td><td></td><td></td><td>0.240</td></tr><tr><td>DRUG_8</td><td>0.681</td><td></td><td></td><td></td><td></td><td></td><td>0.464</td></tr><tr><td>FEM_RIS_MAL_9</td><td></td><td>0.960</td><td></td><td></td><td></td><td></td><td>0.922</td><td>Knowledge of risk<br>factors and causes</td></tr><tr><td>HOM_RIS_ANX_10</td><td></td><td>0.368</td><td></td><td></td><td></td><td></td><td>0.136</td><td></td></tr><tr><td>TCC_13</td><td></td><td></td><td>0.560</td><td></td><td></td><td></td><td>0.314</td><td>Knowledge and beliefs<br>about self-help interventions</td></tr><tr><td>CONFI_DANG_IMM_14</td><td></td><td></td><td>0.514</td><td></td><td></td><td></td><td>0.264</td><td></td></tr><tr><td>INFO_MAL_16</td><td></td><td></td><td></td><td>0.615</td><td></td><td></td><td>0.378</td><td rowspan="5">Knowledge of where to<br>seek information</td></tr><tr><td>ORDI_INFO_17</td><td></td><td></td><td></td><td>0.697</td><td></td><td></td><td>0.485</td></tr><tr><td>QUES_MAL_18</td><td></td><td></td><td></td><td>0.507</td><td></td><td></td><td>0.257</td></tr><tr><td>RESS_INFO_19</td><td></td><td></td><td></td><td>0.756</td><td></td><td></td><td>0.571</td></tr><tr><td>FREQ_DEV_24R</td><td></td><td></td><td></td><td></td><td>0.653</td><td></td><td>0.427</td></tr><tr><td>PAS_FORT_26R</td><td></td><td></td><td></td><td></td><td></td><td>0.775</td><td>0.601</td><td></td></tr><tr><td>NO_AID_27R</td><td></td><td></td><td></td><td></td><td>0.560</td><td></td><td>0.314</td><td></td></tr><tr><td>TRAIT_PAS_EFF_28R</td><td></td><td></td><td></td><td></td><td>0.526</td><td></td><td>0.277</td><td></td></tr><tr><td>HAB_MAL_29</td><td></td><td></td><td></td><td></td><td></td><td>0.623</td><td>0.388</td><td rowspan="6">Stigmatisation</td></tr><tr><td>DISC_MAL_30</td><td></td><td></td><td></td><td></td><td></td><td>0.790</td><td>0.625</td></tr><tr><td>AMI_MAL_31</td><td></td><td></td><td></td><td></td><td></td><td>0.819</td><td>0.670</td></tr><tr><td>TRAV_MAL_32</td><td></td><td></td><td></td><td></td><td></td><td>0.783</td><td>0.612</td></tr><tr><td>MARIA_MAL_33</td><td></td><td></td><td></td><td></td><td></td><td>0.655</td><td>0.430</td></tr><tr><td>POLI_MAL_34</td><td></td><td></td><td></td><td></td><td></td><td>0.618</td><td>0.382</td></tr><tr><td>EMB_MAL_35</td><td></td><td></td><td></td><td></td><td></td><td>0.620</td><td>0.384</td><td></td></tr><tr><td colspan="9">Correlations between factors</td></tr><tr><td>F1</td><td>1.000</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>F2</td><td>0.177*</td><td>1.000</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>F3</td><td>0.993*</td><td>0.187</td><td>1.000</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>F4</td><td>0.383*</td><td>0.004</td><td>0.428*</td><td>1.000</td><td></td><td></td><td></td><td></td></tr><tr><td>F5</td><td>0.319*</td><td>-0.132</td><td>0.306*</td><td>0.378*</td><td>1.000</td><td></td><td></td><td></td></tr><tr><td>F6</td><td>0.257*</td><td>-0.062</td><td>0.321*</td><td>0.252*</td><td>0.413*</td><td>1.000</td><td></td><td></td></tr></tbody></table>

<table><caption>**Table 4.** Measurement of invariance in the CFA model.</caption><thead><tr><th></th><th>WLSMV χ² (df)</th><th>CFI</th><th>RMSEA</th><th>SRMR</th><th>χ² (df)</th><th>p</th><th>CFI</th><th>RMSEA</th><th>SRMR</th></tr></thead><tbody><tr><td>Baseline Models</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Males (n = 152)</td><td>462,441 (284)</td><td>0.852</td><td>0.064</td><td>0.09</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Females (n = 330)</td><td>526,535 (289)</td><td>0.922</td><td>0.05</td><td>0.066</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Measurement Invariance for sex</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Scalar model (H0)</td><td>1,063,427 (653)</td><td>0.898</td><td>0.051</td><td>0.078</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Configural model (H1)</td><td>1,002,902 (580)</td><td>0.895</td><td>0.055</td><td>0.076</td><td>92.519 (73)</td><td>0.0612</td><td>-0.003</td><td>0.004</td><td>-0.002</td></tr></tbody></table>

H0: factor loadings and thresholds free across groups, scale factors fixed at one in all groups, and factor means fixed at zero in all groups. H1: factor loadings and thresholds constrained to be equal across groups, scale factors fixed at one in one group, and free in the other groups, and factor means fixed at zero in one group and free in the other groups.

The fit coefficients of the parameters of the simple and multigroup models were acceptable, and the changes in the absolute values of the fit indices were within the range proposed by Chen [49] for accepting the MI. The description of the distribution of each of the dimension scores obtained and of the total scale as well as the indicators of reliability, internal consistency and test-retest is shown in Table S4. Standard error of measurement was also calculated and found to be 3.03.

### 3.6. Internal Consistency

Values of the internal consistency of the completed questionnaire and of the six dimensions of the original MHLS scale are reported in Table 5. Only the complete questionnaire and factor 7 reported a good internal consistency. Nevertheless, we obtained factors with alpha values that were low or negative (due to reversed items in the translated scale) ranging from -0.827 to 0.815.

<footnote>* significant (*p* < 0.05). F1: Ability to recognize disorders. F2: Knowledge of risk factors and causes. F3: Knowledge and beliefs about self-help interventions. F4: Knowledge of where to seek information. F5 and F6: Stigmatisation.</footnote>
<table><caption>**Table 5.** Scores of the scale and the dimensions: description, reliability, internal consistency, and test-retest reliability.</caption><thead><tr><th></th><th>Total<br>Mean (SD)</th><th>Range</th><th>Cronbach's<br>Alpha</th><th>McDonald<br>Omega</th><th>ICC: IC95% (n = 51)</th></tr></thead><tbody><tr><td>F1</td><td>22.08 (2.78)</td><td>10–28</td><td>0.643</td><td>0.872</td><td>0.772 IC95% = (0.601; 0.870)</td></tr><tr><td>F2</td><td>4.56 (1.33)</td><td>2–8</td><td>0.453</td><td>0.625</td><td>0.637 IC95% = (0.365; 0.793)</td></tr><tr><td>F3</td><td>6.35 (1.10)</td><td>2–8</td><td>0.343</td><td>0.666</td><td>0.291 IC95% = (−0.242; 0.595)</td></tr><tr><td>F4</td><td>15.05 (2.77)</td><td>5–20</td><td>0.608</td><td>0.797</td><td>0.908 IC95% = (0.838; 0.947)</td></tr><tr><td>F5</td><td>17.12 (2.40)</td><td>8–20</td><td>0.595</td><td>0.796</td><td>0.793 IC95% = (0.637; 0.882)</td></tr><tr><td>F6</td><td>25.36 (5.15)</td><td>11–35</td><td>0.815</td><td>0.873</td><td>0.867 IC95% = (0.7671; 0.924)</td></tr><tr><td>Total Scale</td><td>90.52 (8.95)</td><td>60–115</td><td>0.765</td><td>0.961</td><td>0.869 IC95% = (0.770; 0.925)</td></tr></tbody></table>

In order to analyse the effect of the percentage of missing values for the item MAL_MED, we also obtained the alpha coefficient of the factor 6 of the complete questionnaire without including MAL_MED (i.e., with *n* = 482), obtaining a similar result: 0.583 for the factor 6 and 0.765 in total.

### 3.7. Convergent and Discriminant Validity

We tested the hypotheses that we could obtain different scores across sociodemographic characteristics. We hypothesized that female students presented higher scores, however this hypothesis was not confirmed (*p* = 0.212). For age, we hypothesized that older students presented a higher MHL score. Correlation coefficients were small even if some of them were significant maybe because of the sample size, thus confirming our hypothesis. International students presented overall lower scores of MHL than national students (*p* = 0.018). Concerning the field of study, the maximum score was reported in students from Human and Social Sciences, including Psychological Studies (93/119). The second maximum score was reported in students from Health Studies (91.2/119). No statistically significant differences were found with regards to the variables main source of income and monthly all-inclusive resources, *p* = 0.117 for both variables.

All this considered, the MHLS-FR was composed of 26 items with a score from 0 to 119. The items were: PHO_SOC_1; ANX_GEN_2; TROU_PERSO_4; DYS_5; AGORA_6; BIPOL_7; DRUG_8; FEM_RIS_MAL_9; HOM_RIS_ANX_10; TCC_13; CONFI_DANG Imm_14; INFO_MAL_16; ORDI_INFO_17; QUES_MAL_18; RESS_INFO_19; FREQ_DEV_24R; PAS_FORT_26R; NO_AID_27R; TRAIT_PAS_EFF_28R; HAB_MAL_29; DISC_MAL_30; AMI_MAL_31; TRAV_MAL_32; MARIA_MAL_33; POLI_MAL_34; and EMB_MAL_35. Six factors were represented: "ability to recognize disorders" (7 items), "knowledge of sex as a risk factor" (2 items), "knowledge and beliefs about self-help interventions" (2 items), "knowledge of where to seek information" (4 items), and "stigmatisation" (11 items). The acronyms and grouped factors are explained in Table S1 in the Supplementary Material.

## 4. Discussion

The French version of the MHLS presented good psychometric proprieties and was capable of measuring comprehensively MHL in the target population of students. The MHLS-FR was composed of 26 items, 9 less than in the original MHLS. The reduction of the items can be explained by the fact that some of them were highly correlated and could be associated one with the other given their similar contents (e.g., "mental health is fragility" and "mental health is an illness"). In our study, some items were not discriminatory of a specific sub-factor and were removed based on both statistical features and interpretability of the results. The six factors demonstrated reliable psychometric proprieties and fully covered the original representation of the MHL construct. By means of comparison, the Pakistani version of the MHLS was composed of 34 items [54], the Iranian one of 29 items [55], and the Arabic one of 28 items [56]. In the studies conducted in China [57], Portugal [58], South Africa, and Zambia [59], all items were kept.

Concerning descriptive statistics, the mean MHL score calculated in the sample was 90.52/119. This score was higher than in previous studies on students' MHL [21–24] and
even in the study of O'Connor and Casey [5] where the mean score was 127.38/160. Variables such as countries where the study was conducted, cultures, educational training, and language might have influenced the results. Translation of the items may have had an impact on the interpretability of the items. The difficulty of the items must also be considered: the complete scale might include more complex items for young adults compared to the 26-item MHLS-FR. Furthermore, the sample of this study was unbalanced: there were higher proportions of female students and of healthcare students. As reported in the literature [21,27], the MHL scores of these populations are higher than their peers. This study is not representative of all French students and a larger survey would better capture their MHL scores. In any case, a score 30 points lower than the maximum score is not satisfactory and, there is room for improvement. Increasing MHL scores remains a global public health priority [60]. Interventions are needed among students even if at different degrees depending on the MHL scores of concerned populations.

In this study, the Cronbach's alpha was acceptable, 0.744. It scored better in our final scale with 26 items compared to the original scale (0.765). This corroborates our findings: the new MHLS-FR tool was slightly more informative among French students than the original full scale. By means of comparison, the Cronbach's alpha level of the original scale by O'Connor and Casey [5] was of 0.873. For test-retest reliability, results were positive: r(51) = 0.869, *p* < 0.001. The standard error of measurement (SEM) was 4.03. In the original scale [5], the retest results also showed good reliability (r(69) = 0.797, *p* < 0.001). To the same extent as the scores calculated in other versions of the scale, these results must be interpreted with caution. The scales are not fully comparable: translation and different interpretation of the items might have influenced these results.

For convergent and divergent validity, scores of MHL were different only for the variables age and nationality, thus showing that the scale measured MHL independently from sociodemographic characteristics. Similar results were found in the Pakistani study [54].

Strengths of this study include the fact that it produced a golden standard for measuring MHL in a French-speaking population. The use of this scale will benefit several communities of French-speaking young people given the increasing importance of MHL in the field of mental health promotion and disease prevention. The MHLS-FR can be used as a solid and reliable tool to measure the effectiveness of interventions aimed at improving MHL of young people. Its items and factors can also inform the design and development of such interventions. Contents to be covered should be those where students have the lower scores, the objective being to empower them for better mental health outcomes through a process-based learning approach. For instance, increased knowledge makes the student confident to talk to someone about depression, which leads to help-seeking behaviour. To the same extent knowledge leads to self-efficacy, which, in turn, leads to healthy behaviours. The study also answers the need to extend the generalisability of the Australian MHLS with other international samples and provides hints for developing statistically robust norms that can be used to guide the use of this scale [5].

This study is not without limitations. The sample of the retest was recruited on a voluntary basis and was not comparable to the larger sample of the first phase. This limited number of individuals for the retest phase can be explained by the difficulties researchers often face for retaining young people in repeated measure surveys [61,62]. However, the number of 51 was enough for the retest based on statistical power [63]. One value was missing for several respondents, but it was finally removed from the factorial analysis, thus not impacting on the final results. In the final scale of six factors, three items presented (R-square) inferior to 0.25, but, for the sake of interpretability, we included them in the tool.

**5. Conclusions**

The main significance of the study is to provide the first French instrument to measure MHL in students. Its good psychometric proprieties guarantee that the MHLS-FR can be used to collect reliable and robust data. With only 26 items, it is easy and quick to use. For this, it might be included in large surveys on students' health to produce information
advocating for the design and development of MHL education programs. Student Health Services will be alerted to the need for implementing these programs. More broadly, the University Executive Boards would be sensitised to students' MHL.

The MHLS-FR is also meant to be used to evaluate the impact of MHL education programs. It should be administered to assess the change in MHL score before and after the interventions following different designs: randomised controlled trial, pre-post-design, longitudinal study, etc. Effective interventions would promote positive attitudes towards mental health, facilitate access to care, promote help-seeking behaviour and reduce stigma.

Finally, further research is needed to replicate the present findings in other populations. The MHLS-FR offers considerable benefits in mental health research and practice, and its use should be extended beyond students.

**Supplementary Materials:** The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/bs12080259/s1, Figure S1: Representation of the confir- matory factor analyses. Table S1:List of acronyms inspired by the original MHLS according to the classification of O'Connor and Casey (2015). Table S2: Distribution (%) of the answers of the items of the proposed scale (with no inversed items) and test-retest reliability. Table S3: Distribution (%) of the answers of the items of the proposed scale MHLS (with inversed items). Table S4: Scores of the scale and dimensions: description, reliability, internal consistency and test-retest reliability.

**Author Contributions:** I.M. conceptualised the study, interpreted the data, and drafted the paper; J.L.G.C. analysed and interpreted the data and revised the manuscript. All authors have read and agreed to the published version of the manuscript.

**Funding:** This work was supported by the Nouvelle-Aquitaine Regional Council (Conseil Régional Nouvelle-Aquitaine) under grant n°4370420; the Nouvelle-Aquitaine Regional Health Agency (Agence Régionale de Santé Nouvelle-Aquitaine) under grant n°6066R-8; Public Health France (Santé publique France) under grant n°19DPPP023-0; and the National Institute against cancer (INCa) under grant n°INCa_11502. The funding bodies were neither involved in the study design, or in the collection, analysis, or interpretation of the data.

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Comité de Protection des Personnes (CPP, Personal Protection Committee) (protocol code: 2018-A02277-48).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The data that support the findings of this study are available from the corresponding author, I.M., upon reasonable request.

**Acknowledgments:** The authors are indebted to Amélie Capelle for managing the project, Fanny Boutellier for data management, and Rémy Cabannes for his active contribution to the conception and implementation of the study. Special thanks to the i-Share and the Espace Santé Etudiants teams of the University of Bordeaux for their advice and operational support. Finally, we are also indebted to Aude Pouymayou and Edwige Pereira for their statistical advice.

**Conflicts of Interest:** The authors report there are no competing interests to declare.

## References

1. Jorm, A.F.; Korten, A.E.; Jacomb, P.A.; Christensen, H.; Rodgers, B.; Pollitt, P. "Mental health literacy": A survey of the public's ability to recognise mental disorders and their beliefs about the effectiveness of treatment. *Med. J. Aust.* **1997**, *166*, 182–186. [CrossRef] [PubMed]

2. Jorm, A.F. Mental health literacy: Public knowledge and beliefs about mental disorders. *Br. J. Psychiatry* **2000**, *177*, 396–401. [CrossRef] [PubMed]

3. Kelly, C.M.; Jorm, A.F.; Wright, A. Improving mental health literacy as a strategy to facilitate early intervention for mental disorders. *Med. J. Aust.* **2007**, *187*, 26–30. [CrossRef] [PubMed]

4. Garcia, P.; McCarthy, M. *Measuring Health: A Step in the Development of City Health Profiles*; World Health Organization, Regional Office for Europe: Copenhagen, Denmark, 1996; pp. 1–68.

5. O'Connor, M.; Casey, L. The Mental Health Literacy Scale (MHLS): A new scale-based measure of mental health literacy. *Psychiatry Res.* **2015**, *229*, 511–516. [CrossRef]
6. Wei, Y.; McGrath, P.J.; Hayden, J.; Kutcher, S. Measurement properties of tools measuring mental health knowledge: A systematic review. *BMC Psychiatry* **2016**, *16*, 297. [CrossRef]

7. Wei, Y.; McGrath, P.J.; Hayden, J.; Kutcher, S. Measurement properties of mental health literacy tools measuring help-seeking: A systematic review. *J. Ment. Health* **2017**, *26*, 543–555. [CrossRef]

8. Cha, E.S.; Kim, K.H.; Erlen, J.A. Translation of scales in cross-cultural research: Issues and techniques. *J. Adv. Nurs.* **2007**, *58*, 386–395. [CrossRef]

9. Blanco, C.; Okuda, M.; Wright, C.; Hasin, D.S.; Grant, B.F.; Liu, S.M.; Olfson, M. Mental health of college students and their non-college-attending peers: Results from the national epidemiologic study on alcohol and related conditions. *Arch. Gen. Psychiatry* **2008**, *65*, 1429–1437. [CrossRef]

10. Hunt, J.; Eisenberg, D. Mental Health Problems and Help-Seeking Behavior Among College Students. *J. Adolesc. Health* **2010**, *46*, 3–10. [CrossRef]

11. Reddy, P.V. Awareness on Mental Health Problems and Help Seeking Behavior among College Students. *Int. J. Indian Psychol* **2017**, *5*, e82. [CrossRef]

12. Shapiro, M.; Morris, M.; Pumariega, A. Increased rates of mental health service utilization by US college students. *Psychiatr. Serv.* **2019**, *70*, 528. [CrossRef]

13. Storrie, K.; Ahern, K.; Tuckett, A. A systematic review: Students with mental health problems—A growing problem. *Int. J. Nurs. Pract.* **2010**, *16*, 1–6. [CrossRef]

14. Kohler Giancola, J.; Grawitch, M.J.; Borchert, D. Dealing with the stress of college: A model for adult students. *Adult Educ. Q.* **2009**, *59*, 246–263. [CrossRef]

15. Hernández-Torrano, D.; Ibrayeva, L.; Sparks, J.; Lim, N.; Clementi, A.; Almukhambetova, A.; Nurtayev, Y.; Muratkyzy, A. Mental health and well-being of university students: A bibliometric mapping of the literature. *Front. Psychol.* **2020**, *11*, 1226. [CrossRef]

16. Quek, T.T.; Tam, W.W.; Tran, B.X.; Zhang, M.; Zhang, Z.; Ho, C.S.; Ho, R.C. The Global Prevalence of Anxiety Among Medical Students: A Meta-Analysis. *Int. J. Environ. Res. Public Health* **2019**, *16*, 2735. [CrossRef]

17. Auerbach, R.P.; Mortier, P.; Bruffaerts, R.; Alonso, J.; Benjet, C.; Cuijpers, P.; Demyttenaere, K.; Ebert, D.D.; Green, J.G.; Hasking, P.; et al. WHO World Mental Health Surveys International College Student Project: Prevalence and distribution of mental disorder. *J. Abnorm. Psychol.* **2018**, *127*, 623. [CrossRef]

18. Albert, P.R. Why is depression more prevalent in women? *J. Psychiatry Neurosci.* **2015**, *40*, 219–221. [CrossRef]

19. Li, Y.; Wang, A.; Wu, Y.; Han, N.; Huang, H. Impact of the COVID-19 Pandemic on the Mental Health of College Students: A Systematic Review and Meta-Analysis. *Front. Psychol.* **2021**, *12*, 669119. [CrossRef]

20. Arsandaux, J.; Montagni, I.; Macalli, M.; Texier, N.; Pouriel, M.; Germain, R.; Mebarki, A.; Kinouani, S.; Tournier, M.; Schuck, S.; et al. Mental health condition of college students compared to non-students during COVID-19 lockdown: The CONFINS study. *BMJ Open* **2021**, *11*, e053231. [CrossRef]

21. Marwood, M.R.; Hearn, J.H. Evaluating mental health literacy in medical students in the United Kingdom. *J. Ment. Health Train. Educ. Pract.* **2019**, *14*, 339–347. [CrossRef]

22. Gorczynski, P.; Sims-schouten, W.; Hill, D.; Wilson, J.C. Examining mental health literacy, help seeking behaviours, and mental health outcomes in UK university students. *J. Ment. Health Train. Educ. Pract.* **2017**, *12*, 111–120. [CrossRef]

23. Mahmoodi, S.M.H.; Ahmadzad-Asl, M.; Eslami, M.; Abdi, M.; Hosseini Kahnamoui, Y.; Rasoulian, M. Mental Health Literacy and Mental Health Information-Seeking Behavior in Iranian University Students. *Front. Psychiatry* **2022**, *13*, 893534. [CrossRef]

24. Kaneko, Y.; Motohashi, Y. Male gender and low education with poor mental health literacy: A population-based study. *J. Epidemiol.* **2007**, *17*, 114–119. [CrossRef]

25. Singh, S.; Zaki, R.A.; Farid, N.D.N.; Kaur, K. The Determinants of Mental Health Literacy among Young Adolescents in Malaysia. *Int. J. Environ. Res. Public Health* **2022**, *19*, 3242. [CrossRef]

26. Furnham, A.; Cook, R.; Martin, N.; Batey, M. Mental health literacy among university students. *J. Public Ment. Health* **2011**, *10*, 198–210. [CrossRef]

27. Miles, R.; Rabin, L.; Krishnan, A.; Grandoit, E.; Kloskowski, K. Mental health literacy in a diverse sample of undergraduate students: Demographic, psychological, and academic correlates. *BMC Public Health* **2020**, *20*, 1699. [CrossRef]

28. Lo, K.; Gupta, T.; Keating, J.L. Interventions to promote mental health literacy in university students and their clinical educators. A systematic review of randomised control trials. *Health Prof. Educ.* **2018**, *4*, 161–175. [CrossRef]

29. Reis, A.C.; Saheb, R.; Moyo, T.; Smith, C.; Sperandei, S. The Impact of Mental Health Literacy Training Programs on the Mental Health Literacy of University Students: A Systematic Review. *Prev. Sci.* **2022**, *23*, 648–662. [CrossRef] [PubMed]

30. Reavley, N.J.; McCann, T.V.; Jorm, A.F. Mental health literacy in higher education students. *Early Interv. Psychiatry* **2012**, *6*, 45–52. [CrossRef] [PubMed]

31. Ito-Jaeger, S.; Perez Vallejos, E.; Curran, T.; Spors, V.; Long, Y.; Liguori, A.; Warwick, M.; Wilson, M.; Crawford, P. Digital video interventions and mental health literacy among young people: A scoping review. *J. Ment. Health* **2021**, *19*, 1–11. [CrossRef]

32. Liang, M.; Chen, Q.; Guo, J.; Mei, Z.; Wang, J.; Zhang, Y.; He, L.; Li, Y. Mental health first aid improves mental health literacy among college students: A meta-analysis. *J. Am. Coll. Health* **2021**, *9*, 1–10. [CrossRef]

33. Kurki, M.; Gilbert, S.; Mishina, K.; Lempinen, L.; Luntamo, T.; Hinkka-Yli-Salomäki, S.; Sinokki, A.; Upadhyaya, S.; Wei, Y.; Sourander, A. Digital mental health literacy program for the first-year medical students' wellbeing: A one group quasi-experimental study. *BMC Med. Educ.* **2021**, *21*, 563. [CrossRef]
34. Balula Chaves, C.; Sequeira, C.; Carvalho Duarte, J.; Nelas, P.; Gonçalves, A.; Santos, E. Alfabetización en salud mental: Una revisión sistemática de los instrumentos de medición. *Int. J. Educ. Psychol.* **2022**, *3*, 181–194. [CrossRef]

35. Sperber, A.D.; Devellis, R.F.; Boehlecke, B. Cross-Cultural Translation: Methodology and validation. *J. Cross-Cult. Psychol.* **1994**, *25*, 501–524. [CrossRef]

36. Mokkink, L.B.; Terwee, C.B.; Knol, D.L.; Stratford, P.W.; Alonso, J.; Patrick, D.L.; Bouter, L.M.; De Vet, H.C. The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: A clarification of its content. *BMC Med. Res. Methodol.* **2010**, *10*, 22. [CrossRef]

37. Terwee, C.B.; Prinsen, C.A.C.; Chiarotto, A.; Westerman, M.J.; Patrick, D.L.; Alonso, J.; Bouter, L.M.; de Vet, H.C.W.; Mokkink, L.B. COSMIN methodology for evaluating the content validity of patient-reported outcome measures: A Delphi study. *Qual. Life Res.* **2018**, *27*, 1159–1170. [CrossRef]

38. Fleiss, J.L. *Design and Analysis of Clinical Experiments*, 1st ed.; John Wiley & Sons: Hoboken, NJ, USA, 1999; pp. 306–347. [CrossRef]

39. Asparouhov, T.; Muthén, B. Exploratory structural equation modeling. *Struct. Equ. Modeling* **2009**, *16*, 397–438. [CrossRef]

40. Marsh, H.W.; Morin, A.J.S.; Parker, P.D.; Kaur, G. Exploratory Structural Equation Modeling: An Integration of the Best Features of Exploratory and Confirmatory Factor Analysis. *Annu. Rev. Clin. Psychol.* **2014**, *10*, 85–110. [CrossRef]

41. Drasgow, F. Polychoric and Polyserial Correlations. *Stats. Ref.* **2014**, *7*, 68–74. [CrossRef]

42. Meredith, W. Measurement invariance, factor analysis and factorial invariance. *Psychometrika* **1993**, *58*, 525–543. [CrossRef]

43. Millsap, R.E.; Yun-Tein, J. Assessing factorial invariance in ordered-categorical measures. *Multivar. Behav. Res.* **2004**, *39*, 479–515. [CrossRef]

44. Muthén, B.O.; Asparouhov, T. Latent Variable Analysis With Categorical Outcomes: Multiple-Group and Growth Modeling in Mplus. *Mplus. Web. Note* **2002**, *4*, 1–22.

45. Wang, J.; Wang, X. Structural equation modeling: Applications using mplus. In *Structural Equation Modeling: Applications Using Mplus*, 2nd ed.; Wang, J., Wang, X., Eds.; Wiley: Hoboken, NJ, USA, 2019; pp. 1–248. [CrossRef]

46. Reise, S.P.; Widaman, K.F.; Pugh, R.H. Confirmatory factor analysis and item response theory: Two approaches for exploring measurement invariance. *Psychological Bulletin* **1993**, *114*, 552. [CrossRef] [PubMed]

47. Chen, F. Sensitivity of goodness of fit indexes to lack of measurement invariance. *Struct. Equ. Modeling* **2007**, *14*, 464–504. [CrossRef]

48. Cheung, G.W.; Rensvold, R. Evaluating goodness-of-fit indexes for testing measurement invariance. *Struct. Equ. Modeling* **2002**, *9*, 233–255. [CrossRef]

49. Cronbach, L.J. Coefficient alpha and the internal structure of tests. *Psychometrika* **1951**, *16*, 297–334. [CrossRef]

50. Zwick, R.; McDonald, R.P. Test Theory: A Unified Treatment. *J. Am. Stat. Assoc.* **2000**, *95*, 102–1013. [CrossRef]

51. Nunnally, J.; Bernstein, I. *Psychometric Theory*, 3rd ed.; McGraw-Hill: New York, NY, USA, 1994; pp. 1–752.

52. Terwee, C.B.; Bot, S.D.M.; de Boer, M.R.; van der Windt, D.A.W.M.; Knol, D.L.; Dekker, J.; Bouter, L.M.; de Vet, H.C.W. Quality criteria were proposed for measurement properties of health status questionnaires. *J. Clin. Epidemiol.* **2007**, *60*, 34–42. [CrossRef]

53. Akhtar, I.N. Mental Health Literacy Scale: Translation and Validation in Pakistani Context. *Pak. Soc. Sci. Rev.* **2020**, *4*, 722–735. [CrossRef]

54. Nejatian, M.; Tehrani, H.; Momeniyan, V.; Jafari, A. A modified version of the mental health literacy scale (MHLS) in Iranian people. *BMC Psychiatry* **2021**, *21*, 53. [CrossRef]

55. Alshehri, E.; Alosaimi, D.; Rufaidi, E.; Alsomali, N.; Tumala, R. Mental Health Literacy Scale Arabic Version: A Validation Study Among Saudi University Students. *Front. Psychiatry* **2021**, *12*, 741146. [CrossRef]

56. Chen, S.; Chen, K.; Wang, S.; Wang, W.; Li, Y. Initial Validation of a Chinese Version of the Mental Health Literacy Scale Among Chinese Teachers in Henan Province. *Front. Psychiatry* **2021**, *12*, 661903. [CrossRef]

57. Dias Neto, D.; Rocha, I.; Figueiras, M.J.; da Silva, A.N. Measuring mental health literacy: Adaptation and validation of the Portuguese version of the mental health literacy scale (MHLS). *Eur. J. Ment. Health* **2021**, *16*, 64–77. [CrossRef]

58. Korhonen, J.; AXelin, A.; Grobler, G.; Lahti, M. Content validation of Mental Health Literacy Scale (MHLS) for primary health care workers in South Africa and Zambia – a heterogeneous expert panel method. *Glob. Health Action* **2019**, *12*, 1668215. [CrossRef]

59. Almanasef, M. Mental Health Literacy and Help-Seeking Behaviours Among Undergraduate Pharmacy Students in Abha, Saudi Arabia. *Risk. Manag. Healthc. Policy* **2021**, *14*, 1281–1286. [CrossRef]

60. Kutcher, S.; Wei, Y.; Coniglio, C. Mental Health Literacy: Past, Present, and Future. *Can. J. Psychiatry* **2016**, *61*, 154–158. [CrossRef]

61. Foss, E.; Druin, A.; Guha, M.L. Recruiting and retaining young participants: Strategies from five years of field research. In *Proceedings of the 12th International Conference on Interaction Design and Children*, New York, NY, USA, 24–27 June 2013. [CrossRef]

62. Henderson, M.; Wight, D.; Nixon, C.; Hart, G. Retaining young people in a longitudinal sexual health survey: A trial of strategies to maintain participation. *BMC Med. Res. Methodol.* **2010**, *10*, 9. [CrossRef]

63. Bonett, D.G. Sample size requirements for estimating intraclass correlations with desired precision. *Stat. Med.* **2002**, *21*, 1331–1335. [CrossRef]