# A Follow up on the Continuum Theory of Eco-Anxiety: Analysis of the Climate Change Anxiety Scale Using Item Response Theory among French Speaking Population

Taha Hannachi, Sonya Yakimova, Alain Somat

<figure><img src="image_1.png" /></figure>

## To cite this version:

Taha Hannachi, Sonya Yakimova, Alain Somat. A Follow up on the Continuum Theory of Eco-Anxiety: Analysis of the Climate Change Anxiety Scale Using Item Response Theory among French Speaking Population. International Journal of Environmental Research and Public Health, 2024, 21 (9), pp.1158. 10.3390/ijerph21091158. hal-04855387

HAL Id: hal-04855387

https://hal.science/hal-04855387v1

Submitted on 30 Dec 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_3.png" /></figure>

Article

# A Follow Up on the Continuum Theory of Eco-Anxiety: Analysis of the Climate Change Anxiety Scale Using Item Response Theory among French Speaking Population

Taha Hannachi *, Sonya Yakimova and Alain Somat

Laboratory of Psychology, Cognition, Behaviour, Communication (LP3C), Department of Psychology, Faculty of Human Science, Rennes 2 University, 35000 Rennes, France; sonya.yakimova@univ-rennes2.fr (S.Y.); alain.somat@univ-rennes2.fr (A.S.)

* Correspondence: taha.hannachi@univ-rennes2.fr

**Abstract:** The mental health impact of the environmental crisis, particularly eco-anxiety, is a growing research topic whose measurement still lacks consensus. This study aims to use item response theory (IRT) to gain a deeper understanding of the constructs measured by existing questionnaires. To conduct this review, we applied the graded response model with the help of the MIRT package in R on open-access data from the short French version of the Climate Change Anxiety Questionnaire, which measures cognitive-emotional impairment and functional impairment. The models tested in this study are the one, two, and three-factor models, and the bifactor model. After model selection, the psychometric properties of the selected model were tested. Our results suggest that the unidimensional model seems to be the most appropriate for measuring eco-anxiety. The item difficulty parameter extracted from the IRT enabled us to discuss the severity levels of the items comprising this tool. The Climate Change Anxiety Questionnaire appears to be more appropriate for measuring moderate to severe eco-anxiety. Avenues for improving this questionnaire and the measurement of eco-anxiety in general are then discussed.

**Keywords:** eco-anxiety; item response theory; continuum; climate change anxiety scale

<figure><img src="image_4.png" /></figure>

check for
updates

**Citation:** Hannachi, T.; Yakimova, S.; Somat, A. A Follow Up on the Continuum Theory of Eco-Anxiety: Analysis of the Climate Change Anxiety Scale Using Item Response Theory among French Speaking Population. *Int. J. Environ. Res. Public Health* **2024**, *21*, 1158. https://doi.org/10.3390/ijerph21091158

Academic Editor: Paul B. Tchounwou

Received: 8 July 2024

Revised: 21 August 2024

Accepted: 26 August 2024

Published: 30 August 2024

<figure><img src="image_5.png" /></figure>

**Copyright:** © 2024 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).

# 1. Introduction

## 1.1. *The Continuum Hypothesis of Eco-Anxiety*

For several years now, there has been growing interest in the impact of the ecological crisis on mental health. The term often used to describe this phenomenon is eco-anxiety. However, its definition may vary depending on the context of the study and the concepts used to quantify it [1–3]. What is common, however, is the idea that eco-anxiety is a form of psychological distress of varying significance [3–6], ranging from severe distress to what might be described as worry or concern. Thus, eco-anxiety can be situated along a continuum ranging from mild forms of eco-anxiety to more severe forms, generating emotional, cognitive, social, and/or functional difficulties [3,4,7,8]. The term eco-anxiety is often used interchangeably with climate change anxiety, but some researchers choose to distinguish the two [1,3,5]. For instance, Hogg et al. [5] consider that eco-anxiety encompasses the psychological distress related to the ecological crisis in its entirety, including climate change anxiety, which covers anxiety related to anthropogenic climate change. It also encompasses distress from “a multiplicity of environmental calamities, which may or may not be directly caused by climate change, including the elimination of entire ecosystems and plant and animal species, global mass pollution and deforestation” ([5], p. 1). While theoretically the two concepts might be different, there is not enough empirical evidence to suggest that they should be distinguished. In this article, we use the terms “eco-anxiety” and “climate change anxiety” (or “climate anxiety” for short) interchangeably because we believe that,
in all cases, eco-anxiety fundamentally stems from the perception of the human activity's
negative impact, regardless of the specific consequences.

Many authors seem to focus on the most severe forms of eco-anxiety, adopting a pathologizing approach to the concept [9] rather than adopting a continuous perspective that allows climate anxiety to be viewed as a continuum of reactions ranging from mild to severe reactions to the ecological crisis [2,10]. Indeed, even the mildest forms of climate anxiety can have motivational consequences, prompting people to adopt a more eco-responsible lifestyle [11].

Attempting to quantify eco-anxiety is relatively a new task [4,5]. Thus, not all measurement tools incorporate in their development the idea of a continuous conception of eco-anxiety. For example, Lutz et al. [2] demonstrate that the Climate Change Anxiety Scale (CCAS, [4]), one of the most widely used tools in the world, only assesses severe manifestations of eco-anxiety, thus serving as a clinical (even psychologizing) approach to identifying people in need of care.

Several authors have attempted to quantify this psychological discomfort linked to climate change and the environmental crisis in other ways, but no conceptualization takes into account the full spectrum of this discomfort [4,5,12,13]. Beyond the clinical aspect assessed by the CCAS, the psychological reaction to the environmental crisis is assessed either as worry via the Climate Change Worry Scale [12] or as distress and impairment by the Climate Change Distress and Impairment Scale [13]. The Hogg Eco-Anxiety Scale (HEAS, [5]) claims to measure non-pathological eco-anxiety and represents an alternative to the CCAS. However, this questionnaire adapts items from the GAD-7, a tool designed to measure generalized anxiety disorder and considered by Jordan et al. [14] to be primarily suited for diagnosing severe anxiety. Thus, HEAS scores appear to be highly correlated with CCAS scores [15]. Furthermore, these two questionnaires have the same floor effect problem. For both the CCAS [4,16] and the HEAS [5,15], the means reported (with comparable standard deviations slightly below 1) are close to the lower limit of the Likert scale used, with the exception of the last factor of the HEAS. This makes these scales particularly poor at detecting mild forms of eco-anxiety. In summary, while these questionnaires—particularly the CCAS—may be effective in identifying individuals with severe forms of eco-anxiety who require psychotherapeutic intervention due to the physiological, cognitive, and functional symptoms they describe, they are less effective in detecting mild cases that may benefit from other types of interventions. Given the widespread use of the CCAS, applying it to the general population could be counterproductive. This study aims to explore this issue further and offers insights into potential improvements for such a scale.

Logically, if we distinguish between two extremes (weak and strong), intermediate stages of eco-anxiety should also exist. The only attempt to encompass both “ends” of the concept within a single questionnaire is made by Ågoston et al. [17]. The authors pointed out two factors representing the extremes: habitual ecological worry and the negative consequences of eco-anxiety. However, the factor analysis fails to identify a factor that captures an intermediate stage of eco-anxiety.

Consequently, we believe it is important to develop a tool that encompasses the
entire spectrum of anxiety caused by climate change. Such a tool would enable a better
understanding of the reasons behind the variability in eco-anxiety between individuals
(inter variability), or even for the same individual, depending on the context and the time
of measurement (intra variability).

This article represents a first step in exploring the spectrum of eco-anxiety. By applying
statistical methods derived from item response theory (IRT) [18], we aim to examine the
factor structure of the CCAS [4]. We hypothesize that the authors’ original approach
(factor analysis) may hinder the expression of a symptom severity hierarchy. To test
this hypothesis, we will adopt a different statistical perspective (IRT), which allows for
conceptualizing eco-anxiety as a continuum. This approach likely aligns more closely with
the reality of the distress people experience regarding the ecological crisis, as it enables us
to describe the progression of the phenomenon while preserving the scaling properties of
the questionnaire.

## 1.2. *Item Response Theory*

It is very conventional to consider the ordinal categorical modalities of a Likert scale as numerical and continuous, in order to be able to apply linear modeling (principal component analysis). This assumption of linearity can, in the case of principal component analysis, give rise to completely artifactual factors (for a detailed critique on principal component analysis, see [19,20]). In contrast to the frequently used linear model, item response theory (IRT) takes into account the fact that response modalities are categorical and bounded. They are categorial in a sense that the scale is discrete, not continuous, and bounded in the sense that it is finite, not infinite, like the linear model assumes the data to be. Moreover, unlike conventional testing methods, IRT distinguishes between an item's level of difficulty and its ability to discriminate between individuals. More precisely, IRT stipulates that the probability of an individual responding in an extreme manner to an item (the indicator) depends on his or her ability to endorse the phenomenon (latent trait) described by that item. This approach allows IRT to standardize the level of difficulty of each item on the same scale based on people's ability to endorse them, enabling direct comparisons. Consequently, two individuals with comparable scores according to factor analyses may not actually be at the same level of endorsement of the studied phenomenon, depending on the difficulty level of the items.

De Boeck [21] demonstrates, for example, in relation to the use of psychological tests, that the scores of a person's ability to endorse the latent trait produced by IRT models offer us the possibility of identifying a hierarchy of symptoms experienced by people. Factor analyses, on the other hand, are based on averages, which erase information on how participants responded to the items that are most diagnostic of the phenomenon, thus preventing us from identifying the most symptomatic participants.

In the specific case of eco-anxiety, for example, it is claimed that eight out of ten French people suffer from eco-anxiety [22]. IRT provides a solution to this examination by projecting individuals' latent trait scores onto the same graphical space as the difficulty level of the proposed items. This method thus provides a description of the phenomenon that is closer to reality (see [23]).

Furthermore, according to Clayton and Karazsia [4], there is no clear threshold for the transition from adaptive climate anxiety to maladaptive eco-anxiety. They argue that a person with an overall CCAS score above the median is one for whom climate change is having a significant impact on their mental health. Hogg et al. [5] challenge this proposal, believing that only people with extreme overall scores on the HEAS should be considered as suffering from severe eco-anxiety.

## 1.3. *The Factor Structure of the CCAS: A Lack of Consensus*

In its initial validation, the CCAS is a 22-item scale that describes four dimensions: cognitive and emotional difficulties, functional difficulties, experience (direct or indirect of climate change), and environmental engagement. Researchers refer to the first two as measuring eco-anxiety and the last two as control factors [4]. However, several studies have shown that removing the last two dimensions improves the psychometric quality of the scale (e.g., [16,24]). We therefore follow these recommendations and use only the first 13 items of the scale, which cover the dimensions of cognitive and emotional impairment, and functional impairment, which we will refer to as CCAS-13.

In their article validating the CCAS in French, Mouguiama-Daouda et al. [16] carried out only confirmatory factor analyses to test the one- or two-factor structure of the CCAS-13 and the four-dimensional structure (22-item version). They retained the two-factor structure of the 13-item version. We aim to confirm this structure of the CCAS-13 by checking whether there are other models that fit the data better. According to Immekus et al. [25], in addition to theoretical and empirical data, the choice of instrument model must be
made following exclusion of the presence of other models that could lead to an alternative
explanation of the factor structure. Cruz and High [26] explored a second-order model
for the CCAS-13. They found that both CCAS-13 factors are summarized by an overall
second-order factor, i.e., they measure the same construct. Thus, they recommend the
use of a hierarchical structure (one global factor and two first-order factors). On the other
hand, Larionow et al. [27] found that a three-factor solution was better than the two-factor
solution in the Polish context, although they did not report the significance of this difference.
They also observed that correlations between factors are relatively high and recommend
the use of global scale scores. In a sample of participants from three Asian countries and
the USA, Tam et al. [28] demonstrate that a single-factor model does not have a good fit
and argue that the alternative two-factor solution is probably better. Nevertheless, the
inter-factor correlations for the latter solution remain very high (this is true for each of
the four countries represented in this study) and suggest the existence of a hierarchical
structure, such as that mentioned by Cruz and High [26] and Larionow et al. [27]. To
summarize, a lot of models tested on the CCAS and CCAS-13 seem to be plausible but
not consensual.

The aim of this article is to provide some keys to clarifying these fuzzy areas relating
to the conceptualization and measurement of eco-anxiety through the use of IRT. Thus, we
want to explore alternative models that may have a better fit for the data and may provide
a more plausible explanation of the eco-anxiety continuum.

## 2. Materials and Methods

This study utilized pre-existing datasets that were publicly available online. A total
of 1778 responses to the French version of the CCAS were retrieved. The first dataset
originated from study two by Mouguiama-Daouda et al. [16], and the second dataset
originiated from the study by Heeren et al. [29]. The data for the 905 respondents in
Mouguiama-Daouda et al.'s [16] study two were obtained from https://osf.io/m3ygz/
(accessed on 15 November 2022), while the data for 873 respondents in Heeren et al. [29]
were retrieved from https://osf.io/2r659/ (accessed on 15 November 2022). These datasets
were made available by the researchers under the "CC-BY Attribution 4.0 International"
license. The two datasets were selected for this study because they are comparable in size,
were collected in the same context, and employed similar methods of data collection. In the
original studies, data were gathered via social media and listserv advertisements, targeting
a French-speaking population. The participants were predominantly European, with a
small minority (less than 5%) from other French-speaking regions.

The short version of the CCAS (13 items) is evaluated in this article; hence, the
acronym CCAS-13 is used. According to its developers, the CCAS-13 comprises eight items
measuring “cognitive and emotional impairment” and five items measuring “functional
impairment”. Responses are based on a five-point Likert-type frequency scale (1 = “never”,
2 = “rarely”, 3 = “sometimes”, 4 = “often” and 5 = “always”).

The data analysis was performed with the MIRT (multidimensional item response theory) package [30] on R software [31], version 4.3.0. The graded response model (GRM) [32] was used. This model is suitable for the analysis of ordinal data, particularly Likert scales [33]. It is also more flexible than other IRT models in that it allows for the discrimination parameter to be estimated for each item [34]. The estimation method used is the Expectation-Maximization (EM) algorithm, which is used to maximize the likelihood of the model parameters given the observed data [35]. The Oakes method was used as a type of estimation method to calculate the parameter information matrix for computing the standard error (see Chalmers, [36] for details). The item type used was of course “graded” for the GRM.

First, we revisit the unidimensional model, the two-factor model, and the three-factor
model to compare the fit indicators given by IRT with the results of previous research. To
explain the strong correlations between the CCAS-13 dimensions found in the literature, we
will also test the bifactor model [37]. According to Gibbons and Cai [38] (p. 52), the latter
is more appropriate for “self-report measures of health status covering both physical and emotional difficulties”. This model compiles a general factor of the phenomenon and the uncorrelated sub-domains that make it up. It is therefore more flexible than the hierarchical model in explaining correlated factor structures [39]. Indeed, the bifactor model allows us to decide on the relevance of sub-domain scores by looking at the loading force not absorbed by the general factor [40].

Figure 1 shows the different models tested in this article.

<figure><img src="image_6.png" /><figcaption>**Figure 1.** The four models tested in this study: (a) unidimensional model, (b) correlated 2 factor model, (c) correlated 3 factor model, and (d) bifactor model.</figcaption></figure>

Statistical analyses of the data were carried out in two stages: (1) tests of goodness of fit of models and (2) verification of the psychometric properties of the chosen model.

For the first stage, four models were tested: the unidimensional model, the multi-
dimensional two- and three-factor model, and the bifactorial model. The model fit was
tested using the limited information goodness-of-fit statistic *M*₂ [41]. This test is more
appropriate than the Pearson’s test statistic χ², especially in the context of IRT models [41].
The *M*₂ test belongs to the limited information test category. These tests were designed
to solve the problem faced by full information tests like the χ²; they depend on all the
information in the contingency table, especially in models with many variables, where
the probability of combining certain categories is becoming increasingly rare. Indeed,
according to Steinberg and Thissen [42], for a number of categories greater than or equal
to five, the fit approximation becomes invalid for any model as soon as the number of
items exceeds six, regardless of sample size. One of the major interests of IRT is to tackle
the problem of probability sparsity by examining the sample in terms of how participants
responded to difficult answers [43]. The idea behind limited-information tests is to reduce
the dependence on probabilities for rare response combinations [44], which can improve
the power of the tests and the accuracy of the model fit assessment. Probability boxes
removed from the contingency table are, in fact, more likely to appear (see [45] for more on
the difference between limited-information and full-information tests).

Apart from the M₂ test, we also consulted the values of other important indicators. In
particular, we followed the recommendations of Maydeu-Olivares and Joe [46], according
to which an RMSEA (Root Mean Square Error of Approximation) of less than 0.089 and a
SRMR (Standardized Root Mean Square Residual) of less than 0.05 are acceptable model
fits for the IRT. We also took into account the Comparative Fit Index (CFI) and the Tucker
Lewis Index (TLI). They range from 0 to 1, indicating that model fit is acceptable when
their respective values are above 0.90, and that model fit is excellent when their respective
values are above 0.95 [47,48]. We have also reported the values of the Akaike Information Criterion (AIC) [49]. This indicator, which incorporates complexity and level of fit into its equation, is useful when selecting the best model from among several plausible ones [50]. The lower the value of the AIC compared to previous models, the better the model is [51].

Once the most suitable model had been selected, the psychometric qualities of the chosen model were checked. For each item, we evaluated the parameters of discrimination (noted *a*) and difficulty (noted *b*₁, *b*₂, *b*₃, and *b*₄, respectively, relative to the 50% probability of choosing “rarely” over “never”, then “sometimes” over “rarely”, then “often” over “sometimes”, and finally “always” over “often”). Based on the principle that, according to IRT, the latent trait is represented by a continuum going from a low level to a high level, the discrimination parameter (*a*) determines how well an item is able to discriminate individuals along the latent trait continuum [52]. According to Baker [53], item discrimination becomes acceptable when the *a* value is greater than or equal to 0.5. Difficulty parameters (*b*₁...₄) are based on the thresholds at which an individual has a 50% probability or more of selecting any category of an item [54]. The *b*₁...₄ values are standardized on a scale ranging, in our case, from -6 to 6. This choice was made in order to be able to graphically represent the most difficult modalities of the scale with values greater than 3. A *b* value close to -6 reflects the ease of endorsement of the item category in question. Conversely, an item category with a *b* value close to 6 is difficult to endorse.

One of the advantages of IRT is the independence of the estimated item parameters from the sample, and therefore the relative consistency of estimates across samples. Thus, a well-fitted model will always return the same item parameter estimates, regardless of the sample [23]. As a result, the same analyses were carried out separately on the two datasets, with no major differences. Consequently, we will only present the results obtained on a single dataset from the work of Mouguiama-Daouda et al. [16] (n = 905). The full analyses carried out on both datasets are presented in the Supplementary Material.

## 3. Results

### 3.1. *Model Selection*

The results show that model fit statistics are acceptable for all the models tested, with a slightly higher value of the threshold for SRMR for the one-factor model and the two-factor model. All detailed results are displayed in Table 1. The model comparison (Table 2) shows that the two-factor model is significantly better than the one-factor model: $χ^2(1) = 191.234, p < 0.001$. similarly, the three-factor model is significantly better than the two-factor model: $χ^2(2) = 232.094, p < 0.001$. However, we find no significant difference between the three-factor model and the bifactor model. Though, the bifactor model is, like the three-factor model, significantly better than the one-factor and two-factor model. This result is confirmed by the consecutive decrease in the value of the AIC (the decrease in the AIC value is not as important when comparing the three-factor model to the bifactor model as in the previous cases).

For the one-factor model and the two-factor model, the results of the IRT-based dimensionality analysis are comparable to the results of the confirmatory factor analysis of study two by Mouguiama-Daouda et al. [16]. Moreover, the correlation between the two factors (cognitive and emotional impairment (CEI) and functional impairment (FI)) is even stronger at 0.83, compared with 0.66 in Mouguiama-Daouda et al. [16].

This result consolidates Cruz and High's [26] assertion that these two factors are not independent of each other; they could therefore be measuring the same construct: eco-anxiety. This leads us to question the relevance of this two-factor model, which could be modeled differently.

We therefore tested a three-factor model by carrying out an exploratory factor analysis, the results of which are also presented in Table 1.
<table><caption>**Table 1.** Comparison of the fit of CCAS-13 models between the work of Mouguiama-Daouda et al. [16] and the present work. Model Fit Tests Performed by Mouguiama-Daouda et al. [16].</caption><thead><tr><th>Model</th><th>χ<sup>2</sup></th><th>df</th><th><i>p</i></th><th>RMSEA</th><th>SRMR</th><th>TLI</th><th>CFI</th><th>AIC</th></tr></thead><tbody><tr><td>1 factor</td><td>542.26</td><td>65</td><td>&lt;0.01</td><td>0.10</td><td>0.06</td><td>0.81</td><td>0.84</td><td>31,355.91</td></tr><tr><td>2 factors</td><td>390.48</td><td>64</td><td>&lt;0.01</td><td>0.08</td><td>0.05</td><td>0.87</td><td>0.89</td><td>31,170.33</td></tr><tr><td colspan="9">Model fit tests carried out in this work.</td></tr><tr><th>Model</th><th><i>M</i><sub>2</sub></th><th>df</th><th><i>p</i></th><th>RMSEA</th><th>SRMSR</th><th>TLI</th><th>CFI</th><th>AIC</th></tr><tr><td>1 factor</td><td>176.38</td><td>26</td><td>&lt;0.001</td><td>0.08</td><td>0.074</td><td>0.91</td><td>0.94</td><td>27,402.92</td></tr><tr><td>2 factors</td><td>173.14</td><td>25</td><td>&lt;0.001</td><td>0.08</td><td>0.07</td><td>0.90</td><td>0.94</td><td>27,213.69</td></tr><tr><td>3 factors</td><td>3.46</td><td>3</td><td>0.33</td><td>0.01</td><td>0.04</td><td>0.99</td><td>0.99</td><td>26,974.11</td></tr><tr><td>bifactor</td><td>13.89</td><td>13</td><td>0.38</td><td>0.008</td><td>0.049</td><td>0.99</td><td>0.99</td><td>26,959.97</td></tr></tbody></table>

χ2: Pearson's test statistic, *M*₂: the limited information goodness-of-fit test statistic, df: degree of freedom, *p*: *p* value, RMSEA: Root Mean Square Error Approximation, SRMSR: Standardized Root Mean Squared Residual, TLI: Tucker-Lewis Index, CFI: Comparative Fit Index, AIC: Akaike Information Criterion.

<table><caption>**Table 2.** Comparison between the 1 to 2 and 3 dimensional models and the two-factor model.</caption><thead><tr><th>Model</th><th>AIC</th><th>χ<sup>2</sup></th><th>df</th><th><i>p</i></th></tr></thead><tbody><tr><td>1 factor</td><td>27,402.92</td><td></td><td></td><td></td></tr><tr><td>2 factors</td><td>27,213.69</td><td>191.234</td><td>1</td><td>&lt;0.001</td></tr><tr><td>3 factors (Exploratory)</td><td>26,985.59</td><td>232.094</td><td>2</td><td>&lt;0.001</td></tr><tr><td>Bifactor</td><td>26,959.97</td><td>5.857</td><td>10</td><td>0.82</td></tr></tbody></table>

The *M*₂ goodness-of-fit test of the three-dimensional model is insignificant. We therefore retain the null hypothesis that this three-factor model fits the data perfectly. The other indicators (RMSEA, SRMSR, etc.) confirm this finding. Moreover, this model is significantly better than the other two (Table 2).

In fact, the cognitive and emotional difficulties (CEI) factor is divided into two (potentially, physiological symptoms (SYM) and rumination (RUM)).

This three-factor modelling (Table 3) is more faithful to the questionnaire construction process (the first 13 items) adopted by Clayton and Karazsia [4]. As a reminder, the first four items, which correspond here to the SYM factor, were constructed on the basis of reading the scientific literature, but also of people's experiences shared on blogs. The aim of these four items is to reflect "physical symptoms". The next four items belonging to the RUM dimension were indeed adapted from the rumination questionnaire by Treynor et al. [55]. As for the last five (from the CCAS-13 version), representing the functional impairment factor, they were adapted from the Weiss Functional Impairment Rating Scale [56]. This is in line with the three-factor structure validated by Larionow et al. [27] in their Polish-language validation of the CCAS-13.

Despite its perfect fit to the data, this model presents two questionable points. The first concerns the correlations between factors, which remain relatively strong (Table 3: factors correlation). This suggests that the different factors measure almost the same thing. Second, the variance explained by the three-factor model with rotation (55.6%) is not much higher than the variance explained by the unidimensional model (46%). Without rotation, the first factor of the three-factor model explains 41.4% of the variance (Table 4), while the second and third explain only 7.4% and 7.3% of the total variance, respectively. The difference of explained variance between the rotated and the unrotated models might partly be due to the choice of the oblique rotation. The problematic use of rotation in factor analysis and interpretation will be discussed below.
<table><caption>**Table 3.** Three-factor model item loading (with Oblimin rotation).</caption><thead><tr><th></th><th>FI</th><th>SYM</th><th>RUM</th></tr></thead><tbody><tr><td>CCAS1</td><td>-</td><td>0.58</td><td>-</td></tr><tr><td>CCAS2</td><td>-</td><td>0.94</td><td>-</td></tr><tr><td>CCAS3</td><td>-</td><td>0.69</td><td>-</td></tr><tr><td>CCAS4</td><td>-</td><td>0.56</td><td>-</td></tr><tr><td>CCAS5</td><td>-</td><td>-</td><td>0.65</td></tr><tr><td>CCAS6</td><td>-</td><td>-</td><td>0.75</td></tr><tr><td>CCAS7</td><td>-</td><td>-</td><td>0.43</td></tr><tr><td>CCAS8</td><td>-</td><td>-</td><td>0.76</td></tr><tr><td>CCAS9</td><td>0.80</td><td>-</td><td>-</td></tr><tr><td>CCAS10</td><td>0.57</td><td>-</td><td>-</td></tr><tr><td>CCAS11</td><td>0.86</td><td>-</td><td>-</td></tr><tr><td>CCAS12</td><td>0.81</td><td>-</td><td>-</td></tr><tr><td>CCAS13</td><td>0.61</td><td>-</td><td>-</td></tr><tr><td colspan="4">Factor correlations</td></tr><tr><td></td><td>FI</td><td>SYM</td><td>RUM</td></tr><tr><td>FI</td><td>-</td><td>-</td><td>-</td></tr><tr><td>SYM</td><td>0.650</td><td>-</td><td>-</td></tr><tr><td>RUM</td><td>0.736</td><td>0.609</td><td>-</td></tr><tr><td colspan="4">Explained variance</td></tr><tr><td></td><td>FI</td><td>SYM</td><td>RUM</td></tr><tr><td>Explained variance</td><td>22.2%</td><td>18.3%</td><td>15.1%</td></tr></tbody></table>

The bifactorial model, the last model we tested, is likely to provide more of an explanation of the strong correlations between the previously extracted factors. The results show that it fits the data significantly better than the unidimensional model, in contrast to what Cruz and High [26] found. However, it does not show a significant improvement over the three-dimensional model (see Table 2).

However, it is important to note that the general factor absorbs all item loading. The loadings of the scale's sub-domains are low, and even negative for some items. Furthermore, most of the explained variance of the two-factor model is attributed to the global factor (Table 5). This translates into an explained variance of the general factor of the bifactor model comparable to that of the unidimensional model (Table 6). We therefore confirm the recommendation put forward by Cruz and High [26], inviting researchers and practitioners to mainly report the global scores of the CCAS. In line with what these authors suggest, our analyses show that the CEI and FI factor scores are somewhat irrelevant.

Despite the fact that all the models that we tested have acceptable fit statistics, they all have some issues, except for the one-factor model. As seen before, the two and three-factor models suffer from a high correlation value between their factors. Additionally, the bifactor model has weak loadings on its sub-domains, which make their computation and interpretation irrelevant. Thus, it appears that the unidimensional model is more appropriate for measuring climate anxiety by the CCAS. From a theoretical point of view, this choice seems more consistent with the idea of a continuum of eco-anxiety. That is why we concentrate the exploration of the one-factor model's psychometric properties.
<table><caption>**Table 4.** Comparison of factor loadings for the 3-dimensional model without rotation (upper table) and the unidimensional model (lower table).</caption><thead><tr><th></th><th>F1</th><th>F2</th><th>F3</th></tr></thead><tbody><tr><td>CCAS1</td><td>-0.66</td><td>-0.39</td><td>-</td></tr><tr><td>CCAS2</td><td>-0.62</td><td>-0.65</td><td>-</td></tr><tr><td>CCAS3</td><td>-0.49</td><td>-0.47</td><td>0.22</td></tr><tr><td>CCAS4</td><td>-0.53</td><td>-0.38</td><td>0.27</td></tr><tr><td>CCAS5</td><td>-0.55</td><td>-</td><td>0.45</td></tr><tr><td>CCAS6</td><td>-0.69</td><td>-</td><td>0.51</td></tr><tr><td>CCAS7</td><td>-0.39</td><td>-</td><td>0.27</td></tr><tr><td>CCAS8</td><td>-0.50</td><td>-</td><td>0.50</td></tr><tr><td>CCAS9</td><td>-0.82</td><td>-</td><td>-</td></tr><tr><td>CCAS10</td><td>-0.61</td><td>-</td><td>-</td></tr><tr><td>CCAS11</td><td>-0.85</td><td>-</td><td>-</td></tr><tr><td>CCAS12</td><td>-0.83</td><td>-</td><td>-</td></tr><tr><td>CCAS13</td><td>-0.64</td><td>-</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td></tr><tr><th></th><th>F1</th><th>F2</th><th>F3</th></tr><tr><td>Explained variance</td><td>41.4%</td><td>7.4%</td><td>7.3%</td></tr><tr><td></td><td></td><td></td><td></td></tr><tr><th></th><th>F1</th><th></th><th></th></tr><tr><td>CCAS1</td><td>0.73</td><td></td><td></td></tr><tr><td>CCAS2</td><td>0.70</td><td></td><td></td></tr><tr><td>CCAS3</td><td>0.58</td><td></td><td></td></tr><tr><td>CCAS4</td><td>0.63</td><td></td><td></td></tr><tr><td>CCAS5</td><td>0.64</td><td></td><td></td></tr><tr><td>CCAS6</td><td>0.77</td><td></td><td></td></tr><tr><td>CCAS7</td><td>0.45</td><td></td><td></td></tr><tr><td>CCAS8</td><td>0.58</td><td></td><td></td></tr><tr><td>CCAS9</td><td>0.79</td><td></td><td></td></tr><tr><td>CCAS10</td><td>0.60</td><td></td><td></td></tr><tr><td>CCAS11</td><td>0.81</td><td></td><td></td></tr><tr><td>CCAS12</td><td>0.80</td><td></td><td></td></tr><tr><td>CCAS13</td><td>0.65</td><td></td><td></td></tr><tr><td></td><td></td><td></td><td></td></tr><tr><th></th><th>F1</th><th></th><th></th></tr><tr><td>Explained variance</td><td>46%</td><td></td><td></td></tr></tbody></table>

<table><caption>**Table 5.** Comparison of loadings in the unidimensional model and the bifactor model.</caption><thead><tr><th rowspan="2"></th><th colspan="2">Unidimensional Model</th><th colspan="2">Bifactorial Model</th></tr><tr><th>F</th><th>G</th><th>CEI</th><th>FI</th></tr></thead><tbody><tr><td>CCAS1</td><td>0.727</td><td>0.721</td><td>0.300</td><td>-</td></tr><tr><td>CCAS2</td><td>0.700</td><td>0.735</td><td>0.539</td><td>-</td></tr><tr><td>CCAS3</td><td>0.579</td><td>0.615</td><td>0.355</td><td>-</td></tr><tr><td>CCAS4</td><td>0.631</td><td>0.661</td><td>0.250</td><td>-</td></tr><tr><td>CCAS5</td><td>0.635</td><td>0.690</td><td>-0.150</td><td>-</td></tr><tr><td>CCAS6</td><td>0.766</td><td>0.836</td><td>-0.215</td><td>-</td></tr><tr><td>CCAS7</td><td>0.445</td><td>0.464</td><td>-0.179</td><td>-</td></tr><tr><td>CCAS8</td><td>0.581</td><td>0.650</td><td>-0.254</td><td>-</td></tr><tr><td>CCAS9</td><td>0.790</td><td>0.687</td><td>-</td><td>0.443</td></tr><tr><td>CCAS10</td><td>0.597</td><td>0.525</td><td>-</td><td>0.315</td></tr><tr><td>CCAS11</td><td>0.811</td><td>0.690</td><td>-</td><td>0.537</td></tr><tr><td>CCAS12</td><td>0.803</td><td>0.691</td><td>-</td><td>0.502</td></tr><tr><td>CCAS13</td><td>0.651</td><td>0.592</td><td>-</td><td>0.253</td></tr></tbody></table>
<table><caption>**Table 6.** Comparison of variance explained between the unidimensional model and the bifactorial model.</caption><thead><tr><th rowspan="2"></th><th colspan="2">Unidimensional Model</th><th colspan="2">Bifactorial Model</th></tr><tr><th>F</th><th>G</th><th>CEI</th><th>FI</th></tr></thead><tbody><tr><th>Explained variance</th><td>46%</td><td>44.2%</td><td>5.7%</td><td>6.9%</td></tr></tbody></table>

## 3.2. Psychometric Properties of the CCAS-13 Unidimensional Model

Overall, all items have a satisfactory discrimination index (a) (Table 7). However, item 7 has an a-value of less than 1, which means that its wording needs to be readjusted or reformulated. Although acceptable, the discrimination value of this item is not as high as that of the other items, especially as item 7 has proved problematic in several other studies [16,26,27].

<table><caption>**Table 7.** Discrimination and difficulty parameters for CCAS items *n* = 905.</caption><thead><tr><th></th><th>a</th><th>b<sub>1</sub></th><th>b<sub>2</sub></th><th>b<sub>3</sub></th><th>b<sub>4</sub></th></tr></thead><tbody><tr><th>CCAS1</th><td>1.8</td><td>-1.09</td><td>-0.05</td><td>1.19</td><td>2.63</td></tr><tr><th>CCAS2</th><td>1.67</td><td>-0.84</td><td>0.25</td><td>1.47</td><td>2.74</td></tr><tr><th>CCAS3</th><td>1.21</td><td>0.64</td><td>1.85</td><td>3.48</td><td>5.59</td></tr><tr><th>CCAS4</th><td>1.38</td><td>0.58</td><td>1.42</td><td>2.88</td><td>4.51</td></tr><tr><th>CCAS5</th><td>1.4</td><td>-0.86</td><td>0.08</td><td>1.03</td><td>2.71</td></tr><tr><th>CCAS6</th><td>2.03</td><td>-0.29</td><td>0.64</td><td>1.51</td><td>2.84</td></tr><tr><th>CCAS7</th><td>0.84</td><td>0.86</td><td>1.94</td><td>3.3</td><td>5.7</td></tr><tr><th>CCAS8</th><td>1.22</td><td>-0.15</td><td>0.95</td><td>2.06</td><td>4.09</td></tr><tr><th>CCAS9</th><td>2.19</td><td>-0.52</td><td>0.37</td><td>1.49</td><td>2.56</td></tr><tr><th>CCAS10</th><td>1.27</td><td>-1.57</td><td>-0.53</td><td>0.76</td><td>2.6</td></tr><tr><th>CCAS11</th><td>2.36</td><td>-0.22</td><td>0.66</td><td>1.53</td><td>2.51</td></tr><tr><th>CCAS12</th><td>2.3</td><td>-0.15</td><td>0.63</td><td>1.45</td><td>2.34</td></tr><tr><th>CCAS13</th><td>1.46</td><td>-0.41</td><td>0.5</td><td>1.44</td><td>2.78</td></tr></tbody></table>

The IRT classifies the response modalities of each item on the same axis by calculating threshold values corresponding to a 50% probability of choosing a given modality (Table 6 and Figure 2). It also calculates a general difficulty value for each item (Figure 3).

<figure><img src="image_7.png" /><figcaption>**Figure 2.** Projection of thresholds for the modalities of each item and participants' scores standardized on the same scale. **Right panel** represents participants' scores on the eco-anxiety dimension. **Left panel** represents thresholds for having a 50% chance of endorsing a particular modality with A = always, O = often, S = sometimes, R = rarely.</figcaption></figure>
<figure><img src="image_8.png" /><figcaption>**Figure 3.** Wright map: comparison between persons' scores and general items difficulty. **Right panel:** items are sorted by their general level of difficulty. **Left panel:** persons' latent trait level of eco-anxiety.</figcaption></figure>

The overall difficulty level of the items compared with participants' scores on the latent trait (Figure 3) shows that the CCAS-13 mainly provides information on high levels of eco-anxiety. As a result, the questionnaire provides little information on almost half the sample, who identify with little or none of the current CCAS-13 items. For these participants, the CCAS-13 points to an absence of symptoms as described by the items, but tells us nothing about their mental state in relation to climate change.

**4. Discussion**

This study offers novel insights into the measurement of climate anxiety by challenging the existing factor structures of the CCAS-13. By critically analyzing the limitations of multi-factor models and proposing a unidimensional approach, we provide a fresh perspective on how eco-anxiety should be conceptualized and measured. For that purpose, we explored alternative CCAS-13 factor structures that may better explain climate anxiety. We argued in favor of using the one-dimensional model rather than the two- or three-factor model of the CCAS-13. Our results are in line with what has been found in the literature about the goodness of fit of the unidimensional model [16,26]. In fact, values of RMSEA in the literature range from 0.17 to 0.08 for the unidimensional model of CCAS-13 (see Hogg et al. [24] for a synthesis of tested models in the literature) against a value of 0.08 in this study. The SRMR values range from 0.07 to 0.04 against 0.074 for this study.

Indeed, the two- and three-dimensional models in this study, but especially in previous ones, present a problem of strong correlations between their dimensions. This means that the different factors measure almost the same thing. This misinterpretation might partly be due to the almost systematic choice of oblique rotations [57].

Although rotation simplifies model selection and interpretation, it can also be misleading [57]. Factors interpreted a posteriori are merely the result of the chosen rotation. This choice of rotation exacerbates what Van Schuur and Kiers [20] call the "extra factor" phenomenon in exploratory factor analyses. These authors describe how, for a potentially bipolar concept, factor analyses misinterpret the (potentially quadratic) relationship between the latent dimension and its indicators, transforming what should be a unidimensional bipolar phenomenon into a multidimensional one, the factors of which are more or less independent. Furthermore, the discussion around the eco-anxiety continuum, and the distinction between practical eco-anxiety and paralyzing eco-anxiety, raised questions about the multi-factor modeling of this construct. A recurring feature of the CCAS-13 data modelling we carried out without axis rotation is that there is always a dominant
factor that explains most of the variance on its own. The bifactor model, which we also
tested, and which proposes a solution to circumvent the use of rotations, confirms that a
general climate anxiety factor is the most relevant in the case of CCAS-13. Our critique of
the common use of oblique rotations highlights a significant methodological concern that
has not been adequately addressed in previous studies. This critique is crucial for future
research, as it raises awareness about potential misinterpretations because of this practice,
leading to more accurate and reliable measurement tools. Additionally, the introduction of
the bifactor model not only confirms the relevance of a general climate anxiety factor but
also offers a practical solution to the limitations of traditional factor analysis techniques.
This approach can be instrumental for researchers and practitioners in developing more
streamlined and effective assessment tools.

We have also demonstrated the usefulness of the IRT method in providing new
perspectives on the conceptualization of eco-anxiety. It consolidates the tool, but also opens
up prospects for its improvement. Indeed, the analysis results of the unidimensional model
using IRT show that the positions of the items in terms of difficulty are similar across
data sets, thus indicating a good consistency of the eco-anxiety construct as described by
CCAS-13. Consequently, in future studies, it would be more interesting to validate tools
based on a hierarchy of symptom severity.

In this regard, we note that items from the two dimensions (CEI and FI) defined by
Clayton and Karazsia [4] overlap on the eco-anxiety continuum. By interpreting the position
of the items and analyzing their content in depth, we can begin to trace a continuum of the
place eco-anxiety takes in a person's life. According to the results, it starts with (1) having
problems managing one's needs and interest in the sustainable; (2) questioning the way the
environmental situation is handled cognitively and emotionally; (3) having social problems
and a loss of meaning at work; and (4) falling into despair, accompanied by severe physical
symptoms such as crying and nightmares.

In itself, the CCAS-13 serves the authors' objective [4] of identifying severe to clinical
cases of climate change anxiety. However, one part of the continuum remains unexplored:
the part which concerns people with low scores. This limits the use of the CCAS among the
general public and raises questions about how this issue is represented.

Based on the floor effects observed on the distributions of the various items, states
of eco-anxiety other than those described above are likely to be grouped in the non-
anxious category. However, even if these people do not suffer from severe symptoms,
they may nonetheless feel a certain amount of anxiety and stress relative to the perceived
environmental crisis. In a recent qualitative study, Marczak et al. [11] show that people who
are strongly affected by the ecological crisis may not present any of the symptoms described
in the CCAS-13 [11]. If this questionnaire is to be adapted to a general population, we
need to consider adjusting and supplementing it by including items that refer to less severe
manifestations, and thus formulate a tool that could fit in with the idea of a continuum
relating to eco-anxiety.

A number of researchers are now advocating the continuum hypothesis of climate
anxiety [2,7], which makes it possible to identify people who are worried, stressed, or
even anxious about climate change, even though they do not display any manifestations
measured by the CCAS or HEAS. In the current state of the literature, we have several tools
that can be used to identify different levels of discomfort with climate change (i.e., Climate
Change Worry Scale, Climate Change Distress Scale, HEAS). Combining these measurement
instruments with the CCAS would support the idea suggested by Mathé et al. [15] of
developing a unified measurement scale covering the entire eco-anxiety continuum. The
availability of such a tool would make it possible to not only care for people suffering
from eco-anxiety, which they are unable to manage on their own, but also to offer others
the opportunity to engage in action to enable them to manage the discomfort they feel
on their own. The creation of such a scale may help mental health professionals adapt
their intervention to the level of eco-anxiety of the patients, and also help behavioral
change experts and policy makers to take into account the different levels of eco-anxiety of
individuals in studying the social acceptability of new climate policies. This study has a broader objective, which is the advocation for a non-pathologizing approach to eco-anxiety, as, in our point of view, eco-anxiety is a natural outcome for the lack of sense of collective agency and the ability to initiate systemic change while perceiving the need for urgent societal change.

## 4.1. Limitations

This study is not without limitations. First, participants questioned in the samples are mainly French-speaking Europeans. However, a comparison is made with modeling conducted on an American population. In no way can we guarantee the absence of a cultural effect on modeling. Second, the analysis package for bifactorial models is limited. As a result, certain tests could not be carried out. So, the evaluation of the bifactorial model should be carried out again when the appropriate tools are available, since according to Gibbons and Cai [38], the bifactorial model is more promising due to its greater accuracy, especially in the context of supposedly multidimensional data.

## 4.2. Perspectives

In order to validate a questionnaire that could be used with the entire population, we would have to resort to "computer adaptive testing" [58]. This method would require the development of a large panel of questions based on the CCAS-13 items, but also consider what a more moderate level of eco-anxiety encompasses. In this way, we could better identify the threshold for distinguishing between the pathological and motivational dimensions of eco-anxiety relating to the ecological transition [11]. The use of other statistical methodologies, such as the Receiver Operating Characteristic (ROC) [59], could enable the definition of thresholds and optimally detect false positives.

# 5. Conclusions

This study aimed to explore the structure of the Climate Change Anxiety Scale (CCAS-13) using item response theory (IRT), offering new insights into the measurement of eco-anxiety. Our analysis demonstrated that the unidimensional model, which assumes a single underlying factor, provided the best solution for modeling eco-anxiety. These findings challenge previous multi-dimensional models and support the notion that eco-anxiety operates along a single continuum from mild concern to severe anxiety. The IRT analysis also revealed that the CCAS-13 is more effective at detecting severe forms of eco-anxiety, with less sensitivity to milder manifestations.

These results have significant implications for both research and clinical practice. For researchers, our findings suggest that the current tools might be insufficient for capturing the full spectrum of eco-anxiety, particularly its milder forms. This highlights the need for the development of new measurement tools or the refinement of existing ones to ensure they are comprehensive and sensitive to varying levels of eco-anxiety. Clinically, the results underscore the importance of considering the continuum of eco-anxiety when assessing individuals, as the current focus on severe symptoms may overlook those experiencing less intense, yet still impactful, eco-anxiety.

Future research should explore the development of more nuanced instruments that can capture the entire eco-anxiety spectrum and further investigate the factors that contribute to the variability in eco-anxiety experiences across different populations. Understanding these dynamics will be crucial for both improving mental health interventions and fostering more effective public communication strategies about ecological transition policies.
**Author Contributions:** Conceptualization, T.H. and A.S.; methodology, T.H.; software, T.H.; validation, T.H. and S.Y.; formal analysis, T.H.; investigation, T.H.; data curation, T.H.; writing—original draft preparation, T.H.; writing—review and editing, A.S. and S.Y.; supervision, A.S. All authors have read and agreed to the published version of the manuscript.

**Funding:** This study is a part of the first author's Ph.D. project, which is cofounded by the University of Rennes 2 (50%: https://www.univ-rennes2.fr/) and the region of Brittany (50% R930G6 22RCH-G601 RCARE). The latter is the administrative entity responsible for the development of the territory of Brittany in France. This funding sources had no role in the design of this study nor any role during its execution, analyses, interpretation of the data, nor in the decision to submit results.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** Data are contained within the article.

**Conflicts of Interest:** The authors declare no conflicts of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

## References

1. Coffey, Y.; Bhullar, N.; Durkin, J.; Islam, M.S.; Usher, K. Understanding eco-anxiety: A systematic scoping review of current literature and identified knowledge gaps. *J. Clim. Chang. Health* **2021**, *3*, 100047. [CrossRef]

2. Lutz, P.K.; Passmore, H.; Howell, A.J.; Zelenski, J.M.; Yang, Y.; Richardson, M. The Continuum of Eco-Anxiety Responses: A preliminary investigation of its nomological network. *Collabra* **2023**, *9*, 67838. [CrossRef]

3. Pihkala, P. Anxiety and the Ecological Crisis: An analysis of Eco-Anxiety and Climate Anxiety. *Sustainability* **2020**, *12*, 7836. [CrossRef]

4. Clayton, S.; Karazsia, B.T. Development and validation of a measure of climate change anxiety. *J. Environ. Psychol.* **2020**, *69*, 101434. [CrossRef]

5. Hogg, T.L.; Stanley, S.K.; O'Brien, L.; Wilson, M.; Watsford, C. The Hogg Eco-Anxiety Scale: Development and validation of a multidimensional scale. *Glob. Environ. Chang.* **2021**, *71*, 102391. [CrossRef]

6. Verplanken, B.; Marks, E.; Dobromir, A.I. On the nature of eco-anxiety: How constructive or unconstructive is habitual worry about global warming? *J. Environ. Psychol.* **2020**, *72*, 101528. [CrossRef]

7. Passmore, H.; Lutz, P.K.; Howell, A.J. Eco-Anxiety: A CCAScade of fundamental existential anxieties. *J. Constr. Psychol.* **2022**, *36*, 138–153. [CrossRef]

8. Soutar, C.; Wand, A. Understanding the spectrum of anxiety Responses to climate change: A Systematic Review of the Qualitative literature. *Int. J. Environ. Res. Public Health* **2022**, *19*, 990. [CrossRef]

9. Kurth, C.; Pihkala, P.P. Eco-anxiety: What it is and why it matters. *Front. Psychol.* **2022**, *13*, 981814. [CrossRef]

10. Léger-Goodes, T.; Malbœuf-Hurtubise, C.; Mastine, T.; Généreux, M.; Paradis, P.; Camden, C. Eco-anxiety in Children: A scoping review of the mental health impacts of the awareness of climate change. *Front. Psychol.* **2022**, *13*, 872544. [CrossRef]

11. Marczak, M.; Winkowska, M.; Chaton-Østlie, K.; Morote Rios, R.; Klöckner, C.A. "When I say I'm depressed, it's like anger". An exploration of the emotional landscape of climate change concern in Norway and its psychological, social and political implications. *Emot. Space Soc.* **2023**, *4*, 100939. [CrossRef]

12. Stewart, A.E. Psychometric Properties of the Climate Change Worry Scale. *Int. J. Environ. Res. Public Health* **2021**, *18*, 494. [CrossRef]

13. Hepp, J.; Klein, S.A.; Horsten, L.K.; Urbild, J.; Lane, S.P. Introduction and behavioral validation of the climate change distress and impairment scale. *Sci. Rep.* **2023**, *13*, 11272. [CrossRef]

14. Jordan, P.; Shedden-Mora, M.C.; Löwe, B. Psychometric analysis of the Generalized Anxiety Disorder scale (GAD-7) in primary care using modern item response theory. *PLoS ONE* **2017**, *12*, e0182162. [CrossRef] [PubMed]

15. Mathé, M.; Grisetto, F.; Gauvrit, N.; Roger, C. Psychometric validation of the French version of the Hogg Eco-Anxiety Scale (HEAS-FR). *Can. J. Behav. Sci./Rev. Can. Sci. Comport.* **2023**. [CrossRef]

16. Mouguiama-Daouda, C.; Blanchard, M.A.; Coussement, C.; Heeren, A. On the measurement of climate change anxiety: French validation of the climate anxiety scale. *Psychol. Belg.* **2022**, *62*, 123. [CrossRef] [PubMed]

17. Ágoston, C.; Urbán, R.; Nagy, B.; Csaba, B.; Kőváry, Z.; Kovács, K.; Varga, A.; Dúll, A.; Mónus, F.; Shaw, C.A.; et al. The psychological consequences of the ecological crisis: Three new questionnaires to assess eco-anxiety, eco-guilt, and ecological grief. *Clim. Risk Manag.* **2022**, *37*, 100441. [CrossRef]

18. Van Der Linden, W.J. *Handbook of Item Response Theory, Three Volume Set*; Chapman and Hall: London, UK; CRC eBooks: Boca Raton, FL, USA, 2018. [CrossRef]

19. Noel, Y.; Fevrier, F.; Deflandre, A. Two Factors but One Dimension: An Alternative View at the Structure of Mood and Emotion. *PsychArchives* **2018**, *1*, 1–24.
20. Van Schuur, W.H.; Kiers, H.A. Why factor analysis often is the incorrect model for analyzing bipolar concepts, and what model to use instead. *Appl. Psychol. Meas.* **1994**, *18*, 97–110. [CrossRef]

21. De Boeck, P. Psychological testing. In *Handbook of Item Response Theory: Applications*; Van Der Linden, W.J., Ed.; Chapman and Hall: London, UK; CRC Press: Boca Raton, FL, USA, 2018; Volume 3, pp. 313–328.

22. Herbemont, A. ÉCo-anxiété: Huit Français Sur 10 SE Disent Inquiets Face AU Changement Climatique. *Franceinfo*. 2023. Available online: https://www.francetvinfo.fr/replay-radio/le-brief-politique/eco-anxiete-huit-francais-sur-10-se-disent-inquiets-face-au-changement-climatique_6114024.html (accessed on 27 October 2023).

23. Embretson, S.E.; Reise, S.P. *Item Response Theory for Psychologists*; Lawrence Erlbaum Associates Publishers: Mahwah, NJ, USA, 2000.

24. Hogg, T.L.; Stanley, S.K.; O'Brien, L. Synthesising psychometric evidence for the Climate Anxiety Scale and Hogg Eco-Anxiety Scale. *J. Environ. Psychol.* **2023**, *88*, 102003. [CrossRef]

25. Immekus, J.C.; Snyder, K.E.; Ralston, P.A. Multidimensional Item Response Theory for Factor Structure Assessment in Educational Psychology Research. *Front. Educ.* **2019**, *4*, 45. [CrossRef]

26. Cruz, S.M.; High, A.C. Psychometric properties of the climate change anxiety scale. *J. Environ. Psychol.* **2022**, *84*, 101905. [CrossRef]

27. Larionow, P.; Sołtys, A.; Fedyk, M. Methodological triangulation to measure climate change awareness and anxiety among students. *Int. J. Environ. Res. Public Health* **2023**, *20*, 4612.

28. Tam, K.-P.; Chan, H.-W.; Clayton, S. Climate change anxiety in China, India, Japan, and the United States. *J. Environ. Psychol.* **2023**, *87*, 101991. [CrossRef]

29. Heeren, A.; Mouguiama-Daouda, C.; McNally, R.J. A network approach to climate change anxiety and its key related features. *J. Anxiety Disord.* **2023**, *93*, 102625. [CrossRef]

30. Chalmers, R.P. mirt: A Multidimensional Item Response Theory Package for the R Environment. *J. Stat. Softw.* **2012**, *48*, 1–29. [CrossRef]

31. R Core Team. *R: A Language and Environment for Statistical Computing*; R Foundation for Statistical Computing: Vienna, Austria, 2023. Available online: https://www.R-project.org/ (accessed on 8 October 2023).

32. Samejima, F. Estimation of Latent Ability Using a Response Pattern of Graded Scores. *Psychometr. Monogr. Suppl.* **1969**, *34*, 100. [CrossRef]

33. Ayala, R.J. *The Theory and Practice of Item Response Theory*; Guilford Press: New York, NY, USA, 2009.

34. Baker, F.B.; Kim, S.H. *Item Response Theory: Parameter Estimation Techniques*; CRC Press: Boca Raton, FL, USA, 2004.

35. Bock, R.D.; Aitkin, M. Marginal Maximum Likelihood Estimation of Item Parameters: Application of an EM Algorithm. *Psychometrika* **1981**, *46*, 443–459. [CrossRef]

36. Chalmers, R.P. Numerical Approximation of the Observed Information Matrix with Oakes’ Identity. *Br. J. Math. Stat. Psychol.* **2018**, *71*, 415–436. [CrossRef]

37. Gibbons, R.D.; Bock, R.D.; Hedeker, D.; Weiss, D.; Bhaumik, D.K.; Kupfer, D.; Frank, E.; Grochocinski, V.; Stover, A. Full-information item bi-factor analysis of graded response data. *Appl. Psychol. Meas.* **2007**, *31*, 4–19. [CrossRef]

38. Gibbons, R.D.; Cai, L. Dimensionality analysis. In *Handbook of Item Response Theory: Applications*, 2nd ed.; Vander Linden, W.J., Ed.; Chapman and Hall: London, UK; CRC Press: Boca Raton, FL, USA, 2018; Volume 3, pp. 47–65.

39. Rijmen, F. Formal relations and an empirical comparison between the bi-factor, the testlet, and a second order multidimensional IRT model. *J. Educ. Meas.* **2010**, *47*, 361–372. [CrossRef]

40. Reise, S.P. The rediscovery of bifactor measurement models. *Multivar. Behav. Res.* **2012**, *47*, 667–696. [CrossRef]

41. Cai, L.; Hansen, M. Limited-information goodness-of-fit testing of hierarchical item factor models. *Br. J. Math. Stat. Psychol.* **2012**, *66*, 245–276. [CrossRef]

42. Steinberg, L.; Thissen, D. Uses of item response theory and the testlet concept in the measurement of psychopathology. *Psychol. Methods* **1996**, *1*, 81–97. [CrossRef]

43. Reckase, M.D. *Multidimensional Item Response Theory*; Springer: New York, NY, USA, 2009.

44. Bartholomew, D.J.; Tzamourani, P. The goodness of fit of latent trait models in attitude measurement. *Sociol. Methods Res.* **1999**, *27*, 525–546. [CrossRef]

45. Wirth, R.; Edwards, M.C. Item Factor Analysis: Current approaches and future directions. *Psychol. Methods* **2007**, *12*, 58–79. [CrossRef] [PubMed]

46. Maydeu-Olivares, A.; Joe, H. Assessing approximate fit in categorical data analysis. *Multivar. Behav. Res.* **2014**, *49*, 305–328. [CrossRef]

47. Bentler, P.M. Comparative fit indexes in structural models. *Psychol. Bull.* **1990**, *107*, 238–246. [CrossRef] [PubMed]

48. Hu, L.-T.; Bentler, P.M. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. *Struct. Equ. Model.* **1999**, *6*, 1–55. [CrossRef]

49. Akaike, H. Factor analysis and AIC. *Psychometrika* **1987**, *52*, 317–332. [CrossRef]

50. Bozdogan, H. Model selection and Akaike's Information Criterion (AIC): The general theory and its analytical extensions. *Psychometrika* **1987**, *52*, 345–370. [CrossRef]

51. Byrne, B.M. *Structural Equation Modeling with AMOS: Basic Concepts, Applications, and Programming*, 2nd ed.; Routledge: New York, NY, USA, 2013.

52. Embretson, S.E.; Reise, S.P. *Item Response Theory*, 2nd ed.; Psychology Press: New York, NY, USA, 2013.
53. Baker, F.B. *The Basics of Item Response Theory*, 2nd ed.; Eric Clearing House on Assessment and Evaluation: College Park, MD, USA, 2001.

54. Hambleton, R.K.; Swaminathan, H.; Rogers, H.J. *Fundamentals of Item Response Theory*; Sage Publications: Thousand Oaks, CA, USA, 1991.

55. Treynor, W.; Gonzalez, R.; Nolen-Hoeksema, S. Rumination reconsidered: A psychometric analysis. *Cognit. Ther. Res.* **2003**, *27*, 247–259. [CrossRef]

56. Weiss, M.D. *Weiss Functional Impairment Rating Scale (WFIRS) Self-Report*; University of British Columbia: Vancouver, BC, Canada, 2000.

57. Browne, M.W. An overview of analytic rotation in exploratory factor analysis. *Multivar. Behav. Res.* **2001**, *36*, 111–150. [CrossRef]

58. Gibbons, R.D.; Weiss, D.J.; Kupfer, D.J.; Frank, E.; Fagiolini, A.; Grochocinski, V.J.; Bhaumik, D.K.; Stover, A.; Bock, R.D.; Immekus, J.C. Using computerized adaptive testing to reduce the burden of mental health assessment. *Psychiatr. Serv.* **2008**, *59*, 361–368. [CrossRef] [PubMed]

59. Swets, J.A. *Signal Detection Theory and ROC Analysis in Psychology and Diagnostics: Collected Papers*; Lawrence Erlbaum Associates, Inc.: Mahwah, NJ, USA, 1996.

**Disclaimer/Publisher's Note:** The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.