# Validation of visual analog scales of mood and anxiety at the workplace

Frédéric Dutheil, Clara Palgen, Georges Brousse, Thomas Cornet, Martial Mermillod, Ines Lakbar, Guillaume T. Vallet, Julien S Baker, Jeannot Schmidt, Barbara Charbotel, et al.

► To cite this version:

Frédéric Dutheil, Clara Palgen, Georges Brousse, Thomas Cornet, Martial Mermillod, et al. Validation of visual analog scales of mood and anxiety at the workplace. PLoS ONE, 2024, 19 (12), pp.e0316159. 10.1371/journal.pone.0316159. hal-04908693

HAL Id: hal-04908693

https://hal.science/hal-04908693v1

Submitted on 23 Jan 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
RESEARCH ARTICLE

# Validation of visual analog scales of mood and anxiety at the workplace

Frédéric Dutheil¹**, Clara Palgen¹*, Georges Brousse², Thomas Cornet³, Martial Mermillod⁴,⁵, Ines Lakbar⁶, Guillaume Vallet⁷, Julien S. Baker⁸, Jeannot Schmidt⁹, Barbara Charbotel¹⁰, Bruno Pereira¹¹, Louis Delamarre¹⁵

1 University Hospital of Clermont-Ferrand, CHU Clermont-Ferrand, Preventive and Occupational Medicine, Université Clermont Auvergne, Clermont-Ferrand, France, 2 NPSY-SYDO, University Hospital of Clermont-Ferrand, CHU Clermont-Ferrand, Psychiatry, Université Clermont Auvergne, Clermont-Ferrand, France, 3 WittyFit, Paris, France, 4 CNRS, LPNC, Univ. Grenoble Alpes, Grenoble, France, 5 Institut Universitaire de France, Paris, France, 6 Intensive Care Unit, Anesthesia and Critical Care Department (DAR-B), Saint-Eloi Teaching Hospital, University of Montpellier, Montpellier, France, 7 CNRS, LaPSCo, Physiological and Psychosocial Stress, Université Clermont Auvergne, Clermont-Ferrand, France, 8 Centre for Health and Exercise Science Research, Department of Sport, Physical Education and Health, Hong Kong Baptist University, Kowloon Tong, Hong Kong, 9 CNRS, LaPSCo, Physiological and Psychosocial Stress, University Hospital of Clermont-Ferrand (CHU), Emergency Department, Université Clermont Auvergne, Clermont-Ferrand, France, 10 Univ Lyon 1, IFSTTAR, UMRESTTE, UMR_T9405, University of Lyon, Lyon, France, 11 University Hospital of Clermont Ferrand, CHU Clermont-Ferrand, Clinical Research and Innovation Direction, Clermont-Ferrand, France

* These authors contributed equally to this work.

* fdutheil@chu-clermontferrand.fr

<figure><img src="image_2.png" /></figure>

<figure><img src="image_3.png" /></figure>

## OPEN ACCESS

**Citation:** Dutheil F, Palgen C, Brousse G, Cornet T, Mermillod M, Lakbar I, et al. (2024) Validation of visual analog scales of mood and anxiety at the workplace. PLoS ONE 19(12): e0316159. https://doi.org/10.1371/journal.pone.0316159

**Editor:** Alice Mannocci, Telematic University San Raffaele Rome; Universita Telematica San Raffaele Roma, ITALY

**Received:** August 20, 2024

**Accepted:** December 8, 2024

**Published:** December 31, 2024

**Copyright:** © 2024 Dutheil et al. This is an open access article distributed under the terms of the [Creative Commons Attribution License](https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

**Data Availability Statement:** All files are available from the "Validation of VAS Anxiety and Depression" database (URL https://osf.io/6vcsg/?view_only=3f4a891812544f1999da8caaba674c28).

**Funding:** The study is integrally funded by WittyFit (https://www.cegid.com/fr/produits/cegid-wittyfit/), Paris, France. The funding went to FD. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

## Abstract

The Hospital Anxiety and Depression Scale (HADS) is commonly used to detect depressive or anxious states, but its 14-item questionnaire is time-consuming. Visual analog scales (VAS) are easy to use and quick to implement. Although the VAS has been validated to assess pain and occupational stress, VAS scores for anxiety and mood have never been evaluated in the workplace. We aimed to validate the use of visual analog scales (VAS) for anxiety and mood compared to HADS in workers. A HADS self-reported questionnaire associated with VAS assessing perceived anxiety and mood on a horizontal line of 100 mm was administered to 182 workers, with a second test (retest) proposed one week later. Sociodemographic, characteristics of work, sleep, well-being, and stress were also assessed. VAS anxiety and mood correlated with the HADS sub-scores (0.70 and 0.65, respectively). The test-retest reliability was good. Optimal VAS cut-offs were ≥ 60/100 for anxiety and ≤ 60/100 for mood, to define at-risk patients. The VAS is quick to perform, easy to use, and reliable for screening depression and anxiety in occupational medicine. We recommend validated questionnaires for at-risk patients.

**Trial registration.** Clinicaltrials.gov: NCT02596737. Available at: https://www.clinicaltrials.gov/ct2/show/NCT02596737.

## Introduction

The World Health Organization (WHO) estimates that mental illness affects one in five people globally and one in three individuals in his/her lifetime [1]. Mental disorders represent the
**Competing interests:** TC is director of WittyFit. This does not alter our adherence to PLOS ONE policies on sharing data and materials. Other authors have declared that no competing interests exist.

second leading cause of sickness absence [2–4] and the leading cause of disability. Anxiety disorders and mood disorders are common when related to mental illness [5]. The estimated prevalence of anxiety is 18% and mood disorders are 9.5% [6]. Most of them appear at the beginning or during working life [5] and can negatively affect the worker, the organization and the society. Comorbid anxiety and depressive disorders have the greatest impact on long-term work disability and absenteeism, emphasizing the need to address both conditions simultaneously [7]. Psychological ill-health affects human performance and increases the risk of accidents, especially in hazardous industries [8–10]. In manufacturing, an anxious worker is more likely to experience accidents, endangering himself and coworkers, but also increasing downtime and operational costs. Anxiety and depression can lead to both absenteeism [11], and presenteeism [12]. Presenteeism accounts for a significant portion of productivity loss, surpassing direct medical costs for depression [13]. Untreated mental health issues, particularly depression, contribute to a $44 billion annual loss of productive time in the USA [14]. Workplace interventions targeting mental health are moderately effective in improving work outcomes, while programs based on exposure or that combine mental, social and physical health interventions are the most effective [15]. Companies with supportive environments observe a 30% reduction is stress-related absenteeism [16]. For every dollar spent in such programs, companies see up to a $4 return in reduced absenteeism and healthcare costs [17]. Workplace mental health programs and interventions seem to help individual employees but also reduce larger societal costs [17, 18].

Screening and assessing the severity of symptoms is therefore essential to occupational health and should especially happen during consultation by an occupational physician. Among the multiple validated tests assessing anxiety and depression [19–21], the Hospital Anxiety and Depression Scale (HADS) is the most common test. Yet, the questionnaire consists of 14 items and requires extended periods during occupational health consultation, while occupational health physicians have limited time to deal with a large number of workers and worksites [22]. Visual analog scales (VAS) are reproducible tools, fast, and easy to use, with good psychometric characteristics, already validated for pain [23], occupational stress [24, 25], and job satisfaction [26]. Therefore, we hypothesized that a VAS for mood and anxiety would identify at-risk workers with depressive or anxious symptoms, in comparison with the HADS.

Stress, sleep disturbance and working hours have been previously been linked with anxiety and depression scores using HADS [27, 28], and should be associated with VAS anxiety and mood.

The main objective of this study was to validate VAS mood and VAS anxiety vs. HADS in a population of active workers. The external validity was evaluated for both VAS by highlighting their relationships with sociodemographic, professional and well-being characteristics.

# Materials and methods

## Participants

An epidemiological, observational, descriptive and cross-sectional study was conducted between 2016-03-01 and 2016-05-12, for which self-questionnaires were sent to users of the WittyFit software [29]. The name WittyFit comes from Witty and Fitness and reflects the concept of health from the World Health Organization (WHO): to be in good physical and mental health. WittyFit is software whose objective is to promote well-being in companies, with an epidemiological and research conception. WittyFit performs a personalized evaluation by self-questionnaire. Participation in the study required written consent. This observational study received approval from the Ethics Committee of the University Hospital of Clermont-Ferrand, France (clinicaltrials.gov identifier NCT02596737) and from National Commission for
Information Technology and Civil Liberties (CNIL). All data was anonymous, and the employ-
ees' identity was not collected in the database. A human resource created a number to feed the
database, which was then automatically modified by another number.

Participants were invited to participate in the study via the WittyFit software, which deliv-
ered the research invite to the entire workforce of participating companies, with the option to
refuse. Any worker accepting to join the WittyFit research was considered eligible to partici-
pate. No other inclusion criteria were enforced.

**Primary outcome**

The primary objective of this study was to validate the two VAS (anxiety and mood) as an
alternative to the HADS.

Zigmond and Snaith published HADS for the first time in 1983, in English [30–32]. This was then translated and validated in French by Lépine in 1985 and by Ravazi in 1989 [33–35]. This scale is used for screening for common psychopathological disorders. Seven items evaluate depressive symptoms (items 2, 4, 6, 8, 10, 12, 14): one for dysphoria, one for psychomotor slowdown and five for anhedonia. Seven other items measure anxiety (items 1, 3, 5, 7, 9, 11, 13). Each item is comprised of a four-point Likert scale. The scale of answers is sometimes reversed to avoid repetition bias. The questionnaire generates 2 sub-scores, corresponding to the two subscales of anxiety and depression, with two thresholds set at 7 and 11. Scores 0–7 refer to no anxiety or depressive disorders; 8–10 suspected anxiety or depressive disorders; 11–21 proven anxiety or depressive disorders [30–32, 36].

VAS assessed the perceived anxiety and mood of individuals at work, on a horizontal, non-
calibrated line of 100 mm. The scales ranged from very low (0) to very high anxiety (100) for
VAS anxiety; and ranging from very sad mood (0) to very good mood (100) for VAS mood.

**Secondary outcomes**

Secondary outcomes were defined as the measures of association between the HADS or VAS and workers’ personal and work-related characteristics. Sociodemographic data such as age, gender, education level, and marital status was collected. The characteristics of work were also collected, such as occupation, number of working hours per week, seniority within the company, the type of work schedule (fixed, shift, nightshifts) and the number of nightshifts per month. Body mass index was calculated from height and weight. Sleep quantity was evaluated by the number of sleeping hours per night [37]. Sleep quality, well-being, stress at work, and stress at home were evaluated with VAS ranging from very low (0) to very high (100) [22, 25].

**Time of measurements**

Participants could complete the whole questionnaire whenever they wished. All responders
were also invited to complete the HADS and VAS anxiety and mood again a week later, to per-
form the test-retest approach. The handover time was approximately 15 minutes.

**Statistical analysis**

Sample size was determined according to COSMIN recommendations [38, 39] as follows. 1) "Rules-of-thumb vary from four to 10 subjects per variable, with a minimum number of 100 subjects to ensure stability of the variance-covariance matrix" and 2) "Often 0.70 is recommended as a minimum standard for reliability. We give a positive rating for reliability when the ICC or weighted Kappa is at least 0.70 in a sample size of at least 50 patients."
The Stata software (Version 13, StataCorp, College Station, TX) was used, with a two-
sided Type I error of α = 5% to perform statistical analysis. Participants’ characteristics were
expressed as mean ± standard deviation (SD) or median [interquartile range] for continuous
data (assumption of normality assessed by the Shapiro-Wilk test) and as numbers and associ-
ated percentages for categorical parameters.

The test-retest reliability was performed using Pearson correlation coefficient, Lin concor-
dance coefficient and Bland and Altman plots. The external validity was assessed using a corre-
lation coefficient (Pearson or Spearman according to statistical distribution) between VAS and
other psychological measures, such as the HADS score. Then, a ROC receiver operating char-
acteristic (ROC) curve analysis were used to determine the optimal thresholds of VAS to pre-
dict HADS, according to clinical relevance and usual indexes reported in the literature
(Youden index, Liu index and efficiency) [40, 41]. Sensitivity, specificity, positive and negative
predictive values were calculated and presented with 95% confidence intervals. The agreement
between the HADS anxiety and depression sub-score and corresponding VAS, according to
cut-offs determined by ROC curve analysis, was evaluated using agreement rates. Finally,
quantitative variables were compared between independent groups by ANOVA or the Krus-
kal-Wallis test, if ANOVA assumptions were not met (normality and homoscedasticity were
analyzed using the Bartlett test). When appropriate, post-hoc tests were performed considering
multiple comparisons (Tukey-Kramer post ANOVA and Dunn after Kruskal-Wallis). The
comparisons between groups were carried out using the Chi-squared or Fischer’s exact test for
categorical variables. When appropriate, a post-hoc test was used (Marascuillo procedure).
External validity was assessed by comparing the associations of personal and work-related var-
iables (sex, age, BMI, well-being VAS, sleep quality VAS, sleep duration, stress at work VAS,
stress at home VAS, seniority in company, and weekly workload) between participants with
and without risks of depression or anxiety, as measured by HADS and VAS. To assess the
robustness of our primary analysis, a sensitivity analysis was performed on a subset of the pop-
ulation in which the primary outcomes (VAS anxiety and depression as well as HADS-A and
HADS-D), but also the variables used to assess external validity evidence (sex, age, BMI, well-
being VAS, Sleep quality VAS, Sleep duration, Stress at work VAS, stress at home VAS, senior-
ity in company, weekly workload) were complete.

Results

Participants

Among the 1580 workers and users of WittyFit, 222 (14%) agreed to participate. The data for the primary outcomes HADS Anxiety (HADS-A), VAS anxiety, HADS Depression (HADS-D) and VAS mood was missing in 40 participants. The analysis was then conducted on 182 (82%) of the responders. Among them, 86 (47.3%) were women (with 26 missing data). The test-retest approach was calculated on 123 participants who answered twice to the HADS and the two VAS (anxiety and mood) (Fig 1). The average age was 41.4 ± 11.6 years old. Half of the workers were married (44%, 26 missing data). Most included workers were senior executives (57.7%) and had reached a master’s degree or higher (62.6%) (Table 1).

Hospital anxiety and depression scale

To define the threshold on the VAS anxiety and mood compared to the HADS, we assumed
for each sub score of anxiety and depression that a score ≤7 was normal and a score ≥ 8 was
considered abnormally high.

The mean score of HADS-A was 6.9 ± 4.0. 108 (59.3%) participants had no anxiety symp-
toms (score ≤7) and 74 (40.7%) had anxiety symptoms (score ≥8) The mean score of
<figure><img src="image_4.png" /><figcaption>**Fig 1. Flowchart and study design.** Abbreviations: *: The variables considered to define the sensitivity analysis were sex, age, BMI, well-being VAS, Sleep quality VAS, Sleep duration, Stress at work VAS, stress at home VAS, seniority in company, weekly workload. VAS: Visual Analog Scale.</figcaption></figure>

https://doi.org/10.1371/journal.pone.0316159.g001

HADS-D was 3.8 ± 3.0. 157 (86.3%) participants had no depressive symptoms (score ≤7) and 25 (13.7%) had depressive symptoms (score ≥8) (Table 2).

## VAS anxiety and VAS mood: Cut-offs determination and agreement rate

Mean VAS anxiety was 50.4 ± 28.7. A best cut-off value at 63.5 (*p* <0.001) was calculated to assess the risk of anxiety using VAS Anxiety, with a sensibility of 71.6% (95CI: 59.9% to 81.5%) and a specificity of 86.1% (78.1% to 92.0%), an area under the curve of 0.79 (0.73 to 0.85), a positive predictive value of 78% (66.2% to 87.1%), a negative predictive value of 81.6% (73.2% to 88.2%), and an agreement of 80.2%.

Mean VAS mood was 65.4 ± 24. A best cut-off value at 58 (higher risk for values ≤ 58, *p* <0.001) was calculated to assess the risk of depression using VAS mood, with a sensitivity of
<table><caption>Table 1. Characteristics of participants in terms of demographics, education level, marital status, occupational categories and workload.</caption><thead><tr><th>Characteristics of participants</th><th>Sample size (n = 182)</th><th>%</th></tr></thead><tbody><tr><td><strong>Sex</strong></td><td></td><td></td></tr><tr><td>Women</td><td>86</td><td>47.3</td></tr><tr><td>Men</td><td>70</td><td>38.5</td></tr><tr><td>Missing</td><td>26</td><td>14.3</td></tr><tr><td>Age, years (mean ± standard deviation)</td><td colspan="2">41.4 ± 11.6</td></tr><tr><td><strong>Education level</strong></td><td></td><td></td></tr><tr><td>Undergraduate</td><td>10</td><td>5.5</td></tr><tr><td>Bachelor degree or less</td><td>32</td><td>17.6</td></tr><tr><td>Master degree or more</td><td>114</td><td>62.6</td></tr><tr><td>Missing</td><td>26</td><td>14.3</td></tr><tr><td><strong>Marital status</strong></td><td></td><td></td></tr><tr><td>Single</td><td>33</td><td>18.1</td></tr><tr><td>Concubinage</td><td>42</td><td>23.1</td></tr><tr><td>Married</td><td>80</td><td>44</td></tr><tr><td>Widowed</td><td>1</td><td>0.6</td></tr><tr><td>Missing</td><td>26</td><td>14.3</td></tr><tr><td><strong>Occupational categories</strong></td><td></td><td></td></tr><tr><td>Senior executives</td><td>105</td><td>57.7%</td></tr><tr><td>Mid level workers</td><td>16</td><td>8.8%</td></tr><tr><td>Skilled workers</td><td>26</td><td>14.3%</td></tr><tr><td>Unemployed</td><td>3</td><td>1.6%</td></tr><tr><td>Retired</td><td>6</td><td>3.3%</td></tr><tr><td>Missing</td><td>26</td><td>14.3%</td></tr><tr><td><strong>Work schedule</strong></td><td></td><td></td></tr><tr><td>Fixed</td><td>126</td><td>69.2%</td></tr><tr><td>Daytime Shiftwork</td><td>3</td><td>1.7%</td></tr><tr><td>Shiftwork including nighshift</td><td>27</td><td>14.8%</td></tr><tr><td>Missing</td><td>26</td><td>14.3%</td></tr><tr><td><strong>Number of nighshifts per month</strong></td><td></td><td></td></tr><tr><td>0</td><td>129</td><td>70.9%</td></tr><tr><td>1</td><td>6</td><td>3.3%</td></tr><tr><td>2</td><td>5</td><td>2.7%</td></tr><tr><td>3</td><td>1</td><td>0.5%</td></tr><tr><td>4</td><td>2</td><td>1.1%</td></tr><tr><td>5</td><td>5</td><td>2.7%</td></tr><tr><td>6</td><td>2</td><td>1.1%</td></tr><tr><td>7</td><td>4</td><td>2.2%</td></tr><tr><td>8 and more</td><td>2</td><td>1.1%</td></tr><tr><td>Missing</td><td>26</td><td>14.3%</td></tr><tr><td><strong>Number of hours per week</strong></td><td></td><td></td></tr><tr><td>Mean ± SD, in hours</td><td colspan="2">41.5 ± 12.1</td></tr><tr><td>0–10 hours</td><td>4</td><td>2.2%</td></tr><tr><td>11–20 hours</td><td>5</td><td>2.7%</td></tr><tr><td>21–30 hours</td><td>7</td><td>3.8%</td></tr><tr><td>31–40 hours</td><td>75</td><td>41.2%</td></tr><tr><td>41–50 hours</td><td>38</td><td>20.9%</td></tr></tbody></table>

(Continued)
<table><caption>**Table 1.** (Continued)</caption><thead><tr><th>Characteristics of participants</th><th>Sample size (n = 182)</th><th>%</th></tr></thead><tbody><tr><td>51–60 hours</td><td>15</td><td>8.2%</td></tr><tr><td>61–70 hours</td><td>5</td><td>2.7%</td></tr><tr><td>71–80 hours</td><td>1</td><td>0.5%</td></tr><tr><td>81–90 hours</td><td>1</td><td>0.5%</td></tr><tr><td>Missing</td><td>31</td><td>17%</td></tr><tr><td>Seniority in the company (in years)</td><td></td><td></td></tr><tr><td>[0–5[</td><td>61</td><td>33.5%</td></tr><tr><td>[5–10[</td><td>30</td><td>16.5%</td></tr><tr><td>[10–20[</td><td>36</td><td>19.8%</td></tr><tr><td>&gt; = 20</td><td>26</td><td>14.3%</td></tr><tr><td>Missing</td><td>29</td><td>15.9%</td></tr></tbody></table>

https://doi.org/10.1371/journal.pone.0316159.t001

84% (95%CI: 63.9% to 95.5%), a specificity of 72.6% (64.9% to 79.4%), an area under the curve of 0.78 (0.70 to 0.86), a positive predictive value of 32.8% (21.6% to 45.7%) and a negative predictive value of 96.6% (91.5% to 99.1%) and agreement of 74.2%.

Therefore, we propose to set cut-offs at ≥60 for VAS anxiety and ≤60 for VAS mood. 76 (41.8%) workers had a VAS anxiety ≥60, and 67 (36.8%) workers had a VAS mood ≤ 60 (Table 3, Fig 2).

Using the VAS anxiety, data ranged from minimal to maximal values, with a reasonable standard deviation, as suggested by the variation coefficient around 0.57 for VAS anxiety. Using the VAS mood to 0.30, data ranged from 4 to 100, with a reasonable standard deviation (coefficient of variation of 0.37). No real ceiling of floor effects was observed; with 6% of participants for VAS anxiety and 7.7% for VAS mood having the lowest or highest possible score. The correlation between VAS anxiety and VAS mood was large (Spearman rho 0.51,

<table><caption>**Table 2.** Summary statistics for the VAS anxiety, VAS depression, subscale HADS-anxiety, subscale HAD-depression in the study population, segmented according to the chosen cutoff values.</caption><thead><tr><th>Variables</th><th>Sample size<br>n = 182 (%)</th><th>Mean ± Standard Deviation</th></tr></thead><tbody><tr><td><strong>Visual analog scale (VAS)</strong></td><td></td><td></td></tr><tr><td>VAS Anxiety</td><td>182</td><td>50.4 ± 28.7</td></tr><tr><td>&nbsp;&nbsp;&nbsp;&nbsp;&lt;60</td><td>106 (58.2%)</td><td></td></tr><tr><td>&nbsp;&nbsp;&nbsp;&nbsp;≥60</td><td>76 (41.8%)</td><td></td></tr><tr><td>VAS Mood</td><td>182</td><td>65.4 ± 24</td></tr><tr><td>&nbsp;&nbsp;&nbsp;&nbsp;&le; 60 (at risk)</td><td>67 (36.8%)</td><td></td></tr><tr><td>&nbsp;&nbsp;&nbsp;&nbsp;&gt; 60 (not at risk)</td><td>115 (63.2%)</td><td></td></tr><tr><td><strong>Hospital Anxiety and Depression Scale (HADS)</strong></td><td></td><td></td></tr><tr><td><strong>HADS Anxiety (HADS-A)</strong></td><td></td><td></td></tr><tr><td>HADS-A</td><td>182</td><td>6.9 ± 4.0</td></tr><tr><td>&nbsp;&nbsp;&nbsp;&nbsp;&le; 7</td><td>108 (59.3%)</td><td></td></tr><tr><td>&nbsp;&nbsp;&nbsp;&nbsp;≥ 8</td><td>74 (40.7%)</td><td></td></tr><tr><td><strong>HADS Depression (HADS-D)</strong></td><td></td><td></td></tr><tr><td>HADS-D</td><td>182</td><td>3.8 ± 3.0</td></tr><tr><td>&nbsp;&nbsp;&nbsp;&nbsp;&le; 7</td><td>157 (86.3%)</td><td></td></tr><tr><td>&nbsp;&nbsp;&nbsp;&nbsp;≥ 8</td><td>25 (13.7%)</td><td></td></tr></tbody></table>

https://doi.org/10.1371/journal.pone.0316159.t002
<table><caption>Table 3. Contingency tables of VAS mood and anxiety vs their HADS subscales counterparts, after dichotomization of the VAS around the best cut-off values.</caption><thead><tr><th colspan="4">Mood</th></tr><tr><th></th><th></th><th colspan="2">VAS Anxiety—Dichotomized</th><th rowspan="2">Total</th></tr><tr><th></th><th></th><th>Normal &lt; 60</th><th>At-risk ≥ 60</th></tr></thead><tbody><tr><th rowspan="2">Anxiety subscale of HADS—Dichotomized</th><th>Normal ≤ 7</th><td>88 (48.3%)</td><td>20 (11%)</td><td>108 (59.3%)</td></tr><tr><th>At-risk ≥ 8</th><td>18 (9.9%)</td><td>56 (30.8%)</td><td>74 (40.7%)</td></tr><tr><th>Total</th><td></td><td>106 (58.2%)</td><td>76 (41.8%)</td><td>182</td></tr></tbody><thead><tr><th colspan="4">Anxiety</th></tr><tr><th></th><th></th><th colspan="2">VAS Mood—Dichotomized</th><th rowspan="2">Total</th></tr><tr><th></th><th></th><th>Normal &gt; 60</th><th>At-risk ≤ 60</th></tr></thead><tbody><tr><th rowspan="2">Depression subscale of HADS—Dichotomized</th><th>Normal ≤7</th><td>111 (60.9%)</td><td>46 (25.3%)</td><td>157 (86.3%)</td></tr><tr><th>At-risk ≥8</th><td>4 (2.2%)</td><td>21 (11.5%)</td><td>25 (13.7%)</td></tr><tr><th>Total</th><td></td><td>115 (63.2%)</td><td>67 (36.8%)</td><td>182</td></tr></tbody></table>

https://doi.org/10.1371/journal.pone.0316159.t003

p < 0.001). VAS anxiety and mood correlated with their respective HADS sub-scores (0.70 and 0.65, respectively, p < 0.001).

## Validation of VAS anxiety and VAS mood—internal validity

**Correlation between VAS anxiety/VAS mood and HADS-A/HADS-D.** Rank correlation coefficient found high correlation (0.70, *p* < 0.001) between anxiety score from the HADS-A and the VAS anxiety, and moderate correlation (0.65, *p* < 0.001) between depression score from the HADS-D and the VAS mood.

**Test-retest reproducibility.** The analysis was performed on a subsample of 123 participants with available data for both test and retest. Lin concordance coefficient for VAS anxiety was 0.79 (95CI: 0.73 to 0.86) with difference on the retest of -1.4 ± 18.9 (95CI: -38.4 to 35.6). Lin concordance coefficient for VAS mood was 0.72 (95CI: 0.64, 0.81) with difference on the retest of -0.22 ± 18.8 (95CI: -37.1 to 36.7). The Bland and Altman plot is shown in Fig 3.

## Validation of VAS anxiety and VAS mood—external validity

**VAS anxiety.** According to the cut-off for VAS anxiety, age, BMI and seniority in the company were not correlated with anxiety symptoms but women were more affected by anxiety than men (*p* = 0.008). Moreover, a high degree of anxiety was correlated with a lower VAS quality of sleep (*p* < 0.001) but not with duration of sleep (*p* = 0.46). Also, a high degree of anxiety was associated with higher stress at work (*p* < 0.001) and at home (*p* < 0.001) but not with a longer time spent at work (*p* = 0.7). Conversely, a lower VAS anxiety was linked to higher VAS well-being (*p* < 0.001). Relationships were similar using the HADS-A (Table 4).

**VAS mood.** According to the cut-off for VAS mood, age, sex, BMI and seniority in the company were not correlated with anxiety symptoms. A sad mood was associated with a lower VAS quality of sleep (*p* < 0.001) but not with duration of sleep (*p* = 0.2). Also, sad mood was associated with higher stress at work (*p* < 0.001) and at home (*p* = 0.04) but not with longer time spent at work (*p* = 0.18). Conversely, good mood was linked to better VAS well-being (*p* < 0.001). Relationships were similar using the HADS-D (Table 4).

**Sensitivity analysis.** A sensitivity analysis was performed on the group of 144 patients for which complete data were available regarding the primary outcome and defining variables (sex, age, BMI, well-being VAS, Sleep quality VAS, Sleep duration, Stress at work VAS, stress at home VAS, seniority in company, weekly workload), presented in S1 Table. The distributions of HADS-A and HADS-D in this subset was similar to those observed in the entire
<figure><img src="image_5.png" /><figcaption>Fig 2. VAS anxiety and mood.</figcaption></figure>

https://doi.org/10.1371/journal.pone.0316159.g002

cohort. The mean score of HADS-A was 6.9 ± 3.9. 83 (57.6%) participants had no anxiety symptoms (score ≤7) and 61 (42.4%) had anxiety symptoms (score ≥8) The mean score of HADS-D was 3.8 ± 2.96. 126 (87.5%) participants had no depressive symptoms (score ≤7) and 18 (12.5%) had depressive symptoms (score ≥8). Regarding VAS anxiety, the same optimal cutoff of 63.5 was drawn from this sample and yielded similar classification performance. The analysis led of VAS mood led to the same optimal cutoff of 42 and yielded a similar classification performance.

The analysis of test-retest reproducibility was performed on a subsample of 91 participants with available data for both test and retest and yielded similar results compared to those observed in the entire cohort. Lin concordance coefficient for VAS anxiety was 0.79 (95CI: 0.71 to 0.87) with difference on the retest of -1.37 ± 19.5 (95CI: -39.6 to 36.8). Lin concordance coefficient for VAS mood was 0.75 (95CI: 0.65, 0.84) with difference on the retest of -0.42 ± 18.7 (95CI: -36.9 to 36.1). The Bland-Altman analysis is shown in S1 Fig.
A.

<figure><img src="image_6.png" /></figure>

B.

<figure><img src="image_7.png" /><figcaption>Fig 3. Agreement between the test and retest of both visual analog scales according to Bland-Altman analysis. 2.A. Bland-Altman plot for VAS Anxiety, 2.B. Bland-Altman plot for VAS Mood.</figcaption></figure>

https://doi.org/10.1371/journal.pone.0316159.g003

Evidence of external validity were similar in this subset compared to the entire cohort and concordant between the VAS and the HADS assessment for both anxiety and depression. The only discrepancy was observed for VAS stress at work, that was not significantly different between the groups without vs. with suspected or proven depressive disorder (HADS-D $\ge$ 8) (56.3 $\pm$ 22.7 vs. 61.8 $\pm$ 25.1, respectively, p = 0.33, Cohen's d = 0.55), while the groups at-risk of
<table><caption>Table 4. Association between anxiety and depression constructs (VAS or HADS subscales) and participants characteristics (demographics, well-being, sleep, stress and work-related characteristics). The agreement between VAS and HADS subscale is indicated for each characteristic. Abbreviations: Mean ± SD: Mean ± Standard deviation. Statistically significant results are displayed in bold.</caption><thead><tr><th rowspan="3">Variables</th><th colspan="4">Anxiety</th><th colspan="4">Depression</th></tr><tr><th colspan="2">Visual analog scale</th><th colspan="2">HAD-A of HADS</th><th colspan="2">Visual analog scale</th><th colspan="2">HAD-D of HADS</th></tr><tr><th>&lt; 60</th><th>&ge; 60</th><th>&le; 7</th><th>&ge; 8</th><th>&ge; 60</th><th>&lt; 60</th><th>&le; 7</th><th>&ge; 8</th></tr></thead><tbody><tr><td><strong>Age</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Mean ± SD</td><td>42.7 ± 11.6</td><td>39.7 ± 11.5</td><td>42.3 ± 11.9</td><td>40.4 ± 11.1</td><td>42.1 ± 11.5</td><td>38.6 ± 11.8</td><td>41.5 ± 11.8</td><td>41.1 ± 10.8</td></tr><tr><td>p-value</td><td colspan="2">0.12</td><td colspan="2">0.34</td><td colspan="2">0.15</td><td colspan="2">0.94</td></tr><tr><td>Agreement</td><td colspan="4">Yes</td><td colspan="4">Yes</td></tr><tr><td><strong>Sex</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Women (n) / Men (n)</td><td>42 / 49</td><td>44 / 21</td><td>42 / 49</td><td>44 / 21</td><td>68 / 60</td><td>18 / 10</td><td>73 / 63</td><td>13 / 7</td></tr><tr><td>p-value</td><td colspan="2">0.008</td><td colspan="2">0.008</td><td colspan="2">0.3</td><td colspan="2">0.34</td></tr><tr><td>Agreement</td><td colspan="4">Yes</td><td colspan="4">Yes</td></tr><tr><td><strong>Body Mass Index</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Mean ± SD</td><td>24.6 ± 4.8</td><td>23.5 ± 3.9</td><td>24.5 ± 4.3</td><td>23.8 ± 4.6</td><td>24.1 ± 4.4</td><td>24.7 ± 4.9</td><td>23.9 ± 4.2</td><td>25.7 ± 6</td></tr><tr><td>p-value</td><td colspan="2">0.097</td><td colspan="2">0.09</td><td colspan="2">0.76</td><td colspan="2">0.27</td></tr><tr><td>Agreement</td><td colspan="4">Yes</td><td colspan="4">Yes</td></tr><tr><td><strong>VAS Well-being</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Mean ± SD</td><td>67.9 ± 20.1</td><td>53.2 ± 20.5</td><td>68.9 ± 19.6</td><td>51.6 ± 20</td><td>67.4 ± 17.9</td><td>34.8 ± 16</td><td>65.4 ± 19</td><td>38.5 ± 22</td></tr><tr><td>p-value</td><td colspan="2">&lt;0.001</td><td colspan="2">&lt;0.001</td><td colspan="2">&lt;0.001</td><td colspan="2">&lt;0.001</td></tr><tr><td>Agreement</td><td colspan="4">Yes</td><td colspan="4">Yes</td></tr><tr><td><strong>VAS Quality of sleep</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Mean ± SD</td><td>62.7 ± 27</td><td>45.5 ± 24.9</td><td>63 ± 26.8</td><td>44.9 ± 24.9</td><td>60.6 ± 26.5</td><td>31.1 ± 16.7</td><td>58.5 ± 26.8</td><td>35.8 ± 23.8</td></tr><tr><td>p-value</td><td colspan="2">&lt;0.001</td><td colspan="2">&lt;0.001</td><td colspan="2">&lt;0.001</td><td colspan="2">&lt;0.001</td></tr><tr><td>Agreement</td><td colspan="4">Yes</td><td colspan="4">Yes</td></tr><tr><td><strong>Duration of sleep</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Mean ± SD</td><td>427.8 ± 52.3</td><td>422.4 ± 62.1</td><td>425.3 ± 55.5</td><td>423.8 ± 58.5</td><td>428.1 ± 55.9</td><td>413.2 ± 59.4</td><td>427.74 ± 56.5</td><td>411.2 ± 56.3</td></tr><tr><td>p-value</td><td colspan="2">0.46</td><td colspan="2">0.83</td><td colspan="2">0.2</td><td colspan="2">0.14</td></tr><tr><td>Agreement</td><td colspan="4">Yes</td><td colspan="4">Yes</td></tr><tr><td><strong>VAS Stress at home</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Mean ± SD</td><td>28.9 ± 24.4</td><td>46.4 ± 22.2</td><td>29.7 ± 24.8</td><td>45.7 ± 22.1</td><td>34.6 ± 25</td><td>44.3 ± 23.4</td><td>34 ± 24.3</td><td>50.9 ± 24.5</td></tr><tr><td>p-value</td><td colspan="2">&lt;0.001</td><td colspan="2">&lt;0.001</td><td colspan="2">0.04</td><td colspan="2">0.002</td></tr><tr><td>Agreement</td><td colspan="4">Yes</td><td colspan="4">Yes</td></tr><tr><td><strong>VAS Stress at work</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Mean ± SD</td><td>49.5 ± 23.6</td><td>67.8 ± 20.5</td><td>49.2 ± 23.1</td><td>68.6 ± 20.6</td><td>54.6 ± 22.6</td><td>69.6 ± 27.1</td><td>55.6 ± 23.7</td><td>67.5 ± 24.1</td></tr><tr><td>p-value</td><td colspan="2">&lt;0.001</td><td colspan="2">&lt;0.001</td><td colspan="2">&lt;0.001</td><td colspan="2">0.01</td></tr><tr><td>Agreement</td><td colspan="4">Yes</td><td colspan="4">Yes</td></tr><tr><td><strong>Seniority in company</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Mean ± SD</td><td>11.8 ± 11.1</td><td>9.4 ± 9.5</td><td>11.5 ± 10.6</td><td>9.7 ± 10.4</td><td>11.4 ± 10.9</td><td>7.8 ± 8</td><td>11.1 ± 10.8</td><td>8.3 ± 8.2</td></tr><tr><td>p-value</td><td colspan="2">0.29</td><td colspan="2">0.22</td><td colspan="2">0.07</td><td colspan="2">0.4</td></tr><tr><td>Agreement</td><td colspan="4">Yes</td><td colspan="4">Yes</td></tr><tr><td><strong>Number of hours of work per week</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Mean ± SD</td><td>42 ± 11</td><td>40.8 ± 13.5</td><td>41.3 ± 12.4</td><td>41.8 ± 11.8</td><td>42.3 ± 11.8</td><td>37.9 ± 13</td><td>42.2 ± 11.9</td><td>37.2 ± 12.9</td></tr><tr><td>p-value</td><td colspan="2">0.7</td><td colspan="2">0.63</td><td colspan="2">0.18</td><td colspan="2">0.09</td></tr><tr><td>Agreement</td><td colspan="4">Yes</td><td colspan="4">Yes</td></tr></tbody></table>

https://doi.org/10.1371/journal.pone.0316159.t004
depressive disorder according to the VAS Mood exhibited a significantly higher weekly work-load compared to the group not at risk (62.5 ± 24.0 vs. 53.5 ± 21.8, respectively, p = 0.01, Cohen's d = 0.33). Those results are presented in S2 Table.

## Discussion

This study brings validity evidence of visual analog scales of anxiety and mood for clinical use in occupational health, i.e., the assessment of its acceptability, reproducibility, internal and external validity.

## Prevalence and health outcomes

Anxiety and mood disorders are common illnesses in active or non-active populations [1, 42–50]. In our study, 41.8% and 36.8% of workers reported anxiety (VAS anxiety $\ge$ 60/100) and depressive (VAS mood $\le$ 60/100) symptoms. The literature confirms that rates are variable, ranging from 12 to 73% depending on the profession, the country of study, and the questionnaire used [46–48, 50–53]. Those disorders increase the risk of numerous diseases [54], such as metabolic and cardiovascular diseases [55, 56]. Anxiety and depression are also major risk factors for suicide [57, 58]. Psychological disorders are not correctly detected or diagnosed [59–61], and this can significantly impact patients’ professional life such as discrimination in hiring, stigmatization in the workplace, difficulty in maintaining employment, and absenteeism [2–4, 49, 62, 63]. More than half of employers would never accept someone diagnosed with depression for a managerial position, and more than one third of workers would be anxious about such colleagues [64, 65]. Conversely, occupational factors can directly promote anxious-depressive symptomatology such as stress [47, 53, 66–68], damaging social relationships [67] and management [63], imbalanced job design [69], occupational uncertainty, or lack of value and respect in the workplace [66]. The occupational physicians hold an important role in promoting mental health through individual and collective prevention [70], early identification of mental pathologies, and maintenance during return to work [71, 72].

## Acceptability

Ceiling or floor effects happen when 15% of participants have the highest or lowest possible score, indicating a lack of discrepancy for extreme scores [73]. No real ceiling of floor effects were observed; with less than 15% of participants having the lowest or highest possible score, as recommended in the literature [73]. The absence of major floor and ceiling effect demonstrate the absence of over-representation of extreme levels of mood and anxiety, and therefore the pertinence of VAS to discriminate symptoms of anxiety or depression [39, 74]. We retrieve one study mentioning ceiling effects in the group of healthy subjects regarding the use of electronic models of VAS pain, anxiety, fatigue and quality of life [75]. However, most of the available studies did not systematically stipulate floor or ceiling effects [23–26, 76–80]. Only one study evaluated the floor and ceiling effects for HADS and found a floor effect for each item but no ceiling effect. However, this study did not concern active subjects but the general population aged 65 to 80 [81]. Despite the lack of data on the time required to complete the VAS tools and the HADS, we assume that there must be a benefit in completion times, as it takes a few seconds to answer a VAS, while HADS takes up to 5–10 minutes [82–84].

## Internal validity

In our study, the agreement and reproducibility of VAS anxiety and mood measured by Lin concordance coefficients were satisfying and were over 0.70 for the two VAS. The best
sensitivity/specificity compromises were for a threshold of 60/100 for each VAS anxiety and VAS mood: at-risk workers being those for VAS anxiety ≥ 60/100 and those for VAS mood ≤ 60/100. The reliability, the validity and the sensibility of VAS anxiety and VAS mood were also confirmed in the literature versus other questionnaires than HADS, and within specific populations (preanesthesia, dental care for anxiety [76–78, 85], after a stroke or in geriatrics for depression and mood [86–90]. Moreover, VAS anxiety was moderately correlated with VAS mood (coefficient = 0.51; p <0.001), aligned with the literature reporting similar correlations between the HADS-A and the HADS-D (varying from 0.40 to 0.74, mean 0.56) [91–93]. The visual graphical analysis of Bland and Altman for both VAS showed no systematic error but a relatively large random error with a heterogeneous general dispersion; the 95% limits were adequate for high and low values of mean VAS anxiety and mean VAS mood test and retest. We have found similar data in the literature for VAS pain, fatigue, quality of life and anxiety, in a population with multiple sclerosis [75]. The test/retest differences were acceptable [94, 95].

## External validity

This study demonstrated a very good external validity of VAS anxiety/VAS mood: the same relationships were shown between those VAS and other variables, and between HADS-A / HADS-D and those same variables. Women had more anxiety symptoms (both for VAS anxiety and HADS-A) without differences for depressive symptoms (both for VAS mood and HADS-D). The literature also frequently report higher levels of anxiety in women, with more heterogeneous data for mood [32, 48–51, 92]. Whatever the scales considered (VAS anxiety vs. HADS-A, or VAS mood vs. HADS-D), age did not influence depressive and anxiety symptoms. The influence of age on those symptoms is conflicting in the literature, depending on the population studied [28, 48, 53, 92]. Similarly, whatever the scales considered, anxiety and depressive symptoms were associated with a poor sleep quality and high levels of stress at work or at home. In line with the literature, anxiety and mood perturbations may be associated with a poor sleep [48] and work- or home-related stress [46, 96]. In our study, other factors (such as working hours and seniority in the company) were not associated with anxiety and mood. The literature did not have consensus data on the protective [50] or vulnerability [48] effects associated with seniority in the company. However, a positive correlation was found between working hours, anxiety and depression [28]. However, despite no significance, relationships were similar between VAS and HADS, and those factors.

## Limitations

Compared to other French studies using a questionnaire, the response rate may seem low [29, 97–101]. However, the number of included workers made it possible to carry out the statistical analyses with the number of required subjects, determined *a priori* [38, 39]. Also, we had some missing data despite the volume of the survey [102]. Moreover, the proportion of participants who answered both test and retest was higher in our study than in other studies [83], demonstrating the interest of workers for questionnaires on quality of life at work. A limitation arose from participants’ characteristics with more women than men, and a higher representation of executives (57%) compared to mid-level professionals (23%). This class imbalance in terms of socio-economic levels calls for larger studies designed to assess the validity evidence of these VAS in other socio-economic classes. Considering internal validity, our study did not assess sensitivity to changes in anxiety and mood. However, test-retest reliability was acceptable. Moreover, literature previously showed a good sensitivity to changes using VAS anxiety before and after stress [76, 95]. The relatively large random error retrieved at the Bland and Altman
test may traduce the non-negligible probability of a stressful event having occurred between
the test and retest. Also, we suggest that the same evaluation may be repeated for people close
to the at-risk threshold, which is relatively easy considering the instantaneous time required to
answer. Our study may suffer from limitations in terms of external validity because we used
only the HADS whereas other validated assessment tools exist to assess anxiety and depressive
symptoms. However, HADS is classically considered as the gold standard with good external
structure and satisfying discriminant validity [103]. Moreover, the aim of our study was not to
compare the numerous scales but rather to validate the VAS anxiety and mood vs. the refer-
ence test, in a general population. We also used many sociodemographic, occupational, and
clinical data for external validity. Despite a sufficient sample size for the validation of VAS anx-
iety and mood, insufficient data precluded further analyses on at-risk occupations for anxiety
and depressive symptoms. However, all professions can be affected by the common mental dis-
orders of anxiety and depression due to major work changes. Indeed, some studies showed
that intellectual professions or higher degrees have high levels of anxiety and depression [48,
50], while others reported a high risk for lowest occupational categories (workers and technical
classes) [92, 104]. Identifying at-risk workers is necessary for effective preventive strategies.
Assessment of anxiety by a VAS was only described in pre-anesthesia patients [77, 78] or den-
tal care, limiting its generalizability and its validity, as it was not compared with the HADS.
Finally, considering semantics, VAS mood presented in the literature had a specific design.
They used a combination of words and schematic faces indicating different mood states.
Those VAS were designed for certain profiles of patients with neurologic disorders or cogni-
tive impairment [79, 80, 86, 88]. Therefore, we were the first to introduce a unique VAS anxi-
ety and mood model in general population.

Conclusion

Our results show that VAS anxiety and VAS mood are reliable tools for identifying at-risk
workers for anxiety and depression, allowing for a quicker screening and primary prevention
in the workplace. We have determined a cut-off value of 60 for each of the VAS: ≥ 60/100 for
workers at risk of anxiety (Sensitivity 72%, Specificity 86%, ROC AUC 0.79) and ≤ 60/100 for
depression (Sensitivity 84%, Specificity 73%, ROC AUC 0.78). The Anxiety and Mood VAS
yielded satisfying test-retest reliability and their association with participants socio-demo-
graphic and work-related characteristics are similar with those observed for the HADS.

**Supporting information**

S1 Table. Characteristics of participants in the sensitivity analysis, defined by the sub-group in which the primary outcome and the variables used for examining external validity evidence were complete. Those variables are sex, age, BMI, well-being VAS, Sleep quality VAS, Sleep duration, Stress at work VAS, stress at home VAS, seniority in company, weekly workload).
(DOCX)

S2 Table. Agreement between visual analog scales vs. HADS on differences in participants' characteristics in the sensitivity analysis cohort. Abbreviations: Mean ± SD: Mean ± Standard deviation. Statistically significant results are displayed in bold.
(DOCX)

S1 Fig. Agreement between the test and retest of both visual analog scales according to Bland-Altman analysis, in the sensitivity analysis cohort. A. Bland-Altman plot for VAS
Anxiety, B. Bland-Altman plot for VAS Mood.
(DOCX)

**Acknowledgments**

We express our sincere gratitude to Marie-Anne Cousseau who established the public-private partnership between WittyFit and the University Hospital of Clermont-Ferrand, France.

**Author Contributions**

**Conceptualization:** Frédéric Dutheil, Thomas Cornet, Bruno Pereira.

**Data curation:** Frédéric Dutheil, Clara Palgen, Thomas Cornet.

**Formal analysis:** Frédéric Dutheil, Clara Palgen, Bruno Pereira, Louis Delamarre.

**Funding acquisition:** Frédéric Dutheil.

**Investigation:** Frédéric Dutheil, Clara Palgen, Georges Brousse, Martial Mermillod, Ines Lakbar, Guillaume Vallet, Julien S. Baker, Jeannot Schmidt, Barbara Charbotel.

**Methodology:** Frédéric Dutheil, Clara Palgen, Bruno Pereira.

**Software:** Bruno Pereira, Louis Delamarre.

**Supervision:** Frédéric Dutheil.

**Validation:** Frédéric Dutheil, Clara Palgen, Georges Brousse, Martial Mermillod, Ines Lakbar, Guillaume Vallet, Julien S. Baker, Jeannot Schmidt, Barbara Charbotel, Bruno Pereira, Louis Delamarre.

**Visualization:** Clara Palgen, Bruno Pereira.

**Writing – original draft:** Frédéric Dutheil, Clara Palgen.

**Writing – review & editing:** Frédéric Dutheil, Georges Brousse, Thomas Cornet, Martial Mermillod, Ines Lakbar, Guillaume Vallet, Julien S. Baker, Jeannot Schmidt, Barbara Charbotel, Bruno Pereira, Louis Delamarre.

**References**

1. Steel Z, Marnane C, Iranpour C. The global prevalence of common mental disorders: a systematic review and meta-analysis 1980–2013. Int J Epidemiol. 2014; 43: 476–493. https://doi.org/10.1093/ije/dyu038 PMID: 24648481

2. Arends I, Klink JJ, Rhenen W, Boer MR, Bultmann U. Predictors of recurrent sickness absence among workers having returned to work after sickness absence due to common mental disorders. Scand J Work Env Health. 2014; 40: 195–202. https://doi.org/10.5271/sjweh.3384 PMID: 24048675

3. Koopmans PC, Bultmann U, Roelen CA, Hoedeman R, Klink JJ, Groothoff JW. Recurrence of sickness absence due to common mental disorders. Int Arch Occup Env Health. 2011; 84: 193–201. https://doi.org/10.1007/s00420-010-0540-4 PMID: 20449605

4. Koopmans PC, Roelen CA, Groothoff JW. Risk of future sickness absence in frequent and long-term absentees. Occup Med Lond. 2008; 58: 268–274. https://doi.org/10.1093/occmed/kqn040 PMID: 18390880

5. Kessler RC, Petukhova M, Sampson NA, Zaslavsky AM, Wittchen HU. Twelve-month and lifetime prevalence and lifetime morbid risk of anxiety and mood disorders in the United States. Int J Methods Psychiatr Res. 2012; 21: 169–184. https://doi.org/10.1002/pr.1359 PMID: 22865617

6. Kessler RC, Chiu WT, Demler O, Merikangas KR, Walters EE. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005; 62: 617–627. https://doi.org/10.1001/archpsyc.62.6.617 PMID: 15939839
7. Hendriks SM, Spijker J, Licht CMM, Hardeveld F, De Graaf R, Batelaan NM, et al. Long-term work disability and absenteeism in anxiety and depressive disorders. J Affect Disord. 2015; 178: 121–130.
https://doi.org/10.1016/j.jad.2015.03.004 PMID: 25805404

8. Scaife R. Identifying the impact of psychological ill-health on safety performance. 2006.

9. Amati C, Scaife R. The Impact of Psychological Ill-Health on Safety. In: Bust PD, editor. 1st ed. Taylor & Francis; 2020. pp. 551–554. https://doi.org/10.1201/9781003072072-130

10. McCarthy JM, Trougakos JP, Cheng BH. Are anxious workers less productive workers? It depends on the quality of social exchange. J Appl Psychol. 2016; 101: 279–291. https://doi.org/10.1037/apl0000044 PMID: 26375962

11. Bubonya M, Cobb-Clark DA, Wooden M. Mental health and productivity at work: does what you do matter? Melbourne: Melbourne Institute of Applied Economic and Social Research; 2016.

12. De Oliveira C, Saka M, Bone L, Jacobs R. The Role of Mental Health on Workplace Productivity: A Critical Review of the Literature. Appl Health Econ Health Policy. 2023; 21: 167–193. https://doi.org/10.1007/s40258-022-00761-w PMID: 36376610

13. Goetzel RZ, Long SR, Ozminkowski RJ, Hawkins K, Wang S, Lynch W. Health, Absence, Disability, and Presenteeism Cost Estimates of Certain Physical and Mental Health Conditions Affecting U.S. Employers: J Occup Environ Med. 2004; 46: 398–412. https://doi.org/10.1097/01.jom.0000121151.40413.bd PMID: 15076658

14. Stewart WF. Cost of Lost Productive Work Time Among US Workers With Depression. JAMA. 2003; 289: 3135. https://doi.org/10.1001/jama.289.23.3135 PMID: 12813119

15. Wagner S, Koehn C, White M, Harder H, Schultz I, Williams-Whitt K, et al. Mental Health Interventions in the Workplace and Work Outcomes: A Best-Evidence Synthesis of Systematic Reviews. Int J Occup Environ Med. 2016; 7: 1–14. https://doi.org/10.15171/ijoem.2016.607 PMID: 26772593

16. Plaisier I, De Graaf R, De Bruijn J, Smit J, Van Dyck R, Beekman A, et al. Depressive and anxiety disorders on-the-job: The importance of job characteristics for good work functioning in persons with depressive and anxiety disorders. Psychiatry Res. 2012; 200: 382–388. https://doi.org/10.1016/j.psychres.2012.07.016 PMID: 22862911

17. Kelloway EK, Dimoff JK, Gilbert S. Mental Health in the Workplace. Annu Rev Organ Psychol Organ Behav. 2023; 10: 363–87. https://doi.org/10.1146/annurev-orgpsych-120920050527

18. Kessler RC, Frank RG. The impact of psychiatric disorders on work loss days. Psychol Med. 1997; 27: 861–873. https://doi.org/10.1017/s0033291797004807 PMID: 9234464

19. Crocq MA. A history of anxiety: from Hippocrates to DSM. Dialogues Clin Neurosci. 2015; 17: 319–325. https://doi.org/10.31887/DCNS.2015.17.3/macrocq PMID: 26487812

20. Kupfer DJ. Anxiety and DSM-5. Dialogues Clin Neurosci. 2015; 17: 245–246. https://doi.org/10.31887/DCNS.2015.17.3/dkupfer PMID: 26487805

21. Kjaergaard M, Arfwedson Wang CE, Waterloo K, Jorde R. A study of the psychometric properties of the Beck Depression Inventory-II, the Montgomery and Asberg Depression Rating Scale, and the Hospital Anxiety and Depression Scale in a sample from a healthy population. Scand J Psychol. 2014; 55: 83–89.

22. Dutheil F, Pereira B, Moustafa F, Naughton G, Lesage F-X, Lambert C. At-risk and intervention thresholds of occupational stress using a visual analogue scale. Kou YR, editor. PLOS ONE. 2017; 12: e0178948. https://doi.org/10.1371/journal.pone.0178948 PMID: 28586383

23. Boonstra AM, Schiphorst Preuper HR, Reneman MF, Posthumus JB, Stewart RE. Reliability and validity of the visual analogue scale for disability in patients with chronic musculoskeletal pain. Int J Rehabil Res. 2008; 31: 165–169. https://doi.org/10.1097/MRR.0b013e3282fc0f93 PMID: 18467932

24. Lesage FX, Berjot S. Validity of occupational stress assessment using a visual analogue scale. Occup Med Lond. 2011; 61: 434–436. https://doi.org/10.1093/occmed/kqr037 PMID: 21505089

25. Lesage FX, Berjot S, Deschamps F. Clinical stress assessment using a visual analogue scale. Occup Med Lond. 2012; 62: 600–605. https://doi.org/10.1093/occmed/kqs140 PMID: 22965867

26. Ratinaud MC, Chamoux A, Glace B, Coudeyre E. Job satisfaction evaluation in low back pain: a literature review and tools appraisal. Ann Phys Rehabil Med. 2013; 56: 465–481. https://doi.org/10.1016/j.rehab.2013.06.006 PMID: 23928031

27. Gramstad TO, Gjestad R, Haver B. Personality traits predict job stress, depression and anxiety among junior physicians. BMC Med Educ. 2013;13.

28. Afonso P, Fonseca M, Pires JF. Impact of working hours on sleep and mental health. Occup Med Lond. 2017; 67: 377–382. https://doi.org/10.1093/occmed/kqx054 PMID: 28575463
29. Dutheil F, Duclos M, Naughton G. WittyFit-Live Your Work Differently: Study Protocol for a Workplace-Delivered Health Promotion. JMIR Res Protoc. 2017; 6. https://doi.org/10.2196/resprot.6267 PMID: 28408363

30. Snaith RP, Baugh SJ, Clayden AD, Husain A, Sipple MA. The Clinical Anxiety Scale: an instrument derived from the Hamilton Anxiety Scale. Br J Psychiatry. 1982; 141: 518–523. https://doi.org/10.1192/bjp.141.5.518 PMID: 7150890

31. Snaith RP, Zigmond AS. The hospital anxiety and depression scale. Br Med J Clin Res Ed. 1986;292.

32. Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand. 1983; 67: 361–370.

33. Lepine JP, Godchau M, Brun P. Anxiety and depression in inpatients. Lancet. 1985; 2: 1425–1426. https://doi.org/10.1016/s0140-6736(85)92589-9 PMID: 2867417

34. Lepine JP, Godchau M, Brun P, Lemperiere T. Evaluation of anxiety and depression among patients hospitalized on an internal medicine service. Ann Med Psychol Paris. 1985; 143: 175–189.

35. Razavi D, Delvaux N, Farvacques C, Robaye E. Validation de la version française du HADS dans une population de patients cancéreux hospitalisés. [Validation of the French version of the Hospital Anxiety and Depression Scale (HADS) in a population of hospitalized cancer patients. Rev Psychol Appliquée. 1989; 39: 295–307.

36. Olsson I, Mykletun A, Dahl AA. The Hospital Anxiety and Depression Rating Scale: a cross-sectional study of psychometrics and case finding abilities in general practice. BMC Psychiatry. 2005; 5.

37. Dutheil F, Trousselard M, Perrier C. Urinary interleukin-8 is a biomarker of stress in emergency physicians, especially with advancing age—the JOBSTRESS* randomized trial. PLoS One. 2013; 8. https://doi.org/10.1371/journal.pone.0071658 PMID: 23977105

38. Mokkink LB, Terwee CB, Patrick DL. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res. 2010; 19: 539–549. https://doi.org/10.1007/s11136-010-9606-8 PMID: 20169472

39. Terwee CB, Bot SD, Boer MR. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007; 60: 34–42. https://doi.org/10.1016/j.jclinepi.2006.03.012 PMID: 17161752

40. Obuchowski NA, Bullen JA. Receiver operating characteristic (ROC) curves: review of methods with applications in diagnostic medicine. Phys Med Biol. 2018;63. https://doi.org/10.1088/1361-6560/aab4b1 PMID: 29512515

41. Carter JV, Pan J, Rai SN, Galandiuk S. ROC-ing along: Evaluation and interpretation of receiver operating characteristic curves. Surgery. 2016; 159: 1638–1645. https://doi.org/10.1016/j.surg.2015.12.029 PMID: 26962006

42. Kessler RC, Berglund P, Demler O. The epidemiology of major depressive disorder: results from the National Comorbidity Survey Replication (NCS-R. JAMA. 2003; 289: 3095–3105. https://doi.org/10.1001/jama.289.23.3095 PMID: 12813115

43. Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walters EE. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005; 62: 593–602. https://doi.org/10.1001/archpsyc.62.6.593 PMID: 15939837

44. Smith K. Mental health: a world of depression. Nature. 2014;515.

45. Creedy DK, Sidebotham M, Gamble J, Pallant J, Fenwick J. Prevalence of burnout, depression, anxiety and stress in Australian midwives: a cross-sectional survey. BMC Pregnancy Childbirth. 2017;17.

46. Fawzy M, Hamed SA. Prevalence of psychological stress, depression and anxiety among medical students in Egypt. Psychiatry Res. 2017; 255: 186–194. https://doi.org/10.1016/j.psychres.2017.05.027 PMID: 28575777

47. Desouky D, Allam H. Occupational stress, anxiety and depression among Egyptian teachers. J Epidemiol Glob Health. 2017; 7: 191–198. https://doi.org/10.1016/j.jegh.2017.06.002 PMID: 28756829

48. Zhang ZH, Li RH, Li DF. Anxiety and depression status of coal miners and related influencing factors. Zhonghua Lao Dong Wei Sheng Zhi Ye Bing Za Zhi. 2018; 36: 860–863. https://doi.org/10.3760/cma.j.issn.1001-9391.2018.11.018 PMID: 30646656

49. Torske MO, Hilt B, Glasscock D, Lundqvist P, Krokstad S. Anxiety and Depression Symptoms Among Farmers: The HUNT Study, Norway. J Agromedicine. 2016; 21: 24–33. https://doi.org/10.1080/1059924X.2015.1106375 PMID: 26488439

50. Fu A, Liu B, Jiang Y, Zhao J, Zhang G, Liu J. A Mental Health Survey of Different Ethnic and Occupational Groups in Xinjiang, China. Int J Env Res Public Health. 2017;14. https://doi.org/10.3390/ijerph14010046 PMID: 28067780
51. Utzet M, Moncada S, Molinero E, Navarro A. Psychosocial exposures and mental health: distribution by gender and occupational class in a population-based study. J Public Health Oxf. 2016; 38. https://doi.org/10.1093/pubmed/fdv175 PMID: 28158824

52. Theis KA, Roblin DW, Helmick CG, Luo R. Prevalence and causes of work disability among working-age U.S. adults, 2011–2013, NHIS. Disabil Health J. 2018; 11: 108–115. https://doi.org/10.1016/j.dhjo.2017.04.010 PMID: 28476583

53. Tran TTT, Nguyen NB, Luong MA. Stress, anxiety and depression in clinical nurses in Vietnam: a cross-sectional survey and cluster analysis. Int J Ment Health Syst. 2019; 13. https://doi.org/10.1186/s13033-018-0257-4 PMID: 30622629

54. Lépine JP, Briley M. The increasing burden of depression. Neuropsychiatr Treat. 2011; 7: 3–7. https://doi.org/10.2147/NDT.S19617 PMID: 21750622

55. Belialov FI. Depression, anxiety, and stress in patients with coronary heart disease. Ter Arkh. 2017; 89: 104–109.

56. Chauvet-Gelinier JC, Bonin B. Stress, anxiety and depression in heart disease patients: A major challenge for cardiac rehabilitation. Ann Phys Rehabil Med. 2017; 60: 6–12. https://doi.org/10.1016/j.rehab.2016.09.002 PMID: 27771272

57. Bentley KH, Franklin JC, Ribeiro JD, Kleiman EM, Fox KR, Nock MK. Anxiety and its disorders as risk factors for suicidal thoughts and behaviors: A meta-analytic review. Clin Psychol Rev. 2016; 43: 30–46. https://doi.org/10.1016/j.cpr.2015.11.008 PMID: 26688478

58. Bolton JM, Gunnell D, Turecki G. Suicide risk assessment and intervention in people with mental illness. BMJ. 2015;351. https://doi.org/10.1136/bmj.h4978 PMID: 26552947

59. Wang PS, Aguilar-Gaxiola S, Alonso J. Use of mental health services for anxiety, mood, and substance disorders in 17 countries in the WHO world mental health surveys. Lancet. 2007; 370: 841–850. https://doi.org/10.1016/S0140-6736(07)61414-7 PMID: 17826169

60. Wang PS, Angermeyer M, Borges G. Delay and failure in treatment seeking after first onset of mental disorders in the World Health Organization's World Mental Health Survey Initiative. World Psychiatry. 2007; 6: 177–185. PMID: 18188443

61. Wang PS, Berglund P, Olfson M, Pincus HA, Wells KB, Kessler RC. Failure and delay in initial treatment contact after first onset of mental disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005; 62: 603–613. https://doi.org/10.1001/archpsyc.62.6.603 PMID: 15939838

62. Norder G, Hoedeman R, Bruin J, Rhenen W, Roelen CA. Time to recurrence of mental health-related absence from work. Occup Med Lond. 2015; 65: 574–577.

63. Junne F, Michaelis M, Rothermund E. The Role of Work-Related Factors in the Development of Psychological Distress and Associated Mental Disorders: Differential Views of Human Resource Managers. Occup Physicians Prim Care Physicians Psychother Ger Int J Env Res Public Health. 2018;15.

64. Scheid TL. Employment of individuals with mental disabilities: business response to the ADA's challenge. Behav Sci Law. 1999; 17: 73–91. https://doi.org/10.1002/(sici)1099-0798(199901/03)17:1<73::aid-bsl326>3.0.co;2-3 PMID: 10216927

65. Glozier N. Workplace effects of the stigmatization of depression. J Occup Env Med. 1998; 40: 793–800. https://doi.org/10.1097/00043764-199809000-00008 PMID: 9777563

66. Harvey SB, Modini M, Joyce S. Can work make you mentally ill? A systematic meta-review of work-related risk factors for common mental health problems. Occup Env Med. 2017; 74: 301–310. https://doi.org/10.1136/oemed-2016-104015 PMID: 28108676

67. Rusli BN, Edimansyah BA, Naing L. Working conditions, self-perceived stress, anxiety, depression and quality of life: a structural equation modelling approach. BMC Public Health. 2008; 8. https://doi.org/10.1186/1471-2458-8-48 PMID: 18254966

68. Song KW, Choi WS, Jee HJ. Correlation of occupational stress with depression, anxiety, and sleep in Korean dentists: cross-sectional study. BMC Psychiatry. 2017;17.

69. Magnavita N, Fileni A. Association of work-related stress with depression and anxiety in radiologists. Radiol Med. 2014; 119: 359–366. https://doi.org/10.1007/s11547-013-0355-y PMID: 24297590

70. Cleary M, Dean S, Webster S, Walter G, Escott P, Lopez V. Primary health care in the mental health workplace: insights from the Australian experience. Issues Ment Health Nurs. 2014; 35: 437–443. https://doi.org/10.3109/01612840.2013.855853 PMID: 24857527

71. Arends I, Klink JJ, Rhenen W, Boer MR, Bultmann U. Prevention of recurrent sickness absence in workers with common mental disorders: results of a cluster-randomised controlled trial. Occup Env Med. 2014; 71: 21–29. https://doi.org/10.1136/oemed-2013-101412 PMID: 24158311

72. Nigatu YT, Liu Y, Uppal M. Interventions for enhancing return to work in individuals with a common mental illness: systematic review and meta-analysis of randomized controlled trials. Psychol Med. 2016; 46: 3263–3274. https://doi.org/10.1017/S0033291716002269 PMID: 27609709
73. Ravoux H, Pereira B, Brousse G. Work Addiction Test Questionnaire to Assess Workaholism: Validation of French Version. JMIR Ment Health. 2018; 5. https://doi.org/10.2196/mental.8215 PMID: 29439945

74. Wamper KE, Sierevelt IN, Poolman RW, Bhandari M, Haverkamp D. The Harris hip score: Do ceiling effects limit its usefulness in orthopedics? Acta Orthop. 2010; 81: 703–707. https://doi.org/10.3109/17453674.2010.537808 PMID: 21110703

75. Kos D, Raeymaekers J, Remoortel A. Electronic visual analogue scales for pain, fatigue, anxiety and quality of life in people with multiple sclerosis using smartphone and tablet: a reliability and feasibility study. Clin Rehabil. 2017; 31: 1215–1225. https://doi.org/10.1177/0269215517692641 PMID: 28786335

76. Abend R, Dan O, Maoz K, Raz S, Bar-Haim Y. Reliability, validity and sensitivity of a computerized visual analog scale measuring state anxiety. J Behav Ther Exp Psychiatry. 2014; 45: 447–453. https://doi.org/10.1016/j.jbtep.2014.06.004 PMID: 24978117

77. Berghmans JM, Poley MJ, Ende J. A Visual Analog Scale to assess anxiety in children during anesthesia induction (VAS-I): Results supporting its validity in a sample of day care surgery patients. Paediatr Anaesth. 2017; 27: 955–961. https://doi.org/10.1111/pan.13206 PMID: 28707384

78. Facco E, Stellini E, Bacci C. Validation of visual analogue scale for anxiety (VAS-A) in preanesthesia evaluation. Minerva Anestesiol. 2013; 79: 1389–1395. PMID: 23860442

79. Barrows PD, Thomas SA. Assessment of mood in aphasia following stroke: validation of the Dynamic Visual Analogue Mood Scales (D-VAMS). Clin Rehabil. 2018; 32: 94–102. https://doi.org/10.1177/0269215517714590 PMID: 28653547

80. Haley KL, Womack JL, Harmon TG, Williams SW. Visual analog rating of mood by people with aphasia. Top Stroke Rehabil. 2015; 22: 239–245. https://doi.org/10.1179/1074935714Z.0000000009 PMID: 26258449

81. Djukanovic I, Carlsson J, Arestedt K. Is the Hospital Anxiety and Depression Scale (HADS) a valid measure in a general population 65–80 years old? A psychometric evaluation study. Health Qual Life Outcomes. 2017; 15.

82. Fernández-de-las-Peñas C, Rodríguez-Jiménez J, Palacios-Ceña M, de-la-Llave-Rincón AI, Fuensalida-Novo S, Florencio LL, et al. Psychometric Properties of the Hospital Anxiety and Depression Scale (HADS) in Previously Hospitalized COVID-19 Patients. Int J Environ Res Public Health. 2022; 19: 9273. https://doi.org/10.3390/ijerph19159273 PMID: 35954630

83. Reda AA. Reliability and validity of the Ethiopian version of the hospital anxiety and depression scale (HADS) in HIV infected patients. PLoS One. 2011; 6. https://doi.org/10.1371/journal.pone.0016049 PMID: 21283565

84. Guo C, Huang X. Hospital anxiety and depression scale exhibits good consistency but shorter assessment time than Zung self-rating anxiety/depression scale for evaluating anxiety/depression in non-small cell lung cancer. Medicine (Baltimore). 2021; 100: e24428. https://doi.org/10.1097/MD.0000000000024428 PMID: 33663054

85. Appukuttan D, Vinayagavel M, Tadepalli A. Utility and validity of a single-item visual analog scale for measuring dental anxiety in clinical practice. J Oral Sci. 2014; 56: 151–156. https://doi.org/10.2334/josnusd.56.151 PMID: 24930752

86. House ED, Arruda JE, Andrasik F, Grazzi L. The reliability and validity of the Visual Analog Mood Scales in non-English-speaking pain patients. Pain Pr. 2012; 12: 626–632. https://doi.org/10.1111/j.1533-2500.2012.00544.x PMID: 22443497

87. Arruda JE, Stern RA, Somerville JA. Measurement of mood states in stroke patients: validation of the visual analog mood scales. Arch Phys Med Rehabil. 1999; 80: 676–680. https://doi.org/10.1016/s0003-9993(99)90171-5 PMID: 10378494

88. Temple RO, Stern RA, Latham J, Ruffolo JS, Arruda JE, Tremont G. Assessment of mood state in dementia by use of the visual analog mood scales (VAMS). Am J Geriatr Psychiatry. 2004; 12: 527–530. https://doi.org/10.1176/appi.ajgp.12.5.527 PMID: 15353392

89. Kanamori H, Nagai K, Matsubara T. Comparison of the psychosocial quality of life in hemodialysis patients between the elderly and non-elderly using a visual analogue scale: the importance of appetite and depressive mood. Geriatr Gerontol Int. 2012; 12: 65–71. https://doi.org/10.1111/j.1447-0594.2011.00731.x PMID: 21843200

90. Killgore WD. The visual analogue mood scale: can a single-item scale accurately classify depressive mood state? Psychol Rep. 1999; 85: 1238–1243. https://doi.org/10.2466/pr0.1999.85.3f.1238 PMID: 10710979

91. Bjelland I, Dahl AA, Haug TT, Neckelmann D. The validity of the Hospital Anxiety and Depression Scale. An updated literature review. J Psychosom Res. 2002; 52: 69–77. https://doi.org/10.1016/s0022-3999(01)00296-3 PMID: 11832252
92. Bocerean C, Dupret E. A validation study of the Hospital Anxiety and Depression Scale (HADS) in a large sample of French employees. BMC Psychiatry. 2014; 14.

93. Schonberger M, Ponsford J. The factor structure of the Hospital Anxiety and Depression Scale in individuals with traumatic brain injury. Psychiatry Res. 2010; 179: 342–349. https://doi.org/10.1016/j.psychres.2009.07.003 PMID: 20483471

94. HC V, CB T, DL K, LM B. When to use agreement versus reliability measures. J Clin Epidemiol. 2006; 59: 1033–1039. https://doi.org/10.1016/j.jclinepi.2005.10.015 PMID: 16980142

95. Guyatt G, Walter S, Norman G. Measuring change over time: assessing the usefulness of evaluative instruments. J Chronic Dis. 1987; 40: 171–178. https://doi.org/10.1016/0021-9681(87)90069-5 PMID: 3818871

96. Fan LB, Blumenthal JA, Watkins LL, Sherwood A. Work and home stress: associations with anxiety and depression symptoms. Occup Med Lond. 2015; 65: 110–116. https://doi.org/10.1093/occmed/kqu181 PMID: 25589707

97. Tekath M, Dutheil F, Bellini R. Comparison of the ultra-low-dose Veo algorithm with the gold standard filtered back projection for detecting pulmonary asbestos-related conditions: a clinical observational study. BMJ Open. 2014;4. https://doi.org/10.1136/bmjopen-2014-004980 PMID: 24879827

98. Dutheil F, Delaire P, Boudet G. Cost/effectiveness comparison of the vaccine campaign and reduction of sick leave, after vaccination against influenza among the Clermont-Ferrand University Hospital staff. Med Mal Infect. 2008; 38: 567–573.

99. Dutheil F, Kelly C, Biat I. Relation between the level of knowledge and the rate of vaccination against the flu virus among the staff of the Clermont-Ferrand University hospital. Med Mal Infect. 2008; 38: 586–594.

100. Lanhers C, Pereira B, Garde G, Maublant C, Dutheil F, Coudeyre E. Evaluation of “I-Preventive”: a digital preventive tool for musculoskeletal disorders in computer workers-a pilot cluster randomised trial. BMJ Open. 2016;6. https://doi.org/10.1136/bmjopen-2016-011304 PMID: 27660316

101. Lopez V, Chamoux A, Tempier M. The long-term effects of occupational exposure to vinyl chloride monomer on microcirculation: a cross-sectional study 15 years after retirement. BMJ Open. 2013;3. https://doi.org/10.1136/bmjopen-2013-002785 PMID: 23794583

102. Edwards P, Roberts I, Sandercock P, Frost C. Follow-up by mail in clinical trials: does questionnaire length matter? Control Clin Trials. 2004; 25: 31–52. https://doi.org/10.1016/j.cct.2003.08.013 PMID: 14980747

103. Herrmann C. International experiences with the Hospital Anxiety and Depression Scale—a review of validation data and clinical results. J Psychosom Res. 1997; 42: 17–41. https://doi.org/10.1016/s0022-3999(96)00216-4 PMID: 9055211

104. Cohidon C, Imbernon E, Gorlbberg M. Prevalence of common mental disorders and their work consequences in France, according to occupational category. Am J Ind Med. 2009; 52: 141–152. https://doi.org/10.1002/ajim.20656 PMID: 19016268