# Scales used to measure job stressors in intensive care units: are they relevant and reliable? A systematic review

Alexandra Laurent, Florent Lheureux, Magali Genet, Maria Cruz Martin

Delgado, Maria Bocci, Alessia Prestifilippo, Guillaume Besch, Gilles Capellier

► To cite this version:

Alexandra Laurent, Florent Lheureux, Magali Genet, Maria Cruz Martin Delgado, Maria Bocci, et al. Scales used to measure job stressors in intensive care units: are they relevant and reliable? A systematic review. Frontiers in Psychology, 2020, 11, pp.245. 10.3389/fpsyg.2020.00245. hal-03103326

HAL Id: hal-03103326

https://u-bourgogne.hal.science/hal-03103326

Submitted on 12 Jul 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

# Scales Used to Measure Job Stressors in Intensive Care Units: Are They Relevant and Reliable? A Systematic Review

Alexandra Laurent¹,²*, Florent Lheureux³, Magali Genet³, Maria Cruz Martin Delgado⁴, Maria G. Bocci⁵, Alessia Prestifilippo⁶, Guillaume Besch⁷ and Gilles Capellier⁸

¹ Le Laboratoire de Psychologie: Dynamiques Relationnelles Et Processus Identitaires (Psy DREPI), University of Bourgogne Franche-Comté, Dijon, France, ² La Maison des Sciences de l'Homme et de l'Environnement (MSHE) C. N. Ledoux, University of Bourgogne Franche-Comté, Besançon, France, ³ Laboratory of Psychology, University of Bourgogne Franche-Comté, Besançon, France, ⁴ Intensive Care Unit, Hospital Universitario de Torrejón, Madrid, Spain, ⁵ Department of Anesthesiology and Intensive Care, Fondazione Policlinico Universitario Agostino Gemelli IRCCS, Rome, Italy, ⁶ Fondazione Policlinico Universitario Agostino Gemelli IRCCS, Rome, Italy, ⁷ Department of Anesthesiology and Intensive Care Medicine, University Hospital of Besançon, University of Bourgogne Franche-Comté, Besançon, France, ⁸ Medical Intensive Care Unit, University Hospital of Besançon, Besançon, France

## OPEN ACCESS

**Edited by:**
Cesar Merino-Soto,
University of San Martín de
Porres, Peru

**Reviewed by:**
Caterina Ledda,
University of Catania, Italy
Giridhara R. Babu,
Public Health Foundation of
India, India

**\*Correspondence:**
Alexandra Laurent
alexandra.laurent@u-bourgogne.fr

**Specialty section:**
This article was submitted to
Quantitative Psychology and
Measurement,
a section of the journal
Frontiers in Psychology

**Received:** 27 June 2019
**Accepted:** 03 February 2020
**Published:** 12 March 2020

**Citation:**
Laurent A, Lheureux F, Genet M, Martin Delgado MC, Bocci MG, Prestifilippo A, Besch G and Capellier G (2020) Scales Used to Measure Job Stressors in Intensive Care Units: Are They Relevant and Reliable? A Systematic Review. *Front. Psychol.* 11:245. doi: 10.3389/fpsyg.2020.00245

**Background:** Many studies have been conducted in intensive care units (ICUs) to identify the stress factors involved in the health of professionals and the quality and safety of care. The objectives are to identify the psychometric scales used in these studies to measure stressors and to assess their relevance and validity/reliability.

**Methods:** All peer-reviewed full-text articles published in English between 1997 and 2016 and focusing on an empirical quantitative study of job stressors were identified through searches on seven databases and editorial portals.

**Results:** From the 102 studies analyzed, we identified 59 different scales: 17 “all settings scales” (16 validated scales), 20 “healthcare settings scales” (13 validated scales), and 22 “ICU settings scales” (two validated scales). All these scales used measured stressors from at least one of the following eight broad categories: High job demands, Problematic relationships with other professionals, Lack of control over work situations and career, Lack of organizational resources, Problematic situations with users and relatives, Dealing with ethical- and moral-related situations, Risk management issues, and Disadvantages in comparison to other occupational situations. The “all settings scales” and “healthcare settings scales,” the most often validated, did not measure, or only slightly measured, the stressors most specific to ICUs. Where these were taken into account, the authors were forced to develop their own tools or modify existing scales without testing the validity of the tool used.

**Conclusions:** This review highlights the lack of a tool that meets both the criteria of validity and relevance with regard to the specificity of work in ICUs. Future research must focus on developing reliable/valid tools covering all types of relevant stressors to ensure the quality of the studies carried out in this field.

**Keywords:** systematic review, intensive care unit (ICU), job stressors, occupational stressors, psychometrics, job stress scales
# INTRODUCTION

Intensive care unit (ICU) professionals must deal with patients with serious medical conditions that require complex diagnostic and therapeutic procedures. This often requires a considerable level of coordination of human resources. Furthermore, end-of-life decisions are frequent and contribute to an intense emotional charge (Teixeira et al., 2014; Flannery et al., 2016). The ICU is thus fertile ground for the emergence of professional stressors (Donchin and Seagull, 2002; Embriaco et al., 2007a). Assessed by individuals as situations that weaken or are beyond their resources (Lazarus and Folkman, 1984), work-related stressors impact the mental and physical health of workers and the quality and safety of care (Sochalski, 2004; de Cássia Pereira Fernandes et al., 2016; Krämer et al., 2016; Dragano et al., 2017; Vandepla et al., 2017).

To assess these professional stressors, various surveys have been developed that cover either generic scales addressing all professional activities or more specific scales focusing primarily on the healthcare field or on a specific healthcare sector. These healthcare scales are used to measure specific stressors such as “end-of-life decisions” (Ozden et al., 2013; Teixeira et al., 2014), “conflicts,” or the “health culture” (Profit et al., 2014; Garrouste-Orgeas et al., 2015) particularly studied for their involvement in burnout or anxiety-depression.

While the identification of professional stressors is in line with the promotion of well-being at work and the quality of healthcare, the reliability of the psychometric tools is also of great importance. However, it appears that the multiplication of stress assessment tools makes it difficult to choose the most appropriate scale. In this sense, Bonneterre et al. (2008) denounce the use of unsuitable tools or the use of tools with insufficient psychometric qualities, which make the predictive validities between stressors and epidemiological indicators unreliable. Therefore, when researchers or clinicians wish to evaluate the stressors present in the ICU, what types of tools can they find in the literature? First, are these tools psychometrically valid? In addition, are they able to measure all relevant stressors, including those most specific to ICUs? This latter question deals with the issue of ecological validity. The ecological validity concept examined whether a study and its findings are representative of real-life situations (Brewer and Crano, 2014). In particular, ecologically valid studies used material and procedure that satisfy three parameters (Schmuckler, 2001): they reflect situations or events that can actually occur in participants’ everyday lives, reproduce or they refer to features of participants’ current living environments, and stimulate from participants reactions that are already available in their response repertoire. Applied to the measurement of stressors in ICUs, an ecologically valid tool should cover all types of situations actually occurring in ICUs that are likely to cause stress to professionals, with item content that refers to their current working environment (e.g., service functioning, patients and families, tasks to perform). Thus, the ideal tool would respect these principles in addition to present good psychometric properties (e.g., reliability, construct validity). But is it possible to find such an ideal tool in peer-reviewed journals?

To answer these questions, our literature review was guided by the following research objectives:

1. To comprehensively identify the scales and questionnaires that have been used to date to measure perceived job stressors in the ICU (Are there few or numerous measurement tools in the peer-reviewed literature?).

2. To determine the most frequently and least frequently used scales/questionnaires (Is there a consensual use of one or several tools? If so, what may explain this?).

3. To critically examine the ecological validity of these studies, i.e., their ability to take into account all stressors relevant to ICU settings; this means specific stressors as well as more general stressors such as job demands, lack of social support, etc.

4. To critically examine their basic psychometric reliability/validity as evidenced by the use of suitable methodological and statistical procedures such as factor analysis, Cronbach's alpha, etc.

# METHODS

## Study Design

We considered all peer-reviewed full-text articles reporting an empirical study, a literature review, or a meta-analysis published in English between 1997 and 2016. The choice of article in English corresponds to the objective of reporting on the most commonly used stress scales in intensive care at an international level. In this sense, English publications, the common language of international researchers and practitioners, include the largest number of journals with high international visibility. Only the empirical studies that used a quantitative methodology to explicitly measure job/occupational stress factors for healthcare professionals in ICUs were selected.

Noncompliant articles (commentaries, case reports, posters, and editorials) simply reporting perceived stress or focusing on patients instead of healthcare providers were excluded.

## Search Strategy and Study Eligibility

The search for and selection of articles were conducted between March and May 2017 (according to PRISMA guidelines). Seven databases and editorial portals (Medline via Pubmed; PsycInfo and Psychology, and behavioral science collection via EBSCOhost; Elsevier's ScienceDirect, SpringerLink, Sage Journals, and Wiley Online Library) were screened using the following terms in the title, the abstract, or the keywords: (intensive OR critical) AND care AND (professional OR job OR work OR occupational) AND (stress OR stressors OR burnout). The references of 10 literature review articles (Dunser et al., 2006; Embriaco et al., 2007b; Fassier and Azoulay, 2010; Adriaenssens et al., 2015; Karanikola et al., 2015; Van Mol et al., 2015; Chuang et al., 2016; Flannery et al., 2016) were manually screened to identify other relevant studies that had not been initially retrieved.

The initial search, conducted by MG, identified 1,330 records (Figure 1), and 34 were added following a manual search in the references of the articles retrieved. Duplicates (n = 230),
<figure><img src="image_3.png" /><figcaption>FIGURE 1 | Flow diagram of selected articles based on relevance search criteria.</figcaption></figure>

articles in a language other than English (n = 328), or those not conforming to the search criteria (n = 110) were removed. The abstracts were then screened for eligibility separately by two authors (MG and FL). When disagreement or the need for further analysis arose, the entire article was (n = 96) examined and discussions were held between AL, FL, MG, MM, MB, and AP to reach an agreement. From the 696 articles thus selected, 594 were excluded as they did not focus on stress in the ICU (n = 207), did not measure stressors (n = 276), did not include healthcare providers (n = 33), did not use a quantitative methodology (n = 77), or could not be retrieved (n = 1). In total, our review focused on 102 empirical studies (references in **Supplementary Material**).

## Data Extraction and Criteria Used to Assess the Quality and the Relevance of Existing Scales in Intensive Care Unit Contexts

Following the selection of the 102 relevant articles, data were extracted and coded by AL and FL for each scale according to five criteria:

(a) *Identification of the scale used* and the *number of articles* that used it.

(b) The *origin and metrological reliability/validity*. At that level, we made a distinction between *ante hoc* and *ad hoc* scales, with subdivisions within these two broad categories
of scales. This is because “previously-validated scales are generally preferable to *ad hoc* scales” (Furr, 2011, p. 8) given that the quality assessment of *ad hoc* scales rarely goes beyond face validity as perceived by the researchers (i.e., lack of independent evidence of validity from other sources such as experts or participants). Furthermore, the use of *ad hoc* measures reduces the comparability of studies as it introduces a possible confounding factor. Moreover, they generally lack a complete inspection of other forms of validity and reliability, as estimated by internal consistency coefficients (e.g., Cronbach's alpha), high test-retest correlation coefficients, theory-consistent and interpretable factor structure without problematic loadings, or correlations with similar measures (convergent validity) or logically related phenomena (nomological validity). In this respect, “rigorously developed measures have a lower probability of being based on chance or method variance than *ad hoc* measure” Peter and Churchill (1986, p. 3). Finally, *ante hoc* scales offer more the guarantee that they were elaborated independently from any hypothesis-testing purpose (i.e., absence of hypothesis confirmation bias in the generation of items). In addition, we also consider whether *ante hoc* validated scales were modified—or not—by ICU researchers. In fact, as pointed out by Furr (2011, p. 9) “well-validated original scales are preferable to modified scales. Because a modified scale's psychometric properties and quality might differ from those of the original scale, the modified scale is—to some degree—an *ad hoc* scale. As such, its psychometric properties and quality are unclear and suspect.” Nevertheless, because some authors using an *ad hoc* scale have provided validity-related information regarding the origin of items and/or basic statistics (generally Cronbach's alpha and sometimes factor analysis results), we differentiated *ad hoc* scales according to the presence or absence of such information. As a result of the combination of these criteria, we differentiated five types of scales ranging from the probably more reliable/valid to the less reliable/valid: *ante hoc* validated scales (Type 1); *ante hoc* validated scales with *ad hoc* modifications (items removed or rewritten) (Type 2); *ad hoc* scales with validity-related information regarding the origin of items and basic statistics (Type 3); *ad hoc* scales with validity-related information regarding the origin of items only or report of basic statistics only (Type 4); *ad hoc* scales without report of any validity-related information (Type 5).

(c) The target population and the level of generality or specificity according to three levels: (1) scales designed to measure stressors in all work settings (i.e., all settings scales); (2) scales specific to healthcare providers from several types of clinical units/settings (i.e., healthcare settings scales); (3) scales tailored to measure stressors specific to ICU settings.

(d) The number of items and, when relevant and available, their subscales.

(e) The types of stressors measured. An in-depth assessment of the subscales and items enabled us to develop a typology of all stressors that had been measured by the different scales. FL, AL, and MG examined in detail each scale with

regard to its structure (presence and labels of subscales) and item content and subsequently made iterative comparisons between them to identify the broad types and subtypes of stressors measured when considering all scales together. The choice of an inductive methodology to define the typology of stressors made it possible to allow oneself not to be limited to a typology preconceived by a theoretical model, which would risk not covering all the stressors explored in the literature. This typology was then critically examined by AL, GC, and GB and revised subsequently to reach a consensus. Then, each scale was assessed using a binary format “yes/no presence” for each type and subtype of stressors. This criterion was especially considered to assess the ecological validity of scales. Because ICU professionals can be jointly exposed to widely common stressors (e.g., workload, lack of support from colleagues), stressors shared with many other health professions (e.g., lack of recognition from patients and families, administrative hassles), and stressors more typical of ICUs (e.g., end-of-life decisions, constant monitoring of critically ill patients), we consider that the more a scale covers exhaustively the different types and subtypes of stressors, the more it was considered as ecologically valid.

# RESULTS

## What Types of Scales Are Used in the Intensive Care Unit?

From the 102 studies analyzed in our literature review, we identified 59 different scales according to three main categories (Table 1): 17 “generic scales” used in all sectors of professional activities, 20 “healthcare scales” used in the field of human health, and 22 scales specific to ICU settings. Only 28 out of the 59 scales were Type 1 scales (*ante hoc* validated). The majority of them were generic scales (13/17) or healthcare settings scales (13/20); there were only two ICU-specific scales (2/22).

Overall, among the 102 studies seeking to identify stressors in the ICU, 36 used a more or less problematic scale in terms of validity (Types 2–5).

## Types of Stressors Assessed in Intensive Care Unit Studies

Eight major types of stressors, grouping 58 subtypes, have been identified (see Table 2; for the coding of each scale, see Table 1): (1) *High job demands* (40 scales); (2) *Problematic relationships with other professionals* (39 scales); (3) *Lack of control over work situations and career* (31 scales); (4) *Lack of organizational resources* (29 scales); (5) *Problematic situations with users and relatives* (20 scales); (6) *Dealing with ethical and moral-related situations* (19 scales); (7) *Risk management issues* (14 scales); (8) *Disadvantages in comparison to other occupational situations* (11 scales). Two types of stressors do not fall within these eight main categories and have been classified as “other” (measured by three different scales).

The *High job demands* category refers to taxing task-, workflow-, and role-related situations and distinguishes between eight subtypes. The *problematic relationships with*
<table><caption>TABLE 1 | Scale description according to assessment criteria.</caption><thead><tr><th>Scale</th><th>Number of studies</th><th>References in online Supplementary Material</th><th>Metrological validity</th><th>Type of setting</th><th>Type of population</th><th>Stressors defined and measured with numbers of items</th></tr></thead><tbody><tr><td>Job Content Questionnaire (JCQ) and variants</td><td>17 of which 5 used with another scale and 3 not used entirely (12-item version)</td><td>(3, 5-7, 17, 18, 30, 36, 40, 41, 51, 61, 64, 71, 76, 87, 92)</td><td>Type 1 (14 articles)<br>Type 2 (3 articles)</td><td>All settings</td><td>All professionals</td><td>12, 29, or 49 items with three to five subscales: Psychological and physical demands of the job, decision latitude, social support, job insecurity. <i>Types of stressors measured:</i> Lack of positive/supportive relationship with colleagues, Lack of instrumental support from supervisor, Conflict with colleagues, Conflict with supervisor, Workload/time pressure, Interruption/interference/distraction/unanticipated changes and resequencing, Task complexity/High level of attention/performance, Lack of decision authority/autonomy (timing, method, etc.), Lack of growth opportunities, Skill underutilization, Lack of task diversity/interest, Lack of task-role clarity, Lack of job security</td></tr><tr><td>Effort-Reward Imbalance (ERI) questionnaire</td><td>3</td><td>(39, 51, 95)</td><td>Type 1</td><td>All settings</td><td>All professionals</td><td>23 items with 3 subscales: Extrinsic effort, extrinsic reward, and overcommitment. <i>Types of stressors measured:</i> Lack of instrumental support from colleagues, Lack of positive/supportive relationship with supervisor, Workload/time pressure, Interruption/interference/distraction/unanticipated changes and resequencing, High managerial/decisional responsibilities, Lack of growth opportunities, Skill underutilization, Lack of job security</td></tr><tr><td>Workplace Stress Scale (WSS)</td><td>2</td><td>(1, 79)</td><td>Type 1</td><td>All settings</td><td>All professionals</td><td>12 items with 2 subscales: Job demands and Job Resources. <i>Types of stressors measured:</i> Workload/time pressure, Role conflicts/contradictory demands, Lack of decision authority/autonomy (timing, method, etc.), Risky situations for oneself, Lack of job security</td></tr><tr><td>Copenhagen Psychosocial Questionnaire V2 (COPSOQ II)</td><td>2</td><td>(13, 69)</td><td>Type 1</td><td>All settings</td><td>All professionals</td><td>128 items with 3 subscales: work demands, health, job outcomes. <i>Types of stressors measured:</i> Lack of instrumental support from colleagues, Lack of positive/supportive relationship with supervisor, Lack of team cohesion, Lack of instrumental support from supervisor, Conflict with colleagues, Death and suffering and emotion regulation, Communicating with and fulfilling the emotional needs of users or relatives, Conflict with supervisor, Injustice, discrimination, harassment, bullying, Workload/time pressure, Task complexity/High level of attention/performance, Role conflicts/contradictory demands, Lack of participation in workplace and service-level policies, Lack of decision authority/autonomy (timing, method, etc.), Lack of growth opportunities, Lack of task diversity/interest, Lack of a predictable, stable and recovery-propitious schedule, Lack of predictable and stable work relationships and place, Lack of task-role clarity, Lack of task meaning/utility lack of hierarchical role clarity, Observing deviations from safety standards, Lack of job security, Work-Home conflict (e.g., because of night shifts, on site call)</td></tr><tr><td>Daily Hassles Questionnaire (DHQ)</td><td>1</td><td>(100)</td><td>Type 1</td><td>All settings</td><td>All professionals</td><td>118 items with no identified subscales. <i>Types of stressors measured:</i> Lack of positive/supportive relationship with colleagues, Conflict with supervisor, Injustice, discrimination, harassment, bullying, Workload/time pressure, Interruption/interference/distraction/unanticipated changes and resequencing, Task complexity/High level of attention/performance, Skill underutilization, Lack of job security, Work-Home conflict (e.g., because of night shifts, onsite call)</td></tr><tr><td>NASA Task Load Index (NASA-TLX)</td><td>1 used with another scale</td><td>(63)</td><td>Type 1</td><td>All settings</td><td>All professionals</td><td>6 items with 6 subscales: mental demands, physical demands, temporal demands, performance, effort, and frustration. <i>Types of stressors measured:</i> Workload/time pressure, Task complexity/High level of attention/performance, Physical efforts during task performance</td></tr><tr><td>Occupational Stress Indicator (OSI)</td><td>1</td><td>(29)</td><td>Type 1</td><td>All settings</td><td>All professionals</td><td>167 items with 7 subscales: sources of pressure, Type A behavior, locus of control, coping strategies, job satisfaction, mental health, and physical health. <i>Types of stressors measured:</i> Lack of positive/supportive relationship with colleagues, Lack of instrumental support from colleagues, Conflict with colleagues, Conflict with inappropriate expectations or behaviors from users (customer, client, patient, etc.), Workload/time pressure, Taxing work environment (noisy, hectic, crowded, heated, etc.), Role conflicts/contradictory demands, High managerial/decisional responsibilities, Lack of growth opportunities, Lack of task-related skills or preparation (incl. training, knowledge update), Skill underutilization, Lack of a predictable, stable, and recovery-propitious schedule, Lack of task-role clarity, Risky situations for oneself, Lack of pride/self-respect</td></tr><tr><td>Brief job stress questionnaire (B-JSQ)</td><td>1</td><td>(43)</td><td>Type 1</td><td>All settings</td><td>All professionals</td><td>84 items with 4 subscales: job-related stress factors and social support, psychological and somatic symptom. <i>Types of stressors measured:</i> Lack of team cohesion, Lack of or inappropriate inter-services/administrative collaboration, Workload/time pressure, Task complexity/High level of attention/performance, Taxing work environment (noisy, hectic, crowded, heated, etc.), Physical efforts during task performance, Lack of participation in workplace and service-level policies, Lack of decision authority/autonomy (timing, method, etc.), Skill underutilization</td></tr></tbody></table>

(Continued)
<table><caption>TABLE 1 | Continued</caption><thead><tr><th>Scale</th><th>Number of studies</th><th>References in online Supplementary Material</th><th>Metrological validity</th><th>Type of setting</th><th>Type of population</th><th>Stressors defined and measured with numbers of items</th></tr></thead><tbody><tr><td>Rizzo et al. (1970)<br>Role ambiguity and role conflict scales</td><td>1 used with another scale</td><td>(67)</td><td>Type 1</td><td>All settings</td><td>All professionals</td><td>14 items with 2 subscales: role ambiguity and role conflict. <i>Types of stressors measured</i>: Lack of team cohesion, Role conflicts/contradictory demands, Lack of staffing, Lack of information (e.g., to ask patients' questions), Lack of or low-quality or low accessibility to material resources, Lack of task-role clarity, Lack of adequate rules and procedures, Lack of task meaning/utility, Lack of hierarchical role clarity</td></tr><tr><td>Instrument for STress-oriented Analysis (ISTA)</td><td>1</td><td>(46)</td><td>Type 1</td><td>All settings</td><td>All professionals</td><td>30 items with 2 subscales: stressors and resources. <i>Types of stressors measured</i>: Lack of positive/supportive relationship with colleagues, Lack of instrumental support from colleagues, Task complexity/High level of attention/performance, Taxing work environment (noisy, hectic, crowded, heated, etc.), Role conflicts/contradictory demands, Lack of decision authority/autonomy (timing, method, etc.), Lack of task diversity/interest, Lack of or low-quality or low accessibility to material resources, Risk of making severe errors, Risky situations for oneself</td></tr><tr><td>Job Control Scale (JCS)</td><td>1 used with another scale and not used entirely</td><td>(93)</td><td>Type 1</td><td>All settings</td><td>All professionals</td><td>10 items with 2 subscales: timing control and method control. <i>Types of stressors measured</i>: Lack of decision authority/autonomy (timing, method, etc.)</td></tr><tr><td>Negative Acts<br>Questionnaire-Revised (NAQ-R)</td><td>1</td><td>(24)</td><td>Type 1</td><td>All settings</td><td>All professionals</td><td>22 items with subscales about work-related bullying, person-related bullying, or physical intimidation, respectively. <i>Types of stressors measured</i>: Conflict with colleagues, Conflict with supervisor, Injustice, discrimination, harassment, bullying, Workload/time pressure, Lack of participation in workplace and service-level policies, Skill underutilization, Lack of information (e.g., to ask patients' questions)</td></tr><tr><td>Interpersonal Work Relations<br>Scale (in Portuguese ERIT)</td><td>1 used with another scale</td><td>(70)</td><td>Type 1</td><td>All settings</td><td>All professionals</td><td>17 items with 2 subscales: sociability and feeling about oneself. <i>Types of stressors measured</i>: Lack of positive/supportive relationship with colleagues, Lack of instrumental support from colleagues</td></tr><tr><td>Supervisor version of the Survey<br>Perceived Organizational Support (SPOS)</td><td>1</td><td>(93)</td><td>Type 1</td><td>All settings</td><td>All professionals</td><td>4 items (partial) with no subscale. <i>Types of stressors measured</i>: Lack of positive/supportive relationship with supervisor</td></tr><tr><td>NIPG Questionnaire for Work Content-Well-being at Work (in Dutch NOVA-WEBA)</td><td>1 not used entirely</td><td>(90)</td><td>Type 2</td><td>All settings</td><td>All professionals</td><td>7 items (partial) with 2 subscales: autonomy and work pressure. <i>Types of stressors measured</i>: Workload/time pressure, Lack of decision authority/autonomy (timing, method, etc.)</td></tr><tr><td>Work Experience and<br>Assessment Questionnaire (in Dutch VBBA)</td><td>1 not used entirely</td><td>(90)</td><td>Type 2</td><td>All settings</td><td>All professionals</td><td>15 items (partial) with 4 subscales: emotional demands, physical demands, social support, development opportunities. <i>Types of stressors measured</i>: Lack of growth opportunities, Physical efforts during tasks performance, Death and suffering and emotion regulation, Communicating with and fulfilling the emotional needs of users or relatives</td></tr><tr><td>Perceived Job Stressors<br>(PJS)-Negative Subscale</td><td>1 used with another scale and not used entirely</td><td>(78)</td><td>Type 2</td><td>All settings</td><td>All professionals</td><td>9 items with no identified subscales. <i>Types of stressors measured</i>: Workload/time pressure, Interruption/interference/distraction/unanticipated changes and resequencing, Role conflicts/contradictory demands, High managerial/decisional responsibilities, Underload</td></tr><tr><td>Moral Distress Scale Revised<br>(MDS-R)</td><td>15 of which 3 used with another scale</td><td>(11, 15, 16, 19, 20, 33, 42, 57, 60, 62, 68, 82, 86, 98, 99)</td><td>Type 1</td><td>Healthcare settings</td><td>Healthcare professionals</td><td>18 items with 3 subscales: clinical situations, internal constraints, and external constraints. <i>Types of stressors measured</i>: Workload/time pressure, Lack of participation in workplace and service-level policies, Lack of decision authority/autonomy (timing, method, etc.), Lack of assertiveness when confronted with ethical concerns, Observing deviations from safety standards, Unsuitability of care (Futility or over/under aggressiveness of therapeutics), Working with incompetent/inexperienced /negligent staff members, Ignoring patients' preferences and conditions, Unsafe orders/policies from hierarchies or persons in charge</td></tr><tr><td>Nursing Stress Scale (NSS)</td><td>8 of which 2 used with another scale</td><td>(17, 25, 26, 53, 54, 61, 65, 75)</td><td>Type 1</td><td>Healthcare settings</td><td>Nurses only</td><td>34 items with 7 subscales: death and suffering of patients, conflict with doctors, Lack of appropriate training, Lack of social support, conflict with other nurses, excessive workload, and uncertainty about the treatment carried out. <i>Types of stressors measured</i>: Lack of positive/supportive relationship with colleagues, Conflict with colleagues, Death and suffering and emotion regulation, Communicating with and fulfilling the emotional needs of users or relatives, Conflict with supervisor, Workload/time pressure, Lack of task-related skills or preparation (incl. training, knowledge update), Lack of a predictable, stable, and recovery-propitious schedule, Lack of staffing, Lack of information (e.g., to ask patients' questions), Lack of or low-quality or low accessibility to material resources, Lack of predictable and stable work relationships and places, Risk of making severe errors</td></tr></tbody></table>

(Continued)
<table><caption>TABLE 1 | Continued</caption><thead><tr><th>Scale</th><th>Number of studies</th><th>References in online Supplementary Material</th><th>Metrological validity</th><th>Type of setting</th><th>Type of population</th><th>Stressors defined and measured with numbers of items</th></tr></thead><tbody><tr><td>Nursing Work Index (NWI) and variants (PES-NWI/NWI EO/NWI-R)</td><td>6 of which 2 used with another scale and 1 not used entirely</td><td>(7, 12, 48, 56, 67, 77)</td><td>Type 1 (5 articles)<br>Type 2 (1 article)</td><td>Healthcare settings</td><td>Nurses only</td><td>31 to 57 items with 5 subscales: nurse participation in hospital affairs, nursing foundations for quality of care, nurse manager, ability, leadership and support of nurses, staffing and resource adequacy, collegial nurse physician relations. <strong>Types of stressors measured:</strong> Lack of positive/supportive relationship with colleagues, Lack of instrumental support from colleagues, Lack of positive/supportive relationship with supervisor, Lack of team cohesion, Lack of instrumental support from supervisor, Lack of or inappropriate inter-services/administrative collaboration, Lack of value-based team harmony/tolerance, Interruption/interference/distraction/unanticipated changes and resequencing, Lack of participation in workplace and service-level policies, Lack of decision authority/autonomy (timing, method, etc.), Lack of growth opportunities, Lack of a preceptor program for newly hired personnel, Lack of a predictable, stable, and recovery-propitious schedule, Lack of staffing, Lack of information (e.g., unable to question patients?), Lack of predictable and stable work relationships and place, Working with incompetent/inexperienced/negligent staff members</td></tr><tr><td>Nurse Stress Index (NSI)</td><td>4 of which 1 used with another scale</td><td>(9, 47, 52, 70)</td><td>Type 1</td><td>Healthcare settings</td><td>Nurses only</td><td>30 items with 6 subscales: Workload (time management), Workload (managerial demands), lack of organizational support, Work-Home conflict, Confidence and Competence, Dealing with difficult patients and relatives. <strong>Types of stressors measured:</strong> Lack of instrumental support from colleagues, Lack of instrumental support from supervisor, Death and suffering and emotion regulation, Communicating with and fulfilling the emotional needs of users or relatives, Conflict with inappropriate expectations or behaviors from users (customer, client, patient, etc.), Conflict with inappropriate expectations or behaviors from users' relatives, Workload/time pressure, Lack of task-related skills or preparation (incl. training, knowledge update), Work-Home conflict (e.g., because of night shifts, on site call)</td></tr><tr><td>Hospital Survey Of Patient Safety Culture (HSOPSC)</td><td>2</td><td>(96, 97)</td><td>Type 1</td><td>Healthcare settings</td><td>Healthcare professionals</td><td>42 items with 12 subscales: supervisor expectations/actions promoting patient safety, organizational learning continuous improvement, teamwork, communication openness, feedback and error reporting, nonpunitive response to error, staffing, hospital management toward patient safety, teamwork across the hospital, handoffs and transitions, general perception of patient safety, frequency of errors reporting. <strong>Types of stressors measured:</strong> Lack of positive/supportive relationship with colleagues, Lack of instrumental support from colleagues, Lack of team cohesion, Lack of or inappropriate inter-services/administrative collaboration, Workload/time pressure, Lack of participation in workplace and service-level policies, Lack of staffing, Lack of information (e.g., unable to question patients), Observing deviations from safety standards, Unsafe orders/policies from hierarchies or person in charge</td></tr><tr><td>Tummers (2002, 2005)<br>Organizational and work characteristics questionnaire</td><td>2</td><td>(88, 89)</td><td>Type 4 (A)</td><td>Healthcare settings</td><td>Healthcare professionals</td><td>Number of items unavailable. <strong>Types of stressors measured:</strong> Lack of positive/supportive relationship with colleagues, Lack of instrumental support from colleagues, Lack of positive/supportive relationship with supervisor, Lack of instrumental support from supervisor, Conflict with colleagues, Conflict with supervisor, Workload/time pressure, Interruption/interference/distraction/unanticipated changes and resequencing, Task complexity/High level of attention/performance, Lack of decision authority/autonomy (timing, method, etc.), Constant alert and sudden emergencies due to the patient's condition</td></tr><tr><td>Sagie and Krausz (2003)<br>scheduling control scale</td><td>1 used with another scale</td><td>(78)</td><td>Type 3 (A)</td><td>Healthcare settings</td><td>Healthcare professionals</td><td>Number of items unavailable. Nurses' perceptions of having choice and influence on the timing and scheduling of their work. <strong>Types of stressors measured:</strong> Lack of participation in workplace and service-level policies, Lack of decision authority/autonomy (timing, method, etc.)</td></tr><tr><td>Job Stress Scale (JSS)</td><td>1</td><td>(8)</td><td>Type 1</td><td>Healthcare settings</td><td>Healthcare professionals</td><td>22 items with 4 subscales: competence, physical work environment, staffing, team respect. <strong>Types of stressors measured:</strong> Lack of positive/supportive relationship with colleagues, Taxing work environment (noisy, hectic, crowded, heated, etc.), Lack of task-related skills or preparation (incl. training, knowledge update), Lack of staffing</td></tr><tr><td>Work-Related Strain Inventory (WRSI)</td><td>1</td><td>(4)</td><td>Type 1</td><td>Healthcare settings</td><td>Healthcare professionals</td><td>18 items with no identified subscales. <strong>Types of stressors measured:</strong> Lack of positive/supportive relationship with colleagues, Lack of positive/supportive relationship with supervisor, Conflict with colleagues, Conflict with supervisor, Workload/time pressure</td></tr><tr><td>Nursing Job Stressor Scale (NJSS)</td><td>1</td><td>(45)</td><td>Type 1</td><td>Healthcare settings</td><td>Nurses only</td><td>33 items with 7 subscales: Conflict with other nursing staffs, nursing role conflict, conflict with physicians/autonomy, dealing with death and dying, qualitative work load, quantitative work load and conflict with patients. <strong>Types of stressors measured:</strong> Lack of positive/supportive relationship with colleagues, Lack of instrumental support from supervisor, Conflict with colleagues,</td></tr></tbody></table>

(Continued)
<table><caption>TABLE 1 | Continued</caption><thead><tr><th>Scale</th><th>Number of studies</th><th>References in online Supplementary Material</th><th>Metrological validity</th><th>Type of setting</th><th>Type of population</th><th>Stressors defined and measured with numbers of items</th></tr></thead><tbody><tr><td>Modified version of Cooper's Job Stress Questionnaire (CJSQ)</td><td>1</td><td>(66)</td><td>Type1</td><td>Healthcare settings</td><td>Healthcare professionals</td><td>Death and suffering and emotion regulation, Communicating with and fulfilling the emotional needs of users or relatives, Conflict with inappropriate expectations or behaviors from users (customer, client, patient, etc.), Workload/time pressure, Task complexity/High level of attention/performance, Lack of task-related skills or preparation (incl. training, knowledge update), Lack of staffing, Lack of information (e.g., to ask patients' questions), Risk of making severe errors</td></tr><tr><td>Nine Equivalent of nursing Manpower use (NEMS)</td><td>1 used with another scale</td><td>(6)</td><td>Type 1</td><td>Healthcare settings</td><td>Nurses only</td><td>16 items with no identified subscales. Types of stressors measured: Death and suffering and emotion regulation, Communicating with and fulfilling the emotional needs of users or relatives, Assault/aggression from patients or relatives, Lack of patients' recognition, Workload/time pressure, Interruption/interference/distraction/unanticipated changes and resequencing, Taxing work environment (noisy, hectic, crowded, heated, etc.), Risk of making severe errors, Constant alert and sudden emergencies due to the patient's condition, Work-Home conflict (e.g., because of night shifts, on site call)</td></tr><tr><td>Hospital Ethical Climate Scale (HECS-S)</td><td>1 used with another scale</td><td>(98)</td><td>Type 1</td><td>Healthcare settings</td><td>Healthcare professionals</td><td>9 items with no subscales. Types of stressors measured: Workload/time pressure, Task complexity/High level of attention/performance</td></tr><tr><td>Environmental Complexity Scale</td><td>1 used with another scale</td><td>(67)</td><td>Type 1</td><td>Healthcare settings</td><td>Healthcare professionals</td><td>26 items with 5 subscales: relation between colleagues, patients, managers, hospital, and physicians. Types of stressors measured: Lack of positive/supportive relationship with colleagues, Lack of instrumental support from colleagues, Lack of positive/supportive relationship with supervisor, Lack of team cohesion, Lack of instrumental support from supervisor, Lack of or inappropriate inter-services/administrative collaboration, Lack of participation in workplace and service-level policies, Lack of information (e.g., unable to question patients), Observing deviations from safety standards, Working with incompetent/inexperienced/negligent staff members, Ignoring patients' preferences and conditions</td></tr><tr><td>Varjus et al. (2003) Nurses' Autonomy Scale</td><td>1 used with another scale</td><td>(68)</td><td>Type 1</td><td>Healthcare settings</td><td>Nurses only</td><td>18 items with 3 subscales: knowledge, action, value bases of autonomy. Types of stressors measured: Lack of participation in workplace and service-level policies, Lack of decision authority/autonomy (timing, method, etc.)</td></tr><tr><td>Verhaeghe et al. (2008) negative appraisal of recurrent changes in the work environment</td><td>1 used with another scale</td><td>(93)</td><td>Type 4 (A)</td><td>Healthcare settings</td><td>Healthcare professionals</td><td>6 items with no identified subscales. Types of stressors measured: Lack of a predictable, stable, and recovery-propitious schedule, Lack of predictable and stable work relationships and places</td></tr><tr><td>Vessey et al. (2009) Nurses bullying questionnaire</td><td>1</td><td>(94)</td><td>Type 4 (A)</td><td>Healthcare settings</td><td>Nurses only</td><td>30 items with no identified subscales. Type of stressors measured: Injustice, discrimination, harassment, bullying</td></tr><tr><td>Casado et al. (2008) Occupational factors questionnaire</td><td>1</td><td>(10)</td><td>Type 4 (A)</td><td>Healthcare settings</td><td>Healthcare professionals</td><td>Number of items unavailable. Types of stressors measured: Lack of positive/supportive relationship with colleagues, Lack of positive/supportive relationship with supervisor, Workload/time pressure, Lack of a predictable, stable, and recovery-propitious schedule</td></tr><tr><td>Amin et al. (2009) potential stressors questionnaire</td><td>1</td><td>(2)</td><td>Type 5</td><td>Healthcare settings</td><td>Healthcare professionals</td><td>6 items with no identified subscales. Types of stressors measured: Lack of positive/supportive relationship with colleagues, Lack of instrumental support from colleagues, Lack of instrumental support from supervisor, Workload/time pressure, Lack of a predictable, stable, and recovery-propitious schedule, Lack of job security, Insufficient remuneration</td></tr><tr><td>Fujita et al. (2011) workplace violence questionnaire</td><td>1</td><td>(23)</td><td>Type 5</td><td>Healthcare settings</td><td>Healthcare professionals</td><td>3 questions about workplace violence, the work environment, and other topics. Type of stressors measured: Assault/aggression from patients or relatives</td></tr><tr><td>Safety Attitudes Questionnaire-ICU (SAQ-ICU)</td><td>6</td><td>(32, 37, 50, 72, 74, 102)</td><td>Type 1</td><td>ICU settings</td><td>Healthcare professionals</td><td>32 items with 6 subscales: teamwork climate, safety climate, job satisfaction, stress recognition, perception of management, working conditions. Types of stressors measured: Lack of positive/supportive relationship with colleagues, Lack of instrumental support from colleagues, Lack of positive/supportive relationship with supervisor, Lack of team cohesion, Lack of instrumental support from supervisor, Lack of or inappropriate inter-services/administrative collaboration, Lack of participation in workplace and service-level policies, Lack of staffing, Lack of information (e.g., to ask patients' questions), Observing deviations from safety standards, Working with incompetent/inexperienced/negligent staff members, Unsafe orders/policies from hierarchies or person in charge</td></tr><tr><td>Embriaco et al. (2007a) Working conditions questionnaire</td><td>4 of which 1 used with another scale</td><td>(21, 22, 59, 84)</td><td>Type 4 (A)</td><td>ICU settings</td><td>Healthcare professionals</td><td>Number of items unavailable. Types of stressors measured: Conflict with colleagues, Death and suffering and emotion regulation, Conflict with supervisor, Conflict with inappropriate</td></tr></tbody></table>

(Continued)
<table><caption>TABLE 1 | Continued</caption><thead><tr><th>Scale</th><th>Number of studies</th><th>References in online Supplementary Material</th><th>Metrological validity</th><th>Type of setting</th><th>Type of population</th><th>Stressors defined and measured with numbers of items</th></tr></thead><tbody><tr><td>Teixeira et al. (2014)<br>Ethical decision experiences of IC professionals</td><td>2 of which 1 used with another scale</td><td>(83, 84)</td><td>Type 4 (A)</td><td>ICU settings</td><td>Healthcare professionals</td><td>expectations or behaviors from users (customer, client, patient, etc.), Conflict with inappropriate expectations or behaviors from users' relatives, Workload/time pressure, Lack of a predictable, stable, and recovery-propitious schedule, Decisional dilemmas/uncertainty regarding patients' survival and end-of-life care, Risk of making severe errors</td></tr><tr><td>Coomber et al. (2002)<br>ICU-related stressors questionnaire</td><td>2</td><td>(14, 91)</td><td>Type 4</td><td>ICU settings</td><td>Healthcare professionals</td><td>Number of items unavailable. <i>Type of stressors measured:</i> Decisional dilemmas/uncertainty regarding patients' survival and end-of-life care</td></tr><tr><td>ICU nursing stress audit</td><td>1</td><td>(27)</td><td>Type 1</td><td>ICU settings</td><td>Nurses only</td><td>30 items with no identified subscales. <i>Types of stressors measured:</i> Lack of positive/supportive relationship with colleagues, Lack of positive/supportive relationship with supervisor, Lack of team cohesion, Communicating with and fulfilling the emotional needs of users or relatives, Conflict with supervisor, Lack of or inappropriate inter-services/administrative collaboration, Workload/time pressure, High managerial/decisional responsibilities, Lack of a predictable, stable, and recovery-propitious schedule, Lack of or low-quality or low accessibility to material resources, Decisional dilemmas/uncertainty regarding patients' survival and end-of-life care, Observing deviations from safety standards, Unsuitability of care (Futility or over/under aggressiveness of therapeutics, Risky situations for oneself, Work-Home conflict (e.g., because of night shifts, on site call))</td></tr><tr><td>Piers et al. (2011a)<br>Perceived ethical environment questionnaire</td><td>1 used with another scale</td><td>(71)</td><td>Type 3 (A)</td><td>ICU settings</td><td>Healthcare professionals</td><td>47 items with 9 subscales: management, interpersonal, relationships, patient care, knowledge, skill, work environment, self-perception, and administrative uncertainties. <i>Types of stressors measured:</i> Lack of positive/supportive relationship with colleagues, Lack of instrumental support from colleagues, Lack of positive/supportive relationship with supervisor, Lack of team cohesion, Conflict with colleagues, Death and suffering and emotion regulation, Communicating with and fulfilling the emotional needs of users or relatives, Lack of or inappropriate inter-services/administrative collaboration, Workload/time pressure, Interruption/interference/distraction/unanticipated changes and resequencing, Taxing work environment (noisy, hectic, crowded, heated, etc.), High managerial/ decisional responsibilities, Lack of growth opportunities, Lack of task-related skills or preparation (incl. training, knowledge update), Lack of a predictable, stable, and recovery-propitious schedule, Lack of staffing, Lack of adequate rules and procedures, Lack of task meaning/utility, Unsuitability of care (Futility or over/under aggressiveness of therapeutics), Working with incompetent/inexperienced/negligent staff members, Risk of making severe errors, Constant alert and sudden emergencies due to the patient's condition, Lack of job security, Insufficient remuneration</td></tr><tr><td>Hays et al. (2006) list of 13 stressors in ICU</td><td>1</td><td>(35)</td><td>Type 3 (A, B)</td><td>ICU settings</td><td>Healthcare professionals</td><td>7 items with no identified subscales. <i>Types of stressors measured:</i> Lack of positive/supportive relationship with colleagues, Lack of team cohesion, Lack of value-based team concordance /tolerance, Lack of participation in workplace and service-level policies</td></tr><tr><td>Comprehensive Nursing Intervention Score (CNIS)</td><td>1</td><td>(101)</td><td>Type 3 (A)</td><td>ICU settings</td><td>Nurses only</td><td>13 items with no identified subscales. <i>Types of stressors measured:</i> Death and suffering and emotion regulation, Lack of or inappropriate inter-services/administrative collaboration, Conflict with inappropriate expectations or behaviors from users' relatives, Taxing work environment (noisy, hectic, crowded, heated, etc.), Lack of staffing, Unsuitability of care (Futility or over/under aggressiveness of therapeutics), Working with incompetent/inexperienced/negligent staff members, Risk of making severe errors, Constant alert and sudden emergencies due to the patient's condition</td></tr><tr><td>Azoulay et al. (2009)<br>Nurse-physician conflicts questionnaire</td><td>1 used with another scale</td><td>(5)</td><td>Type 3 (B)</td><td>ICU settings</td><td>Healthcare professionals</td><td>73 items with 8 subscales: Monitoring, Transfusion of blood/fluids, Injections, Respiratory management, Assisted circulation, Drainage tube management, Special therapy, Basic nursing care. <i>Types of stressors measured:</i> Communicating with and fulfilling the emotional needs of users or relatives, Workload/time pressure, Task complexity/High level of attention/performance, Physical efforts during tasks performance</td></tr></tbody></table>

(Continued)
<table><caption>TABLE 1 | Continued</caption><thead><tr><th>Scale</th><th>Number of studies</th><th>References in online Supplementary Material</th><th>Metrological validity</th><th>Type of setting</th><th>Type of population</th><th>Stressors defined and measured with numbers of items</th></tr></thead><tbody><tr><td>Well-Being of Intensive Care nurses (WEBIC)-questionnaire</td><td>1</td><td>(49)</td><td>Type 3 (A)</td><td>ICU settings</td><td>Nurses only</td><td>Working with incompetent/inexperienced/negligent staff members, Ignoring patients' preferences and conditions</td></tr><tr><td>Van dam et al. (2012) Scale</td><td>1</td><td>(90)</td><td>Type 4 (A)</td><td>ICU settings</td><td>Healthcare professionals</td><td>Number of items unavailable. <i>Types of stressors measured</i>: Workload/time pressure, Task complexity/High level of attention/performance</td></tr><tr><td>Hansen et al. (2009) Perceptions of end-of-life care factors</td><td>1</td><td>(34)</td><td>Type 4 (A)</td><td>ICU settings</td><td>Healthcare professionals</td><td>38 items with 10 subscales. Dealing with night shifts, Technical orientation, Emotional demands, Physical demands, Threats from relatives, Social support, Autonomy, Development opportunities, Work pressure and Turnover intention. <i>Types of stressors measured</i>: Conflict with inappropriate expectations or behaviors from users' relatives, Lack of task diversity/interest, Lack of a predictable, stable, and recovery-propitious schedule</td></tr><tr><td>End-of-life DECision-making and staff Stress (EIDECS) questionnaire</td><td>1</td><td>(80)</td><td>Type 4</td><td>ICU settings</td><td>Healthcare professionals</td><td>30 items with 5 subscales: knowledge and ability, work environment, support for staff, support for patients and patients' families, and work stress related to specific end-of-life situations. <i>Types of stressors measured</i>: Lack of positive/supportive relationship with colleagues, Lack of instrumental support from colleagues, Lack of positive/supportive relationship with supervisor, Lack of team cohesion, Lack of instrumental support from supervisor, Communicating with and fulfilling the emotional needs of users or relatives, Lack of task-related skills or preparation (incl. training, knowledge update), Lack of information (e.g., to ask patients' questions), Lack of or low-quality or low accessibility to material resources, Decisional dilemmas/uncertainty regarding patients' survival and end-of-life care, Unsuitability of care (Futility or over/under aggressiveness of therapeutics), Ignoring patients' preferences and conditions, Lack of participation in end-of-life decisions or strong disagreement</td></tr><tr><td>Questionnaire of Performance Obstacles of Intensive Care Nurses (QPO-ICN)</td><td>1</td><td>(63)</td><td>Type 4</td><td>ICU settings</td><td>Nurses only</td><td>37 items with no identified subscales. <i>Types of stressors measured</i>: Lack of positive/supportive relationship with colleagues, Lack of positive/supportive relationship with supervisor, Lack of team cohesion, Death and suffering and emotion regulation, Lack of or inappropriate inter-services/administrative collaboration, Workload/time pressure, Interruption/interference/distraction/unanticipated changes and resequencing, Lack of participation in workplace and service-level policies, Lack of information (e.g., to ask patients' questions), lack of task-role clarity</td></tr><tr><td>Piers et al. (2011) (b) Perceived Inappropriateness of Care Questionnaire</td><td>1 used with another scale</td><td>(71)</td><td>Type 4</td><td>ICU settings</td><td>Healthcare professionals</td><td>53 items with no identified subscales. <i>Types of stressors measured</i>: Lack of instrumental support from colleagues</td></tr><tr><td>Hamric and Blackhall (2007)</td><td>1 used with another scale</td><td>(33)</td><td>Type 4 (A)</td><td>ICU settings</td><td>Healthcare professionals</td><td>7 items with no subscales. <i>Types of stressors measured</i>: Observing deviations from safety standards, Unsuitability of care (Futility or over/under aggressiveness of therapeutics), Ignoring patients' preferences and conditions, Lack of participation in end-of-life decisions or strong disagreement, Inaccurately informed patients and families</td></tr><tr><td>Malaquin et al. (2016) well-being at work</td><td>1</td><td>(58)</td><td>Type 5</td><td>ICU settings</td><td>Healthcare professionals</td><td>Number of items unavailable, 4 subscales: ethical environment, end-of-life communication, satisfaction with quality of care, collaboration. <i>Types of stressors measured</i>: Lack of instrumental support from colleagues, Lack of team cohesion, Decisional dilemmas/uncertainty regarding patients' survival and end-of-life care, Observing deviations from safety standards</td></tr><tr><td>Kincey et al. (2010) Potential sources of pressure in ICM and healthcare settings</td><td>1</td><td>(44)</td><td>Type 5</td><td>ICU settings</td><td>Healthcare professionals</td><td>Number of items unavailable. <i>Types of stressors measured</i>: Lack of positive/supportive relationship with colleagues, Lack of instrumental support from colleagues, Lack of positive/supportive relationship with supervisors, Lack of instrumental support from supervisors, Conflict with colleagues, Conflict with supervisors, Workload/time pressure, Lack of a predictable, stable, and recovery-propitious schedule, Finding time for research</td></tr><tr><td>Grzeskowiak et al. (2012) 10 most common real situations in the PICU</td><td>1</td><td>(31)</td><td>Type 5</td><td>ICU settings</td><td>Healthcare professionals</td><td>40 items with no identified subscales. <i>Types of stressors measured</i>: Lack of team cohesion, Workload/time pressure, Lack of a predictable, stable, and recovery-propitious schedule, Lack of staffing, Lack of or low-quality or low accessibility to material resources, Lack of predictable and stable work relationships and places, Decisional dilemmas/uncertainty regarding patients' survival and end-of-life care, Observing deviations from safety standards, Risk of making severe errors, Constant alert and sudden emergencies due to patients' conditions, Work-Home conflict (e.g., because of night shifts, on site call)</td></tr><tr><td></td><td></td><td></td><td></td><td></td><td></td><td>10 items with no identified subscales. <i>Types of stressors measured</i>: Death and suffering and emotion regulation, Communicating with and fulfilling the emotional needs of users or relatives, Supervisor's evaluation, Decisional dilemmas/uncertainty regarding patients' survival and end-of-life care, Risk of making severe errors</td></tr></tbody></table>

(Continued)
<table><caption>TABLE 1 | Continued</caption><thead><tr><th>Scale</th><th>Number of studies</th><th>References in online Supplementary Material</th><th>Metrological validity</th><th>Type of setting</th><th>Type of population</th><th>Stressors defined and measured with numbers of items</th></tr></thead><tbody><tr><td>Poncet et al. (2007) work-related factors questionnaire</td><td>1</td><td>(73)</td><td>Type 5</td><td>ICU settings</td><td>Healthcare professionals</td><td>23 items with no identified subscales. Types of stressors measured: Conflict with colleagues, Death and suffering and emotion regulation, Conflict with supervisor, Conflict with inappropriate expectations or behaviors from users (customer, client, patient, etc.), Conflict with inappropriate expectations or behaviors from users' relatives, Lack of participation in workplace and service-level policies, Lack of a predictable, stable, and recovery-prophylactic schedule, Decisional dilemmas/uncertainty regarding patients' survival and end-of-life care</td></tr><tr><td>Shehabi et al. (2009)</td><td>1</td><td>(81)</td><td>Type 5</td><td>ICU settings</td><td>Healthcare professionals</td><td>Number of items unavailable. Type of stressors measured: Workload/time pressure</td></tr><tr><td>Self-reported 12-weekly averaged workload pattern</td><td>1</td><td>(38)</td><td>Type 5</td><td>ICU settings</td><td>Healthcare professionals</td><td>15 items no subscales identified. Types of stressors measured: Conflict with colleagues, Death and suffering and emotion regulation, Communicating with and fulfilling the emotional needs of users or relatives, Conflict with inappropriate expectations or behaviors from users' relatives, Lack of staffing, Decisional dilemmas/uncertainty regarding patients' survival and end-of-life care, Unsuitability of care (Futility or over/under aggressiveness of therapeutics), Constant alert and sudden emergencies due to the patient's condition, Risky situations for oneself</td></tr><tr><td>Janda et Jandova (2015)</td><td>1</td><td></td><td></td><td></td><td></td><td></td></tr></tbody></table>

Type 1: Ante hoc validated scales.

Type 2: Ante hoc validated scales with ad hoc modifications.

Type 3: Ad hoc scales with validity-related information on item origin and basic statistics.

Type 4: Ad hoc scales with validity-related information on item origin or basic statistics.

Type 5: Ad hoc scales without any validity-related information.

(A): Ad hoc items derived from literature and/or another scale.

(B): Ad hoc items derived from interviews or conference consensus with healthcare professionals.

other professionals category includes 11 subtypes of stressors related to the lack of support, communication/collaboration problems, or conflicts with colleagues, superiors, or other departments. The lack of control over work situations and career category includes the different control and skill-related factors which may prevent professionals from being responsive, efficient, and committed. This category of stressors includes eight subtypes. The lack of organizational resources category includes nine subtypes of resources the healthcare organization might fail to sufficiently provide to promote the quality of life and the quality of care of ICU professionals. The problematic situations with service users and relatives category includes six subtypes of stressors which all involve one or more people whose behavior or health condition is likely to exceed the resources of healthcare professionals. The dealing with ethical- and moral-related situations category refers to eight subtypes of situations where patients' well-being and safety are likely to be harmed by unsuitable decisions/actions of the ICU team. The risk management issues category refers to risky situations for both patients and professionals with a substantial probability of making critical errors (three subtypes included). Lastly, the disadvantages in comparison to other occupational situations category of stressors concerns the extent to which the professional considers his/her current occupational situation is disadvantaged in comparison to other career choices (e.g., other jobs, other employers, other professions) (three measured subtypes).

## Ability of Scales to Take Into Account the Specificity of Intensive Care Unit Stressors

There are important differences concerning the extent to which the three types of scales (i.e., all settings, healthcare settings, and ICU settings scales) address the broad range of the types and subtypes of stressors mentioned above. To highlight these differences, we compared the proportion of each type of scale that measured a given stressor relative to the overall number of scales that measured it. However, given that there were more ICU settings scales (N = 22) than healthcare settings scales (N = 20) and all settings scales (N = 17), we applied a correction to make the comparisons valid<fnref n="1" />.

As illustrated by **Figure 2**, high job demands (52%), disadvantages in comparison to other occupational situations

<footnote n="1">¹The formula was: (N scales of a given type that measure the stressor/Overall N scales that measure the stressor)/(N all settings scales that measure the stressor/17) + (N healthcare settings scales that measure the stressor/20) + (N ICU settings scales that measure the stressor/22). For instance, there were nine scales that measured the “taxing work environment (noisy, hectic, crowded, heated, etc.)” stressor type, including three all settings scales, three healthcare settings scales, and three ICU settings scales. Without the correction, this stressor appeared to be measured to the same extent by the three types of scales (3/9). However, this conclusion is incorrect because the ratio 3/17 (17.65%) for all settings scales is higher than the ratio of 3/20 (15%) for healthcare settings scales and higher than the ratio of 3/22 (13.63%) for ICU settings scales. After the correction which took into account the unequal number of scale types (17 vs. 20 vs. 22), it emerged that this stressor was measured slightly more frequently by all settings scales [(3/17)/((3/17) + (3/20) + (3/22)) = 38%] than by healthcare settings scales [(3/20)/((3/17) + (3/20) + (3/22)) = 35%] and ICU settings scales [(3/22)/((3/17) + (3/20) + (3/22)) = 29%].</footnote>
<table><caption>TABLE 2 | Broad types and subtypes of stressors measured by each type of scales used in ICU studies.</caption><thead><tr><th rowspan="2">Broad types of stressors</th><th rowspan="2">Subtypes of stressors</th><th colspan="2">All scales (N = 59)</th><th colspan="2">All settings scales (n = 17)</th><th colspan="2">Healthcare settings scales (n = 20)</th><th colspan="2">ICU settings scales (n = 22)</th></tr><tr><th>N</th><th>%</th><th>N</th><th>Corrected %<sup>a</sup></th><th>N</th><th>Corrected %<sup>a</sup></th><th>N</th><th>Corrected %<sup>a</sup></th></tr></thead><tbody><tr><td rowspan="8">High job demands<br>40 scales with at least one subtype</td><td>1. Workload/time pressure</td><td>31</td><td>53</td><td>11</td><td>40</td><td>11</td><td>34</td><td>9</td><td>25</td></tr><tr><td>2. Interruption/interference/distraction/ unanticipated changes and resequencing</td><td>12</td><td>20</td><td>4</td><td>38</td><td>5</td><td>40</td><td>3</td><td>22</td></tr><tr><td>3. Task complexity/High level of attention/ performance</td><td>11</td><td>19</td><td>6</td><td>59</td><td>3</td><td>25</td><td>2</td><td>15</td></tr><tr><td>4. Taxing work environment (noisy, hectic, crowded, heated, etc.)</td><td>9</td><td>15</td><td>3</td><td>38</td><td>3</td><td>32</td><td>3</td><td>29</td></tr><tr><td>5. Role conflicts/contradictory demands</td><td>8</td><td>14</td><td>7</td><td>90</td><td>0</td><td>0</td><td>1</td><td>10</td></tr><tr><td>6. High managerial/decisional responsibilities</td><td>5</td><td>8</td><td>3</td><td>66</td><td>0</td><td>0</td><td>2</td><td>34</td></tr><tr><td>7. Physical efforts during task performance</td><td>4</td><td>7</td><td>3</td><td>80</td><td>0</td><td>0</td><td>1</td><td>20</td></tr><tr><td>8. Underload</td><td>1</td><td>2</td><td>1</td><td>100</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td colspan="2">TOTAL</td><td>81</td><td>18</td><td>38</td><td>52</td><td>22</td><td>26</td><td>21</td><td>22</td></tr><tr><td rowspan="10">Problematic relationships with other professionals<br>39 scales with at least one subtype</td><td>1. Lack of positive/supportive relationship with colleagues</td><td>25</td><td>42</td><td>7</td><td>32</td><td>10</td><td>39</td><td>8</td><td>29</td></tr><tr><td>2. Lack of instrumental support from colleagues</td><td>19</td><td>32</td><td>6</td><td>36</td><td>6</td><td>31</td><td>7</td><td>33</td></tr><tr><td>3. Lack of positive/supportive relationship with supervisor</td><td>17</td><td>29</td><td>4</td><td>28</td><td>6</td><td>35</td><td>7</td><td>37</td></tr><tr><td>4. Lack of team cohesion</td><td>15</td><td>25</td><td>3</td><td>24</td><td>3</td><td>20</td><td>9</td><td>56</td></tr><tr><td>5. Lack of instrumental support from supervisor</td><td>14</td><td>24</td><td>4</td><td>33</td><td>6</td><td>42</td><td>4</td><td>25</td></tr><tr><td>6. Conflict with colleagues</td><td>14</td><td>24</td><td>4</td><td>33</td><td>4</td><td>28</td><td>6</td><td>39</td></tr><tr><td>7. Conflict with supervisor</td><td>12</td><td>20</td><td>4</td><td>38</td><td>3</td><td>24</td><td>5</td><td>37</td></tr><tr><td>8. Lack of/or inappropriate inter-services/ administrative collaboration</td><td>11</td><td>19</td><td>1</td><td>11</td><td>3</td><td>28</td><td>7</td><td>60</td></tr><tr><td>9. Injustice, discrimination, harassment, bullying</td><td>4</td><td>7</td><td>3</td><td>78</td><td>1</td><td>22</td><td>0</td><td>0</td></tr><tr><td>10. Lack of value-based team concordance / tolerance</td><td>2</td><td>3</td><td>0</td><td>0</td><td>1</td><td>52</td><td>1</td><td>48</td></tr><tr><td colspan="2">TOTAL</td><td>134</td><td>33</td><td>36</td><td>31</td><td>43</td><td>32</td><td>55</td><td>37</td></tr><tr><td rowspan="10">Lack of control over work situations and career<br>31 scales with at least one subtype</td><td>1. Lack of participation to workplace and service-level policies</td><td>14</td><td>24</td><td>3</td><td>25</td><td>6</td><td>43</td><td>5</td><td>32</td></tr><tr><td>2. Lack of decision authority/autonomy (timing, method, etc.)</td><td>12</td><td>20</td><td>7</td><td>62</td><td>5</td><td>38</td><td>0</td><td>0</td></tr><tr><td>3. Lack of growth opportunities</td><td>7</td><td>12</td><td>5</td><td>75</td><td>1</td><td>13</td><td>1</td><td>12</td></tr><tr><td>4. Lack of tasks-related skills or preparation (incl. training, knowledge update)</td><td>7</td><td>12</td><td>1</td><td>17</td><td>4</td><td>57</td><td>2</td><td>26</td></tr><tr><td>5. Skill underutilization</td><td>6</td><td>10</td><td>6</td><td>100</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>6. Lack of task diversity/interest</td><td>4</td><td>7</td><td>3</td><td>80</td><td>0</td><td>0</td><td>1</td><td>20</td></tr><tr><td>7. Lack of a preceptor program for newly hired personnel</td><td>1</td><td>2</td><td>0</td><td>0</td><td>1</td><td>100</td><td>0</td><td>0</td></tr><tr><td>8. Lack of assertiveness in front of ethical concerns</td><td>1</td><td>2</td><td>0</td><td>0</td><td>1</td><td>100</td><td>0</td><td>0</td></tr><tr><td colspan="2">TOTAL</td><td>52</td><td>12</td><td>25</td><td>53</td><td>18</td><td>32</td><td>9</td><td>15</td></tr><tr><td rowspan="3">Lack of organizational resources<br>29 scales with at least one subtype</td><td>1. Lack of a predictable, stable, and recovery-propitious schedule</td><td>14</td><td>24</td><td>2</td><td>17</td><td>5</td><td>36</td><td>7</td><td>46</td></tr><tr><td>2. Lack of staffing</td><td>12</td><td>20</td><td>1</td><td>10</td><td>5</td><td>43</td><td>6</td><td>47</td></tr><tr><td>3. Lack of information (e.g., to ask patients' questions)</td><td>11</td><td>19</td><td>2</td><td>21</td><td>5</td><td>45</td><td>4</td><td>33</td></tr></tbody></table>

(Continued)
<table><caption>TABLE 2 | Continued</caption><thead><tr><th rowspan="2">Broad types of stressors</th><th rowspan="2">Subtypes of stressors</th><th colspan="2">All scales (N = 59)</th><th colspan="2">All settings scales (n = 17)</th><th colspan="2">Healthcare settings scales (n = 20)</th><th colspan="2">ICU settings scales (n = 22)</th></tr><tr><th>N</th><th>%</th><th>N</th><th>Corrected %<sup>a</sup></th><th>N</th><th>Corrected %<sup>a</sup></th><th>N</th><th>Corrected %<sup>a</sup></th></tr></thead><tbody><tr><td></td><td>4. Lack of/or low-quality or low accessibility to material resources</td><td>7</td><td>12</td><td>2</td><td>34</td><td>1</td><td>14</td><td>4</td><td>52</td></tr><tr><td></td><td>5. Lack of predictable and stable work relationships and place</td><td>5</td><td>8</td><td>1</td><td>23</td><td>3</td><td>59</td><td>1</td><td>18</td></tr><tr><td></td><td>6. Lack of task-role clarity</td><td>5</td><td>8</td><td>4</td><td>84</td><td>0</td><td>0</td><td>1</td><td>16</td></tr><tr><td></td><td>7. Lack of adequate rules and procedures</td><td>3</td><td>5</td><td>1</td><td>39</td><td>0</td><td>0</td><td>2</td><td>61</td></tr><tr><td></td><td>8. Lack of task meaning/utility</td><td>3</td><td>5</td><td>2</td><td>72</td><td>0</td><td>0</td><td>1</td><td>28</td></tr><tr><td></td><td>9. Lack of hierarchical role clarity</td><td>2</td><td>3</td><td>2</td><td>100</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td></td><td>TOTAL</td><td>62</td><td>14</td><td>17</td><td>32</td><td>19</td><td>30</td><td>26</td><td>38</td></tr><tr><td rowspan="6">Problematic situations with service users and relatives 20 scales with at least one subtype</td><td>1. Death and suffering and emotion regulation</td><td>13</td><td>22</td><td>2</td><td>19</td><td>4</td><td>31</td><td>7</td><td>50</td></tr><tr><td>2. Communicating with and fulfilling emotional needs of users or relatives</td><td>12</td><td>20</td><td>2</td><td>20</td><td>4</td><td>34</td><td>6</td><td>46</td></tr><tr><td>3. Conflict with inappropriate expectations or behaviors from users (customer, client, patient, etc.)</td><td>7</td><td>12</td><td>2</td><td>33</td><td>3</td><td>42</td><td>2</td><td>25</td></tr><tr><td>4. Conflict with inappropriate expectations or behaviors from users' relatives</td><td>6</td><td>10</td><td>0</td><td>0</td><td>1</td><td>18</td><td>5</td><td>82</td></tr><tr><td>5. Assault/aggression from users or relatives</td><td>2</td><td>3</td><td>0</td><td>0</td><td>2</td><td>100</td><td>0</td><td>0</td></tr><tr><td>6. Lack of users' recognition</td><td>1</td><td>2</td><td>0</td><td>0</td><td>1</td><td>100</td><td>0</td><td>0</td></tr><tr><td></td><td>TOTAL</td><td>41</td><td>10</td><td>6</td><td>18</td><td>15</td><td>37</td><td>20</td><td>45</td></tr><tr><td rowspan="6">Dealing with ethical and moral-related situations 19 scales with at least one subtype</td><td>1. Decisional dilemmas/uncertainty regarding patients' survival and end-of-life care</td><td>10</td><td>17</td><td>0</td><td>0</td><td>0</td><td>0</td><td>10</td><td>100</td></tr><tr><td>2. Observing deviations from safety standards</td><td>10</td><td>17</td><td>1</td><td>12</td><td>3</td><td>31</td><td>6</td><td>57</td></tr><tr><td>3. Unsuitability of care (Futility or over/under aggressiveness of therapeutics)</td><td>8</td><td>14</td><td>0</td><td>0</td><td>1</td><td>14</td><td>7</td><td>86</td></tr><tr><td>4. Working with incompetent/unexperienced/ negligent staff members</td><td>7</td><td>12</td><td>0</td><td>0</td><td>3</td><td>45</td><td>4</td><td>55</td></tr><tr><td>5. Ignoring patient's preferences and conditions</td><td>5</td><td>8</td><td>0</td><td>0</td><td>2</td><td>42</td><td>3</td><td>58</td></tr><tr><td>6. Unsafe orders/policies from hierarchies or person in charge</td><td>3</td><td>5</td><td>0</td><td>0</td><td>2</td><td>69</td><td>1</td><td>31</td></tr><tr><td></td><td>7. Lack of participation in end-of-life decisions/ or strong disagreement</td><td>2</td><td>3</td><td>0</td><td>0</td><td>0</td><td>0</td><td>2</td><td>100</td></tr><tr><td></td><td>8. Inaccurately informed patients and families</td><td>1</td><td>2</td><td>0</td><td>0</td><td>0</td><td>0</td><td>1</td><td>100</td></tr><tr><td></td><td>TOTAL</td><td>46</td><td>10</td><td>1</td><td>3</td><td>11</td><td>26</td><td>34</td><td>72</td></tr><tr><td rowspan="3">Risk management issues 14 scales with at least one subtype</td><td>1. Risk of making severe errors</td><td>9</td><td>15</td><td>1</td><td>13</td><td>3</td><td>34</td><td>5</td><td>52</td></tr><tr><td>2. Constant alert and sudden emergencies due to the patient's condition</td><td>6</td><td>10</td><td>0</td><td>0</td><td>2</td><td>35</td><td>4</td><td>65</td></tr><tr><td>3. Risky situations for oneself</td><td>5</td><td>8</td><td>3</td><td>66</td><td>0</td><td>0</td><td>2</td><td>34</td></tr><tr><td></td><td>TOTAL</td><td>20</td><td>4</td><td>4</td><td>24</td><td>5</td><td>25</td><td>11</td><td>51</td></tr><tr><td rowspan="3">Job disadvantages in comparison to other occupational situations 11 scales with at least one subtype</td><td>1. Lack of job security</td><td>7</td><td>12</td><td>5</td><td>75</td><td>1</td><td>13</td><td>1</td><td>12</td></tr><tr><td>2. Work-Home conflict (e.g., because of night shifts, on site call)</td><td>6</td><td>10</td><td>2</td><td>38</td><td>2</td><td>32</td><td>2</td><td>29</td></tr><tr><td>3. Insufficient remuneration</td><td>3</td><td>5</td><td>1</td><td>38</td><td>1</td><td>32</td><td>1</td><td>29</td></tr><tr><td></td><td>TOTAL</td><td>16</td><td>4</td><td>8</td><td>55</td><td>4</td><td>23</td><td>4</td><td>21</td></tr><tr><td rowspan="2">Other 3 scales with at least one subtype</td><td>1. Making time for research</td><td>2</td><td>3</td><td>0</td><td>0</td><td>0</td><td>0</td><td>2</td><td>100</td></tr><tr><td>2. Lack of pride/self-respect</td><td>1</td><td>2</td><td>1</td><td>100</td><td>0</td><td>0</td><td>0</td><td>0</td></tr></tbody></table>

<footnote><sup>a</sup>The correction accounts for the unequal numbers of scales of each type (17, 20, and 22) to allow a valid comparison of row percentages. The formula was (N scales of the target type that measured the stressor/Overall N scales that measured the stressor)/((N all settings scales that measured the stressor/17) + (N healthcare settings scales that measured the stressor/20) + (N ICU settings scales that measured the stressor/22)).</footnote>
<figure><img src="image_4.png" /><figcaption>**FIGURE 2** | Comparison of the three types of scales regarding their propensity to measure the eight broad types of stressors (% is the corrected proportion of scales of a type covering at least one stressor in the target category).</figcaption></figure>

(55%), and lack of control over work situations and career (53%) were more frequently measured by the all settings scales than by the other two scales (ranging from 15 to 32%). However, all settings scales were rarely used to measure risk management issues (24%) and problematic situations with service users and relatives (18%) and were hardly used to measure dealing with ethical- and moral-related situations (3%; conversely, ICU settings scales were used more frequently to measure these broad types of stressors (51, 45, and 72%, respectively). Healthcare settings scales were used to measure problematic situations with service users and relatives (37%) almost as often as ICU settings scales, but less frequently to measure risk management issues (25%) and dealing with ethical- and moral-related situations (26%). The three types of scales measured lack of organizational resources (30–38%) and

problematic relationships with other professionals (31–37%) to the same extent.

Beyond these eight broad types of stressors, there were a number of important differences concerning their subtypes. For instance, some stressors were more frequently measured by ICU scales than by the other two scales: decisional dilemmas/uncertainty regarding patients’ survival and end-of-life care (100%), unsuitability of care (futility or over/under aggressiveness of therapeutics) (86%), conflict and inappropriate expectations or behavior from relatives (82%), or constant alert and sudden emergencies due to the patient’s condition (65%).

In addition, in terms of item content, ICU settings scales and, to a lesser extent, healthcare settings scales generally refer more to precise and tailored features of the working environment of ICU professionals than all settings scales.
# Scales Most Frequently Used in Intensive Care Unit Studies and Comparison of Their Metrological Properties and Specificities

Approximately two thirds of the retrieved studies used at least one of the eight following scales (out of the 59 scales identified) (Table 1): 17 studies used the Job Content Questionnaire (JCQ) or one of its variants (Karasek et al., 1998; Piers et al., 2011); 15 studies used the Moral Distress Scale (MDS) or one of its variants (Corley et al., 2001; Hamric et al., 2012); eight studies used the Nursing Stress Scale (NSS) (Gray-Toft and Anderson, 1981); six studies used the Safety Attitudes Questionnaire-ICU version (SAQ-ICU) (Sexton et al., 2006); six studies used the Nursing Work Index (NWI) or one of its variants (Kramer and Hafner, 1989; Aiken and Patrician, 2000; Lake, 2002; Bonneterre et al., 2011); four studies used the Nurses Stress Index (NSI) (Harris, 1989); four studies used the scale of Embriaco et al. (2007a); and three studies used the Effort-Reward Imbalance (ERI) questionnaire (Siegrist et al., 2004). Other scales such as the Daily Hassles Questionnaire (DHQ) (Kanner et al., 1981), Workplace Stress Scale (WSS) (The Marlin Company and the American Institute of Stress, 2011), Hospital Survey Of Patient Safety Culture (HSOPSC) (Sorra and Dyer, 2010), Copenhagen Psychosocial Questionnaire V2 (COPSOQ2) (Pejtersen et al., 2010), ICU-related stressors questionnaire (Coomber et al., 2002), and the scales of Tummers et al. (2002) or Teixeira et al. (2013) were used in two studies each. Finally, the 44 other scales were used in one article each, including 10 all settings scales (four Type 1 and four Type 2 scales, Rizzo et al., 1970; Kanner et al., 1978; Cooper and Williams, 1991; Dhondt and Houtman, 1992; Jackson et al., 1993; Veldhoven and Meijman, 1994; Semmer et al., 1995; Shimomitsu et al., 2000; Stinglhamber and Vandenberghe, 2003; Hart, 2006), 15 healthcare settings scales (10 Type 1 scales, Hinshaw and Atwood, 1985; Revicki et al., 1991; Miranda et al., 1997; O'Brien-Pallas et al., 1997; Tyssen et al., 2001; Kitaoka-Higashiguchi and Nakagawa, 2003; Varjus et al., 2003; Olson, 2007; Einarsen et al., 2009; one Type 3, three Type 4, and two Type 5 scales), and 19 ICU settings scales (one Type 1, Bailey et al., 1980; five Type 3, six Type 4, and six Type 5 scales).

An analysis of the 59 scales reveals that no scale covered all the main stressors in ICUs (Table 2), not even those with satisfactory metrological properties. For instance, the JCQ, the most used across the world, is a Type 1 scale (ante hoc with satisfying validity-related statistics) which has primarily been used to measure the following types of stressors: lack of control over work situations and career, high job demands, and problematic relationships with other professionals. However, this scale does not cover stressors such as dealing with ethical- and moral-related situations, problematic situations with service users and relatives, and risk management issues and measures only one stressor in the broad category of the lack of organizational resources (note: the full 49-item JCQ also measures Lack of job security, a stressor associated with the disadvantages in comparison with other occupational situations category, but this version was not used in the retrieved ICU studies). Furthermore, four of the 17 studies retrieved combined it with a healthcare settings scale [NSS, NWI, Nine Equivalent of nursing Manpower use (NEMS)] or an

ICU settings scale such as the “Perceived ethical environment questionnaire” and the “Perceived inappropriateness of care questionnaire” developed by Piers et al. (2011).

The second most frequently used scale in ICU studies—the MDS—also presents certain limitations regarding its coverage of the different types of stressors. It is a Type 1 scale, but it is tailored to measure stressors relevant in many healthcare settings which focus primary on stressors related to *dealing with ethical- and moral-related situations*. It also measures stressors relating to the *lack of control over work situations and career*, as well as one stressor relating to *high job demands*. However, this scale does not measure stressors relating to the following categories: *problematic relationships with other professionals, problematic situations with service users and relatives, lack of organizational resources, risk management issues, and disadvantages in comparison to other occupational situations*. This led authors of two out of the 16 studies which used the MDS to combine it with either a healthcare settings scale (Varjus et al., 2003) or an *ad hoc* ICU-specific questionnaire (Hamric and Blackhall, 2007) to measure additional stressors.

Similar observations can be made with regard to the third most frequently used scale, i.e., the NSS. This Type 1 scale specifically targets nurses (i.e., it is unsuitable for other healthcare professions). It primarily covers stressors from four broad types: *high job demands, lack of organizational resources, problematic relationships with other professionals, and problematic situations with service users and relatives*. However, it measures one stressor from the *lack of control over work situations and career* category to a limited extent and none from the categories *dealing with ethical- and moral-related situations, risk management issues, and disadvantages in comparison with other occupational situations*.

The same comments were made for all other scales. No scale covered the eight broad types of stressors in a comprehensive manner, and many of them were problematic in terms of validity/reliability. It is worth mentioning that 14 articles analyzed combined two or three scales to increase stressor coverage.

# DISCUSSION

From the 102 studies analyzed, we identified 59 different scales. Only 28 out of the 59 scales were validated (Type 1), and two ICU settings scales out of 22 were validated. Our review of the literature highlights the wide variability across scales used to identify stressors in the ICU with regard to their level of generality/specificity (scales for all types of professional contexts or scales targeting healthcare contexts or more specifically ICU professionals), their psychometric qualities (five levels of validity/reliability), and the type of stressors covered by scales.

This variability sheds light on the constraints that appear to guide investigators' methodological choices. The advantage of using a generic scale validated internationally is that authors are able to carry out epidemiological studies allowing interprofessional and international comparisons (El Khamali et al., 2018). However, generic scales, primarily used in studies, appear ill adapted to measure the stressors more specific to the professional activity in ICUs. Indeed, the stressors in the categories *Dealing with ethical- and moral-related situations* and
*Risk management issues*, represented primarily in specific scales, did not appear or were relatively few within the generic and healthcare scales.

For instance, the scale most commonly used by the authors was the JCQ scale, despite the fact that no factor related to *dealing with ethical- and moral-related situations* or *risk management issues* was covered. Many authors, however, raised the question of the importance of the issues related to risk management and patient safety in relation to high-tech care and the severity of the pathology (Aiken and Patrician, 2000; Adriaenssens et al., 2015). The difficulties associated with ethical dimensions have also been extensively studied in the ICU context in relation to end-of-life situations (Laurent et al., 2017). The SAQ-ICU scale combines interesting criteria to identify stress factors in ICU settings. The scale has been validated and covers factors relative to the *problematic relationships with other professionals* and *lack of organizational resources* categories. However, factors relating to stressors such as *job disadvantages, problematic situations with users and relatives, high job demands*, and *risk management issues* are absent, despite the fact that the latter two factors of stress have been widely reported by ICU professionals (Pastores et al., 2019).

Thus, investigators who chose to use a scale reflecting these more specific stressors of ICUs were forced to develop their own tools whose psychometric qualities were yet to be tried and tested (tools based on literature reviews, interviews with professionals, or items from different scales). Beyond the measurement level, these observations highlight the crucial limitation of the uncritical use of general theories of occupational stress (e.g., the job strain model of Karasek, the effort-reward imbalance model of Siegrist) to analyze job stressors in ICUs. Indeed, as advocated by a number of other scholars (Borteyrou et al., 2014), these theories/models of stress must be contextualized to enhance their ecological validity as they only account for generic stressors in professional settings and not the specific ones.

Our systematic review shows that a number of factors are absent from all the scales, for instance, diagnostic/admission decisions, the training and supervision of students, the lack of space related to rooms for break or family discussions (Blanch et al., 2016; Trevick et al., 2016; Pastores et al., 2019). We noted that the healthcare and ICU scales targeted either all healthcare professionals or nurses and that there was no scale developed specifically for physicians.

It is worth nothing that some scales allow the assessment of the effects on outcomes (e.g., job burnout) of the interaction between some types of stressors. This is the case of the JCQ and its variants which can estimate whether resources like job control (decision latitude) and social support (from colleagues and supervisors) moderate the effect of job demands. This is also the case of the ERI scale, which is based on the postulate that the imbalance (interaction) between the rewards obtained in exchange of job efforts is critical to explain outcomes, more than their isolated main effects. However, accounting for these interaction effects generally did not increase the explained variance of outcomes in empirical studies (e.g., Brough and Biggs, 2015; Gorgievski et al., 2019).

This study has a number of limitations. Several studies, and therefore stress scales, may not have been considered in our review if the objective of identifying ICU stressors was not clearly

reflected in the article title, keywords, or abstract. In addition, we have focused mainly on studies published in English. Thus, scales developed at a more local level may have been excluded from our review, and a more extensive search seems warranted to cover more the important issue of international, intercultural, and health-system comparisons in terms of stressor identification and measurement.

## CONCLUSION

Our review of the literature clearly raises the question of the relevance of the scales used in studies measuring stressors in intensive care settings. Indeed, no available tool meets both the criteria of metrological validity and of ecological validity (i.e., covering all relevant stressors in ICUs, particularly those that are the most specific to them). Thus, researchers and practicians currently face a methodological dilemma, as they are forced to make a choice between the two or to make some unfortunate “bricolage,” such as *ad hoc* elaboration, removal, or modification of items or the combination of different scales.

There is an urgent need to propose a validated tool capable of taking into account the whole professional reality of ICU settings. This tool would make it possible to compare the respective impacts of generic vs. specific stressors in the etiology of outcomes (e.g., burnout, job satisfaction, turnover intentions). A better identification of stress factors should make it possible to define a more appropriate care policy in particular to prevent burnout and its associated effects such as depression, suicidal tendencies, addictive behaviors, and physical impairment (e.g., Lheureux et al., 2016; Vandeval et al., 2017). Finally, the more comprehensive and thus more able to approximate the reality of the activities undertaken by professionals the tool is, the easier it will be to use the factors identified to implement effective training needs and target the necessary prevention and support measures.

## DATA AVAILABILITY STATEMENT

All datasets generated for this study are included in the article/Supplementary Material.

## AUTHOR CONTRIBUTIONS

AL: abstracts screened for eligibility, screened full text sectioned, examination in detail of each scale, examination of typology, and writing of the article. FL: abstracts screened for eligibility, examination in detail of each scale, and writing of the article. MG: initial search and examination in detail of each scale. MM: screened full text sectioned and proofreading. MB: screened full text sectioned and proofreading. AP: screened full text sectioned and proofreading. GB: examination of typology and proofreading. GC: examination of typology and proofreading.

## SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg.2020.00245/full#supplementary-material
REFERENCES

Adriaenssens, J., De Gucht, V., and Maes, S. (2015). Determinants and prevalence of burnout in emergency nurses: a systematic review of 25 years of research. *Int. J. Nurs. Stud.* 52, 649–661. doi: 10.1016/j.ijnurstu.2014.11.004

Aiken, L. H., and Patrician, P. A. (2000). Measuring organizational traits of hospitals: the revised nursing work index. *Nurs. Res.* 49, 146–153. doi: 10.1097/00006199-200005000-00006

Bailey, J. T., Steffen, S. M., and Grout, J. W. (1980). The stress audit: identifying the stressors of ICU nursing. *J. Nurs. Educ.* 19, 15–25.

Blanch, L., Abillama, F. F., Amin, P., Christian, M., Joynt, G. M., Myburgh, J., et al. (2016). Triage decisions for ICU admission: Report from the Task Force of the World Federation of Societies of Intensive and Critical Care Medicine. *J. Crit. Care* 36, 301–305. doi: 10.1016/j.jcrc.2016.06.014

Bonneterre, V., Ehlinger, V., Balducci, F., Caroly, S., Jolivet, A., Sobaszek, A., et al. (2011). Validation of an instrument for measuring psychosocial and organisational work constraints detrimental to health among hospital workers: The NWI-EO questionnaire. *Int. J. Nurs. Stud.* 48, 557–567. doi: 10.1016/j.ijnurstu.2010.09.006

Bonneterre, V., Liaudy, S., Chatellier, G., Lang, T., and de Gaudemaris, R. (2008). Reliability, validity, and health issues arising from questionnaires used to measure psychosocial and organizational work factors (POWFs) among hospital nurses: a critical review. *J. Nurs. Meas.* 16, 207–230. doi: 10.1891/1061-3749.16.3.207

Borteyrou, X., Truchot, D., and Rascle, N. (2014). Development and validation of the work stressor inventory for nurses in oncology: preliminary findings. *J. Adv. Nurs.* 70, 443–453. doi: 10.1111/jan.12231

Brewer, M. B., and Crano, W. D. (2014). "Research design and issues of validity," in *Handbook of Research Methods in Social and Personality Psychology, 2nd edn.*, eds H. T. Reis and C. M. Judd (New York, NY: Cambridge University Press), 11–26. doi: 10.1017/CBO9780511996481.005

Brough, P., and Biggs, A. (2015). Job demands × job control interaction effects: do occupation-specific job demands increase their occurrence? *Stress Health* 31, 138–149. doi: 10.1002/smi.2537

Chuang, C. H., Tseng, P. C., Lin, C. Y., Lin, K. H., and Chen, Y. Y. (2016). Burnout in the intensive care unit professionals. *Medicine* 95:e5629. doi: 10.1097/MD.0000000000005629

Coomber, S., Todd, C., Park, G., Baxter, P., Firth-Cozens, J., and Shore, S. (2002). Stress in UK intensive care unit doctors. *Br. J. Anaesth.* 89, 873–881. doi: 10.1093/bja/aef273

Cooper, C. L., and Williams, J. (1991). A validation study of the OSI on a blue-collar sample. *Stress Med.* 7, 109–112. doi: 10.1002/smi.2460070210

Corley, M. C., Elswick, R. K., Gorman, M., and Clor, T. (2001). Development and evaluation of a moral distress scale. *J. Adv. Nurs.* doi: 10.1037/t37284-000

de Câssia Pereira Fernandes, R., da Silva Pataro, S. M., de Carvalho, R. B., and Burdorf, A. (2016). The concurrence of musculoskeletal pain and associated work-related factors: a cross sectional study. *BMC Pub. Health* 16:628. doi: 10.1186/s12889-016-3306-4

Dhondt, S., and Houtman, I. (1992). *NIPG Onderzoeksvragenlijst Arbeidsinhoud: Constructie en Eerste Toets op Betrouwbaarheid en Validiteit.* Leiden: TNO, rapportnummer 92.088.

Donchin, Y., and Seagull, F. J. (2002). The hostile environment of the intensive care unit. *Curr. Opin. Crit. Care* 8, 316–320. doi: 10.1097/00075198-200208000-00008

Dragano, N., Siegrist, J., Nyberg, S. T., Lunau, T., Fransson, E. I., Alfredsson, L., et al. (2017). Effort-reward imbalance at work and incident coronary heart disease: a multicohort study of 90,164 individuals. *Epidemiol. Camb. Mass* 28, 619–626. doi: 10.1097/EDE.0000000000000666

Dunser, M. W., Baelani, I., and Ganbold, L. (2006). A review and analysis of intensive care medicine in the least developed countries. *Crit. Care Med.* 34, 1234–1242. doi: 10.1097/01.CCM.0000208360.70835.87

Einarsen, S., Hoel, H., and Notelaers, G. (2009). Measuring exposure to bullying and harassment at work: Validity, factor structure and psychometric properties of the Negative Acts Questionnaire-Revised. *Work Stress* 23, 24–44. doi: 10.1080/02678370902815673

El Khamali, R., Mouaci, A., Valera, S., Cano-Chervel, M., Pinglis, C., Sanz, C., et al. (2018). Effects of a multimodal program including simulation on job

strain among nurses working in intensive care units: a randomized clinical trial. *JAMA* 320, 1988–1997. doi: 10.1001/jama.2018.14284

Embriaco, N., Azoulay, E., Barrau, K., Kentish, N., Pochard, F., Loundou, A., et al. (2007a). High level of burnout in intensivists: prevalence and associated factors. *Am. J. Respir. Crit. Care Med.* 175, 686–692. doi: 10.1164/rccm.200608-11840C

Embriaco, N., Papazian, L., Kentish-Barnes, N., Pochard, F., and Azoulay, E. (2007b). Burnout syndrome among critical care healthcare workers. *Curr. Opin. Crit. Care* 13, 482–488. doi: 10.1097/MCC.0b013e3282efd28a

Fassier, T., and Azoulay, E. (2010). Conflicts and communication gaps in the intensive care unit. *Curr. Opin. Crit. Care* 16, 654–665. doi: 10.1097/MCC.0b013e32834044f0

Flannery, L., Ramjan, L. M., and Peters, K. (2016). End-of-life decisions in the Intensive Care Unit (ICU) – Exploring the experiences of ICU nurses and doctors – A critical literature review. *Aust. Crit. Care* 29, 97–103. doi: 10.1016/j.aucc.2015.07.004

Furr, R. M. (2011). *Scale Construction and Psychometrics for Social and Personality Psychology*. London: SAGE Publications Ltd. doi: 10.4135/9781446287866

Garrouste-Orgeas, M., Perrin, M., Soufir, L., Vesin, A., Blot, F., Maxime, V., et al (2015). The Iatroref study: medical errors are associated with symptoms of depression in ICU staff but not burnout or safety culture. *Intens. Care Med.* 41, 273–284. doi: 10.1007/s00134-014-3601-4

Gorgievski, M. J., Van der Heijden, B. I. J. M., and Bakker, A. B. (2019). Effort-reward imbalance and work-home interference: a two-wave study among european male nurses. *Work Stress.* 33, 315–333. doi: 10.1080/02678373.2018.1503358

Gray-Toft, P., and Anderson, J. G. (1981). The nursing stress scale: development of an instrument. *J. Behav. Assess.* 3, 11–23. doi: 10.1007/BF01321348

Hamric, A. B., and Blackhall, L. J. (2007). Nurse-physician perspectives on the care of dying patients in intensive care units: collaboration, moral distress, and ethical climate. *Crit. Care Med.* 35, 422–429. doi: 10.1097/01.CCM.0000254722.50608.2D

Hamric, A. B., Borchers, C. T., and Epstein, E. G. (2012). Development and testing of an instrument to measure moral distress in healthcare professionals. *AJOB Prim. Res.* 3, 1–9. doi: 10.1080/21507716.2011.652337

Harris, P. E. (1989). The nurse stress index. *Work Stress* 3, 335–346. doi: 10.1080/02678378908256952

Hart, S. G. (2006). NASA-task load index (NASA-TLX); 20 years later. *Proc. Hum. Factors Ergon. Soc. Annu. Meet.* 50, 904–908. doi: 10.1177/1541931206050000909

Hinshaw, A. S., and Atwood, J. R. (1985). *Anticipated Turnover Among Nursing Staff Study: Final Report (R01NU00908)*. Bethesda, MD: National Institutes of Health, National Center for Nursing Research.

Jackson, P. R., Wall, T. D., Martin, R., and Davids, K. (1993). New measures of job control, cognitive demand, and production responsibility. *J. Appl. Psychol.* 78, 753–762. doi: 10.1037/0021-9010.78.5.753

Kanner, A. D., Coyne, J. C., Schaefer, C., and Lazarus, R. S. (1981). Comparison of two modes of stress measurement: daily hassles and uplifts versus major life events. *J. Behav. Med.* 4, 1–39. doi: 10.1007/BF00844845

Kanner, A. D., Kafry, D., and Pines, A. (1978). Conspicuous in its absence: the lack of positive conditions as a source of stress. *J. Human Stress* 4, 33–39. doi: 10.1080/0097840X.1978.10545984

Karanikola, M., Giannakopoulou, M., Mpouzika, M., Kaite, C. P., Tsiaousis, G. Z., and Papathanassoglou, E. D. E. (2015). Dysfunctional psychological responses among Intensive Care Unit nurses: a systematic review of the literature. *Rev. Esc. Enferm. USP* 49, 847–857. doi: 10.1590/S0080-623420150000 500020

Karasek, R., Brisson, C., Houtman, I., Bongers, P., and Amick, B. (1998). The Job Content Questionnaire (JCQ): an instrument for internationally comparative assessments of psychosocial job characteristics. *J. Occup. Health Psychol.* 3, 322–355. doi: 10.1037/1076-8998.3.4.322

Kitaoka-Higashiguchi, K., and Nakagawa, H. (2003). Job strain, coping, and burnout among Japanese nurses. *Jpn. J. Health Hum. Ecol.* 69, 66–79. doi: 10.3861/jshhe.69.66

Kramer, M., and Hafner, L. P. (1989). Shared values: impact on staff nurse job satisfaction and perceived productivity. *Nurs. Res.* 38, 172–177. doi: 10.1097/00006199-198905000-00019

Krämer, T., Schneider, A., Spieß, E., Angerer, P., and Weigl, M. (2016). Associations between job demands, work-related strain and perceived quality
of care: a longitudinal study among hospital physicians. *Int. J. Qual. Health Care J. Int. Soc. Qual. Health Care* 28, 824–829. doi: 10.1093/intqhc/mzw119

Lake, E. T. (2002). Development of the practice environment scale of the nursing work index. *Res. Nurs. Health* 25, 176–188. doi: 10.1002/nur.10032

Laurent, A., Bonnet, M., Capellier, G., Aslanian, P., and Hebert, P. (2017). Emotional impact of end-of-life decisions on professional relationships in the ICU: an obstacle to collegiality? *Crit. Care Med.* 45, 2023–2030. doi: 10.1097/CCM.0000000000002710

Lazarus, R. S., and Folkman, S. (1984). *Stress, Appraisal, and Coping*. New-York, NY: Springer Publishing Company.

Lheureux, F., Truchot, D., and Borteyrou, X. (2016). Suicidal tendency, physical health problems and addictive behaviors among general practitioners: the effect of burnout. *Work Stress*. 30, 173–192. doi: 10.1080/02678373.2016.1171806

Miranda, D. R., Moreno, R., and Iapichino, G. (1997). Nine equivalents of nursing manpower use score (NEMS). *Intensive Care Med*. 23, 760–765. doi: 10.1007/s001340050406

O'Brien-Pallas, L., Irvine, D., Peereboom, E., and Murray, M. (1997). Measuring nursing workload: understanding the variability. *Nurs. Econ*. 15, 171–182.

Olson, L. L. (2007). Hospital nurses' perceptions of the ethical climate of their work setting. *Image J. Nurs. Sch.* 30, 345–349. doi: 10.1111/j.1547-5069.1998.tb01331.x

Ozden, D., Karagozoglu, S., and Yildirim, G. (2013). Intensive care nurses' perception of futility: Job satisfaction and burnout dimensions. *Nurs. Ethics* 20, 436–447. doi: 10.1177/0969733012466002

Pastores, S. M., Kvetan, V., Coopersmith, C. M., Farmer, J. C., Sessler, C., Christman, J. W., et al. (2019). Workforce, workload, and burnout among intensivists and advanced practice providers: a narrative review. *Crit. Care Med.* 1:3637. doi: 10.1097/CCM.0000000000003637

Pejtersen, J. H., Kristensen, T. S., Borg, V., and Bjorner, J. B. (2010). The second version of the Copenhagen Psychosocial Questionnaire. *Scand. J. Public Health.* 38(3 Suppl), 8–24. doi: 10.1177/140349809349858

Peter, J. P., and Churchill, G. A. (1986). Relationships among research design choices and psychometric properties of rating scales: a meta-analysis. *J. Mark. Res.* 23, 1–10. doi: 10.1177/002224378602300101

Piers, R. D., Azoulay, E., Ricou, B., Dekeyser Ganz, F., Decruyenaere, J., Max, A., et al. (2011). Perceptions of appropriateness of care among European and Israeli intensive care unit nurses and physicians. *JAMA* 306, 2694–2703. doi: 10.1001/jama.2011.1888

Profit, J., Sharek, P. J., Abspoker, A. B., Kowalkowski, M. A., Nisbet, C. C., Thomas, E. J., et al. (2014). Burnout in the NICU setting and its relation to safety culture. *BMJ Qual. Saf.* 23, 806–813. doi: 10.1136/bmjqs-2014-002831

Revicki, D. A., May, H. J., and Whitley, T. W. (1991). Reliability and validity of the work-related strain inventory among health professionals. *Behav. Med.* 17, 111–120. doi: 10.1080/08964289.1991.9937554

Rizzo, J. R., House, R. J., and Lirtzman, S. I. (1970). Role conflict and ambiguity in complex organizations. *Adm. Sci.* Q. 15:150. doi: 10.2307/2391486

Schmuckler, M. A. (2001). What is ecological validity? A dimensional analysis. *Infancy* 2, 419–436. doi: 10.1207/S15327078IN0204_02

Semmer, N., Zapf, D., and Dunckel, H. (1995). "Assessing stress at work: a framework and an instrument," in *Work and Health: Scientific Basis of Progress in the Working Environment*, eds. O. Svane and C. Johansen [Bruxelles, BE: Directorate-General for Employment, Social Affairs and Inclusion (European Commission), Directorate-General for Research and Innovation (European Commission)], 105–113.

Sexton, J. B., Helmreich, R. L., Neilands, T. B., Rowan, K., Vella, K., Boyden, J., et al. (2006). The Safety Attitudes Questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Serv. Res. 6:44. doi: 10.1186/1472-6963-6-44

Shimomitsu, T., Yokoyama, K., Ohno, H., Maruta, T., and Tanigawa, T. (2000). [Manual of the Brief Job Stress Questionnaire. Reports on the Study of Job Stress and its Effects on Health in the Workplace: The Research Grant for the Prevention of Work-Related Diseases from the Japan Ministry of Labor]. Tokyo: The Ministry of Labor (2000), 17–27 (in Japanese).

Siegrist, J., Starke, D., Chandola, T., Godin, I., Marmot, M., Niedhammer, I., et al. (2004). The measurement of effort-reward imbalance at work: European comparisons. *Soc. Sci. Med.* 58, 1483–1499. doi: 10.1016/S0277-9536(03)00351-4

Sochalski, J. (2004). Is more better?: the relationship between nurse staffing and the quality of nursing care in hospitals. *Med. Care* 42, II67–II73. doi: 10.1097/01.mlr.0000109127.76128.aa

Sorra, J. S., and Dyer, N. (2010). Multilevel psychometric properties of the AHRQ hospital survey on patient safety culture. BMC Health Serv. Res. 10:199. doi: 10.1186/1472-6963-10-199

Stinglhamber, F., and Vandenberghe, C. (2003). Organizations and supervisors as sources of support and targets of commitment: a longitudinal study. *J. Organ. Behav.* 24, 251–270. doi: 10.1002/job.192

Teixeira, C., Ribeiro, O., Fonseca, A. M., and Carvalho, A. S. (2013). Burnout in intensive care units - a consideration of the possible prevalence and frequency of new risk factors: a descriptive correlational multicentre study. BMC Anesthesiol. 13:38. doi: 10.1186/1471-2253-13-38

Teixeira, C., Ribeiro, O., Fonseca, A. M., and Carvalho, A. S. (2014). Ethical decision making in intensive care units: a burnout risk factor? Results from a multicentre study conducted with physicians and nurses. *J. Med. Ethics* 40, 97–103. doi: 10.1136/medethics-2012-100619

The Marlin Company and the American Institute of Stress (2011). *Attitudes in the American Workplace VII. The Seventh Annual Labor Day Survey*. Available online at: https://www.stress.org/wp-content/uploads/2011/08/2001Attitude-in-the-Workplace-Harris.pdf

Trevick, S., Kim, M., and Naidech, A. (2016). Communication, leadership, and decision-making in the neuro-ICU. *Curr. Neurol. Neurosci. Rep.* 16:99. doi: 10.1007/s11910-016-0699-5

Tummers, G. E. R., van Merode, G. G., and Landeweerd, J. A. (2002). The diversity of work: differences, similarities and relationships concerning characteristics of the organisation, the work and psychological work reactions in intensive care and non-intensive care nursing. *Int. J. Nurs. Stud.* 39, 841–855. doi: 10.1016/S0020-7489(02)00020-2

Tyssen, R., Vaglum, P., Grønvold, N. T., and Ekeberg, Ø. (2001). The impact of job stress and working conditions on mental health problems among junior house officers. A nationwide Norwegian prospective cohort study. *Med. Educ.* 34, 374–384. doi: 10.1046/j.1365-2923.2000.00540.x

Van Mol, M. M. C., Kompanje, E. J. O., Benoit, D. D., Bakker, J., Nijkamp, M. D., and Seedat, S. (2015). The prevalence of compassion fatigue and burnout among healthcare professionals in intensive care units: a systematic review. PLoS ONE 10:e0136955. doi: 10.1371/journal.pone.0136955

Vandevala, T., Pavey, L., Chelidoni, O., Chang, N. F., Creagh-Brown, B., and Cox, A. (2017). Psychological rumination and recovery from work in intensive care professionals: associations with stress, burnout, depression and health. *J. Intensive Care* 5:16. doi: 10.1186/s40560-017-0209-0

Varjus, S. L., Suominen, T., and Leino-Kilpi, H. (2003). Autonomy among intensive care nurses in Finland. *Intensive Crit. Care Nurs.* 19, 31–40. doi: 10.1016/S0964-3397(03)00007-7

Veldhoven, M. V., and Meijman, T. (1994). *Het Meten van Psychosociale Arbeidsbelasting Met een Vragenlijst: de Vragenlijs: TNO Repository*. Amsterdam: Nederlands Instituut voor Arbeidsomstandigheden (NIA).

**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2020 Laurent, Lheureux, Genet, Martin Delgado, Bocci, Prestifilippo, Besch and Capellier. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.