<figure><img src="image_1.png" /></figure>

HAL
open science

# Crisis management in the operating room: A systematic review of simulation training to develop non-technical skills

Inas D Redjem, Arnaud Huaulmé, Pierre Jannin, Estelle Michinov

► To cite this version:

Inas D Redjem, Arnaud Huaulmé, Pierre Jannin, Estelle Michinov. Crisis management in the operating room: A systematic review of simulation training to develop non-technical skills. Nurse Education Today, 2025, 147, pp.106583. 10.1016/j.nedt.2025.106583. hal-04920266

HAL Id: hal-04920266

https://hal.science/hal-04920266v1

Submitted on 30 Jan 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_3.png" /></figure>

Review

# Crisis management in the operating room: A systematic review of simulation training to develop non-technical skills

<figure><img src="image_4.png" /></figure>

Inas D. Redjem<sup>a,b,*</sup>, Arnaud Huaulmé<sup>b</sup>, Pierre Jannin<sup>b</sup>, Estelle Michinov<sup>a</sup>

ARTICLE INFO

Keywords:

Simulation

Healthcare training

Operating room

Non-technical skills

Crew resource management

Systematic review

ABSTRACT

**Aims:** Crises in the operating room, often resulting from human factors, endangers patient safety. Simulation-based training to develop non-technical skills shows promise in managing these crises. This review examines the simulation techniques, targeted healthcare professionals, non-technical skills, crisis scenarios, and evaluation metrics used in operating room crisis management training.

**Design:** Systematic review.

**Data sources:** MEDLINE, APA PsycInfo and Web of Science databases were searched for peer-reviewed articles published between January 2004 and March 2024.

**Review methods:** This systematic review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. The review protocol has been registered on the Open Science Framework (OSF) (https://osf.io/7bsc8). The inclusion criteria were as follows: (1) The study population comprised healthcare and medical professionals or students; (2) the intervention involved a simulated learning or training experience; (3) the outcomes focused on non-technical skills or crew resource management; (4) the training setting was the operating room (simulated or real); and (5) the learning scenarios depicted a crisis or an adverse event.

**Results:** This systematic review identified 29 eligible articles. The findings highlight the predominance of high-fidelity simulations, primarily targeting medical staff rather than nurses or other healthcare professionals. Training focused on communication, teamwork, situation awareness, problem solving, and decision making, with scenarios mostly addressing patient deterioration. Assessments reached up to Kirkpatrick's Level 3, demonstrating a positive training impact through learners' reactions and learning metrics rather than behavior and organizational results.

**Conclusions:** Despite their effectiveness, current training practices exhibit limitations. Incorporating nurses and other paramedical staff in interprofessional training, as well as emphasizing team-related scenarios and evaluating behavioral changes in practice, could enhance training effectiveness. This has implications for interprofessional healthcare education and skills transfer to real-world settings, ultimately improving patient safety.

# 1. Introduction

Adverse events during hospitalization can affect patients' health (Panagioti et al., 2019). Defined as unexpected incidents resulting from medical management, perioperative adverse events include equipment or communication failures, as well as procedural mistakes occurring during surgical procedures (Jung et al., 2019). A large portion of adverse events are considered preventable because they result from human factors (Gordo et al., 2021; Smits et al., 2021).

A significant portion of the errors leading to these “Never Events” can

be attributed to a lack of non-technical skills (Allard et al., 2020; Koleva, 2020), defined as “cognitive, social and personal resource skills that complement technical skills and contribute to safe and efficient task performance” (Flin et al., 2008). Non-technical skills are generally grouped into two categories: social skills (communication, teamwork, leadership, and coordination) and cognitive skills (situation awareness and decision-making; Allard et al., 2020). It is well established that a lack of non-technical skills can delay the response to critical medical situations, leading to adverse events (Yucel et al., 2020).

Therefore, crew resource management – a set of principles dealing

<footnote><sup>a</sup> Univ Rennes, LP3C (Laboratoire de Psychologie: Cognition, Comportement, Communication), F-35000 Rennes, France</footnote>

<footnote><sup>b</sup> Univ Rennes, INSERM, LTSI (Laboratoire du Traitement du Signal et de l'Image) - UMR 1099, F-35000 Rennes, France</footnote>

<footnote>* Corresponding author at: Université Rennes 2, Département de Psychologie, Place du Recteur Henri le Moal, 35043 Rennes Cedex, France.
E-mail addresses: inas.redjem@univ-rennes2.fr (I.D. Redjem), estelle.michinov@univ-rennes2.fr (E. Michinov).</footnote>
with cognitive and interpersonal skills that contribute to optimal team performance (Buljac-Samardzic et al., 2020) – has played a crucial role in healthcare education since its introduction to the curricula in the 1990s (Howard et al., 1992). Crew resource management provides contextualized training in non-technical skills, with the objective of providing predefined responses to critical incidents and guidance on the coordinated use of all available resources to maximize safe patient outcomes (Buljac-Samardzic et al., 2020). Recent systematic reviews have shown that simulation-based training is the most common method for improving team effectiveness and non-technical skills (Buljac-Samardzic et al., 2020; Ounounou et al., 2019). Both methodologies allow healthcare teams to develop these skills in realistic scenarios without risking patient safety.

Existing systematic reviews examining simulation-based training for developing non-technical skills in crisis management are limited and largely outdated (Fung et al., 2015; Merién et al., 2010). The most recent reviews are narrow in scope, primarily focusing on emergency departments (Alsbri et al., 2022; Brogaard et al., 2022; Thim et al., 2022; Yucel et al., 2020). There is a noticeable lack of comprehensive coverage on crisis management in the operating room, with only a few reviews addressing team training (Brogaard et al., 2022; Cumin et al., 2013) and non-technical skills in surgery (Dedy et al., 2013; Robertson et al., 2017), providing limited details on training methodologies and skills measurements.

This systematic review seeks to fill the gaps in the existing literature by comprehensively summarizing the research on simulation training on crisis management in the operating room aimed at healthcare professionals and students. The research questions (RQs) are:

RQ1. In what research contexts have simulations for managing crises in operating rooms been implemented?

RQ2. What simulation methodologies have been used and how are these simulations organized?

RQ3. What specific scenarios of crisis management and skills are assessed within simulations?

RQ4. What are the evaluation levels according to Kirkpatrick's model (Kirkpatrick and Kirkpatrick, 2006)?

## 2. Method

### 2.1. Registration and reporting guidelines

This systematic review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (Page et al., 2021). The review protocol has been registered on OSF (https://osf.io/7bsc8). The PRISMA checklist is provided in Supplementary Material 1.

### 2.2. Data source and search strategy

The Population, Intervention, Comparison and Outcome (PICO) framework (Richardson et al., 1995) was used to structure a clear review question. The population was defined as healthcare and medical professionals or students; the intervention as a simulated learning or training experience; the outcomes as non-technical skills or crew resource management; and the setting as the operating room.

The following comprehensive research equation was developed: (health* OR nursing professional* OR nursing* OR surgeon* OR nursing student* OR medical student* OR “clinical”) AND (“simulation” OR “training” OR “simulation-based training” OR “crew resource management” OR “CRM” OR “crisis management training” OR “virtual reality” OR immersive environment* OR “immersive virtual reality” OR “VR” OR “HMD” OR immer* OR patient safety training) AND (“operating room” OR “operating theater” OR “surgery”) AND (human factor* OR soft skill* OR non-technical skill* OR “decision making” OR “stress management” OR “leadership” OR “communication” OR team* OR “risk

management” OR “crisis management” OR “crisis” OR critical event* OR adverse event* OR “CRM” OR medical error*). Full search strategies are provided in Supplementary Material 2.

Three databases covering the fields of interest (healthcare, simulation, and non-technical skills) were queried: Medline, Web of Science and APA PsycInfo via the EBSCO platform. Web of Science provides an interdisciplinary perspective, encompassing research in simulation and education. APA PsycInfo focuses on behavioral sciences and psychology, which are key to studying non-technical skills, and Medline is a recognized reference in healthcare. This combination helps reduce publication bias by offering a diverse range of relevant studies. The availability of these databases through institutional access ensures the consistent use of the same Boolean search function across all three. The final data extraction was performed on March 6, 2024.

### 2.3. Eligibility criteria

Reviews, case reports, opinion pieces, viewpoints, conceptual frameworks, and conference abstracts were excluded. Empirical peer-reviewed original works written in English and published between January 2004 and March 2024 were included. This time limit was selected as it corresponds to the establishment of the Society for Simulation in Healthcare. The inclusion criteria were as follows: (1) The study population comprised healthcare and medical professionals or students; (2) the intervention involved a simulated learning or training experience; (3) the outcomes focused on non-technical skills or crew resource management; (4) the training setting was the operating room (simulated or real); and (5) the learning scenarios depicted a crisis or an adverse event.

### 2.4. Data extraction and analysis

A data extraction sheet was developed to compile the selected articles and their key characteristics. The initial screening of articles was conducted by the first author (I.R.), who extracted details such as authors, publication year, country, study design, participant occupation, type of simulation, study objectives, outcomes, and results. For study objectives, outcomes, and results, only data related to non-technical skills were included. Relevant information pertaining to the four research questions was also collected. To address the research questions, the results of the systematic review were synthesized based on the research context, simulation methodology and organization, crisis simulation scenarios and skills measured, as well as the levels of evaluation according to Kirkpatrick's model (Kirkpatrick and Kirkpatrick, 2006). The data extraction tables for information pertaining to the four research questions are provided in Supplementary Material 3.

### 2.5. Study selection

The study selection was performed according to PRISMA guidelines. A total of 741 articles were initially retrieved, including 301 duplicates that were later removed. The titles and abstracts of the articles were first screened by I.R. in order to exclude those that did not meet all the eligibility criteria. Out of the 440 articles, 388 were excluded based on title or abstract as they did not meet the inclusion criteria. Of the remaining 52 articles, full texts for three were unavailable, leaving 49 articles for the full-text analysis.

Full-text screening was independently performed by two authors (I.R and E.M), who agreed on the inclusion of 23 articles and the exclusion of 20 articles. Disagreement arose over six articles (Cohen K = 0.76) centered around the definition of a crisis (Bracq et al., 2021; Burkhart et al., 2013; Gardner and AbdelFattah, 2017; Raison et al., 2018; Ramjeeawon et al., 2020; Sirihorachai et al., 2018). It was ultimately agreed that for a simulation scenario to qualify as crisis situation training, it must include stressors leading to adverse events or depict a perioperative adverse event.
Following this consensus, the authors re-evaluated the six disputed articles and determined that they all met this refined criterion. Specifically, two articles (Burkhart et al., 2013; Raison et al., 2018) depicted clear perioperative adverse events related to patient health. Another two articles (Bracq et al., 2021; Sirihorachai et al., 2018) focused on training

to manage interruptions caused by stressors, noting that repeated interruptions can increase the risk of adverse events. These stressors included equipment malfunctions, identity monitoring errors, disruptions during surgery, and interruptions from beepers, radio calls, or people entering and exiting the room, as well as irrelevant

Identification of studies via databases and registers

<figure><img src="image_5.png" /><figcaption>**Fig. 1.** Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) flow diagram.</figcaption></figure>
communication. The final two articles (Gardner and AbdelFattah, 2017; Ramjeeawon et al., 2020) depicted adverse events associated with stressors such as equipment failure, which were also linked to patient health issues.

The authors unanimously agreed to include the articles that had initially been subject to disagreement. As a result, 17 articles were excluded since they did not involve a crisis situation, and three further articles were excluded since they did not include any assessment of non-technical skills. This systematic review ultimately includes 29 articles (Fig. 1).

## 2.6. Quality appraisal

The Mixed Methods Appraisal Tool (MMAT) 2018 (Hong et al., 2018) was used to assess the quality of the studies included after full-text screening. The MMAT consists of two screening questions and five core quality criteria adapted to five specific study design categories: qualitative, randomized controlled, non-randomized, quantitative descriptive, and mixed methods. Studies were considered to be of high quality if they met 100 % of the criteria, moderate quality if they met 80–99 % of the criteria, average quality if they met 60–79 % of the criteria, low quality if they met 40–59 % of the criteria, and very low quality if they met <39 % of the criteria. The overall MMAT scores ranged from 20 % to 100 %, with a mean quality score of 73 % (SD = 22). The MMAT total scores for studies are presented in Table 1 and the detailed scoring for each criterion is provided in Supplementary Material 4.

# 3. Results

## 3.1. RQ1. Research contexts

### 3.1.1. Study characteristics

The majority of the articles (18 articles) reported on quantitative non-randomized studies. Four articles were quantitative descriptive studies, four articles were mixed method studies, and three articles were quantitative randomized controlled trials. The articles were published between 2006 and 2023 with a median publication rate of one article per year (min = 1, max = 5). Notably, there was a peak in publication activity between 2015 and 2021. The articles included are predominantly from the United States (16 articles), followed by the United Kingdom (5 articles). Table 1 summarizes the characteristics of the 29 studies included in the systematic review.

### 3.1.2. Surgery specialties

A wide range of surgical specialties were featured in the included articles, listed in order of frequency: gastrointestinal/abdominal (5 articles, Abelson et al., 2015; Gardner and AbdelFattah, 2017; Nguyen et al., 2015; Pena et al., 2015; Sirihorachai et al., 2018), trauma/emergency (5 articles, Acero et al., 2012; Lehner et al., 2017; Pena et al., 2015; Ramjeeawon et al., 2020; Wunder et al., 2020), thoracic (4 articles, Bierer et al., 2018; Burkhart et al., 2013; Pena et al., 2015; Wunder et al., 2020) and general surgery (4 articles, Dedy et al., 2016; Pena et al., 2015; Sirihorachai et al., 2018; Wongsirimeteekul et al., 2018), urology (3 articles, Gettman et al., 2009; Goldenberg et al., 2018; Raison et al., 2018), vascular (3 articles, Moorthy et al., 2006; Ramjeeawon et al., 2020; Taaffe et al., 2021) and head and neck (3 articles, Mai et al., 2020; Rochlen et al., 2019; Sirihorachai et al., 2018), orthopedics (1 article, Edwards et al., 2023), pediatric surgery (1 article, Lehner et al., 2017), and ophthalmology (1 article, Wood et al., 2023). Some articles encompassed multiple specialties (5 articles, Lehner et al., 2017; Pena et al., 2015; Ramjeeawon et al., 2020; Sirihorachai et al., 2018; Wunder et al., 2020), highlighting a non-exclusive focus, while several articles did not specify a surgical specialty reflecting a more generalized or non-disciplinary approach to the subject matter (7 articles, Bracq et al., 2021; Gao et al., 2021; Leithead et al., 2019; Paige et al., 2014; Shi et al., 2021;

Truong et al., 2022; Weller et al., 2015).

### 3.1.3. Participants' occupations

The participants in the articles came from a variety of occupations (see Table 1). In the majority of articles, participants were medical trainees with various levels of expertise (22 articles, Abelson et al., 2015; Acero et al., 2012; Burkhart et al., 2013; Dedy et al., 2016; Edwards et al., 2023; Gettman et al., 2009; Goldenberg et al., 2018; Leithead et al., 2019; Mai et al., 2020; Moorthy et al., 2006; Nguyen et al., 2015; Paige et al., 2014; Pena et al., 2015; Raison et al., 2018; Ramjeeawon et al., 2020; Rochlen et al., 2019; Shi et al., 2021; Taaffe et al., 2021; Truong et al., 2022; Weller et al., 2015; Wongsirimeteekul et al., 2018; Wood et al., 2023). The two second most represented populations were nurses (Acero et al., 2012; Lehner et al., 2017; Mai et al., 2020; Rochlen et al., 2019; Shi et al., 2021; Sirihorachai et al., 2018; Truong et al., 2022; Weller et al., 2015; Wongsirimeteekul et al., 2018) and attending surgeons (Abelson et al., 2015; Bierer et al., 2018; Gardner and AbdelFattah, 2017; Lehner et al., 2017; Rochlen et al., 2019; Shi et al., 2021; Truong et al., 2022; Weller et al., 2015; Wood et al., 2023), targeted as the study population in nine articles each. Other allied health professionals were also enrolled as participants (7 articles, Acero et al., 2012; Edwards et al., 2023; Mai et al., 2020; Rochlen et al., 2019; Truong et al., 2022; Weller et al., 2015; Wongsirimeteekul et al., 2018), although in fewer studies, alongside anesthesiologist trainees (7 articles, Acero et al., 2012; Gao et al., 2021; Mai et al., 2020; Rochlen et al., 2019; Shi et al., 2021; Truong et al., 2022; Wongsirimeteekul et al., 2018), attending anesthetists (4 articles, Lehner et al., 2017; Rochlen et al., 2019; Truong et al., 2022; Weller et al., 2015), nursing students (3 articles, Edwards et al., 2023; Leithead et al., 2019; Paige et al., 2014), student registered nurse anesthetists (3 articles, Leithead et al., 2019; Paige et al., 2014; Wunder et al., 2020), scrub nurses (2 articles, Acero et al., 2012; Bracq et al., 2021), and certified registered nurse anesthetists (1 article, Mai et al., 2020).

### 3.1.4. Simulation groups

The median sample size of participants involved in a simulation study was 31. In half of the included articles, participants underwent the simulation individually, rather than in groups (15 articles, Abelson et al., 2015; Bracq et al., 2021; Dedy et al., 2016; Edwards et al., 2023; Gardner and AbdelFattah, 2017; Goldenberg et al., 2018; Nguyen et al., 2015; Pena et al., 2015; Raison et al., 2018; Ramjeeawon et al., 2020; Sirihorachai et al., 2018; Taaffe et al., 2021; Truong et al., 2022; Wood et al., 2023; Wunder et al., 2020). In simulations involving several learner participants concurrently working on a task (14 articles), the groups were either composed of (1) medical staff, anesthesia staff, and nurses (8/14 articles, Acero et al., 2012; Gao et al., 2021; Leithead et al., 2019; Mai et al., 2020; Paige et al., 2014; Shi et al., 2021; Weller et al., 2015; Wongsirimeteekul et al., 2018); (2) medical staff and nurses (2/14 articles, Bierer et al., 2018; Lehner et al., 2017); (3) residents together (2/14 articles, Burkhart et al., 2013; Gettman et al., 2009); (4) a combination of medical staff, anesthesia staff, nurses, and other allied health professionals (1/14 articles, Rochlen et al., 2019); or (5) medical staff and other allied health professionals (1/14 articles, Edwards et al., 2023). When multiple learners were engaged simultaneously, the average minimum group size was 4.54 (SD = 1.50) participants.

## 3.2. RQ2. Simulation methodologies and organization

### 3.2.1. Operating room team simulation

Engaging multiple participants in a scenario can effectively simulate an operating room team. This was often achieved by using a team composed of participants and scripted confederates or actors collaborating within the environment (13 articles, Bierer et al., 2018; Dedy et al., 2016; Edwards et al., 2023; Gardner and AbdelFattah, 2017; Gettman et al., 2009; Goldenberg et al., 2018; Mai et al., 2020; Moorthy et al., 2006; Pena et al., 2015; Raison et al., 2018; Ramjeeawon et al.,
<table><caption>**Table 1**

Characteristics of studies included in the systematic review.</caption><thead><tr><th>Article, (year), country</th><th>Design</th><th>MMAT (%)</th><th>Participants, n</th><th>Type</th><th>Objectives</th><th>Outcome</th><th>Results</th></tr></thead><tbody><tr><td>Abelson et al. (2015), USA</td><td>NR</td><td>60</td><td>Attending Surgeon, Medical Trainee, n = 33</td><td>VR</td><td>Evaluate simulator for construct and face validity</td><td>Decision Making</td><td>Attendings completed the simulation faster and had a higher pass rate than trainees, but this difference was not statistically significant</td></tr><tr><td>Acero et al. (2012), USA</td><td>NR</td><td>80</td><td>Nurse, Scrub Nurse, Medical Trainee, Anesthesiologist Trainee, OAHP, n = 171</td><td>HF</td><td>Improve team performance during emergency response activation</td><td>Communication, Decision Making</td><td>Understanding of roles and activation of protocol in OR emergencies increased from 50 % to 98 %, with significant improvements in task completion time</td></tr><tr><td>Bierer et al. (2018), Canada</td><td>NR</td><td>60</td><td>Attending Surgeon, n = 12</td><td>IS</td><td>Develop simulation for effective interprofessional communication and teamwork</td><td>Communication, Teamwork, Situation Awareness, Decision Making, Leadership, Team Management, Situation Monitoring, Mutual support, Team structure</td><td>Consultants scored higher than fellows in NTS, effectively differentiating experts from learners</td></tr><tr><td>Bracq et al. (2021), France</td><td>Mixed</td><td>60</td><td>Scrub Nurse, n = 26</td><td>VR</td><td>Assess error recognition in a virtual operating room</td><td>Situation Awareness</td><td>Participants with higher situation awareness detected more errors and were faster</td></tr><tr><td>Burkhart et al. (2013), USA</td><td>NR</td><td>60</td><td>Medical Trainee, n = 23</td><td>HF</td><td>Test simulation to improving confidence, proficiency, and crisis management</td><td>Situation Awareness, Decision Making, Confidence</td><td>Confidence and crisis management improved, with success rates in tasks rising from 40 % to 69 %</td></tr><tr><td>Dedy et al. (2016), Canada</td><td>RCT</td><td>100</td><td>Medical Trainee, n = 22</td><td>HF</td><td>Evaluate the effectiveness of structured training on NTS in an OR</td><td>Communication, Teamwork, Situation Awareness, Decision Making, Leadership</td><td>Curriculum-trained residents improved significantly in knowledge, attitude and NTS performance compared to conventionally trained residents</td></tr><tr><td>Edwards et al. (2023), UK</td><td>RCT</td><td>100</td><td>Nurse Student, Medical Trainee, OAHP, n = 40</td><td>HF VR</td><td>Compare multiplayer immersive VR with single-player for NTS acquisition</td><td>Communication, Teamwork, Situation Awareness, Decision Making, Leadership, Task Management</td><td>Team-trained participants outperformed solo groups on all NTS</td></tr><tr><td>Gao et al. (2021), China</td><td>NR</td><td>100</td><td>Anesthesiologist Trainee, n = 31</td><td>HF</td><td>Enhance student learning awareness through simulation training</td><td>Communication, Situation Awareness, Teamwork, Decision Making, Team Management, Confidence, Task Management</td><td>Students improved their responses and completed tests and scenarios significantly faster after training</td></tr><tr><td>Gardner and AbdelFattah (2017), USA</td><td>NR</td><td>60</td><td>Attending Surgeon, n = 26</td><td>HF</td><td>Examine role of simulation in documenting resident NTS compared to faculty ratings</td><td>Communication, Decision Making, Leadership, Situation Awareness, Task Management</td><td>Simulation assessments did not align well with traditional clinical rotation evaluations for NTS</td></tr><tr><td>Gettman et al. (2009), USA</td><td>NR</td><td>80</td><td>Medical Trainee, n = 19</td><td>HF</td><td>Evaluate teamwork and communication in a simulated OR</td><td>Communication, Teamwork, Decision Making, Leadership, Situation Awareness, Team Management</td><td>Significant improvements in teamwork and adherence to best practices were observed between scenarios</td></tr><tr><td>Goldenberg et al. (2018), Canada</td><td>NR</td><td>60</td><td>Medical Trainee, n = 15</td><td>HF</td><td>Create simulation tool for safe NTS assessment</td><td>Communication, Teamwork, Decision Making, Leadership, Situation Awareness</td><td>Senior residents outperformed juniors in NTS</td></tr><tr><td>Lehner et al. (2017), Germany</td><td>NR</td><td>60</td><td>Attending Anesthesiologist, Attending Surgeon, Nurse, n = 18</td><td>HF</td><td>Establish interdisciplinary simulation-based training to improve trauma in pediatric surgery</td><td>Communication, Leadership, Team Management</td><td>Participants reported improvements in NTS</td></tr><tr><td>Leithead et al. (2019), USA</td><td>NR</td><td>40</td><td>Nurse student, Student Registered Nurse Anesthetist, Medical Trainee, n = 152</td><td>HF</td><td>Investigate impact of simulation on OR team training across different professions</td><td>Teamwork, Confidence</td><td>Both medical and nursing students showed significant improvements in team-based attitudes and collaboration scores</td></tr><tr><td>Mai et al. (2020), USA</td><td>QD</td><td>20</td><td>Nurse, Certified Registered Nurse Anesthetist, Medical Trainee, Anesthesiologist Trainee, OAHP, n = 86</td><td>IS</td><td>Develop simulation to identify and manage OR fire, reduce adverse outcomes, and apply crisis management principles</td><td>Teamwork</td><td>Most participants found the simulation realistic, relevant, and effective in promoting teamwork</td></tr><tr><td>Moorthy et al. (2006), UK</td><td>NR</td><td>80</td><td>Medical Trainee, n = 20</td><td>HF</td><td>Develop and evaluate feasibility, realism, and validity of surgical crisis simulation</td><td>Communication, Teamwork, Situation Awareness, Decision Making</td><td>No significant differences in NTS between groups, but leadership showed a non-significant trend and</td></tr></tbody></table>

(continued on next page)
<table><caption>**Table 1 (continued)**</caption><thead><tr><th>Article, (year), country</th><th>Design</th><th>MMAT (%)</th><th>Participants, n</th><th>Type</th><th>Objectives</th><th>Outcome</th><th>Results</th></tr></thead><tbody><tr><td>Nguyen et al. (2015), USA</td><td>NR</td><td>80</td><td>Medical Trainee, n = 11</td><td>HF</td><td>Examine effect of simulation-based training on NTS</td><td>Leadership, Team Management, Communication, Professionalism</td><td>utterances frequency difference was significant Residents showed significant improvement in completing perioperative and intraoperative checklists after training</td></tr><tr><td>Paige et al. (2014), USA</td><td>NR</td><td>80</td><td>Nurse student, Student Registered Nurse Anesthetist, Medical Trainee, n = 66</td><td>HF</td><td>Examine impact of HF OR team training on team-related attitudes and behaviors</td><td>Teamwork, Confidence</td><td>Statistically significant gains were observed in self-efficacy and teamwork performance after simulation training</td></tr><tr><td>Pena et al. (2015), Australia</td><td>NR</td><td>100</td><td>Medical Trainee, n = 40</td><td>IS</td><td>Explore effects of simulation-based training, alone or combined on NTS</td><td>Communication, Teamwork, Situation Awareness, Decision Making, Leadership</td><td>Both groups improved significantly in NTS, with no differences between the simulation-only and workshop groups</td></tr><tr><td>Raison et al. (2018), UK</td><td>RCT</td><td>100</td><td>Medical Trainee, n = 62</td><td>HF</td><td>Investigate the effectiveness of motor imagery for NTS training</td><td>Communication, Teamwork, Situation Awareness, Decision Making, Leadership</td><td>No significant differences were found between motor imagery and standard training in NTS</td></tr><tr><td>Ramjeeawon et al. (2020), UK</td><td>NR</td><td>80</td><td>Medical Trainee, n = 16</td><td>HF</td><td>Assess if fully-immersive simulation training with structured debriefing improves the lead surgeon team-work</td><td>Communication, Teamwork, Situation Awareness, Leadership, Team Management, Task Management, Situation Monitoring, Mutual support</td><td>Structured debriefing significantly improved teamwork performance across all domains.</td></tr><tr><td>Rochlen et al. (2019), USA</td><td>NR</td><td>80</td><td>Attending Anesthesiologist, Attending Surgeon, Nurse, Medical Trainee, Anesthesiologist Trainee, OAHP, n = 31</td><td>IS</td><td>Evaluate feasibility and effectiveness of interprofessional crisis management simulation on improvements in NTS</td><td>Communication, Teamwork, Situation Awareness, Decision Making, Leadership, Team Management</td><td>All teams improved in NTS scores post-intervention, but the changes were not statistically significant.</td></tr><tr><td>Shi et al. (2021), USA</td><td>Mixed</td><td>60</td><td>Attending Surgeon, Nurse, Medical Trainee, Anesthesiologist Trainee, n = 14</td><td>IS</td><td>Explore OR professionals' perspective on teamwork and communication pre- and post-interprofessional OR simulation</td><td>Communication, Self Efficacy</td><td>Participants showed high agreement on their communication skills and ability to speak up in both emergency and non-emergency situations. Experienced nurses were more likely to allow interruptions during time-outs, using cognitive strategies to manage distractions.</td></tr><tr><td>Sirihorachai et al. (2018), USA</td><td>Mixed</td><td>100</td><td>Nurse, n = 30</td><td>HF</td><td>Develop simulation scenarios to understand decision-making in response to interruptions</td><td>Interruption Management</td><td>The vascular surgery crisis simulation yielded similar results for residents and fellows, with a total score of 83 %.</td></tr><tr><td>Taaffe et al. (2021), USA</td><td>NR</td><td>80</td><td>Medical Trainee, n = 40</td><td>S</td><td>Assess feasibility of a vascular surgery crisis simulator and evaluate the performance of residents and fellows</td><td>Situation Awareness, Decision Making</td><td>Confidence</td></tr><tr><td>Truong et al. (2022), USA</td><td>QD</td><td>100</td><td>Attending Anesthesiologist, Attending Surgeon, Nurse, Medical Trainee, Anesthesiologist Trainee, OAHP, n = 180</td><td>VR</td><td>Evaluate responses to VR-simulated OR fire, examine impact of experience, confidence, and role on the number of attempts needed to pass simulation</td><td>Confidence</td><td>Confidence in fire response increased significantly post-training, with most participants passing the simulation within five attempts.</td></tr><tr><td>Weller et al. (2015), New Zealand</td><td>Mixed</td><td>80</td><td>Attending Anesthesiologist, Attending Surgeon, Nurse, Medical Trainee, OAHP, n = 120</td><td>HF</td><td>Explore the feasibility and effectiveness of simulation-based course for OR team</td><td>Communication</td><td>There was a significant improvement in BMRI scores between the first and last scenarios.</td></tr><tr><td>Wongsirimeteekul et al. (2018), USA</td><td>QD</td><td>20</td><td>Nurse, Medical Trainee, Anesthesiologist Trainee, OAHP, n = 91</td><td>IS</td><td>Develop simulation to apply core concepts of crisis resource management.</td><td>Teamwork</td><td>Participants reported that the simulation improved teamwork skills and promoted interprofessional learning.</td></tr><tr><td>Wood et al. (2023), UK</td><td>NR</td><td>60</td><td>Attending Surgeon, Medical Trainee, n = 17</td><td>IS</td><td>Design a simulation-based training for managing posterior capsule rupture and analyze NTS changes</td><td>Communication, Teamwork, Situation Awareness, Decision Making, Leadership, Professionalism</td><td>NTS improved significantly after simulation-based training for managing posterior capsule rupture.</td></tr><tr><td>Wunder et al. (2020), USA</td><td>QD</td><td>80</td><td>Student Registered Nurse Anesthetist, n = 32</td><td>HF AR</td><td>Evaluate NTS after mixed reality simulation of OR fire</td><td>Teamwork, Situation Awareness, Decision Making, Team Management</td><td>NTS performance was 91.25 % after the mixed reality simulation of OR fire.</td></tr></tbody></table>
Note. The design section in coded such as NR = Non-Random; RCT = Randomized Controlled Trial; QD = Quantitative Descriptive. In the participant section, OAHP = Other Allied Health Professional. The simulation section is coded such as VR = Virtual Reality; HF = High-Fidelity; IS = In Situ; S = Stations; AR = Augmented Reality. OR = operating room. NTS = non-technical skills.

2020; Sirihorachai et al., 2018; Wunder et al., 2020). An alternative approach involved assigning specific roles to each learner within the operating room team (12 articles, Acero et al., 2012; Burkhart et al., 2013; Edwards et al., 2023; Gao et al., 2021; Lehner et al., 2017; Leithead et al., 2019; Nguyen et al., 2015; Paige et al., 2014; Rochlen et al., 2019; Shi et al., 2021; Weller et al., 2015; Wongsirimeteekul et al., 2018). Two articles (Abelson et al., 2015; Edwards et al., 2023) introduced an innovative approach by using avatars to represent team members. Finally, four articles focused on the experiences of individual learners working alone in the operating room (Bracq et al., 2021; Taaffe et al., 2021; Truong et al., 2022; Wood et al., 2023).

### 3.2.2. Operating room environment simulation

Simulation environments included in situ settings, virtual reality, mixed reality, simulation stations, and high-fidelity simulations. In situ simulation was sometimes used in the included articles (7 articles, Bierer et al., 2018; Mai et al., 2020; Pena et al., 2015; Rochlen et al., 2019; Shi et al., 2021; Wongsirimeteekul et al., 2018; Wood et al., 2023), while virtual reality simulation (4 articles, Abelson et al., 2015; Bracq et al., 2021; Edwards et al., 2023; Truong et al., 2022), simulation stations (1 article, Taaffe et al., 2021) and augmented reality simulation (1 article, Wunder et al., 2020) were less represented. However, the most reported modality was a form of "simulated operating room" (18 articles), which does not fall into the previous categories and could be linked to high-fidelity simulation. Among these 18 articles, two did not provide sufficient details of their definition of "simulated operating room" (Gardner and AbdelFattah, 2017; Sirihorachai et al., 2018).

Within the remaining 16 articles simulating an operating room in high-fidelity, three elements were commonly simulated: the operating room environment, the operating room equipment, and the patient. First, a considerable portion of these articles simulated the patient only (3/16 articles, Acero et al., 2012; Dedy et al., 2016; Lehner et al., 2017). In each case, this was done using a full-scale high-fidelity patient simulator mannequin. Other studies describe the simulated environment and equipment (2/16 articles, Raison et al., 2018; Ramjeeawon et al., 2020). In two articles, the simulation was performed using a simulated patient in a simulated OR but without sufficient information about the instruments (Burkhart et al., 2013; Wunder et al., 2020). Finally, the majority of articles (9/16 articles) describe a simulated operating room with both simulated equipment and a simulated patient: the operating room was consistently an immersive mock operating room, and the patient simulation was predominantly carried out using a human patient simulator mannequin. All of these nine articles used real operating room instruments (Edwards et al., 2023; Gao et al., 2021; Gettman et al., 2009; Goldenberg et al., 2018; Leithead et al., 2019; Moorthy et al., 2006; Nguyen et al., 2015; Paige et al., 2014; Weller et al., 2015).

Before 2018, high-fidelity simulation appears to be the predominant method for training non-technical skills in crisis management, as evidenced by articles published during that period (9/11 articles, Acero et al., 2012; Dedy et al., 2016; Gardner and AbdelFattah, 2017; Gettman et al., 2009; Lehner et al., 2017; Moorthy et al., 2006; Nguyen et al., 2015; Paige et al., 2014; Weller et al., 2015). However, this trend shifted after 2018, with an increasing adoption of in-situ simulation (6/17 articles, Bierer et al., 2018; Mai et al., 2020; Rochlen et al., 2019; Shi et al., 2021; Wongsirimeteekul et al., 2018; Wood et al., 2023), equaling the use of high-fidelity simulation. A notable shift occurred in 2020, characterized by the digitalization of training practices, with a predominant focus on the use of virtual reality, augmented reality, and simulation stations in the published articles (5/8 articles, Bracq et al., 2021; Edwards et al., 2023; Taaffe et al., 2021; Truong et al., 2022; Wunder et al., 2020).

### 3.2.3. Briefing phase

In most studies, a briefing was provided to participants before the simulation (18 articles). Briefings in simulations can encompass various elements, often combining multiple aspects: orientation regarding the patient's medical history (13/18 articles, Acero et al., 2012; Bracq et al., 2021; Dedy et al., 2016; Gardner and AbdelFattah, 2017; Goldenberg et al., 2018; Mai et al., 2020; Nguyen et al., 2015; Paige et al., 2014; Ramjeeawon et al., 2020; Weller et al., 2015; Wongsirimeteekul et al., 2018; Wood et al., 2023; Wunder et al., 2020), simulation objectives (8/18 articles, Bracq et al., 2021; Gardner and AbdelFattah, 2017; Goldenberg et al., 2018; Mai et al., 2020; Moorthy et al., 2006; Sirihorachai et al., 2018; Wongsirimeteekul et al., 2018; Wunder et al., 2020), and introduction to the simulation equipment (9/18 articles, Bracq et al., 2021; Dedy et al., 2016; Gettman et al., 2009; Mai et al., 2020; Moorthy et al., 2006; Ramjeeawon et al., 2020; Wongsirimeteekul et al., 2018; Wood et al., 2023; Wunder et al., 2020). Some briefings also included theoretical clinical aspects required for the scenario (3/18 articles, Mai et al., 2020; Wongsirimeteekul et al., 2018; Wunder et al., 2020).

### 3.2.4. Debriefing phase

A debriefing was also provided for participants after the simulation in most of the included articles (22 articles). Various professionals conducted the debriefings: moderators (5/22 articles, Acero et al., 2012; Lehner et al., 2017; Leithead et al., 2019; Paige et al., 2014; Wood et al., 2023), medical staff (6/22 articles, Bierer et al., 2018; Nguyen et al., 2015; Pena et al., 2015; Ramjeeawon et al., 2020; Rochlen et al., 2019; Shi et al., 2021), faculty members or teachers (4/22 articles, Gardner and AbdelFattah, 2017; Mai et al., 2020; Wongsirimeteekul et al., 2018; Wunder et al., 2020), and researchers (2/22 articles, Moorthy et al., 2006; Weller et al., 2015). One debriefing team was composed of both researchers and teachers (1/22 articles, Bracq et al., 2021). One debriefing was conducted by a content expert, without further precision about profession or role (Nguyen et al., 2015). Among this subset of 22 articles, four articles did not specify the composition of the debriefing team (Dedy et al., 2016; Gao et al., 2021; Gettman et al., 2009; Sirihorachai et al., 2018). Debriefings were often video-based (11/22 articles, Acero et al., 2012; Gardner and AbdelFattah, 2017; Gettman et al., 2009; Lehner et al., 2017; Mai et al., 2020; Moorthy et al., 2006; Nguyen et al., 2015; Pena et al., 2015; Sirihorachai et al., 2018; Wongsirimeteekul et al., 2018; Wood et al., 2023) and a portion of the studies used specific methodologies or frameworks to guide the process (7/22 articles, Bracq et al., 2021; Gardner and AbdelFattah, 2017; Leithead et al., 2019; Paige et al., 2014; Pena et al., 2015; Wongsirimeteekul et al., 2018; Wunder et al., 2020). When debriefing techniques were not specified (4/22 articles), there was a consistent emphasis on non-technical skills and crew resource management, performed by reviewing teamwork behaviors (Mai et al., 2020), strategies for enhancing communication (Rochlen et al., 2019), the interprofessional nature of communication and teamwork (Shi et al., 2021), and the exploration of mechanisms and barriers to information sharing (Weller et al., 2015).

### 3.3. RQ3. Crisis simulation scenarios and skills measured

The crisis scenarios in the included studies can be classified into three main categories (Fig. 2). The most reported type of crisis is patient health-related failure (22 articles, Acero et al., 2012; Bierer et al., 2018; Burkhart et al., 2013; Dedy et al., 2016; Edwards et al., 2023; Gao et al., 2021; Gardner and AbdelFattah, 2017; Gettman et al., 2009; Goldenberg et al., 2018; Lehner et al., 2017; Leithead et al., 2019; Moorthy et al., 2006; Nguyen et al., 2015; Paige et al., 2014; Pena et al., 2015; Raison et al., 2018; Rochlen et al., 2019; Shi et al., 2021; Taaffe et al., 2021;
<figure><img src="image_6.png" /><figcaption>(a) Patient health-related failures</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>(b) Medical team-related failures</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>(c) Equipment-related failures</figcaption></figure>

<caption>**Fig. 2.** Scenario distribution within each failure type. Panel (a): Patient health-related failures. Panel (b): Medical team-related failures. Panel (c): Equipment-related failures.</caption>

Weller et al., 2015; Wongsirimeteekul et al., 2018; Wood et al., 2023), which refers to crisis scenarios in which a patient experiences a sudden or critical perioperative deterioration in their health status. These scenarios typically require the surgical team to quickly recognize, diagnose, and manage the crisis to prevent further harm or adverse outcomes.

Another type of scenario is equipment-related failure (11 articles, Abelson et al., 2015; Bracq et al., 2021; Gardner and AbdelFattah, 2017; Gettman et al., 2009; Mai et al., 2020; Nguyen et al., 2015; Pena et al., 2015; Shi et al., 2021; Sirihorachai et al., 2018; Truong et al., 2022; Wood et al., 2023; Wunder et al., 2020). This refers to crisis scenarios where the malfunction, misuse, or unavailability of medical equipment disrupts the surgical procedure or patient care. These scenarios challenge the surgical team to quickly identify, troubleshoot, and resolve the issue to minimize delays, prevent harm, and maintain patient safety.

The third category is medical team dynamics failure (7 articles, Acero et al., 2012; Dedy et al., 2016; Nguyen et al., 2015; Pena et al., 2015; Raison et al., 2018; Sirihorachai et al., 2018; Wood et al., 2023). This refers to crisis scenarios where breakdowns in communication, coordination, leadership, or decision-making within the surgical team contribute to a critical situation. These failures can lead to delays, errors, and compromised patient safety.

These crisis scenarios were used to train a wide range of non-technical skills. Table 1 presents a summary of the non-technical skills addressed in each included article. The non-technical skills receiving the most training in crisis simulations were communication (18 articles, Acero et al., 2012; Bierer et al., 2018; Dedy et al., 2016; Edwards et al., 2023; Gao et al., 2021; Gardner and AbdelFattah, 2017; Gettman et al., 2009; Goldenberg et al., 2018; Lehner et al., 2017; Moorthy et al., 2006; Nguyen et al., 2015; Pena et al., 2015; Raison et al., 2018; Ramjeeawon et al., 2020; Rochlen et al., 2019; Shi et al., 2021; Weller et al., 2015; Wood et al., 2023), teamwork (17 articles, Bierer et al., 2018; Dedy et al., 2016; Edwards et al., 2023; Gao et al., 2021; Gettman et al., 2009; Goldenberg et al., 2018; Leithead et al., 2019; Mai et al., 2020; Moorthy et al., 2006; Paige et al., 2014; Pena et al., 2015; Raison et al., 2018; Ramjeeawon et al., 2020; Rochlen et al., 2019; Wongsirimeteekul et al., 2018; Wood et al., 2023; Wunder et al., 2020), decision making or

problem-solving (17 articles, Abelson et al., 2015; Acero et al., 2012; Bierer et al., 2018; Burkhart et al., 2013; Dedy et al., 2016; Edwards et al., 2023; Gao et al., 2021; Gardner and AbdelFattah, 2017; Gettman et al., 2009; Goldenberg et al., 2018; Moorthy et al., 2006; Pena et al., 2015; Raison et al., 2018; Rochlen et al., 2019; Taaffe et al., 2021; Wood et al., 2023; Wunder et al., 2020), situation awareness (17 articles, Bierer et al., 2018; Bracq et al., 2021; Burkhart et al., 2013; Dedy et al., 2016; Edwards et al., 2023; Gao et al., 2021; Gardner and AbdelFattah, 2017; Gettman et al., 2009; Goldenberg et al., 2018; Moorthy et al., 2006; Pena et al., 2015; Raison et al., 2018; Ramjeeawon et al., 2020; Rochlen et al., 2019; Taaffe et al., 2021; Wood et al., 2023; Wunder et al., 2020), and leadership (13 articles, Bierer et al., 2018; Dedy et al., 2016; Edwards et al., 2023; Gardner and AbdelFattah, 2017; Gettman et al., 2009; Goldenberg et al., 2018; Lehner et al., 2017; Moorthy et al., 2006; Pena et al., 2015; Raison et al., 2018; Ramjeeawon et al., 2020; Rochlen et al., 2019; Wood et al., 2023).

Additionally, other non-technical skills that do not align directly with established non-technical skills rating scales were also addressed in the training. These include team management (8 articles, Bierer et al., 2018; Gao et al., 2021; Gettman et al., 2009; Lehner et al., 2017; Moorthy et al., 2006; Ramjeeawon et al., 2020; Rochlen et al., 2019; Wunder et al., 2020), confidence or self-efficacy (6 articles, Burkhart et al., 2013; Gao et al., 2021; Leithead et al., 2019; Nguyen et al., 2015; Shi et al., 2021; Truong et al., 2022), task management (4 articles, Edwards et al., 2023; Gao et al., 2021; Gardner and AbdelFattah, 2017; Raison et al., 2018), situation monitoring (2 articles, Bierer et al., 2018; Ramjeeawon et al., 2020), mutual support (2 articles, Bierer et al., 2018; Ramjeeawon et al., 2020), professionalism (2 articles, Nguyen et al., 2015; Wood et al., 2023), team structure (1 article, Bierer et al., 2018), and management of interruptions (1 article, Sirihorachai et al., 2018).

Among the 29 articles that focused on non-technical skills, 27 attempted to assess these skills using various methods and tools. Observation-based assessment was the most commonly used method (21/27 articles, Acero et al., 2012; Bierer et al., 2018; Burkhart et al., 2013; Dedy et al., 2016; Edwards et al., 2023; Gao et al., 2021; Gardner and AbdelFattah, 2017; Gettman et al., 2009; Goldenberg et al., 2018;
Moorthy et al., 2006; Nguyen et al., 2015; Paige et al., 2014; Pena et al., 2015; Raison et al., 2018; Ramjeeawon et al., 2020; Rochlen et al., 2019; Sirihorachai et al., 2018; Taaffe et al., 2021; Weller et al., 2015; Wood et al., 2023; Wunder et al., 2020). Observation was often paired with a behavioral rating system, such as the Non-Technical Skills for Surgeons (NOTSS; Yule et al., 2009) used in six articles (Bierer et al., 2018; Dedy et al., 2016; Edwards et al., 2023; Goldenberg et al., 2018; Pena et al., 2015; Raison et al., 2018), Modified Theatre Team Non-Technical Skills Scoring System (NOTECHS-II; Robertson et al., 2014) used in three articles (Edwards et al., 2023; Moorthy et al., 2006; Rochlen et al., 2019), Operating Room Teamwork Assessment Scales (ORTAS; Paige et al., 2009a) used in one article (Paige et al., 2014), Objective Structured Assessment of Nontechnical Skills (OSANTS; Dedy et al., 2015) used in one article (Dedy et al., 2016), Operating Room Communication Assessment (ORCA; Rudolph et al., 2006) used in one article (Gardner and AbdelFattah, 2017), Team Strategies and Tools to Enhance Performance and Patient Safety (TeamSTEPPS; Guimond et al., 2009) used in one article (Bierer et al., 2018), Anaesthetists' Non-Technical Skills (ANTS; Fletcher et al., 2003) used in one article (Wunder et al., 2020), HUman Factors in intraoperative Ophthalmic Emergencies Scoring System (HUFOES; Wood et al., 2021) used in one article (Wood et al., 2023), Observational Teamwork Assessment for Surgery (OTAS; Undre et al., 2007) used in one article (Ramjeeawon et al., 2020), Behavioural Marker of Risk Index (BMRI; Mazzocco et al., 2009) used in one article (Weller et al., 2015), and Scrub Practitioners' List of Intraoperative Non-Technical Skills (SPLINTS; Mitchell et al., 2012) used in one article (Edwards et al., 2023).

While simulation sessions were video-recorded in the majority of studies (18/27 articles, Acero et al., 2012; Bierer et al., 2018; Dedy et al., 2016; Edwards et al., 2023; Gardner and AbdelFattah, 2017; Gettman et al., 2009; Goldenberg et al., 2018; Lehner et al., 2017; Mai et al., 2020; Moorthy et al., 2006; Nguyen et al., 2015; Pena et al., 2015; Raison et al., 2018; Ramjeeawon et al., 2020; Sirihorachai et al., 2018; Wongsirimeteekul et al., 2018; Wood et al., 2021; Wunder et al., 2020), a smaller proportion used these recordings for observation-based assessment purposes (14/27 articles, Acero et al., 2012; Bierer et al., 2018; Dedy et al., 2015; Edwards et al., 2023; Gardner and AbdelFattah, 2017; Gettman et al., 2009; Goldenberg et al., 2018; Moorthy et al., 2006; Nguyen et al., 2015; Pena et al., 2015; Raison et al., 2018; Ramjeeawon et al., 2020; Wood et al., 2023; Wunder et al., 2020). When the observation-based assessment was not conducted on recorded video, it was performed live during the simulation itself.

In other studies, non-technical skills were assessed using self-assessment Likert scales (7/27 articles), some of which were homemade (5/7 articles, Lehner et al., 2017; Mai et al., 2020; Paige et al., 2014; Shi et al., 2021; Truong et al., 2022). Other scales used were the Interprofessional Teamwork Scale (Paige et al., 2009b) used in one article (Leithead et al., 2019), the Teamwork Assessment Scales (Garbee et al., 2013) used in one article (Leithead et al., 2019), the TeamSTEPPS teamwork attitudes questionnaire (Baker et al., 2010) used in one article (Dedy et al., 2016), the Situation Awareness Rating System (Taylor, 2011) used in one article (Bracq et al., 2021). Finally, non-technical skills were also evaluated with quizzes to assess knowledge of non-technical skills and patient safety in the operating room (Dedy et al., 2016).

## 3.4. RQ4. Levels of evaluation per Kirkpatrick's model

The Kirkpatrick model's four levels (Kirkpatrick and Kirkpatrick, 2006) were used to investigate training outcomes. The first level (Reaction), addressed in the majority of studies, examines participants' immediate reactions, highlighting their self-reported satisfaction and engagement (20/29 articles, Abelson et al., 2015; Acero et al., 2012; Bierer et al., 2018; Bracq et al., 2021; Burkhart et al., 2013; Dedy et al., 2016; Gao et al., 2021; Gettman et al., 2009; Lehner et al., 2017; Mai et al., 2020; Moorthy et al., 2006; Nguyen et al., 2015; Paige et al., 2014;

Pena et al., 2015; Ramjeeawon et al., 2020; Rochlen et al., 2019; Truong et al., 2022; Weller et al., 2015; Wongsirimeteekul et al., 2018; Wood et al., 2023).

The second level (Learning), also addressed in the majority of included studies, focuses on skills, attitudes, and knowledge acquisition as a result of training (25/29 articles, Abelson et al., 2015; Acero et al., 2012; Bierer et al., 2018; Bracq et al., 2021; Burkhart et al., 2013; Dedy et al., 2016; Edwards et al., 2023; Gao et al., 2021; Gardner and AbdelFattah, 2017; Gettman et al., 2009; Goldenberg et al., 2018; Lehner et al., 2017; Leithead et al., 2019; Moorthy et al., 2006; Nguyen et al., 2015; Paige et al., 2014; Pena et al., 2015; Raison et al., 2018; Ramjeeawon et al., 2020; Rochlen et al., 2019; Shi et al., 2021; Taaffe et al., 2021; Weller et al., 2015; Wood et al., 2023; Wunder et al., 2020). The measurement tools were mostly third-party observation-based assessments, self-administered questionnaires, and quizzes.

The third level (Behavior) was addressed in a few studies. It investigates changes in participants' behavior or practice as a result of the training (6/29 articles, Dedy et al., 2016; Gettman et al., 2009; Paige et al., 2014; Pena et al., 2015; Ramjeeawon et al., 2020; Weller et al., 2015). In this review, Level 3 was considered to be achieved when skills acquired during training were successfully applied in scenarios beyond the initial training context. The transfer of skills was consistently assessed through third-party observation of non-technical skills performances.

The fourth level (Results) was not addressed in any of the included studies. This level aims to measure the broader impact of training on organizational outcomes, capturing the ultimate effectiveness of training in achieving its intended goals.

# 4. Discussion

## 4.1. Summary of evidence

This systematic review summarizes research on simulation training for healthcare professionals and students in crisis management within the operating room. A comprehensive database search yielded 29 relevant articles. The results show that high-fidelity simulation is the most common method used for crisis management training, which is conducted across various surgical specialties and often involving interdisciplinary collaboration. However, there is a predominant focus on medical professionals and students, to the detriment of nurses and other healthcare professionals. Communication was identified as the most critical non-technical skill targeted for training, followed by teamworking, decision-making and situation awareness. Training scenarios were categorized into three main types: patient health-related, equipment-related, and team-related challenges, with team-related challenges being notably underutilized for training non-technical skills. The metrics most employed to evaluate the training were third-party observation-based assessments of non-technical skills performance. According to Kirkpatrick's model, assessments reached a maximum of Level 3 (behavioral change), with Level 1 (participant reactions) and Level 2 (learning outcomes) almost always assessed. However, the third level was measured less frequently, and notably, the fourth level, which would indicate results in terms of organizational practice or patient outcomes, was never assessed in the studies included in this review.

## 4.2. Simulation methods

This systematic review highlights that high-fidelity simulation is the primary method used in healthcare education, aligning with existing literature (Hanshaw and Dickerson, 2020). This prevalence can be explained by the fact that the method provides immersive and risk-free environments that enhance both technical and non-technical skills (Alshehri et al., 2023; Hanshaw and Dickerson, 2020) by supporting experiential learning, scenario replication, immediate feedback, and the management of emotional and cognitive load (Newton and Smith, 2024;
Rogers and Franklin, 2021). However, its implementation involves substantial costs related to scenario complexity, equipment, and facilitator training (Alshehri et al., 2023), as well as logistical challenges such as team availability and travel to simulation centers, which could limit its accessibility (Gardner et al., 2020).

Additionally, in situ simulation has gained popularity (Martin et al., 2020) since 2018, as evidenced by the findings of this systematic review. This approach enhances realism by allowing participants to train in their actual work environment, leading to potential improvements in clinical outcomes (Calhoun et al., 2024). Despite these advantages, in situ simulation poses challenges related to time constraints, resource allocation, and its primary suitability for continuing education rather than initial training (Gardner et al., 2020).

Faced with these limitations, virtual reality emerges as a promising alternative. The findings highlight the gradual digitalization of training since 2020. This shift can be attributed not only to the increasing accessibility of these technologies for healthcare education organizations but also to the inherent advantages of virtual reality. Virtual reality provides cost-effective, repeatable, and standardized training experiences (Pottle, 2019) while facilitating interaction with virtual patients, colleagues, and crisis scenarios, ultimately enhancing clinical and non-technical skills (Bracq et al., 2019; Mistry et al., 2023). Virtual reality also supports tele-simulation, enabling broader access and global collaboration (Almousa et al., 2021). Its ability to provide high-frequency, low-stress repetitions makes it particularly suitable for crisis management scenarios (Heldring et al., 2024).

Nevertheless, the under-adoption of virtual reality can be explained by several barriers, including initial investment costs, technical expertise requirements, and resistance to change in traditional educational settings (Kyaw et al., 2019; Lee et al., 2023).

### 4.3. Target population

With approximately 45 % of the included studies featuring multidisciplinarity, this review highlights the potential for improvement in fostering interprofessional education within simulation training. However, there remains a noticeable skew towards medical personnel such as medical trainees, surgeons, and anesthetists at the expense of paramedical staff like nurses or allied health professionals. This imbalance may reflect traditional institutional biases regarding team roles within surgical settings.

The prevailing view positions the surgeon as the de facto leader (Peters et al., 2020), a role that is reinforced by the leadership demands of the operating room. Surgeons are expected to guide the team and make critical decisions (Murtagh and Bezemer, 2021), as outlined in the NOTSS taxonomy (Yule et al., 2008). Despite this traditional hierarchy, effective performance relies on teamwork (Schmutz et al., 2019) and the collective expertise of the entire team, including nurses, who represent a significant portion of the surgical team, as well as other paramedical staff (Watkins and Hensley, 2023). This dynamic reinforces the argument for multidisciplinary simulation-based training programs (Saragih et al., 2024; Scherer and Winokur, 2019), as evidenced by systematic reviews on the impact of crisis resource management simulation-based training for interprofessional and interdisciplinary teams (Fung et al., 2015).

### 4.4. Outcomes

This systematic review identified three categories of crisis scenarios used to train non-technical skills in operating room crisis management: patient health-related failures, equipment-related failures, and team dynamics failures. These categories align with the classifications of perioperative adverse events described in the literature, which include diagnostic errors, tissue injuries, and technical issues related to patient health; equipment malfunctions, medication errors, and wrong-site/patient procedures associated with equipment or checklist failures;

and communication failures linked to team dynamics (Jung et al., 2019).

However, the results of this review reveal a pronounced emphasis on patient health crises over medical team dynamics or equipment malfunctions despite literature indicating that perioperative errors are often attributable to non-technical skills related to team dynamics issues (Allard et al., 2020; Koleva, 2020). This highlights the critical need for training in team-related crisis management to better prepare healthcare professionals. Scenarios focused on patient health crises may emphasize technical skills and immediate clinical management, sidelining non-technical skills. To address this imbalance, integrating scenarios unrelated to patient health, such as equipment failures or team-based decision-making, could shift the focus away from technical tasks and better reinforce non-technical skills. Effective crisis management relies on seamless team collaboration, making it crucial to balance scenarios that reflect real-world communication challenges, such as role ambiguity, miscommunication under pressure, and leadership conflicts. Prioritizing team dynamics could enhance performance and help reduce adverse events caused by communication breakdowns.

Another interesting result is that training programs often focus on non-technical skills like communication, teamwork, and situation awareness, reflecting priorities deemed essential for crisis management. Indeed, the Interprofessional Education Collaborative (2023) identifies communication and teamwork as core competencies that enhance healthcare education. Programs focusing on these skills improve crisis management competence (Buljac-Samardzić et al., 2021), while communication significantly reduces medical errors (Weller and Webster, 2021), and situation awareness is crucial for recognizing and reacting to shifts in patient condition during crises (Lei and Palm, 2024). Teamwork involves broader non-technical skills related to inter-member interactions, which could explain why it is consistently prioritized in training.

Finally, over half of the articles in the systematic review report a positive effect of non-technical skills training. However, these results rely on diverse methodological frameworks, therefore a nuanced interpretation of this effectiveness is required. Positive impacts of training are observed at Kirkpatrick's Level 1 (reaction) and Level 2 (learning), consistent with the literature (Buljac-Samardzić et al., 2021). The transfer of trained skills, as expressed by Kirkpatrick's Level 3, was scarcely evaluated, and no studies achieved Kirkpatrick's Level 4. This highlights a gap in linking training to actual skills transfer and organizational outcomes. Further, longer-term follow-up is required to determine if the short-term benefits guide future practice and translate into improvements in the workplace (Hanshaw and Dickerson, 2020).

In summary, this systematic review shows that the predominant use of high-fidelity simulation primarily targets medical personnel, while neglecting other essential team members, thereby limiting the potential for effective interprofessional crisis management. This narrow focus reduces the effectiveness of training in real-world team dynamics, as evidenced by the underutilization of team-based scenarios. The limited emphasis on interprofessional collaboration is further reflected in the lack of assessments at higher Kirkpatrick levels, indicating a gap between training exercises and the complex realities encountered in practice. Consequently, the simulations fall short of fully replicating the intricate challenges of real-world situations.

Future research should explore strategies for leveraging virtual reality to overcome the limitations of high-fidelity simulation and examine how trainer and learner preferences impact its integration into training programs. As technological advancements progress, virtual reality holds substantial potential to revolutionize healthcare education by offering scalable, immersive, and collaborative training solutions. Future studies could also focus on interprofessional simulation to strengthen non-technical skills, particularly in communication and teamwork. An exciting avenue for investigation is in the development of multi-user virtual reality simulations (Edwards et al., 2023), which enable the study of team dynamics in high-risk environments. Collaborative surgical team training in immersive virtual reality represents a particularly
promising area for assessing the effectiveness of this approach in enhancing team performance

## 5. Limitations

This systematic review has limitations, including a restricted search across three databases for articles written exclusively in English, which may have limited the scope of study inclusion. Incorporating other databases that index journals specializing in nursing and allied health sciences, such as CINAHL, might have yielded more comprehensive information. However, CINAHL employs a different Boolean search methodology, and was not accessible through the authors' institutional resources. Moreover, the review did not include gray literature such as conference proceedings and project reports, potentially omitting additional insights into the real-world applications of simulation training for crisis management. These choices were primarily influenced by time and resource constraints. Additionally, the reporting of non-technical skills in the source articles was inconsistent, making their identification challenging at times. A narrative synthesis of the included studies was conducted due to the lack of comparability in design, interventions and heterogeneous outcomes, making it impossible to apply a single summary measure. A meta-analysis was not feasible due to the diversity of interventions and outcomes.

## 6. Theoretical and practical implications

The findings of this review offer several theoretical and practical implications. First, they highlight the need for greater interprofessional education, with increased involvement of nurses and allied healthcare professionals to foster collaborative practices in crisis management. Second, training in team-related crises should be conducted in order to prepare healthcare professionals to effectively handle the situations they most commonly encounter during adverse events. Finally, demonstrating real-world skill transfer remains paramount. It is vital to conduct follow-up evaluations of training programs using level 3 and level 4 Kirkpatrick metrics to confirm sustained behavioral changes that ultimately enhance organizational performance and patient outcomes.

## 7. Conclusion

This systematic review offers valuable insights into the current research landscape regarding simulation programs for developing non-technical skills to manage crises in surgical settings. It reveals a strong preference for high-fidelity approaches, which are tailored to various surgical specialties and primarily focus on enhancing skills such as communication, teamwork, situation awareness and decision-making. While the scenarios address a broad range of health, equipment, and team-based failures, they mainly focus on health failures, despite the fact that adverse events in practice are often due to team-based failures. Although training shows positive reactions and skills acquisition, genuine behavioral change assessments are scarce, and the impact on organizational outcomes remains unclear. By summarizing interprofessional involvement, evaluation practices, crisis training scenarios and real-world skill transfers, this review highlights promising pathways to optimize non-technical skills training, thereby enhancing the education of healthcare professionals and elevating care standards in surgery.

### CRediT authorship contribution statement

**Inas D. Redjem:** Writing – review & editing, Writing – original draft, Visualization, Methodology, Investigation, Data curation, Conceptualization. **Arnaud Huaulmé:** Writing – review & editing, Supervision, Conceptualization. **Pierre Jannin:** Writing – review & editing, Supervision, Methodology, Funding acquisition, Conceptualization. **Estelle Michinov:** Writing – review & editing, Supervision, Methodology, Funding acquisition, Conceptualization.

### Funding statement

This work was supported by state aid managed by the French National Research Agency under the France 2030 program, bearing the reference ANR-21-DMES-0001.

### Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

## Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.nedt.2025.106583.

### Data availability

Data and supplementary materials are available on our OSF project page: https://osf.io/jp94c.

### References

Abelson, J.S., Silverman, E., Banfelder, J., Naides, A., Costa, R., Dakin, G., 2015. Virtual operating room for team training in surgery. Am. J. Surg. 210 (3), 585–590. https://doi.org/10.1016/j.amjsurg.2015.01.024.

Acerro, N.M., Motuk, G., Luba, J., Murphy, M., McKelvey, S., Kolb, G., Dumon, K.R., Resnick, A.S., 2012. Managing a surgical exsanguination emergency in the operating room through simulation: an interdisciplinary approach. J. Surg. Educ. 69 (6), 759–765. https://doi.org/10.1016/j.jsurg.2012.06.022.

Allard, M.-A., Blanéi, A., Brouquet, A., Benhamou, D., 2020. Learning non-technical skills in surgery. J. Visc. Surg. 157 (3, Supplement 2), S131–S136. https://doi.org/10.1016/j.vivisc Surg. 2020.03.001.

Almousa, O., Zhang, R., Dimma, M., Yao, J., Allen, A., Chen, L., Heidari, P., Qayumi, K., 2021. Virtual reality technology and remote digital application for tele-simulation and global medical education: an innovative hybrid system for clinical training. Simul. Gaming 52 (5), 614–634. https://doi.org/10.1177/10468781211008258.

Alsabri, M., Boudi, Z., Lauque, D., Dias, R.D., Whelan, J.S., Östlundh, L., Alinier, G., Oneyej, C., Michel, P., Liu Jr., S.W., Camargo, C.A., Lindner, T., Slagman, A., Bates, D.W., Tazarourte, K., Singer, S.J., Toussi, A., Grossman, S., Bellou, A., 2022. Impact of teamwork and communication training interventions on safety culture and patient safety in emergency departments: a systematic review. J. Patient Saf. 18 (1), e351–e361. https://doi.org/10.1097/PTS.0000000000000782.

Alshehri, F.D., Jones, S., Harrison, D., 2023. The effectiveness of high-fidelity simulation on undergraduate nursing students' clinical reasoning-related skills: a systematic review. Nurse Educ. Today 121, 105679. https://doi.org/10.1016/j.nedt.2022.105679.

Baker, D.P., Amodeo, A.M., Krokos, K.J., Slonim, A., Herrera, H., 2010. Assessing teamwork attitudes in healthcare: development of the TeamSTEPPS teamwork attitudes questionnaire. Qual. Saf. Health Care 19 (6), e49. https://doi.org/10.1136/qshc.2009.036129.

Bierer, J., Memu, E., Leeper, W.R., Fortin, D., Fréchette, E., Inculet, R., Malthaner, R., 2018. Development of an in situ thoracic surgery crisis simulation focused on nontechnical skill training. Ann. Thorac. Surg. 106 (1), 287–292. https://doi.org/10.1016/j.athoracsurg.2018.01.058.

Bracq, M.-S., Michinov, E., Jannin, P., 2019. Virtual reality simulation in nontechnical skills training for healthcare professionals: a systematic review. Simul. Healthc. 14 (3), 188–194. https://doi.org/10.1097/SIH.0000000000000347.

Bracq, M.-S., Michinov, E., Le Duff, M., Arnaldi, B., Gouranton, V., Jannin, P., 2021. Training situational awareness for scrub nurses: error recognition in a virtual operating room. Nurse Educ. Pract. 53, 103056. https://doi.org/10.1016/j.nepr.2021.103056.

Brogaard, L., Glerup Lauridsen, K., Løfgren, B., Krogh, K., Paltved, C., Boie, S., Hvidman, L., 2022. The effects of obstetric emergency team training on patient outcome: a systematic review and meta-analysis. Acta Obstet. Gynecol. Scand. 101 (1), 25–36. https://doi.org/10.1111/aogs.14263.

Buljac-Samardzic, M., Doekhie, K.D., van Wijngaarden, J.D.H., 2020. Interventions to improve team effectiveness within health care: a systematic review of the past decade. Hum. Resour. Health 18, 2. https://doi.org/10.1186/s12960-019-0411-3.

Buljac-Samardzic, M., Dekker-van Doorn, C.M., Maynard, M.T., 2021. What do we really know about crew resource management in healthcare?: an umbrella review on crew resource management and its effectiveness. J. Patient Saf. 17 (8), e929–e958. https://doi.org/10.1097/PTS.0000000000000816.

Burkhart, H.M., Riley, J.B., Lynch, J.J., Suri, R.M., Greason, K.L., Joyce, L.D., Nuttall, G. A., Stulak, J., Schaff, H.V., Dearani, J.A., 2013. Simulation-based postcardiotomy extracorporeal membrane oxygenation crisis training for thoracic surgery residents. Ann. Thorac. Surg. 95 (3), 901–906. https://doi.org/10.1016/j.athoracsurg.2012.12.015.
Calhoun, A.W., Cook, D.A., Genova, G., Motamedi, S.M.K., Waseem, M., Carey, R., Hanson, A., Chan, J.C.K., Camacho, C., Harwayne-Gidansky, I., Walsh, B., White, M., Geis, G., Monachino, A.M., Maa, T., Posner, G., Li, D.L., Lin, Y., 2024. Educational and patient care impacts of in situ simulation in healthcare: a systematic review. Simul. Health. 19 (1S), S23-S31. https://doi.org/10.1097/SIH.0000000000000773.

Cumin, D., Boyd, M.J., Webster, C.S., Weller, J.M., 2013. A systematic review of
simulation for multidisciplinary team training in operating rooms. Simul. Health. 8
(3), 171-179. https://doi.org/10.1097/SIH.0b013e31827e2f4c.

Dedy, N.J., Bonrath, E.M., Zevin, B., Grantcharov, T.P., 2013. Teaching nontechnical skills in surgical residency: a systematic review of current approaches and outcomes. Surgery 154 (5), 1000-1008. https://doi.org/10.1016/j.surgery.2013.04.034.

Dedy, N.J., Szasz, P., Louridas, M., Bonrath, E.M., Husslein, H., Grantcharov, T.P., 2015. Objective structured assessment of nontechnical skills: reliability of a global rating scale for the in-training assessment in the operating room. Surgery 157 (6), 1002-1013. https://doi.org/10.1016/j.surg.2014.12.023.

Dedy, N.J., Bonrath, E.M., Ahmed, N., Grantcharov, T.P., 2016. Structured training to improve nontechnical performance of junior surgical residents in the operating room: a randomized controlled trial. Ann. Surg. 263 (1), 43-49. https://doi.org/10.1097/SLA.00000000000001186.

Edwards, T.C., Soussi, D., Gupta, S., Khan, S., Patel, A., Patil, A., Liddle, A.D., Cobb, J.P., Logishetty, K., 2023. Collaborative team training in virtual reality is superior to individual learning for performing complex open surgery: a randomized controlled trial. Ann. Surg. 278 (6), 850-857. https://doi.org/10.1097/SLA.0000000000000679.

Fletcher, G., Flin, R., McGeorge, P., Glavin, R., Maran, N., Patey, R., 2003. Anaesthetists' non-technical skills (ANTS): evaluation of a behavioural marker system. Br. J. Anaesth. 90 (5), 580-588. https://doi.org/10.1093/bja/aeg112.

Flin, R., O'Connor, P., Crichton, M., 2008. Safety at the Sharp End: A Guide to Non-technical Skills. Ashgate Publishing, Ltd.

Fung, L., Boet, S., Bould, M.D., Qosa, H., Perrier, L., Tricco, A., Tavares, W., Reeves, S., 2015. Impact of crisis resource management simulation-based training for interprofessional and interdisciplinary teams: a systematic review. J. Interprof. Care 29 (5), 433-444. https://doi.org/10.3109/13561820.2015.1017555.

Gao, P., Wang, C., Liu, S., Tran, K.C., Wen, Q., 2021. Simulation of operating room crisis management—hypotension training for pre-clinical students. BMC Med. Educ. 21 (1), 60. https://doi.org/10.1186/s12909-020-02477-8.

Garbee, D., Paige, J., Bonanno, L., 2013. Effectiveness of teamwork and communication education using an interprofessional high-fidelity human patient simulation critical care code. J. Nurs. Educ. 3, 1-12.

Gardner, A.K., AbdelFattah, K., 2017. Comparison of simulation-based assessments and faculty ratings for general surgery resident milestone evaluation: are they telling the same story? Am. J. Surg. 214 (3), 547-553. https://doi.org/10.1016/j.amsurg.2016.07.025.

Gardner, A., DeSandro, S., Pillow, M.T., Ahmed, R., 2020. Optimizing interprofessional education with in situ simulation. In: Paige, J.T., Shones, S.C., Garbee, D.D., Bonanno, L.S. (Eds.), Comprehensive Healthcare Simulation: InterProfessional Team Training and Simulation. Springer International Publishing, pp. 105-120. https://doi.org/10.1007/978-3-030-28845-7.

Gettman, M.T., Pereira, C.W., Lipsky, K., Wilson, T., Arnold, J.J., Leibovich, B.C.,
Karnes, R.J., Dong, Y., 2009. Use of high fidelity operating room simulation to assess
and teach communication, teamwork and laparoscopic skills: initial experience.
J. Urol. 181 (3), 1289-1296. https://doi.org/10.1016/j.juro.2008.11.018.

Goldenberg, M.G., Fok, K.H., Ordon, M., Pace, K.T., Lee, J.Y., 2018. Simulation-based laparoscopic surgery crisis resource management training—predicting technical and nontechnical skills. J. Surg. Educ. 75 (4), 1113-1119. https://doi.org/10.1016/j.jsurg.2017.11.011.

Gordo, C., Mateo, R., Labiano, J., Núñez-Córdoba, J.M., 2021. Ranking quality and patient safety challenges: a nationwide survey of healthcare quality experts from general hospitals in Spain. J. Patient Saf. 17 (8), 541-547. https://doi.org/10.1097/PTS.0000000000000675.

Guimond, M.E., Sole, M.L., Salas, E., 2009. TeamSTEPPS. Am. J. Nurs. 109 (11), 66. https://doi.org/10.1097/01.NAJ.0000363359.84377.27.

Hanshaw, S.L., Dickerson, S.S., 2020. High fidelity simulation evaluation studies in nursing education: a review of the literature. Nurse Educ. Pract. 46, 102818. https://doi.org/10.1016/j.nepr.2020.102818.

Heldring, S., Lindström, V., Jirwe, M., Wihlborg, J., 2024. Exploring ambulance clinicians' clinical reasoning when training mass casualty incidents using virtual reality: a qualitative study. Scand. J. Trauma Resusc. Emerg. Med. 32 (1), 90. https://doi.org/10.1186/s13049-024-01255-5.

Hong, Q.N., Fábregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M.-P., Griffiths, F., Nicolau, B., O'Cathain, A., Rousseau, M.-C., Vedel, I., Pluye, P., 2018. The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers. Educ. Inf. 34 (4), 285-291. https://doi.org/10.3233/EFI-180221.

Howard, S.K., Gaba, D.M., Fish, K.J., Yang, G., Samquist, F.H., 1992. Anesthesia crisis resource management training: teaching anesthesiologists to handle critical incidents. Aviat. Space Environ. Med. 63 (9), 763-770.

Interprofessional Education Collaborative, 2023. IPEC Core Competencies for Interprofessional Collaborative Practice: Version 3.

Jung, J.J., Elfassy, J., Jüni, P., Grantcharov, T., 2019. Adverse events in the operating room: definitions, prevalence, and characteristics. A systematic review. World J. Surg. 43 (10), 2379-2392. https://doi.org/10.1007/s00268-019-05048-1.

Kirkpatrick, D.L., Kirkpatrick, J.D., 2006. Evaluating Training Programs: The Four Levels. Berrett-Koehler Publishers.

Koleva, S.I., 2020. A literature review exploring common factors contributing to never events in surgery. J. Perioper. Pract. 30 (9), 256-264. https://doi.org/10.1177/1750458919886182.

Kyaw, B.M., Saxena, N., Posadzki, P., Vseleckova, J., Nikolaou, C.K., George, P.P., Divakar, U., Masiello, I., Kononowicz, A.A., Zary, N., Tudor Car, L., 2019. Virtual reality for health professions education: systematic review and meta-analysis by the digital health education collaboration. J. Med. Internet Res. 21 (1), e12959. https://doi.org/10.2196/12959.

Lee, K., Wang, F., Reeder, B., 2023. Development of an immersive virtual reality platform for innovation in nursing education. In: Stephanidis, C., Antona, M., Ntoa, S., Salvendy, G. (Eds.), HCI International 2023 Posters. Springer Nature Switzerland, pp. 227-234. https://doi.org/10.1007/978-3-031-36004-631.

Lehner, M., Heimberg, E., Hoffmann, F., Heinzel, O., Kirschner, H.-J., Heinrich, M., 2017. Evaluation of a pilot project to introduce simulation-based team training to pediatric surgery trauma room care. Int. J. Pediatr. 2017, 1-6. https://doi.org/10.1155/2017/9732316.

Lei, C., Palm, K., 2024. Crisis resource management training in medical simulation. In: StatPearls. StatPearls Publishing. http://www.ncbi.nlm.nih.gov/books/NBK551708/.

Leithead, J., Garbee, D.D., Yu, Q., Rusnak, V.V., Kiselov, V.J., Zhu, L., Paige, J.T., 2019. Examining interprofessional learning perceptions among students in a simulation-based operating room team training experience. J. Interprof. Care 33 (1), 26-31. https://doi.org/10.1080/13561820.2018.1513464.

Mai, C.L., Wongsirimeteekul, P., Petrusa, E., Minehart, R., Hemingway, M., Pian-Smith, M., Eromo, E., Phitayakorn, R., 2020. Prevention and management of operating room fire: an interprofessional operating room team simulation case. MedEDPORTAL, 10871. https://doi.org/10.15766/medep-2374-8265.10871.

Martin, A., Cross, S., Attoe, C., 2020. The use of in situ simulation in healthcare education: current perspectives. Adv. Med. Educ. Pract. 11, 893-903. https://doi.org/10.2147/AMEP.S188258.

Mazzocco, K., Petitti, D.B., Fong, K.T., Bonacum, D., Brookey, J., Graham, S., Lasky, R.E., Sexton, J.B., Thomas, E.J., 2009. Surgical team behaviors and patient outcomes. Am. J. Surg. 197 (5), 678-685. https://doi.org/10.1016/j.amsjurg.2008.03.002.

Meríán, A.E.R., van de Ven, J., Mol, B.W., Houterman, S., Oei, S.G., 2010. Multidisciplinary team training in a simulation setting for acute obstetric emergencies: a systematic review. Obstet. Gynecol. 115 (5), 1021-1031. https://doi.org/10.1097/AOG.0b013e3181d9f4cd.

Mistry, D., Brock, C.A., Lindsey, T., 2023. The present and future of virtual reality in medical education: a narrative review. Cureus 15 (12), e51124. https://doi.org/10.7759/cureus.51124.

Mitchell, L., Flin, R., Yule, S., Mitchell, J., Coutts, K., Youngson, G., 2012. Evaluation of the scrub practitioners' list of intraoperative non-technical skills system. Int. J. Nurs. Stud. 49 (2), 201-211. https://doi.org/10.1016/j.ijnurstu.2011.08.012.

Moorthy, K., Munz, Y., Forrest, D., Pandey, V., Undre, S., Vincent, C., Darzi, A., 2006. Surgical crisis management skills training and assessment: a stimulation-based approach to enhancing operating room performance. Ann. Surg. 244 (1), 139-147. https://doi.org/10.1097/01.sla.0000217618.307441.

Murtagh, G.M., Bezemer, J., 2021. 'Hm no-one says anything, did you notice?'?: communication in the operating theatre amid a hierarchy of expertise. Commun. Med. 17 (1). https://doi.org/10.1558/cam.15138.

Newton, J., Smith, A.D.A.C., 2024. Developing emotional preparedness and mental resilience through high-fidelity simulation: a "bridge too far" for institutions teaching major trauma management and mass-casualty medicine? BMC Med. Educ. 24 (1), 544. https://doi.org/10.1186/s12909-024-05526-8.

Nguyen, N., Elliott, J.O., Watson, W.D., Dominguez, E., 2015. Simulation improves nontechnical skills performance of residents during the perioperative and intraoperative phases of surgery. J. Surg. Educ. 72 (5), 957-963. https://doi.org/10.1016/j.jsurg.2015.03.005.

Ounounou, E., Aydin, A., Brunckhorst, O., Khan, M.S., Dasgupta, P., Ahmed, K., 2019. Nontechnical skills in surgery: a systematic review of current training modalities. J. Surg. Educ. 76 (1), 14-24. https://doi.org/10.1016/j.jsurg.2018.05.017.

Page, M.J., McKenzie, J.E., Bossuyt, P.M., Boutron, I., Hoffmann, T.C., Mulrow, C.D., Shamseer, L., Tetzlaff, J.M., Akl, E.A., Brennan, S.E., Chou, R., Glanville, J., Grimshaw, J.M., Hróbjartsson, A., Lalu, M.M., Li, T., Loder, E.W., Mayo-Wilson, E., McDonald, S., Moher, D., 2021. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 372, n71. https://doi.org/10.1136/bmj.n71.

Paige, J.T., Aaron, D.L., Yang, T., Howell, D.S., Chauvin, S.W., 2009a. Improved operating room teamwork via SAFETY prep: a rural community hospital's experience. World J. Surg. 33 (6), 1281. https://doi.org/10.1007/s00268-009-9952-2.

Paige, J.T., Kozmenko, V., Yang, T., Paragi Gururaja, R., Hilton, C.W., Cohn, I., Chauvin, S.W., 2009b. High-fidelity, simulation-based, interdisciplinary operating room team training at the point of care. Surgery 145 (2), 138-146. https://doi.org/10.1016/j.surg.2008.09.010.

Paige, J.T., Garbee, D.D., Kozmenko, V., Yu, Q., Kozmenko, L., Yang, T., Bonanno, L., Swartz, W., 2014. Getting a head start: high-fidelity, simulation-based operating room team training of interprofessional students. J. Am. Coll. Surg. 218 (1), 140-149. https://doi.org/10.1016/j.jamcollsurg.2013.09.006.

Panagioti, M., Khan, K., Keers, R.N., Abuzour, A., Phipps, D., Kontopantelis, E., Bower, P., Campbell, S., Haneef, R., Avery, A.J., Ashcroft, D.M., 2019. Prevalence, severity, and nature of preventable patient harm across medical care settings: systematic review and meta-analysis. BMJ (Clin. Res. Ed.) 366, l4185. https://doi.org/10.1136/bmj.l4185.

Pena, G., Altree, M., Field, J., Sainsbury, D., Babidge, W., Hewett, P., Maddern, G., 2015. Nontechnical skills training for the operating room: a prospective study using
simulation and didactic workshop. Surgery 158 (1), 300–309. https://doi.org/
10.1016/j.surg.2015.02.008.

Peters, W., Picchioni, A., Fleshman, J.W., 2020. Surgical leadership. Clin. Colon Rectal
Surg. 33 (4), 233–237. https://doi.org/10.1055/s-0040-1712976.

Pottle, J., 2019. Virtual reality and the transformation of medical education. Future Healthc. J. 6 (3), 181–185. https://doi.org/10.7861/fhj.2019-0036.

Raison, N., Ahmed, K., Abe, T., Brunckhorst, O., Novara, G., Buffi, N., McIlhenny, C., Van Der Poel, H., Van Hemelrijck, M., Gavazzi, A., Dasgupta, P., 2018. Cognitive training for technical and non-technical skills in robotic surgery: a randomised controlled trial. BJU Int. 122 (6), 1075–1081. https://doi.org/10.1111/bju.14376.

Ramjeeawon, A., Sharrock, A.E., Morbi, A., Martin, G., Riga, C., Bicknell, C., 2020. Using fully-immersive simulation training with structured debrief to improve nontechnical skills in emergency endovascular surgery. J. Surg. Educ. 77 (5), 1300–1311. https://doi.org/10.1016/j.jsurg.2020.03.023.

Richardson, W.S., Wilson, M.C., Nishikawa, J., Hayward, R.S., 1995. The well-built clinical question: a key to evidence-based decisions. ACP J. Club 123 (3), A12-A13.

Robertson, E.R., Hadi, M., Morgan, L.J., Pickering, S.P., Collins, G., New, S., Griffin, D., McCulloch, P., Catchpole, K.C., 2014. Oxford NOTECHS II: a modified theatre team non-technical skills scoring system. PLoS One 9 (3), e90320. https://doi.org/
10.1371/journal.pone.0090320.

Robertson, J.M., Dias, R.D., Yule, S., Smink, D.S., 2017. Operating room team training with simulation: a systematic review. J. Laparoendosc. Adv. Surg. Tech. A 27 (5), 475–480. https://doi.org/10.1089/lap.2017.0043.

Rochlen, L.R., Malloy, K.M., Chang, H., Kim, S., Guichard, L., Cassidy, R., Zisblatt, L., 2019. Pilot one-hour multidisciplinary team training simulation intervention in the operating room improves team nontechnical skills. J. Educ. Perioper. Med. 21 (2). https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6972973/.

Rogers, B.A., Franklin, A.E., 2021. Cognitive load experienced by nurses in simulation-based learning experiences: an integrative review. Nurse Educ. Today 99, 104815. https://doi.org/10.1016/j.nedt.2021.104815.

Rudolph, J.W., Simon, R., Dufresne, R.L., Raemer, D.B., 2006. There's no such thing as "nonjudgmental" debriefing: a theory and method for debriefing with good judgment. Simul. Healthc. 1 (1), 49–55. https://doi.org/10.1097/01266021-200600110-00006.

Saragih, I.D., Tarihoran, D.E.T.A.U., Lin, W.-T., Lee, B.-O., 2024. Outcomes of scenario-
based simulation courses in nursing education: a systematic review and meta-
analysis. Nurse Educ. Today 136, 106145. https://doi.org/10.1016/j.
nedt.2024.106145.

Scherer, K., Winokur, R.S., 2019. Multidisciplinary team training simulation in interventional radiology. Tech. Vasc. Interv. Radiol. 22 (1), 32–34. https://doi.org/
10.1053/j.tvir.2018.10.007.

Schmutz, J.B., Meier, L.L., Manser, T., 2019. How effective is teamwork really? The relationship between teamwork and performance in healthcare teams: a systematic review and meta-analysis. BMJ Open 9 (9), e028280. https://doi.org/10.1136/
bmjopen-2018-028280.

Shi, R., Marin-Nevarez, P., Hasty, B., Roman-Micek, T., Hirx, S., Anderson, T., Schmiederer, I., Fanning, R., Goldhaber-Fiebert, S., Austin, N., Lau, J.N., 2021. Operating room in situ interprofessional simulation for improving communication and teamwork. J. Surg. Res. 260, 237–244. https://doi.org/10.1016/j.
jsr.2020.11.051.

Sirihorachai, R., Aebersold, M.L., Sarter, N.B., Harris, M., Marsh, V., Redman, R.W., 2018. Examining interruptions in the operating room using simulation. Clin. Simul. Nurs. 24, 14–24. https://doi.org/10.1016/j.ecns.2018.08.004.

Smits, M., Langelaan, M., de Groot, J., Wagner, C., 2021. Examining causes and prevention strategies of adverse events in deceased hospital patients: a retrospective patient record review study in the Netherlands. J. Patient Saf. 17 (4), 282–289. https://doi.org/10.1097/PTS.0000000000000586.

Taaffe, J.P., Kabbani, L.S., Goltz, C.J., Bath, J., Mattos, M.A., Caputo, F.J., Singh, P., Vogel, T.R., 2021. Feasibility and evaluation of surgical simulation with developed crisis scenarios: a comparison of performance by vascular surgery training paradigms. J. Surg. Educ. 78 (6), 2110–2116. https://doi.org/10.1016/j.
jsurg.2021.05.007.

Taylor, R.M., 2011. Situational awareness rating technique (Sart): the development of a tool for aircrew systems design. In: Situational Awareness. Routledge.

Thim, S., Henriksen, T.B., Laursen, H., Schram, A.L., Paltved, C., Lindhard, M.S., 2022. Simulation-based emergency team training in pediatrics: a systematic review. Pediatrics 149 (4), e2021054305. https://doi.org/10.1542/peds.2021-054305.

Truong, H., Qi, D., Ryason, A., Sullivan, A.M., Cudmore, J., Alfred, S., Jones, S.B., Parra, J.M., De, S., Jones, D.B., 2022. Does your team know how to respond safely to an operating room fire? Outcomes of a virtual reality, AI-enhanced simulation training. Surg. Endosc. 36 (5), 3059–3067. https://doi.org/10.1007/s00464-021-08602-y.

Undre, S., Sevdalis, N., Healey, A.N., Darzi, A., Vincent, C.A., 2007. Observational teamwork assessment for surgery (OTAS): refinement and application in urological surgery. World J. Surg. 31 (7), 1373–1381. https://doi.org/10.1007/s00268-007-9053-z.

Watkins, S.C., Hensley, N.B., 2023. Team dynamics in the operating room: how is team performance optimized? Anesthesiol. Clin. 41 (4), 775–787. https://doi.org/
10.1016/j.anclin.2023.05.004.

Weller, J.M., Webster, C.S., 2021. Normalising good communication in hospital teams. Br. J. Anaesth. 126 (4), 758–760. https://doi.org/10.1016/j.bja.2020.12.036.

Weller, J., Cumin, D., Torrie, J., Boyd, M., Civil, I., Madell, D., MacCormick, A., Gurisinghe, N., Crossan, M., Ng, W.L., Johnson, S., Corter, A., Lee, T., Selander, L., Cokorilo, M., Merry, A.F., 2015. Multidisciplinary operating room simulation-based team training to reduce treatment errors: a feasibility study in New Zealand hospitals, 128 (1418).

Wongsirimeetekul, P., Mai, C.L., Petrusa, E., Minehart, R., Hemingway, M., Pian-Smith, M., Eromo, E., Phitayakorn, R., 2018. Identifying and managing intraoperative arrhythmia: a multidisciplinary operating room team simulation case. MedEdPORTAL, 10688. https://doi.org/10.15766/mep_2374-8265.10688.

Wood, T.C., Maqsood, S., Zoutewelle, S., Nanavaty, M.A., Rajak, S., 2021. Development of the HUman Factors in intraoperative Ophthalmic Emergencies Scoring System (HUFOES) for non-technical skills in cataract surgery. Eye (Lond.) 35 (2), 616–624. https://doi.org/10.1038/s41433-020-0921-1.

Wood, T.C., Maqsood, S., Saunders, A., Sancha, W., Nanavaty, M.A., Wearne, M., Rajak, S., 2023. Non-technical skills simulation-based training model for managing intraoperative posterior capsule rupture during cataract surgery. Eye 37 (3), 474–479. https://doi.org/10.1038/s41433-022-01962-2.

Wunder, L., Gomez, N.A.G., Gonzalez, J.E., Mitzova-Vladinov, G., Cacchione, M., Mato, J., Foronda, C.L., Groom, J.A., 2020. Fire in the operating room: use of mixed reality simulation with nurse anesthesia students. Informatics 7 (4), 40. https://doi.org/10.3390/informatics7040040.

Yucel, C., Hawley, G., Terzioglu, F., Bogossian, F., 2020. The effectiveness of simulation-based team training in obstetrics emergencies for improving technical skills: a systematic review. Simul. Healthc. 15 (2), 98–105. https://doi.org/10.1097/SIH.0000000000000416.

Yule, S., Flin, R., Maran, N., Rowley, D., Youngson, G., Paterson-Brown, S., 2008. Surgeons' non-technical skills in the operating room: reliability testing of the NOTSS behavior rating system. World J. Surg. 32 (4), 548–556. https://doi.org/10.1007/s00268-007-9320-z.

Yule, S., Rowley, D., Flin, R., Maran, N., Youngson, G., Duncan, J., Paterson-Brown, S., 2009. Experience matters: comparing novice and expert ratings of non-technical skills using the NOTSS system. ANZ J. Surg. 79 (3), 154–160. https://doi.org/
10.1111/j.1445-2197.2008.04833.x.