Minimizing Peri-operative Time Delays, Maximizing Operating Room Efficiency: A Pilot Study
Conditions: Operating Room Efficiency
Sponsor: Unity Health Toronto

 
 
 
 
 
 
OR Black Box: Pilot Study 
Version 27-Feb-20 
1
Minimizing OR Cancellations, Maximizing OR Utilization: A Pilot 
Study  
TABLE OF CONTENTS 
I. 
Statement of Purpose 
II. 
Background and Rationale 
III. 
Research Objectives 
IV. 
Hypothesis and Novelty 
V. 
Methods 
VI. 
Ethical Considerations 
VII. 
Relevance 
 
Appendix 1 
Appendix 2 
Appendix 3 
Appendix 4 
Appendix 5 
Appendix 6 
Appendix 7 
 
References 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
OR Black Box: Pilot Study 
Version 27-Feb-20 
2
I. Statement of Purpose 
The purpose of this pilot study is to prospectively identify operating room time delays, and 
assess the contributing impact of non-technical skills, team performance, and 
environmental factors in the operating room. This analysis will be used to identify root 
causes leading to delays and define areas for improvement.  
 
II. Background and Rationale 
Time delays in the operating room play a role in increasing wait time length, number of 
cancelled cases, health care expenditure and resources. Furthermore, they inconvenience 
patients and families, and delay care. Minimally invasive surgery (laparoscopy) affords 
same-day discharge, faster recovery, and decreased morbidity. However, performing 
laparoscopy requires special equipment and multiple steps during set-up that can lead to 
time delays if not performed efficiently. Multiple factors have been identified to 
contribute to inefficient care delivery including (1): communication error, equipment 
failure, lack of teamwork among nursing, surgical or anesthesia teams, and duplication of 
tasks. 
 
The Institute of Medicine (IOM) report “To err is human,” released in 2000, set off 
numerous studies to identify errors in healthcare and to develop strategies to avoid adverse 
outcome (2). The most commonly used types of interventions in surgery to date have been 
preoperative “surgical safety checklist” for aviation-style briefing (3-5). Other 
interventions have focused on training isolated components of non-technical skills, such as 
communication (6, 7).  
 
The systems approach to human error is based on the assumption that human error is 
inevitable and looks for failures in the system rather than in the individual. This novel 
approach has led to the current concept of Crew Resource Management training as a 
strategy to enhance non-technical skills in order to avoid, trap and mitigate error and 
ultimately increase safety (8-10). This concept can be extended to the area of interest of 
time delays. Unanticipated events such as human resource absenteeism, equipment failure 
or malfunction, and unscheduled emergency add-on cases as examples, inevitably will arise 

 
 
 
 
 
 
OR Black Box: Pilot Study 
Version 27-Feb-20 
3
to cause time delays. These delays can occur in the OR directly, or in other areas of the 
hospital which can in turn affect the flow in the OR. Efficient use of the OR requires expert 
technical skills of the surgical team, but more importantly teamwork within the peri-
operative team and administration that are prepared to accommodate an array of potential 
unforeseen circumstances that can lead to time delays. Understanding what are the most 
common types of delays, the frequency and trends in which they occur, and the root causes 
for the delays can help prevent delay by being prepared, that is, anticipating delay 
possibilities and having a plan in place to minimize delay. Although factors outside of the 
OR can influence the intra-operative period, this study focuses specifically on determining 
which factors within the OR can be targeted to improve efficiency.   
 
An innovative technology, the OR Black Box, has been developed for the OR to mimic the 
blackbox in airline industry. It has the ability to record the surgical field, the anesthetic 
equipment, and the level of noise in the OR. This technology also has the ability to time 
stamp all activities that take place in the OR. Using this technology, we will be able to 
record all of the important check points that take place in the course of a complex 
laparoscopic procedure (i.e. total laparoscopic hysterectomy). This will in turn highlight 
several potential areas for examining efficiency that can be generalized to other surgical 
specialities. This analysis will be used to identify root cause(s) leading to OR delays and 
define areas for improvement. The causes of delays can be reported to the team members 
through formative performance-based feedback (11). 
 
Summary 
OR delays are a result of a chain of events spanning multiple phases of operative patient 
care. Currently, there are no reported methods available to objectively observe and evaluate 
causes for OR efficiency including non-technical skills factors. Delays are tracked by the 
nursing staff which is a subjective process fraught with inconsistency and associated biases. 
Objective measures of intra-operative OR delays, identification of their root causes, and 
creation of an intervention to improve OR efficiency has the potential for the development 
of a standardized framework to minimize OR efficiency, which could be generalized to 
any surgical discipline to improve OR efficiency. 

 
 
 
 
 
 
OR Black Box: Pilot Study 
Version 27-Feb-20 
4
 
III. Research Objectives 
1) To prospectively identify patterns or chain of events leading to prolonged case duration 
in total laparoscopic hysterectomy procedure in a gynecologic surgical population. 
2) To assess how non-technical skills and team performance of surgeon, nursing and 
anesthesiology teams correlate with case duration and subsequent cancellations. 
 
 
IV. Hypothesis & Novelty 
We hypothesize that non-technical skills and teamwork in the OR are correlated with 
operating room efficiency. Furthermore, we hypothesize that time delays can be tracked to 
a chain of events that is influenced by non-technical skills as well as environmental factors 
in addition to technical skills. Together, this knowledge can be used to identify areas for 
organizational, technological, and team or individual improvement, and to design a 
standardized framework to improve individual and team performance and ultimately 
increase OR efficiency that can be applied to any surgical department. The novelty of this 
approach is the prospective assessment of non-technical, team performance, and 
environmental factors in the OR as well as delays, inefficiencies, and patient outcome in 
order to identify areas for improvement and characterize a chain of events.  
 
V. Methods 
Experimental Design 
1. Prospective observational study  
2. Randomized trial (Sub-study/branch of pilot study: Measuring the impact of 
standardized efficiency framework to optimize OR utilization) 
 
General Design 
Study Subjects 
A minimum of sixty elective total laparoscopic hysterectomy procedures performed in one 
operating room of an academic tertiary care center during a twelve-month period will be 
included in this study. Procedures will only be excluded if data collection is in any part 

 
 
 
 
 
 
OR Black Box: Pilot Study 
Version 27-Feb-20 
5
incomplete or patient or OR staff consent was declined or withdrawn. Studying total 
laparoscopic hysterectomy was selected as a marker to analyze OR efficiency as this is a 
commonly performed procedure that requires moderate degree of complexity in equipment 
set-up and can be generalized to other laparoscopic procedures. Health professionals that 
will be participating in the study are OR nursing staff, technical (scrub) assistants, 
anesthesia staff, and surgeons. The nursing team involved in each procedure typically 
consists of one scrub technical assistant or nurse and one circulating nurse. The anesthesia 
team is typically represented by either a staff anesthesiologist or an anesthesia resident 
(who is always supervised by a staff anesthesiologist). The surgical team is comprised of 
the staff surgeon, and two other surgeons, typically a fellow and a surgical resident. The 
staff surgeon will be present during all cases and perform the majority of procedures 
however, some full procedures, or particular steps of procedures, will be performed by a 
clinical fellow or a surgical resident under staff surgeon’s direct supervision, which is 
standard practice in teaching hospitals. Medical students will occasionally also be part of 
the surgical OR team as part of their surgical rotation during clinical clerkships.  
 
Recruitment and consent of patients 
All patients that are scheduled for elective surgery will be eligible for participation in the 
study. Minors and patients with mental disability or impairment are not eligible for the 
study. Pregnant women will also not be enrolled. All eligible patients will be informed 
about the study by the treating surgeon during their appointment in the clinics. Each patient 
will then receive written information (see Patient Consent form) to read at home in order 
to provide adequate time for consideration. All patients that have been scheduled for 
surgery routinely return for a second presentation for a consult with Anesthesiology, this 
is typically two to three weeks prior to the surgery. Upon this pre-admission appointment, 
the Research Coordinator, who is not involved in patient treatment, will provide additional 
information regarding the details of the study to the patients and will obtain informed 
consent. Patients that are scheduled for elective total laparoscopic hysterectomy surgery 
will be enrolled in the study, provided that informed consent has been obtained. It is 
furthermore important to note that participation in the study will not result in any deviation 

 
 
 
 
 
 
OR Black Box: Pilot Study 
Version 27-Feb-20 
6
of the standard treatment of patients in our unit. This includes all visits prior to the 
procedure, the surgery and hospital stay, and postoperative care. 
 
Recruitment of nursing staff 
Prior to the start of the study, approval will be obtained from the clinical manager of 
perioperative services (OR leader). Nursing staff will be identified through the roster and 
addressed by the Clinical Leader Managers of Perioperative Services, via their St. 
Michael’s Hospital email addresses. All nursing staff scheduled to work with this unit will 
be invited by the clinical manager via email to attend an informative presentation given by 
one of the researchers (see Appendix 1). After the presentation, the Research Coordinator 
will then obtain written informed consent. All staff members who did not have the 
opportunity to attend the presentation will be informed of the study by the Research 
Coordinator. Nursing students are occasionally scheduled in the unit’s OR, mostly to 
observe the procedure for training purposes (i.e., not active participants). Nursing students 
who are scheduled to attend one of the unit’s OR lists will be informed of the study by the 
Nurse Educator and informed consent will be obtained by the Research Coordinator. If any 
member of the staff declines consent or withdraws consent at any point during the study, 
procedures involving that staff member will not be recorded from that point on. A reminder 
e-mail will be sent to staff who are scheduled in the OR but who did not provide informed 
consent or were unable to attend the Grand Rounds presentation (see “E-mail consent 
reminder”). This e-mail will remind them of the study and provide another opportunity to 
sign the consent form if they wish to participate.  
 
Recruitment of anesthesia staff 
Anesthesia staff (including residents and fellows) who are scheduled in the unit’s OR will 
be informed of the study by staff anesthetists via their St. Michael’s email addresses (see 
Appendix 1). All anesthesia staff will be invited to join an informative presentation about 
the study that will be held by one of the researchers at departmental Grand Rounds. After 
the presentation, the Research Coordinator will obtain written informed consent from all 
anesthesiology staff who wish to participate. If consent is declined or withdrawn by an 
anesthesia staff member, resident, or fellow, procedures will not be recorded when the 

 
 
 
 
 
 
OR Black Box: Pilot Study 
Version 27-Feb-20 
7
respective person is scheduled for the unit’s OR list. Anesthesia staff, residents, and fellows 
will be informed that a decision not to participate in the study will not affect their rotation 
or professional standing at St. Michael’s Hospital in any way.  A reminder e-mail will also 
be sent to anesthesiology staff who are scheduled in the OR, but who did not provide 
informed consent or were unable to attend the Grand Rounds presentation 
 
Recruitment of surgical staff 
The project investigator as well as the participating staff surgeons in the Department of 
Gynecologic Surgery at St. Michael’s Hospital will be study subjects in all observed 
procedures, and therefore will also sign the written informed consent. All gynecology 
surgical staff (including fellows and trainees) will be initially contacted about the study via 
e-mail and will be invited to attend an informative presentation at the next Grand Rounds 
session (see Appendix 1).  Surgical residents will be contacted via e-mail by the Program 
Director’s office. Occasionally, medical students participate in the unit’s OR for clinical 
clerkships. The rotation plan of surgical clerkship students is known to the undergraduate 
medical education coordinator for Fitzgerald academy, St. Michael’s Hospital, medical 
students scheduled for the unit’s OR lists will be contacted through the medical student 
speaker as a student representative. Informed consent will be obtained from all surgical 
staff who wish to participate in the study. This will be obtained by the Research 
Coordinator at the end of the Grand Rounds presentation. Prior to the day of scheduled 
surgery, a reminder email will be sent to all staff who have not yet provided informed 
consent or who were unable to attend the Grand Rounds presentation.  
 
Declined or withdrawn consent 
The informed consent of all participants (patients, healthcare staff, residents and students) 
has the highest priority for the researchers. If any team member in the OR has not given 
consent or has previously withdrawn consent, the procedure will not be recorded. If a staff 
member agrees, he or she may be scheduled for a different OR. To ensure that all members 
of the operating team have given consent, the Research Coordinator will monitor the OR 
lists and staff schedules one day prior to the list to ensure valid informed consent is 
available of all team members scheduled for that day. For that purpose, the Research 

 
 
 
 
 
 
OR Black Box: Pilot Study 
Version 27-Feb-20 
8
Coordinator will keep a master log that contains all names and roles of participants that 
have given informed consent (Appendix 7). If an OR team member has not given valid 
consent or has withdrawn consent, the Research Coordinator will ensure that on that 
particular day no recording is made. Consent of staff members will be valid for 12 months. 
This is explained in the staff consent form. Every study participant can withdraw consent 
at any point in time during the study. If a participating staff member withdraws consent, 
no procedures will be recorded from that point on, if the respective staff member is present. 
A patient participating in the study can also withdraw consent at any point before her 
procedure. Both staff members and patients can withdraw consent for procedures that have 
already been recorded, if they do so within 48 hours of the procedure. In order to maintain 
the scientific integrity of the study and avoid bias, the recordings may not be withdrawn 
beyond the 48-hour timeframe, nor may the other data collected up to the point of 
withdrawal. Data from procedures that have already been video-audio recorded and 
evaluated while valid consent existed cannot be withdrawn. This is a requirement to 
maintain the scientific integrity of the study and this will be clarified in the consent forms. 
 
The researchers acknowledge that the OR staff may feel pressured by the recording 
and the assessment of their performance. Nursing staff will only be evaluated with regards 
to non-technical and team skills. It is important to acknowledge, that individual 
performance of staff members will not be evaluated. Instead, the respective roles in the OR 
(primary surgeon, secondary surgeons, circulating nurse, scrub nurse) are evaluated, thus 
no names or subject numbers will be linked to the analysis of OR delays. Each procedure 
will only be linked to a patient code, to ensure procedure, team performance, and outcome 
can be correlated to this case. Healthcare staff will not appear on any data collection forms. 
Consent forms and the staff consent master log are the only documents containing 
healthcare staff names and will be kept secure in a locked office at all times that can only 
be accessed by the researchers. Consent forms of healthcare staff cannot be linked to a 
particular case. 
 
To avoid any feeling of coercion or pressure to participate, the Research 
Coordinator who is not involved in surgical procedures can be contacted by the participants 
in order to withdraw informed consent. Contact details will be provided on the final version 
of the consent form. The Research Coordinator will keep the name of the person who has 

 
 
 
 
 
 
OR Black Box: Pilot Study 
Version 27-Feb-20 
9
withdrawn consent anonymous and delete their name from the master log to ensure no 
further procedures will be recorded when this staff member is in the OR. As an additional 
safety barrier, a sign will be affixed on all doors of the unit’s OR to inform any person 
entering the OR that recording may be in progress (Appendix 9). 
 
Data collection 
All eligible procedures will be continuously video- and audio- recorded. A panoramic view 
of the entire OR will be obtained using multiple surveillance cameras. The inside view of 
the laparoscopic camera will be recorded, as it is already standard practice. Video and audio 
recordings will start once the patient has entered the operating room and the user has started 
capture. Recordings will end when the patient exits the room and the user ends capture. 
The video and audio recordings will be automatically encrypted at the point of capture and 
stored behind a secure firewall within St. Michael’s hospital. The encrypted videos will be 
transferred by secure socket connection to a St. Michael’s Hospital research center - the 
International Centre for Surgical Safety (ICSS) – which is also protected by a secure 
firewall, where the videos will remain in a ‘badge-access-only’ office, on a multiple-
password-protected computer. Next, a single study staff member with access to this 
computer will decrypt the videos and “blur” any frames in which the patient’s face and/or 
sensitive body parts (i.e. genital area, breasts) are visible in order to preserve the patient’s 
anonymity and dignity. The study staff member will then re-encrypt the files which will 
remain encrypted in the same protected environment until a certified clinical data analyst 
(i.e. a non-study-staff employee of ICSS who is rigorously trained to review operative 
videos) decrypts the videos and analyzes the videos for variables of interest as described 
in detail below.  
 
Patient demographics as well as diagnoses and risk factors will be collected on an 
electronic patient data sheet (Appendix 2). The information on the patient data sheet is 
important for the study to stratify the data for patient risk factors and to identify 
organizational factors such as delays and environmental influences such as time of day. 
Video- and audio- recordings as well as all other recordings of data will capture time but 
not date. The time of day is important, since some factors such as vigilance can be affected 

 
 
 
 
 
 
OR Black Box: Pilot Study 
Version 27-Feb-20 
10
by circadian rhythms of individuals. Additionally, we will record if any cases were 
cancelled following the hysterectomy that was captured. All data records, documentation 
or information containing patient data will be de-identified with study numbers to ensure 
that persons outside of the study will not be able to identify the participating patients. The 
code will be kept on a password protected hard drive that will only be accessible by the 
Research Coordinator. The researchers will however be given access to the data sheets as 
required to enter additional information such as follow-up data.  
The recorded, analyzed, de-identified video files will be retained until completion 
of the recording review period (30 days). After 30 days the video and audio files will be 
deleted. A small sample of “segments of interest” (i.e. short video segments deemed to be 
highly educational) may be kept beyond 30 days, however these will not contain any 
identifying information for either the patient or staff members. Any retained “segments of 
interest” will undergo further de-identification processes in which all faces are blurred and 
all voices are pitched, thus preserving the anonymity of all members of the operating room 
team throughout these segments. The research data will be stored and the code linking the 
case data to a patient name will be deleted at 30 days. After that, the data will be 
permanently anonymized. 
 
Hawthorne effect 
The Hawthorne effect describes a phenomenon of an unintentional change in behavior in 
response to the presence of an observer. The Hawthorne effect can affect any study that 
involves observation of the study subjects and can therefore not be completely eliminated 
(12). It is known that the Hawthorne effect typically fades with time, as the subjects are 
getting used to the observation, especially if the presence of an observer is not directly 
visible. Two factors may reduce the Hawthorne effect in our study: 1) long duration of the 
study over 12 months, which is likely to lead to an acclimatization of the subjects once the 
novelty of the cameras wears off. 2) Video recordings will be made with small surveillance 
cameras that are mounted on the wall. The unobtrusive nature of the observation is likely 
going to accelerate the habituation, further reducing the Hawthorne effect. 
 
Disclosure of errors and adverse events 

 
 
 
 
 
 
OR Black Box: Pilot Study 
Version 27-Feb-20 
11
Technical errors in our study are defined as the smallest measurable unit reflecting a non-
deliberate action or deviation from a plan. These alone may not have the potential to lead 
to adverse outcome. Such errors may be the dropping the needle whilst suturing or the 
incorrect orientation of the needle driver, or communication that does not follow a closed 
loop reconfirmation. “Events” represent the next level of error analysis and these have the 
potential to influence outcome, if not rectified. Examples would be a serosal injury of small 
bowel, which would require suture repair. An event in communication representing 
miscommunication for example would be poor instructions leading to the opening of the 
wrong sterile instrument causing additional cost. In the case of patient events needing 
medical rectification these will naturally as per standard practice be described in the OR 
notes and documented. This is the standard operating procedure for these types of events. 
In cases with more severe consequences the intraoperative complication will be discussed 
with the patient as per standard of care. 
 
Study data is abstract and at this point not linked to outcome. Only factors, which 
actually influence the treatment or need rectification will be discussed with the patient. 
This is already the standard procedure and will not be changed during the study. Incidental 
findings during the surgery will naturally be discussed with the patient since this is standard 
medical care. 
 
Sample size 
This study represents a pilot study to establish knowledge about time delay rates, surgical 
technical and non-technical skill components as well as to determine failures in technology 
(equipment failure). These numbers are currently unknown and we therefore hope to obtain 
an adequate knowledge base for future research in this field. What is known is that 
operating room cancellations and delays occur. These represent aberrations from the ideal 
task execution, which occur non-deliberately. They do not necessarily cause measureable 
effects in outcome. It is more likely, that an accumulation of events (that results from an 
issue that requires corrective measure) can influence a number of outcome parameters such 
as OR time, cost, and even patient outcome. But the delay rate itself may be influenced by 
other parameters such as patient factors (weight, gender, comorbidities, etc.).  
 

 
 
 
 
 
 
OR Black Box: Pilot Study 
Version 27-Feb-20 
12
Setting 
The study will be conducted at St. Michael’s Hospital – a University of Toronto affiliated, 
tertiary-care teaching hospital. 
 
Evaluation of collected data 
1) Time delays 
Data for the time period that the patient enters and exits the OR will be captured with the 
OR Black Box technology. Recordings will begin once the patient enters the operating 
room and will be stopped once she leaves. Blurring of her face and sensitive body parts 
will be performed by a study team member prior to data analysis. The objective time stamps 
and occurrence of events to be collected are outlined in Appendix 3.   
 
2) Non-Technical Skills 
Video- and audio-recordings of all procedures will be evaluated by the OR Black Box data 
analysts with formal training in CRM-evaluation using SPLINTS(13), NOTSS (14), and/or 
the ANTS rating scales (Appendix 4, 5, 6). The SPLINTS and the NOTSS rating scales 
were derived from frameworks for the observation of team behaviors and have been 
modified for use in surgical teams (13-15). The ANTS framework has been developed to 
assess the non-technical skills of the anesthesia team. These rating scales have previously 
been shown to have construct validity and good inter-rater reliability. Recordings of any 
patients that encounter complications directly related to the index procedure will be 
reviewed and screened again using the SPLINTS, ANTS and the NOTSS rating scales 
(Appendix 4, 5, 6). Twenty percent of recorded procedures will be randomly selected for 
evaluation by a second observer using these tools. The second observer for the blinded 
evaluation of the audiovisual recordings will be a surgeon with experience in the use of 
behavioural rating scales. The intra-class correlation coefficient (ICC) will be determined 
for the results of both observers. An ICC equal or greater than 0.75 will be considered 
acceptable.  
 
 
Primary Outcome Measure 

 
 
 
 
 
 
OR Black Box: Pilot Study 
Version 27-Feb-20 
13
The primary objective of this study is to measure the incidence and categorization of 
intra-operative time delays. Secondary outcome measures will be the incidence of 
events leading to time delays when evaluating in retrospection, the interaction of 
non-technical skills and their impact on events leading to time delays, correlation of 
time delay to its root cause, the amount of time taken to complete a procedure, the 
number of canceled cases, and OR utilization (ratio of number of canceled/completed 
cases). 
 
Statistical Analysis 
This observational study represents a pilot study. Therefore, an a priori analysis of 
statistical power and sample size cannot be performed (see also chapter sample size). 
Descriptive statistical analysis of defined events and outcomes will be performed. 
Correlation statistic will be performed to evaluate the team’s conscious awareness of 
incidents and adverse events. Logistic regression analysis will be performed to identify 
influencing non-technical and team performance factors. A biostatistician will be involved 
in the statistical analysis of the results. 
 
VI. Ethics 
Ethics approval for this study will be obtained from the Research and Ethics Boards from 
the participating hospital and the University of Toronto.  
 
 
VII. Relevance 
OR delays due to inefficiency can result from a chain of intraoperative events including 
technical factors (ie. equipment failure or case complexity) and from non-technical human 
factors (ie. communication errors or lack of coordinated teamwork within and between OR 
teams). Objective measures of intraoperative OR delays and identification of their causes 
can lead to the development of a standardized framework that could be generalized to any 
surgical discipline to improve OR efficiency. Anticipated outcomes include: reduced 
cancellation of cases, increased OR utilization, reduced time delays, increased number of 
completed cases, improved patient flow within peri-operative department, and 

 
 
 
 
 
 
OR Black Box: Pilot Study 
Version 27-Feb-20 
14
improvement in non-technical skills of surgeons, anesthetists, nursing, and administrative 
team when facing challenges in time delays. 
 
References 
1. 
Kimbrough, C. et al. Improved operating room efficiency via constraint 
management: experience of a teritary-care academic medical center. Journal of Americal 
College of Surgeons. February 2015. Vol. 221, No. 1, pp 154-162 
2.  
Kohn LT CJ, Donaldson MS, editor. To err is human: building a safer health 
system. 6 ed. Washington DC: National Academies Press; 2000 
3.  
Gururaja RP, Yang T, Paige JT, Chauvin SW. Examining the Effectiveness of 
Debriefing at the Point of Care in Simulation-Based Operating Room Team Training. 
Advances in Patient Safety: New Directions and Alternative Approaches (Vol 3: 
Performance and Tools). 2008. 
4.  
McGreevy JM, Otten TD. Briefing and debriefing in the operating room using 
fighter pilot crew resource management. J Am Coll Surg. 2007;205(1):169-76. 
5. 
Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AH, Dellinger EP, et al. A 
surgical safety checklist to reduce morbidity and mortality in a global population. N Engl 
J Med. 2009;360(5):491-9. 
6. 
Halverson AL, Casey JT, Andersson J, Anderson K, Park C, Rademaker AW, et al. 
Communication failure in the operating room. Surgery. 2011;149(3):305-10. 
7. 
Lingard L, Regehr G, Orser B, Reznick R, Baker GR, Doran D, et al. Evaluation of 
a preoperative checklist and team briefing among surgeons, nurses, and anesthesiologists 
to reduce failures in communication. Arch Surg. 2008;143(1):12-7; discussion 8. 
8. 
Reason J. The contribution of latent human failures to the breakdown of complex 
systems. Philos Trans R Soc Lond B Biol Sci. 1990;327(1241):475-84. 
9. 
Helmreich RL, Merritt AC, Wilhelm JA. The evolution of Crew Resource 
Management training in commercial aviation. Int J Aviat Psychol. 1999;9(1):19-32. 
10. 
Reason J. Human error: models and management. Bmj. 2000;320(7237):768-70. 
11. 
Porta, C. et al. January 2013. Operating room efficiency improvement after 
implementation of a postoperative team assessment. Journal of Surgical Research. Vol 180, 
pp. 15-20. 
12. 
Kohli E, Ptak J, Smith R, Taylor E, Talbot EA, Kirkland KB. Variability in the 
Hawthorne effect with regard to hand hygiene performance in high- and low-performing 
inpatient care units. Infect Control Hosp Epidemiol. 2009;30(3):222-5. 
13. 
Mishra A, Catchpole K, McCulloch P. The Oxford NOTECHS System: reliability 
and validity of a tool for measuring teamwork behaviour in the operating theatre. Qual Saf 
Health Care. 2009;18(2):104-8. 
14. 
Yule S, Flin R, Paterson-Brown S, Maran N, Rowley D. Development of a rating 
system for surgeons' non-technical skills. Med Educ. 2006;40(11):1098-104. 
15. 
Yule S, Flin R, Maran N, Rowley D, Youngson G, Paterson-Brown S. Surgeons' 
non-technical skills in the operating room: reliability testing of the NOTSS behavior rating 
system. World J Surg. 2008;32(4):548-56.