Evaluation of the Effects of Virtual Reality Learning Environment on Nursing Student Non-technical Skills Development: A Quasi-Experimental Study
Phase: NA
Conditions: Educational Problems
Sponsor: The Hong Kong Polytechnic University

Study Protocol: Evaluation of the Effects of Virtual Reality Learning Environment on 
Nursing Student Non-technical Skills Development: A Quasi-Experimental Study   
Background and information 
A Virtual Reality (VR) Ward Training System with a virtual environment was installed 
in the School of Nursing, The Hong Kong Polytechnic University. The objective of the VR 
system is to enhance students’ situation awareness and communication skills. A complete 
set of 3D rendering scenes are created for three learning modules. In addition, there is real-
time animation interacting with controllers.  
Virtual Hospital is a laboratory mimicking a real hospital in a virtual platform to 
allow students to attend their clinical practice. The VR technology can be applied to simulate a 
clinical environment for students to experience as if they were learning in a hospital while 
they are in fact on campus. The equipment will allow stages of learning through beginners 
and graduating students. 
Currently, students will have simulation sessions as the usual teaching method on the 
clinical reasoning subject to train their clinical communication and situation awareness 
skills. The VR ward training system will be used to improve students’ learning experience.  
The proposed research study investigates whether the VR teaching pedagogy will have 
added value in developing students’ situation awareness, communication skills, and self-
confidence in learning.  
Virtual reality (VR) uses a computer system to simulate real-life events, and the users can 
interact with the virtual environment (Alrehaili & Osman, 2019). The main characteristics of 
the 
VR applications included immersion, interaction and imagination (Burdea & Coffiet, 2003). VR 
has been widely studied in education and has shown promising results, especially in 
improving 
learner motivation, engagement and attention  (Virvou et al., 2005). VR environments 
merging 
with game-based approaches are more commonly used in recent years. Merchant et al. 
(2014) 
conducted a meta-analysis on the effectiveness of VR technology in three areas: virtual worlds, 
Document Date: January 1, 2022

games and simulation environments. The VR context improves knowledge acquisition in the 
game’s environment among these three areas. The study also found that the users who played the 
games had a higher retention level. Alrehaili & Osman (2019) also revealed that VR serious 
games could improve students’ knowledge retention and motivation in learning.  
Study Aims  
This study evaluates the benefits of the Virtual Reality ward training system. It aims to 
explore the impact of the VR system on students’ situation awareness, communication skills and 
self-confidence in learning.  
The primary question is: Does Virtual Reality improve student’s learning outcome (1) 
situation awareness; (2) communication skills; and (3) self-confidence in learning?  
Methodology  
Study design 
The study employs a quasi-experimental, pretest-posttest non-equivalent waitlist control 
group design. It is unethical to forbid a group of students from receiving a potential beneficial 
educational intervention for their learning purposes. In contrast, a waitlist design is appropriate 
to address the concern. The control group will not receive the intervention until after data has 
been collected. A pretest-posttest method will be used because of possible differences between 
the two groups.  
Two intact groups, which cannot be randomly assigned, serve as the control and 
intervention groups. The intervention group will be having the VR training, whereas the control 
group will not until the study is completed.  
 
Document Date: January 1, 2022

Participants 
Students studying Bachelor of Science in Nursing who have completed the subject 
“Clinical reasoning and decision making in action part 1” are eligible to participate in the study 
(n=200). The researcher accepts a margin of error at 5%, confidence interval at 95% and 
standard deviation of 0.5. By using the Z-score formula (Necessary Sample Size = (Z-score)2 * 
StdDev*(1-StdDev) / (margin of error)2), the sample size should be N = 132. To avoid dropout 
and attrition from the participants, the researcher will recruit 150 samples to participate in the 
study.  
Convenience sampling will be adopted to recruit subjects. Participants will be assigned to 
two study groups, the control and the experimental group. The Control group will have usual 
teaching activities (simulation), whereas the participants of the intervention group, will have VR 
sessions.  
Procedure 
This study implements a quantitative data collection method. Participants will be invited 
to participate in the study through email. The researcher will send the invitation email to year 
three nursing students with inclusion criteria being those who have completed the subject 
“Clinical reasoning and decision making in action part 1”.  
Participants will be invited to participate in the questionnaires on two-time points along 
the semester: before the teaching activity as the baseline and immediately after the usual teaching 
activity or intervention.  
Before administering the questionnaire, an explanation of the research will be given to 
the students regarding the purpose of the study. They have the right to withdraw from the study 
at any time without any consequences. The data collected by this research will be treated as 
Document Date: January 1, 2022

entirely confidential. Participation is completely voluntary, and participants must agree to 
provide informed consent to participate in this research. Upon their agreement to participate, 
they will conduct three assessments throughout their academic year. The details can refer to in 
the Consort flow diagram as below.  
Document Date: January 1, 2022

 
 
 
 
 
Assessed for eligibility  
Allocation   
Waitlist Control Group 
Intervention Group 
Intervention:  
VR 
Conventional Training  
Intervention Group – 
Follow up 
Waitlist Control Group – 
Follow up 
Intervention:  
VR 
Waitlist Control Group – 
Follow up 
Total Analysed  
Total Analysed  
Enrollment  
Allocation   
Baseline Assessment 
(Time 1)    
Follow-up Assessment 
(Time 2)    
Follow-up Assessment 
(Time 3)    
Document Date: January 1, 2022

Instrumentation  
The study aims to assess students’ situation awareness, communication skills and self-
confidence in learning. Two validated instruments are used to measure the dependent variables.  
The first and second dependent variables, student’s situation awareness and 
communication skills, will be tested using the revised “NOn-TECHnical skills (NOTECHS)” 
scale (Sevidalis et al., 2008).  The scale is a widely accepted instrument to measure health care 
workers non-technical skills. The scale consists of 22 items with five subscales, including 
communication and interaction, situation awareness and vigilance, cooperation and team skills, 
leadership and managerial skills and decision making. The correlation coefficients for the five 
subscales reported range from (α .77 - .87), demonstrated promising evidence of internal 
consistency of all subscales.   
The third dependent variable, student’s self-confidence in learning, will be tested using 
the “Student satisfaction and self-confidence in Learning” inventory (National League for 
Nursing, 2005).  A total of 13 items assess the attitudes toward satisfaction with instruction and 
self-confidence in learning in simulation. A “satisfaction with instruction” subscale contains five 
items measuring satisfaction with teaching methods, diversity of learning materials, facilitation, 
motivation, and overall suitability of simulation. A “self-confidence with learning” subscale 
contains eight items measuring self-confidence in content mastery, content necessity, skills 
development, available resources, and knowledge of how to obtain help to solve clinical 
problems in simulation. Participants indicate their personal feelings about a statement that 
describes their attitudes or beliefs for each item. Response options include 1) strongly disagree, 2) 
disagree, 3) undecided, 4) agree, and 5) strongly agree according to Likert-style scale. 
Cronbach’s alpha was reported as 0.94 for the satisfaction subscale and 0.87 for the self-
Document Date: January 1, 2022

confidence subscale (Jeffries and Rizzolo, 2006). Scores are calculated by summing responses; 
higher scores indicate more satisfaction and more self-confidence, respectively.  
Apart from the instruments, students’ demographic data will be collected, including the 
prior experience in VR use to see whether previous VR experiences will affect the result.  
Qualitative questions will be asked during the debriefing sessions after the VR experience 
to supplement the quantitative results. In addition, the debriefing guiding questions will be 
provided to the facilitators. Besides, the debriefing session will be video recorded for data 
analysis purposes.  
Data Analysis  
The quantitative data will be analyzed using IBM SPSS for Windows version 26. 
Frequency and/or mean will be used to describe the demographic data. Descriptive data such as 
mean and median will describe the prevalence of attitudes and self-confidence, level of situation 
awareness and communication skills. Mann-Whitney U test will be used to examine the changes 
in mean scores of variables (attitudes and self-confidence, level of situation awareness and 
communication skills) between the control and experimental groups. A content analysis of the 
qualitative data will be conducted for the data collected from the debriefing session.  
Ethical consolidation  
This study will follow the Hong Kong Polytechnic University ethics procedure and 
ethical clearance will be applied. All data collected will be stored in a secured area. The 
researcher will also reassure the participants of the right to withdraw from the study at any time. 
The video recorded for review will be destroyed after data analysis. 
 
Document Date: January 1, 2022

Conclusion  
Non-technical skills in the healthcare industry are vital to maintaining patient safety and 
preventing errors. The effectiveness of the technology-assisted teaching pedagogy is well studied. 
However, the use of VR to develop students’ non-technical skills in healthcare education field is 
not common. The proposed study investigates the effectiveness of the VR training system on 
situation awareness, communication skills and self-confidence in learning development in 
university nursing students.  
 
Document Date: January 1, 2022

Reference  
Alrehaili, & Al Osman, H. (2019). A virtual reality role-playing serious game for experiential 
learning. Interactive Learning Environments, 1–14. 
https://doi.org/10.1080/10494820.2019.1703008 
Burdea, G. C., & Coiffet, P. (2003). Virtual reality technology. Hoboken, New Jersey: John 
Wiley & Sons 
Jeffries, P. R., & Rizzolo, M. A. (2006). Designing and implementing models for the innovative 
use of simulation to teach nursing care of ill adults and children: A national, multi-site, 
multi-method study. New York, NY: National League for Nursing. 
Merchant, Goetz, E. T., Cifuentes, L., Keeney-Kennicutt, W., & Davis, T. J. (2014). 
Effectiveness of virtual reality-based instruction on students’ learning outcomes in K-12 
and 
higher 
education: 
A 
meta-analysis. Computers 
& 
Education., 70, 
29–40. 
https://doi.org/10.1016/j.compedu.2013.07.033 
National League for Nursing (2015). vSim for nursing implementation guide for 
faculty.https://thepoint.lww.com/vsim 
Sevdalis, Davis, R., Koutantji, M., Undre, S., Darzi, A., & Vincent, C. A. (2008). Reliability of a 
revised NOTECHS scale for use in surgical teams. The American Journal of 
Surgery., 196(2), 184–190. https://doi.org/10.1016/j.amjsurg.2007.08.070 
Virvou, Katsionis, G., & Manos, K. (2005). Combining Software Games with Education: 
Evaluation of its Educational Effectiveness. Educational Technology & Society, 8(2), 54–
65. 
 
 
 
Document Date: January 1, 2022