Effectiveness of an Immersive Virtual Reality Environment on Curricular Training for Complex Cognitive Skills in Liver Surgery: A Multicentric Crossover Randomized Trial
Phase: NA
Conditions: Hepatobiliary Neoplasm
Sponsor: University Hospital, Ghent

 
 
B6702020000275; Version No. 03, July 15, 2020 
 
1
 
  Study Protocol 
 
Effectiveness of an Immersive Virtual Reality Environment 
on Curricular Training for Complex Cognitive Skills in Liver Surgery: 
A Multicentric Crossover Randomized Trial 
 
 
Protocol Version and Date: Version No. 03, July 15, 2020 
 
 
Principal Investigators: 
Wouter Willaert MD PhD, Ghent University Hospital, Belgium. 
Roberto Ivan Troisi MD PhD, Federico II University Hospital, Naples, Italy. 
 
Co-Investigator: 
Nikdokht Rashidian MD, Ghent University Hospital, Belgium. 
 
 

 
  
Version No. 03, July 15, 2020 
 
 
 2
Contents 
1. General description of project 
1.1        Introduction 
1.2        Rationale and objectives 
1.3        Hypotheses 
2. Material and Methods 
2.1. 
Training and assessment methods 
2.2. 
Basic cognitive training 
2.3. 
Three-dimensional anatomy training and decision making 
2.4. 
Assessment tools 
2.5. 
Study population 
2.6. 
Sample size calculation 
2.7. 
Statistical analysis 
2.8. 
Randomization 
2.9. 
Criteria for selecting the radiologic imaging 
3. Endpoints 
3.1. 
Primary outcome 
3.2. 
Secondary outcome 
4. Cross-over randomized trial 
5. Ethics 
6. References 

 
  
Version No. 03, July 15, 2020 
 
 
 3
 
1. General description of project  
1.1 Introduction 
Continual advancements in technology have dramatically changed the ways of surgical 
education. Multimedia-based technologies and different computer-based simulators 
have been studied to enhance the efficacy of training and to improve the quality of care 
by reducing surgical complications.1,2 The adoption of web technologies for cognitive 
online training can be a useful way of optimizing educational programs, allowing for 
anytime/anywhere training.3 Computer-based surgical simulation, including three-
dimensional (3D) models and virtual reality (VR) immersive environments, also hold 
enormous potential in surgical education.4  
Liver surgery is considered an essential part of the general surgery curriculum, and 
trainees in this field need guidance to learn the complex liver anatomy, pathophysiology, 
patient management and necessary operative skills to successfully manage patients. 
Despite the implementation of various training models to improve skills, there is currently 
absence of high-qualitative research that really demonstrates that training results in 
improvement of skills in the clinical setting. 
1.2 Rationale and Objectives 
In liver surgery, to perform safe and accurate hepatectomy, it is important to understand 
the complex liver anatomy and the relationship between the tumor and the intrahepatic 
structures. To date, preoperative assessment of patients with liver tumors have been 

 
  
Version No. 03, July 15, 2020 
 
 
 4
mainly reliant on Computerized Tomography scan (CT scan) and Magnetic Resonance 
(MR) images. These modalities are limited to 2D displays, which require surgeons to 
reconstruct 3D spatial images based on their clinical knowledge and experience. This 
process needs a vast cognitive knowledge of liver anatomy and pathophysiology, and 
possibly leads to a significant amount of cognitive load. Cognitive load refers to the 
amount of information being processed by the working memory resources at any point in 
time.5,6 To address this cognitive load in liver surgery, different desktop interface (DI) 
based 3D models and VR solutions have been developed.7,8 Evidence suggests that for 
experts 3D reconstruction is beneficial for planning complex liver surgery but also for 
improving residents' understanding of liver anatomy and surgical planning.6,9,10 The 
presentation of these 3D models is mostly as 3D PDFs on 2D screens and, therefore, it still 
has the limitation of depending on the viewer's mental ability to reconstruct images into 
3D structures. In light of recent developments in digitalization of the healthcare system 
and surgical education, affordable VR solutions have been introduced for reviewing 3D 
models. Recently, immersive VR environments with context-sensitive presentation 
integrating relevant patient data have been introduced as a possible solution to improve 
surgical training and preoperative liver surgery planning.11-13 While these studies provide 
subjective data on the potential of VR to enhance complex cognitive skills training, 
objective data to demonstrate its effectiveness as it relates to high judgmental skills are 
lacking. 
 

 
  
Version No. 03, July 15, 2020 
 
 
 5
1.3        Hypotheses 
• We hypothesize that an immersive VR experience is superior to 3D imaging 
reconstruction to understand the complex liver anatomy and to devise precise pre-
operative planning for general surgery residents. 
• We hypothesize that a stepwise training model will enable general surgery residents 
to understand liver anatomy, surgical approach to liver tumors and pre-operative 
evaluation of patients. 
 
2. Material and Methods 
2.1. 
Training and assessment methods 
• Basic cognitive training: by means of 3 web-based learning modules. 
• Three-dimensional anatomy training and operation planning: by means of 3D 
modeling for planning liver surgery (Synapse 3D workstation vs Immersive VR 
environment) 
• Assessment: by means of MCQs and real case scenarios  
2.2. 
Basic cognitive training 
Module 1. Liver anatomy 
Module 2. Surgical approach to liver tumors 
Module 3. Pre-operative evaluation 
2.3. 
Three-dimensional anatomy training and decision making 

 
  
Version No. 03, July 15, 2020 
 
 
 6
The trainees will be trained individually by an investigator in order to be able to 
interact with 3D software and VR environment. To orient them to the technology and 
allow them to learn how to manipulate the model, a sample 3D model will be provided 
in both environments. The familiarization will be considered enough if they can 
perform the following features independently:  
• Rotating of 3D model 360 degrees in both vertical and horizontal axes.  
• Making the liver surface transparent to show the intrahepatic structures in 
relation to the liver surface. 
• Isolating each intrahepatic structure to better analysis their configuration. 
• Labeling all the structures (portal vein, hepatic artery, hepatic vein, tumor). 
• Volumetry of liver base on portal branches. 
• Virtual hepatectomy based on portal branches. 
• Virtual hepatectomy using extract scoop region (wedge resection). 
• Decision-making based on reviewed data 
2.4. 
Assessment tools  
1. A validated multiple choice questionnaire (MCQ) will be used to test whether 
residents will acquire the appropriate cognitive knowledge.  
Proficiency level is defined as 80% of the average expert score (≥ 12 of 15-MCQs 
correct). 
2. Ten real patient cases will be used as an assessment tool to test whether residents 
will acquire the appropriate skills in clinical decision making.  

 
  
Version No. 03, July 15, 2020 
 
 
 7
Proficiency level is defined as 80% of the average expert score (≥ 8 of 10 cases correct). 
2.5. 
Study population 
All surgical residents at Ghent University, Belgium, and Federico II University, Naples, 
Italy, irrespective of their postgraduate level (1-6), will be invited to participate in the 
program. Participation of residents will be on voluntary basis, and they are free to 
withhold their data from the research data set. 
2.6. 
Sample size calculation 
For alpha of 0.05, power of 0.80, an estimated standard deviation of 0.6, and 
moderately expected effect size that is compatible with pooled analysis of previous 
studies comparing VR with other digital training environments,14  a sample size of 46 
trainees is required to be able to show a statistically significant difference in results. 
Considering a potential dropout rate of 10%, a total sample size of 50 participants is 
needed. 
2.7. 
Statistical analysis 
Data analysis will be performed using R version 3.6.1 (2019, The R Foundation for 
Statistical Computing). Categorical data will be presented as absolute frequency and 
associated percentage rate. Symmetrically distributed data will be presented as mean 
± standard deviation (SD). For independent samples, t-Test or Mann–Whitney U test, 
and for dependent samples, paired t-Test or paired Wilcoxon signed-rank test will be 
used to compare the performance differences according to the type and distribution 

 
  
Version No. 03, July 15, 2020 
 
 
 8
of variables. Reckoning with the study's crossover design, the comparison of 
continuous outcomes will be conducted using a linear mixed model, which assumes 
treatment (VR vs DI-based training) and period as fixed effects and resident as a 
random effect. To evaluate the risk of a carryover and learning effect between two 
sequences of tests, an interaction term treatment x period will be included in the 
mixed model to test for the presence of a carryover effect. Finally, the influence of 
resident gender and level (senior vs junior) on the outcomes will be explored by 
including in turn these variables into the model as fixed effects. An α level of P < .05 
will be considered statistically significant and P values will be calculated 2-sided. 
2.8. 
Randomization 
Using computer-generated randomization stratified by postgraduate (PG) level 
(Junior (PG 1-2) vs Senior (PG ≥ 3)) and institution, residents will be randomized in a 
1:1 allocation to one of two assessment sequences to devise a surgical plan for 10 
clinical scenarios. In group A, the first five cases will be evaluated via a DI, and the 
second five cases will be evaluated in the VR environment; while in group B, the 
sequence of cases will be reversed. We consider one-hour washout time between the 
first and the second sequence of cases. 
 
2.9. 
Criteria for selecting the radiologic imaging 
CT scans used in this study will be taken from a library of CT scans of patients with 
liver tumors who underwent liver surgery at Ghent university hospital. Ten patient 

 
  
Version No. 03, July 15, 2020 
 
 
 9
cases weill be selected based on quality images that provide adequate visualization 
for surgical planning and reflect the intraoperative findings, as the final surgery 
performed was in line with the preoperative plan. Each preoperative image will be 
independently reviewed by two hepatobiliary surgeons to provide expert opinion on 
the optimal surgical plan, and cases with differing opinions will be excluded.  
A total of 10 cases will be selected and patient data will be anonymized.  
CT images will be imported into the FUJIFILM synapse 3D software, to produce 3D 
surface renderings of the liver surface, hepatic veins, portal veins, hepatic arteries and 
tumor. The same 3D model will be uploaded to the VR environment. 
Various liver cases (benign and malignant) will be considered, and the final 10 cases 
will include the following surgical plan: 
1. Left lateral resection 
2. Left hepatectomy 
3. Right hepatectomy 
4. Extended right hepatectomy 
5. Multiple wedge resection (cherry-picking metastasectomy) 
6. Right posterior sectionectomy 
7. Unresectable tumor due to main portal vein invasion 
8. Central hepatectomy 
9. Anatomical resection of segment 7 
10. Unresectable tumor due to low volume of future liver remnant 

 
  
Version No. 03, July 15, 2020 
 
 
 
10
 
3. Endpoints 
3.1. 
Primary outcome 
• The difference in accuracy of the residents' decision-making when reviewing 
patient data along with 3D liver models visualized via DI or VR, measured by 
comparing their performance to experts' opinion. 
3.2. 
Secondary outcomes: 
• Percentage of residents achieving proficiency in clinical decision-making after 
completing the training curriculum 
• The difference in the timing to devise treatment plans using DI or VR 
• Percentage of correct anatomical landmark and liver segments identification 
using DI or VR 
• Knowledge retention assessed by comparing MCQs score before and after e-
learning training 
4. Cross-over randomized trial 
After signing informed consent, all participants will complete a pre-training questionnaire 
to determine their demographic information and their experience in liver surgery. Then, 
they will answer a set of 15 MCQs to assess their baseline cognitive knowledge. 
Subsequently, they will receive the 3 e-learning models to study. After completion of e-

 
  
Version No. 03, July 15, 2020 
 
 
 
11
learnings, 15-MCQ will be provided. When the average MCQ score (i.e. at least 80%) of 
the experts (benchmark) has been achieved, they can progress to the next training step. 
If they will not accomplish the required score, they are referred to the e-learning platform 
and once they feel ready, a second different set of MCQ should be completed until they 
reach the benchmark score. They will have one month time to study the modules and 
associated tests on their own pace. 
For the next step, residents will be invited to an individualized training session for 3D 
anatomy training and clinical decision-making. Participants will be trained individually by 
an investigator to be able to interact with SYNAPSE VINCENT and LiVeR Trainer. After 
completing the training session, the two groups will be compared for accuracy of their 
surgical planning when they used DI or VR to review 3D models of liver during surgical 
planning. Group A will receive the first 5 cases along with 3D models via DI and the second 
5 cases in the VR environment, while group B will receive a reverse sequence of cases. 
 
5. Ethics 
The institutional review board at Ghent University Hospital reviewed and approved the 
protocol (IRB approval No. B6702020000275). Participation of trainees will be voluntary 
and informed consent will be obtained from all of them.  
6. References 
1. 
Grantcharov TP. Is virtual reality simulation an effective training method in surgery? Nat 
Clin Pract Gastroenterol Hepatol. 2008;5(5):232-233. 

 
  
Version No. 03, July 15, 2020 
 
 
 
12
2. 
Madani A, Jones DB, Fuchshuber P, Robinson TN, Feldman LS. Fundamental Use of 
Surgical Energy™ (FUSE): a curriculum on surgical energy-based devices. Surg Endosc. 
2014;28(9):2509-2512. 
3. 
Maertens H, Madani A, Landry T, Vermassen F, Van Herzeele I, Aggarwal R. Systematic 
review of e-learning for surgical training. Br J Surg. 2016;103(11):1428-1437. 
4. 
Haluck RS, Krummel TM. Computers and Virtual Reality for Surgical Education in the 21st 
Century. Archives of Surgery. 2000;135(7):786-792. 
5. 
Crossingham JL, Jenkinson J, Woolridge N, Gallinger S, Tait GA, Moulton C-AE. 
Interpreting three-dimensional structures from two-dimensional images: a web-based 
interactive 3D teaching model of surgical liver anatomy. HPB : the official journal of the 
International Hepato Pancreato Biliary Association. 2009;11(6):523-528. 
6. 
Yeo CT, MacDonald A, Ungi T, et al. Utility of 3D Reconstruction of 2D Liver Computed 
Tomography/Magnetic Resonance Images as a Surgical Planning Tool for Residents in 
Liver Resection Surgery. Journal of Surgical Education. 2018;75(3):792-797. 
7. 
Quero G, Lapergola A, Soler L, et al. Virtual and Augmented Reality in Oncologic Liver 
Surgery. Surgical Oncology Clinics of North America. 2019;28(1):31-44. 
8. 
Oshiro Y, Yano H, Mitani J, et al. Novel 3-dimensional virtual hepatectomy simulation 
combined with real-time deformation. World journal of gastroenterology. 
2015;21(34):9982-9992. 
9. 
Procopio F, Cimino M, Viganò L, et al. Prediction of remnant liver volume using 3D 
simulation software in patients undergoing R1vasc parenchyma-sparing hepatectomy 
for multiple bilobar colorectal liver metastases: reliability, clinical impact, and learning 
curve. HPB (Oxford). 2020. 
10. 
Radtke A, Sotiropoulos GC, Molmenti EP, et al. Computer-Assisted Surgery Planning for 
Complex Liver Resections: When Is It Helpful? A Single-Center Experience Over an 8-Year 
Period. Annals of Surgery. 2010;252(5). 
11. 
Kenngott HG, Pfeiffer M, Preukschas AA, et al. IMHOTEP: cross-professional evaluation 
of a three-dimensional virtual reality system for interactive surgical operation planning, 
tumor board discussion and immersive training for complex liver surgery in a head-
mounted display. Surg Endosc. 2021. 
12. 
Boedecker C, Huettl F, Saalfeld P, et al. Using virtual 3D-models in surgical planning: 
workflow of an immersive virtual reality application in liver surgery. Langenbeck's 
Archives of Surgery. 2021. 
13. 
Pfeiffer M, Kenngott H, Preukschas A, et al. IMHOTEP: virtual reality framework for 
surgical applications. International Journal of Computer Assisted Radiology and Surgery. 
2018;13(5):741-748. 
14. 
Kyaw BM, Saxena N, Posadzki P, et al. Virtual Reality for Health Professions Education: 
Systematic Review and Meta-Analysis by the Digital Health Education Collaboration. J 
Med Internet Res. 2019;21(1):e12959.