Validity and Effectiveness of a Virtual Reality Simulator for Manual Small Incision Cataract Surgery: The Orbis-FundamentalVR MSICS Simulator.
Phase: NA
Conditions: Surgical Education
Sponsor: Orbis

 
 
Orbis-FVR – Study Protocol         
Version 3.0         
April 12,  2021 
 
 
 
Fundamental Virtual Reality Simulation for Manual Small Incision Cataract 
Surgery Validity: Evaluation, Efficacy and Acceptability Study in Bangladesh, 
China, Ethiopia, India, Mongolia, Togo, United Kingdom and United States of 
America.  
 
A Prospective Multi-Centre Mixed Methods Study. 
 
 
 
STUDY PROTOCOL 
 
 
 
Principle Investigators   
Dr William H Dean1 
 
 
Co-Investigators:  
Amelia Geary2 
Professor Nathan Congdon2,3 
 
Collaborating Institutions: 
 
1. London School of Hygiene and Tropical Medicine, UK 
2. Orbis International, New York, USA 
3. Queen’s University Belfast, Northern Ireland, UK 
4. FundamentalVR, London, UK 
5. Chittagong Eye Infirmary and Training Complex, Chittagong, Bangladesh 
6. Addis Ababa University, department of Ophthalmology, Menelik II Hospital, Addis Ababa, Ethiopia 
7. University of Gondar, Gondar, Ethiopia 
8. Dr. Shroff’s Charity Eye Hospital (SCEH), New Delhi, India 
9. H.V. Desai Eye Hospital, Pune, India 
10. The First Central Hospital, Ulaanbaatar, Mongolia 
11. Shenyang He Eye Specialist Hospital, China. 
12. Lumiere Devine Eye Hospital, Lome, Togo 
13. Emory Eye Center, Atlanta, GA USA 
 
 
 
 
 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
2
 
This protocol describes the Orbis-FVR Study, and provides information about procedures for selecting 
participants, training involved, and the methods for validation. 
 
 
The protocol should not be used as a replacement curriculum for current cataract surgical training.  
 
 
Questions relating to this educational-intervention study should be referred, in the first instance, to the 
primary investigators Dr William Dean: will.dean@lshtm.ac.uk or Amelia Geary: Amelia.geary@orbis.org 
 
 
This trial will adhere to the principles outlined in the International Conference on Harmonization Good 
Clinical Practice (ICH GCP) guidelines, and all applicable local and training program regulations. 
 
 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
3
Table of Contents 
Glossary of Abbreviations ............................................................................................................ 5 
Keywords..................................................................................................................................... 5 
General Information .................................................................................................................... 6 
Project Title ............................................................................................................................................................................ 6 
Identifying numbers .......................................................................................................................................................... 6 
Principle Investigators ..................................................................................................................................................... 6 
Coordinating Research Institution............................................................................................................................. 6 
Collaborating Training Institutions .......................................................................................................................... 6 
Study Sponsor ........................................................................................................................................................................ 7 
Study Funders ........................................................................................................................................................................ 7 
Study Summary............................................................................................................................ 8 
Exclusion Criteria (trainee) ................................................................................................................................................ 9 
Executive Summary .................................................................................................................... 10 
Background ............................................................................................................................... 11 
Cataract and Cataract Surgery globally .......................................................................................................................... 11 
Simulation in Ophthalmic Surgical Training ................................................................................................................ 11 
Development of the Training Curriculum .................................................................................................................. 12 
Methodology ............................................................................................................................. 13 
Design Summary ...................................................................................................................................................................... 13 
Study Setting .............................................................................................................................................................................. 13 
Study Duration .......................................................................................................................................................................... 13 
Study Participants ................................................................................................................................................................... 13 
Inclusion / Exclusion Criteria............................................................................................................................................. 13 
Inclusion Criteria (trainee participant) ..................................................................................................................... 13 
Exclusion Criteria (trainee) ............................................................................................................................................. 13 
Informed Consent .................................................................................................................................................................... 14 
Withdrawal Criteria ................................................................................................................................................................ 14 
Baseline Data ............................................................................................................................................................................. 14 
Resident Trainees ................................................................................................................................................................ 14 
Attending Consultants (experts).................................................................................................................................... 14 
Randomisation .......................................................................................................................................................................... 14 
Sequence generation .......................................................................................................................................................... 14 
Allocation Concealment .................................................................................................................................................... 15 
Randomization Implementation ................................................................................................................................... 15 
Table 2: Orbis-FVR Training Program ............................................................................................................................ 16 
Outcomes ..................................................................................................................................................................................... 19 
Primary Outcome – Orbis-FVR Study........................................................................................................................... 19 
Secondary Outcomes / Qualitative Outcomes / Additional Exploratory Analysis: .................................... 19 
Analysis ........................................................................................................................................................................................ 19 
Statistical analysis .............................................................................................................................................................. 19 
Qualitative analysis ............................................................................................................................................................ 20 
Cost-effectiveness ................................................................................................................................................................ 20 
Construct Validity .................................................................................................................................................................... 20 
Face Validity ............................................................................................................................................................................... 20 
Sample size ................................................................................................................................................................................. 21 
Prevention of Bias ................................................................................................................................................................... 21 
Observer Bias ...................................................................................................................................................................... 21 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
4
Benefits of the Study .............................................................................................................................................................. 22 
Benefits to the study participants ................................................................................................................................. 22 
General benefits ................................................................................................................................................................... 22 
Risks and Limitations............................................................................................................................................................. 22 
Study Timetable: ...................................................................................................................................................................... 23 
Data Management ..................................................................................................................... 24 
Expected Outcomes of the Study ................................................................................................ 24 
Quality Assurance ...................................................................................................................... 24 
Data management .............................................................................................................................................................. 24 
Project Management ................................................................................................................. 25 
Study Management ................................................................................................................................................................. 25 
Advisory Panel ...................................................................................................................................................................... 25 
Funding ........................................................................................................................................................................................ 25 
Medical Registration .............................................................................................................................................................. 25 
Data and safety management ............................................................................................................................................. 25 
Ethical Considerations ................................................................................................................ 25 
Ethical Approval & Trial Registration............................................................................................................................. 25 
Participant / Trainee Informed Consent ....................................................................................................................... 26 
Dissemination of Results and Publication Policy .......................................................................... 27 
References ................................................................................................................................. 28 
Appendices ................................................................................................................................ 30 
Appendix 1 Participant Consent Form ........................................................................................................................... 30 
Appendix 2 Participant Information Sheet................................................................................................................... 30 
Appendix 3 Training Intervention ................................................................................................................................... 30 
Appendix 4 MSICS Sim-OSSCAR ........................................................................................................................................ 30 
Appendix 5 Acceptability and Utility Survey ............................................................................................................... 30 
Appendix 6 Proposed Budget ............................................................................................................................................. 30 
........................................................................................................................... 31 
Appendix 1  
 Participant Consent Form (Orbis-FVR) ................................................................ 31 
Appendix 2  
Participant Information Sheet – Orbis-FVR .......................................................... 33 
Appendix 3. 
Training Intervention ......................................................................................... 37 
Appendix 4.   
MSICS Sim-OSSCAR ............................................................................................ 40 
Appendix 5. 
Acceptability and Utility Survey & Questionnaire ................................................ 41 
Appendix 6. 
Proposed Budget ............................................................................................... 42 
 
 
 
 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
5
Glossary of Abbreviations 
 
 
 
ACGME  
Accreditation Council for Graduate Medical Education 
CCC 
 
Continuous curvilinear capsulorrhexis 
ECCE 
 
Extracapsular cataract extraction 
FRCOphth 
Fellow of the Royal College of Ophthalmologists (UK) 
FVR 
 
Fundamental VR 
GCP 
 
Good Clinical Practice 
IAPB 
 
International Agency for the Prevention of Blindness 
ICEH 
 
International Centre for Eye Health 
INO 
 
Instituto Nacional de Oftalmología 
IRO 
 
Instituto Regional de Oftalmología 
LSHTM  
London School of Hygiene & Tropical Medicine 
MD 
 
Doctor of Medicine 
MEd 
 
Masters in Education 
MSICS  
Manual small incision cataract surgery 
OASIS  
Objective assessment of skills in intra-ocular surgery 
OSSCAR 
Ophthalmic Simulated Surgical Competency Assessment Rubric 
PCR 
 
Posterior capsule rupture 
PhD 
 
Doctor of Philosophy 
PI 
 
Principal investigator 
QUB 
 
Queen’s University Belfast 
RCOphth 
The Royal College of Ophthalmologists, UK 
SCEH 
 
Shroff’s Charity Eye Hospital 
UK 
 
United Kingdom 
USA 
 
United States of America 
VA 
 
Visual acuity 
VL 
 
Vitreous loss 
VR 
 
Virtual reality  
WHO 
 
World Health Organization 
 
 
Keywords 
 
Virtual reality, Simulation, Surgical Education, Cataract 
 
 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
6
General Information 
 
Project Title 
 
Fundamental Virtual Reality Simulation for Manual Small Incision Cataract Surgery Validity: Evaluation, 
Efficacy and Acceptability Study in Bangladesh, China, Ethiopia, India, Mongolia, Togo, United Kingdom 
and United States of America.  
  
Identifying numbers 
LSHTM Application Reference Number:  
Principle Investigators 
Dr William Dean PhD, FRCOphth, MEd, MBChB, BSc 
 
Co-Investigators  
Amelia Geary 
Professor Nathan Congdon  PhD, MD 
 
 
International Centre for Eye Health, Clinical Research Department, Faculty of Infectious and Tropical 
Diseases, London School of Hygiene & Tropical Medicine, Keppel Street, London, WC1E 7HT, UK 
 
 
Coordinating Research Institution 
Orbis International 
 
Collaborating Training Institutions  
 
1. London School of Hygiene and Tropical Medicine, UK 
2. Orbis International, New York, USA 
3. Queen’s University Belfast, Northern Ireland, UK 
4. FundamentalVR, London, UK 
5. Chittagong Eye Infirmary and Training Complex, Chittagong, Bangladesh 
6. Addis Ababa University, department of Ophthalmology, Menelik II Hospital, Addis Ababa, Ethiopia 
7. University of Gondar, Gondar, Ethiopia 
8. Dr. Shroff’s Charity Eye Hospital (SCEH), New Delhi, India 
9. H.V. Desai Eye Hospital, Pune, India 
10. The First Central Hospital,, Ulaanbaatar, Mongolia 
11. Shenyang He Eye Specialist Hospital, China. 
12. Lumiere Devine Eye Hospital, Lome, Togo 
13. Emory Eye Center, Atlanta, GA USA 
 
 
 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
7
Study Sponsor 
London School of Hygiene & Tropical Medicine is the main research sponsor for this study.  For further 
information regarding the sponsorship conditions, please contact the Research Governance and Integrity 
Office: 
 
 
 
London School of Hygiene & Tropical Medicine 
Keppel Street 
London WC1E 7HT, UK 
Tel: +44 207 927 2626 
 
Study Funders 
• 
Orbis International 
 
 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
8
Study Summary 
 
Title 
Fundamental Virtual Reality Simulation for Manual Small Incision Cataract Surgery Validity: 
Evaluation, Efficacy and Acceptability Study in Bangladesh, China, Ethiopia, India, 
Mongolia, Togo, United Kingdom and United States of America.  
 
Design 
Prospective Pre-Post single-masked education-intervention randomized controlled study 
of intense virtual reality (VR) simulation-based surgical education of ophthalmologists in 
Bangladesh, China, Ethiopia, India, Mongolia, Togo, UK and USA. 
 
Construct validity study of assessment scores generated by the VR simulator for novices 
versus experts.  
 
Qualitative study of face validity of VR simulator, and acceptability questionnaire survey of 
users.  
 
Aims 
To investigate the efficacy of intense VR simulation-based surgical education using the 
Orbis-FVR simulator. To examine whether it improves competence; is acceptable and has 
validity. To assess the construct validity of the VR simulator’s assessment capacity. 
 
 
Intervention 
The ‘intervention training’ is a structured course of training using the Orbis-FVR MSICS 
simulator. This training is in addition to, and an enhancement of the trainees’ normal 
current standard conventional training, and not designed to replace it.  
 
Outcome 
measures 
Assessments and follow-up time points are at baseline and 1 month. 
 
Primary outcome measure: mean specific simulated surgical competency assessment 
score at one-month post-training intervention between groups (assessed using Sim-
OSSCAR) (Appendix 4).1 
 
 
Secondary outcome measures: 
• 
Assessment scores generated by Orbis-FVR Simulator pre- and post-intervention. 
• 
Step-specific analysis: mean differences in Sim-OSSCAR score for the seven 
interactive steps assessed (incision, tunnel, AC entry, CCC, mobilization, removal 
and IOL). Total possible score 14. 
• 
Self-reported confidence ratings (10-point Likert scale) in MSICS skills, and 
ophthalmic surgical skills 
 
Further Exploratory Analysis: 
• 
Construct validity study comparing scores generated by Orbis-FVR simulator 
between two cohorts of novice and expert MSICS surgeons. 
• 
Face validity study subjectively assessing a group of expert MSICS surgeons’ views 
of the Orbis-FVR simulator as representing MSICS 
• 
Qualitative study of the acceptability and utility of the Orbis-FVR simulator as a 
training or assessment tool. 
 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
9
Population 
The VR simulation surgical training, construct and face validity studies will be conducted in 
ophthalmology institutions in the Americas, Europe, Africa, and Asia. Participants will have 
follow-up assessments in their home training or work institutions in Bangladesh, China, 
Ethiopia, India, Mongolia, Togo, UK and USA. 
 
 
 
 
Eligibility 
 Inclusion criteria for trainee:  
• 
Trainee ophthalmologist (year 1, 2, or 3) in collaborating institution. 
• 
Agree to undertake simulation procedure assessments 
• 
Agree to undertake and complete the intense VR simulation training course. 
• 
Performed 0 MSICS as primary surgeon and assisted or part-performed less than 
10 MSICS cases 
Exclusion Criteria (trainee) 
• 
Performed 1 or more MSICS cataract surgeries as primary surgeon; and assisted or 
part-performed 10 or more. 
 
Inclusion Criteria (Expert) 
1. Participants in the construct validity study are two cohorts: novice MSICS surgeons 
having performed 0 MSICS procedures as primary surgeon and assisted in less than 
10. The second cohort is expert MSICS surgeons having performed a minimum of 
1,000 MSICS procedures as primary surgeon. 
2. Participants in face validity study are expert MSICS surgeons with over 1,000 
MSICS procedures performed. 
 
Duration 
The anticipated overall project duration is one year. The fieldwork will take about nine 
months. 
 
 
 
 
 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
10
Executive Summary 
 
Cataract is the leading cause of blindness, representing 35% of global blindness.2  There is a huge need to 
perform high volumes of cataract surgery globally, to tackle the backlog of cataract cases. There is a great 
need to train many eye surgeons safely, efficiently, effectively, and to an acceptable level of competence.   
 
Currently, surgical training is often conducted using the traditional “apprentice model”, where a trainee 
observes a qualified surgeon and learns from them, and then the surgeon supervises the trainee 
performing surgery on a patient. We believe that this conventional model has substantial limitations and 
drawbacks, making surgical training less efficient and less safe. Simulation-based surgical education in 
cataract and glaucoma surgery has been shown to be effective in improving competence and confidence 
of trainee eye surgeons.3 4 However, the role of an immersive haptic virtual reality learning environment 
has not been established for manual small incision cataract surgery.  
 
We will test the hypothesis that intense virtual reality (VR) simulation-based ophthalmic surgical training 
impacts initial acquisition of competence in key stages of MSICS. Orbis International has partnered with 
FundamentalVR and expert cataract surgeons to create a MSICS simulator, using virtual reality software 
combined with existing gaming technology.  The result is a VR simulator available at a fraction of the cost 
of cataract surgical simulators currently on the market.5 6  We are proposing a pilot randomized multi-
country study evaluating the efficacy and acceptability of the simulator training, using simulation alone. 
This mix method study will combine qualitative and quantitative data collection; no patients are involved. 
 
All the training within the ‘educational intervention’ of this study will be performed using simulation.  
There is no testing or interventional surgical training on patients. When three anonymized and non-
identifiable recordings of supervised cataract surgical procedures are video recorded (at one month), no 
patients are involved, and these assessments are, as for the baseline assessments, using validated artificial 
model eyes and assessment rubric.1 3 
 
The results of this study will inform the design of a future multi-center randomized-controlled clinical trial.  
 
 
 
 
 
 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
11
Background  
 
Cataract and Cataract Surgery globally 
 
Globally there are 36 million people who are blind and a further 219 million with moderate or severe 
vision impairment (MSVI).2 Cataract remains the most common cause of blindness. Approximately 80% of 
blindness is preventable or treatable, and 90% of the burden is in low- and middle-income countries 
(LMIC). Only around half of the 230,000 ophthalmologists in the world perform surgery, and regions with 
the greatest burden have the lowest ratios of ophthalmologists to population.7  One reason for having 
surgeons who do not perform surgery is that they are inadequately trained to perform it. For example, in 
India and China, consultants do not trust their novice apprentices to perform safe surgery leading to 
residents performing fewer than a dozen independent surgeries in their entire training. Ophthalmologists 
who have graduated with inadequate surgical skills find fewer opportunities to practice their skills after 
graduation.  
 
The procedure of manual small-incision cataract surgery (MSICS) is the most commonly performed 
cataract surgery procedure in many regions of the world and is the main standard of care.8 9 It is the 
commonest technique of cataract surgery taught in residency training programs in South Asia.10 The 
technique uses a smaller wound compared to the older technique of sutured extra-capsular cataract 
extraction (ECCE).11 There is less post-operative astigmatism, and fewer suture-related problems for SICS 
versus ECCE. The surgical and visual outcomes of phacoemulsification cataract surgery and MSICS are 
comparable.12-14 MSICS is an appropriate, safe, and affordable technique for blindness prevention.  
Simulation in Ophthalmic Surgical Training 
 
A recent randomized controlled trial (RCT) of intense simulation-based surgical education for MSICS using 
artificial eyes showed a rapid and sustained increase in competence, a 72% reduction in complication 
rates, and a doubling of confidence in participants.3 An RCT of simulation-based surgical education in 
glaucoma has also illustrated a rapid and sustained increase in surgical competence as well as confidence.4  
 
The use of computerized simulation models has been validated for cataract15-17 and retinal surgery.18 
Three computerized simulators have been used for cataract surgical training in ophthalmology: the Eyesi 
(VRMagic Holding AG, Mannheim, Germany), MicroVisTouch (ImmersiveTouch, Chicago, USA), and 
PhacoVision (Melerit Medical, Linkoping, Sweden).  
 
A simulation-based performance test and certification for phacoemulsification cataract surgery has been 
established for use with the Eyesi simulator. The test showed evidence of validity, and appeared to be a 
useful and reliable assessment tool, both for cataract procedure-specific as well as general micro-surgical 
skills.19  HelpMeSee (New York, USA) recently launched a full-immersion surgical training simulator for the 
use within high capacity surgical education programs for small-incision cataract surgery.5  This was 
launched in November 2020.20 However, the simulator is currently unavailable in full, and the cost per 
unit is around US$250,000.6 
 
 
 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
12
Development of the Training Curriculum 
 
 
Table 1: Training Course Curriculum & Objectives 
Pre-Course 
 
• 
Study information and consent 
• 
Video of procedure (MSICS) 
• 
Introduction video to Orbis-FVR simulator  
 
Course 
Curriculum 
 
• 
Video of procedure (MSICS) 
• 
Surgical procedure specifics  
• 
Orbis Fundamentals of MSICS course 
• 
Detailed illustration of Orbis-FVR simulator and instruction course 
• 
Intense Orbis-FVR simulator training: Course completion to minimum 5 
sequential benchmarked score. 
 
 
 
 
 
 
 
 
 
  

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
13
Methodology 
 
Design Summary 
This research program involves a prospective, single-masked education-intervention randomized 
controlled study of intense virtual reality (VR) simulation-based surgical education of ophthalmologists in 
Bangladesh, China, Ethiopia, India, Mongolia, Togo, UK and USA.  Participants will be randomized, and 
simulation surgical competency will be assessed at baseline and 1-month between groups.  Competency 
will be assessed at using artificial model eyes: both groups will perform three full MSICS cases using model 
eyes at baseline.  At one-month post intervention, both groups will again perform three full MSICS using 
model eyes (primary outcome).   
 
The study will assess the construct validity study of assessment scores generated by the VR simulator for 
novices versus experts. Both Novices and Experts will perform three full VR cases using the simulator.  The 
first score will be disregarded to allow for familiarization with the hardware, and the mean of the second 
and third cases will be recorded.   
 
Qualitative study of face validity of VR simulator, and acceptability questionnaire survey of users. An 
anonymous survey administered to experts and novices, using a Likert scale.    
 
Study Setting 
This is a multi-center study. We will enroll trainee ophthalmologists (doctors who have graduated from 
medical school and are currently undergoing ophthalmology specialist training) in 9 ophthalmology 
training institutions in in Bangladesh, China, Ethiopia, India, Mongolia, Togo, UK and USA. 
 
Study Duration 
The training will be conducted during 2021 and 2022.  
 
Study Participants 
Current trainees (between January 2021 and June 2022) in all 9 training institutions will be selected 
according to the inclusion and exclusion criteria.  
 
Inclusion / Exclusion Criteria 
Orbis-FVR Training Study: 
Inclusion Criteria (trainee participant) 
• 
Trainee ophthalmologist (year 1, 2 or 3) in collaborating institution. 
• 
Agree to undertake simulation surgical procedure assessments. 
• 
Agree to undertake and complete the intense VR simulation training course. 
• 
Performed 0 MSICS as primary surgeon and assisted or part-performed less than 10. 
 
Exclusion Criteria (trainee) 
• 
Performed 1 or more MSICS cataract surgeries as primary surgeon and assisted or part-performed 
10 or more. 
 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
14
Inclusion Criteria (Expert) 
3. Participants in the construct validity study are two cohorts: novice MSICS surgeons having 
performed 0 MSICS procedures as primary surgeon and assisted in less than 10. The second cohort 
is expert MSICS surgeons having performed a minimum of 1000 MSICS procedures 
4. Participants in face validity study are expert MSICS surgeons with over 1,000 MSICS procedures 
performed. 
Informed Consent 
Potential participant trainees will be informed of the training opportunity and the study. Heads of 
Department will be involved in the process. Trainee participants will be informed in detail about the 
nature of the education-intervention study; that the training offered offers no official qualification and 
will not be recorded in their annual review of clinical national training evaluation. All surgeons 
participating will be free to leave the study at any time.  See Appendices 1 to 2 for detailed Information 
and Consent Forms. 
 
Withdrawal Criteria 
Trainee participants are free to leave the study at any time. If this is the case for any participant, no effort 
will be made to recover any costs incurred. Data collected up to the point of withdrawal of consent will 
have been anonymized and securely stored and will still be held and included in data analysis.  
 
Baseline Data 
Resident Trainees 
• 
Age 
• 
Sex 
• 
Year of training 
• 
Training Institution 
• 
Number of MSICS surgeries performed as primary surgeon 
• 
Number of MSICS surgeries assisted in or part performed 
• 
Total number of other ophthalmic surgeries performed (disaggregated by type) 
 
Attending Consultants (experts) 
• 
Sex 
• 
Training institution 
• 
Number of MSICS surgeries performed as primary surgeon 
 
Randomisation 
Sequence generation 
The randomization sequences will be computer generated and administered centrally by a statistician 
based at the QUB who is independent of all other aspects of the trial. We will use block randomization 
(block size 2 or 4), with a separate sequence for each of the ten recruitment sites, to ensure balance. The 
statistician will generate the code / sequence (as a block of 2 or 4). 
 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
15
Allocation Concealment 
The statistician will not have access to information about subsequent allocation, and the individual 
potential participants.  The PI, co-investigators, and participants will have no prior access to the random 
sequence.   
 
Randomization Implementation 
Trainees within the same training institution, who have met the appropriate inclusion and exclusion (as 
detailed previously), will be eligible for randomization to the ‘intervention’ or ‘control’ arm. Each group 
of four trainee participants will be agreed by the Head of Training Program or Department.  
 
For example (Blocks will be in groups of 4, 6 or 8 depending on the institution and the number of residents 
that meet the inclusion criteria): 
 
A block of four potential participants is identified in Chittagong Eye Institute Training Center (CEITC), 
Bangladesh. Cards with the allocation or a block of four (two intervention and two control) were printed 
and placed in sealed opaque envelopes. Physically, in Bangladesh, a block of four identical envelops (e.g. 
block number 11) will be selected.  Participants will be invited by the Head of Department to pick one of 
the four envelops.  In this example, CEITC randomization block 11 allocation might be: 
 
IRO1101 
Intervention 
IRO1102 
Control 
IRO1103 
Control 
IRO1104 
Intervention 
 
 
 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
16
Table 2: Orbis-FVR Training Program 
 
Pre-Learning (one month in advance) 
• 
Surgical video review 
 
Pre-Course Assignment (one week in advance) 
• 
Complete and record three full MSICS cases using the Philips Studio MSICS model eye21 and 
following the steps of the Sim-OSSCAR (Appendix 4). 
• 
Upload the 3 recorded simulated surgical procedures to the Cybersight platform 
 
The Orbis-FVR Virtual Reality MSICS simulator has 19 steps in total.  Currently, 10 steps are animated 
and 9 are interactive, as follows: 
1. Patient Preparation – Animated Step 
2. Conjunctival Peritomy – Animated Step 
3. Paracentesis – Aminated Step 
4. Formation of the AC – Animated Step 
5. Anterior Chamber Maintainer – Animated Step 
6. Marking the Scleral Incision – Interactive Step 
7. Scleral Incision – Interaction Step 
8. Sclerocorneal tunnel – Interactive Step 
9. Anterior Chamber (AC) Entry – Interactive Step  
10. Continuous Curvilinear Capsulorrhexis (CCC) – Interactive Step 
11. Hydrodissection – Animated Step 
12. Mobilization of the nucleus – Interactive Step 
13. Releasing the nucleus – Animated Step 
14. Nucleus removal – Interactive Step 
15. Cortex removal – Animated Step 
16. IOL insertion – Interactive Step 
17. IOL Dialing and Implantation – Interactive Step 
18. Inspection of the wound – Animated Step 
19. Closure of the wound – Animated Step 
 
Course Overview 
Participants must attend and successfully complete all sessions.  All simulator tasks are to be completed 
in sequential order, in line with the course outline.  A required minimum score of 85 points for each 
interactive step must be reached reliably three times in a row to advance to the next step in the course. 
 
For each step within the simulation the following is recorded:  
Step Time - The total time spent on a step including ALL attempts. 
Pass / Fail - The interpreted pass or fail result of the last attempt of the step. 
Step score - Each step is scored between 0 and 100 points, based on user performance in completing the 
surgical objective, taking into account any adverse events. 
• 
Adverse events –For each step, if triggered, all of the adverse events are recorded. 
• 
No. of attempts - The number of attempts (inc. resets) of the step. 
 
Course Content 
• 
Each course will combine autonomous learning and live mentorship.   
• 
Autonomous learning will consist of a series of 15-minute video presentations, which include: 
o Narrated demonstration video of specific VR step(s) 
o PowerPoint presentations 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
17
o Instrument use demonstrations 
• 
Live mentorship will be delivered in two ways: 
o Local mentor who provides in-person mentorship 
o International mentor from technical advisory group provides live tele-mentorship 
 
Content hosting 
• 
All content will be hosted on the Orbis Cybersight Learning Management System platform 
(cybersight.org) 
• 
Data will be randomized and anonymized by the Cybersight Program manager, who will not be 
involved in any other aspect of the research study.   
 
Course delivery 
Each day: 
• 
Local mentor will facilitate introductions in each morning session.  This will cover: 
o An overview of the day’s training plan 
o Outlining the learning objectives for the day’s training plan 
o Introduction of the content covered in the day’s training plan  
• 
Local mentor will also facilitate discussion and answer questions on autonomous content 
viewed in the morning and midday sessions  
• 
Local mentor will provide periodic check ins throughout the day’s session 
• 
International mentor will provide tele-mentorship for the first hour of the 2-5pm session 
• 
Trainees will do self-practice for the remainder of the afternoon session 
• 
Each trainee will complete each interactive step on the VR simulator a minimum of 5 times with 
a reliability gate (3) and minimum score of 85 
• 
Each trainee will complete at least 5 full cases on the VR simulator with a reliability gate (3) and 
minimum score of 85 
 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
18
Course outline  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Attendance to be recorded as follows: 
• 
Confirmation of three simulation video uploads pre-intervention 
• 
Simulator use and assessment logs 
• 
Attendance records taken per daily session 
• 
Confirmation of three simulation video uploads post-intervention 
 
 
Day 
 
Morning 
8:00 – 10:30 
Midday 
11:00 – 1:00 
Afternoon 
2:00 – 5:00 
Monday 
 
 
 
 
Introduction 
Orientation to 
simulator. 
Review of animated 
steps 1-5 
Marking the scleral 
Incision  
Scleral Incision  
Sclerocorneal tunnel 
 
Mentored practice 
Self-Practice  
Tuesday 
 
 
 
 
Introduction 
Review  
CCC 
AC entry 
Hydrodissection 
Releasing the Nucleus 
Nucleus Removal  
Cortex Removal  
Mentored practice 
Self-Practice 
Wednesday 
 
 
 
Introduction 
Review 
IOL Insertion  
IOL Dialling and 
implantation  
Inspection of the 
wound 
Closure of the wound 
Mentored practice 
Self-Practice 
Thursday  
Introduction 
Review of entire SICS 
procedure. 
Putting it all together: 
Full Cases 
 
Mentored practice 
Self-Practice 
Friday  
Introduction 
Assessment and 
Scoring  
 
Final Review 
 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
19
Outcomes  
 
In the Fundamental Virtual Reality MSICS study, participants will be assessed on one occasion after 
recruitment (in addition to baseline), at one month: three simulation MSICS procedures recorded, 
anonymized and remotely assessed in a masked fashion (primary outcome). On the baseline and 1-month 
assessment, simulation MSICS procedures will be recorded (with masked assessment using the Sim-
OSSCAR).1  
 
Primary Outcome – Orbis-FVR Study 
The primary outcome measure of the Orbis-FVR study will be the procedure specific repeated measures 
analysis of Sim-OSSCAR score performed three times at 1-month. The analysis of the primary outcome 
measure will be based on the mean differences in the Sim-OSSCAR scores between the intervention and 
control groups. This score is derived from an assessment matrix or rubric of procedure specific and general 
microsurgical skill indices (see appendix 4).1 Each item in the matrix is graded on a modified Dreyfus score 
(novice, advanced beginner, and competent). The total possible score is 40 points. 
 
Simulation assessment will be recorded using a standard microscope and recording device with all 
participants wearing similar latex-free surgical gloves. Recordings will be given an anonymous number to 
give no indication as to in which arm the surgeon is.  Assessments of the surgical video will be conducted 
separately by two masked observers, watching the recorded surgery performed by the trainee at a 
separate time and place. Both observers are experienced eye surgeons and surgical trainers. Intra- and 
Inter-observer reliability studies will be conducted.   
 
Secondary Outcomes / Qualitative Outcomes / Additional Exploratory Analysis: 
1. Step-specific analysis: Secondary analysis of the primary outcome measure: mean differences in Sim-
OSSCAR score for the seven interactive steps assessed (incision, tunnel, AC entry, CCC, mobilization, 
removal and IOL). Total possible score 14. 
2. Assessment scores generated by Orbis-FundamentalVR MSICS simulator for two cohorts of surgeon: 
novice and expert (construct validity). 
3. Self-reported confidence ratings (10-point Likert scale) in MSICS skills, and ophthalmic surgical skills. 
4. Face validity assessment of Orbis-FVR simulator by international group of MSICS expert surgeons, 
performed through survey, using 5-point Likert scale. 
5. Further qualitative study including acceptability assessment of Orbis-FVR simulator by international 
group of MSICS expert surgeons; and subjective improvement analysis and feedback by both experts 
and trainees. 
6. Cost-effectiveness analysis of simulation-based training 
 
Analysis 
 
Statistical analysis 
 
The primary outcome measure (mean Sim-OSSCAR score at 1 month) will be analyzed using a t-test. 
Performances will be compared between intervention and control groups: these are a continuous variable 
in each group. We will compare these mean Sim-OSSCAR1 scores by t-tests (which is unadjusted) and 
linear regression (adjusting for training center as a fixed effect). An alpha level of p<0.05 will be considered 
statistically significant, and a kappa coefficient of ≥0.75 for inter-rater agreement is considered 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
20
excellent.226 
 
Qualitative analysis 
 
Structured anonymous survey (appendix 8), using a 5-point Likert scale, to assess acceptability of VR 
simulator by Novice and Expert study participants.   
 
Face validity assessment of Orbis-FVR simulator by international group of MSICS expert surgeons, 
performed through survey, using 5-point Likert scale. 
 
Cost-effectiveness 
 
Cost effectiveness will be evaluated by calculating the incremental cost-effectiveness ration (for the 
primary outcome of Sim-OSSCAR scores of surgical competency: 
 
ICER = cost of Orbis-FVR Simulation module per group of trainees divided by the difference in their 
outcomes (Scores). 
 
Cost-effectiveness will also be assessed against the expense of the VR simulator and model eyes with 
trainers time, versus trainers time alone; and compared at different safety and learning curves for 
competency outcomes of participants per group. 
 
Willingness-to-pay will be assessed as a crude monetary value per simulator, then per trainee outcome. 
 
 
 
Construct Validity 
 
Construct validity essentially assesses the sharpness of a tool, in this context addressing the question: can 
the simulator be used to discriminate between a novice and an expert. Mean assessment scores 
generated by Orbis-FundamentalVR MSICS simulator will be compared between two cohorts of surgeon: 
novice and expert. There will be 5 in each cohort, each performing 3 cases. The first case will be 
disregarded and the mean of the remaining two cases will be recorded and analyzed.   
 
Participants in the construct validity of assessment scores will be recruited from three centers. 
Face Validity  
 
Face Validity will be assessed by survey and response to the questions:  
• 
“Do you think the Orbis-FVR MSICS simulator is an appropriate way for trainees to learn manual 
small incision cataract surgery?” 
• 
“Do you think the Orbis-FVR MSICS simulator is an appropriate way to assess trainees’ MSICS 
surgical skill?” 
 
Responses will be evaluated on a 5-point Likert scale anchored at ‘1=strongly disagree’ and ‘5=strongly 
agree’. 
 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
21
A total of ten experts will be selected as participants in the face validity of the VR simulator. Recruited 
from all locations participating in the primary randomized control trial (Bangladesh, China, Ethiopia, India, 
Bangladesh, Togo, UK and USA). 
 
Sample size  
 
Based on pilot and published data we anticipate the mean Sim-OSSCAR score to be 10/40 (standard 
deviation (SD) 6 points) at baseline. We anticipate an effect size of 2.5 SD in the mean Sim-OSSCAR 
between groups. We assume a correlation between these observations of 0.8. Variation between clusters 
(training institutions) will be accounted for with a co-efficient of variation of 0.5.  
 
Therefore, a sample of 52 individuals in total (26 in each arm) would have 80% power and 95% confidence 
to detect a difference of 15 points (2.5SD). We will recruit 60 participants for the Orbis-FVR training study 
to provide 8 extra participants as we anticipate a moderate loss to follow-up.   
 
We and our collaborators consider this sample size of 60 participants to be feasible within the available 
time and financial resources. It would take longer (an extra academic year) if we needed to recruit many 
more, and if the impact of COVID-19 on training continued throughout the recruitment year. 
 
For the construct validity of the FVR assessment capacity, we will recruit a total of 5 experts and 5 novices 
from participating centers.  Sample sizes of content, face and construct validity study are based on data 
from recent validation studies of ophthalmic simulation surgical competency assessment rubrics.1 237 
Additionally, as we predict a large differences between the two participating groups, small cohort 
numbers suffice.   For face validity, we will recruit ten experts from multiple centers, likewise, this was 
determined based on data from similar face validation studies.     
 
Prevention of Bias 
 
It is accepted that there will be variability in individual participants’ inherent or natural surgical aptitude.  
All efforts will be made to standardize the training offered.  The intense simulation course will be hosted 
on the same standardized surgical training platform: Cybersight.  All intervention and control participants 
will wear surgical gloves 
 
Video recordings of procedures will be allocated a random 7-digit number, and subsequently stored onto 
an encrypted computer, and a separate encrypted hard drive.  This random number will be the only 
identifiable information available when the simulation surgical procedure is assessed, thus masking the 
assessor to the participant’s intervention/control arm.  
 
Observer Bias 
 
Recordings will be converted to an MP4 format, and coded.  At CyberSight, the recording will be renamed 
as a randomly generated seven-digit number (e.g. 6253815).  The code sheet will be generated by an 
independent statistician and only be known to them and the CyberSight administrator. Once assessors 
are notified that the video is ready for marking, this random number will be the only identifiable 
information available when the simulation surgical procedure is assessed, thus completely masking the 
assessor to the participant’s intervention/control arm and personal identity.  
 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
22
Benefits of the Study 
 
Benefits to the study participants 
Participants are expected to benefit from the additional intense VR simulation training in MSICS, as well 
as the online educational content of Cybersight.  
 
General benefits 
These may include the further validation and refinement of the educational platform, as well as 
additional evidence to the utility of VR simulation-based ophthalmic surgical education. 
 
Risks and Limitations 
 
There are no clinical risks within this study, as all the intervention training is using simulation.  No patients 
are involved in any of the training, or any of the assessments.   
 
There are a number of broad risks in conducting this study. 
 
• 
Trainees, meeting the inclusion criteria, not being available for enrolment (due to examinations, 
closure of training institutions, personal reasons).  
• 
Connectivity issues undermining access to the VR software, orientation and training 
• 
New technology: users’ comfortability with engaging in new technology 
• 
Civil unrest (including national elections). 
• 
Customs delays in the import of equipment or consumables. 
 
There are expected limitations to this study: 
• 
All training and assessments are using simulation. No patients are involved. Predictive validity 
(how the simulation training impacts live surgical competence) will not be assessed. A future 
prospective randomized-controlled educational-intervention clinical trial will be needed. 
.

 
 
Orbis-FVR – Study Protocol         
Version 3.0         
April 12,  2021 
Study Timetable: 
 
 
Figure 1. Detailed timeline of recruitment, assessment, and training. 
 
 
2021/2022: 
Feb 
Mar 
Apr 
May 
Jun 
Jul 
Aug 
Sep 
Oct 
Nov 
Dec 
Jan 
Feb 
Mar 
Orbis research concept template 
X 
 
 
 
 
 
 
 
 
 
 
 
 
 
Protocol development 
X 
X 
 
 
 
 
 
 
 
 
 
 
 
 
Collaborating institution MOUs 
 
X 
X 
 
 
 
 
 
 
 
 
 
 
 
Orbis-FVR 
simulator 
training 
course 
 
 
X 
X 
 
 
 
 
 
 
 
 
 
 
Equipment 
procurement 
& 
shipping 
X 
X 
X 
X 
 
 
 
 
 
 
 
 
 
 
Ethics submission 
 
X 
X 
 
 
 
 
 
 
 
 
 
 
 
Recruitment 
 
 
 
 
X 
X 
X 
X 
X 
 
 
 
 
 
Baseline assessments 
 
 
 
 
X 
X 
X 
X 
X 
X 
 
 
 
 
Construct validity study (novices) 
 
 
 
 
X 
X 
X 
X 
X 
X 
 
 
 
 
Orbis-FVR simulator training 
 
 
 
 
X 
X 
X 
X 
X 
X 
X 
 
 
 
1-month follow-up assessments 
 
 
 
 
 
X 
X 
X 
X 
X 
X 
X 
 
 
Face validity study 
 
 
 
 
 
 
X 
X 
X 
X 
 
 
 
 
Construct validity study (experts) 
 
 
 
 
 
 
X 
X 
X 
X 
 
 
 
 
Qualitative / acceptability data 
 
 
 
 
 
 
 
X 
X 
X 
X 
 
 
 
Data analysis  
 
 
 
 
 
 
 
 
 
X 
X 
X 
 
 
Final report  
 
 
 
 
 
 
 
 
 
 
 
 
X 
 
Presentation & Publication 
 
 
 
 
 
 
 
 
 
 
 
 
X 
X 
 
 
 
 

 
 
Orbis-FVR – Study Protocol         
Version 3.0         
April 12,  2021 
Data Management  
 
All recordings of surgeries (either simulated or real) will be anonymized.  Recordings will be kept on an 
encrypted computer hard drive, and a separate back-up encrypted hard drive in a safe in a locked office 
by the Principal Investigator, and numerically randomized.  Any identifiable information (of the 
performing surgeon) will be kept separately on an encrypted spreadsheet.  No patient identifiable 
information will be recorded at any time, as no patients are involved in the study. Recordings will be 
transported on an encrypted hard-drive if physical transportation is necessary. If this is not practical (in 
terms of delivering the videos to a masked assessor), then the videos will be uploaded to the secure 
CyberSight website. The website will send a notification to the assessor that a video has been uploaded 
and is ready for assessment, however the assessor will need a login name and password to access the 
website and video.   
 
Expected Outcomes of the Study 
 
The outcome of this study is to test the Null Hypothesis that there is no association or relationship 
between the educational intervention of ‘intense virtual reality simulation-based surgical education’ for 
manual small incision cataract surgery. 
 
If the analyzed data from this study does indeed statistically prove the alternate hypothesis, this study has 
the potential of proving, and providing the robust data, that simulation-based VR surgical education in 
MSICS improves competence and has face and construct validity. 
 
Quality Assurance 
 
Data management 
 
All data collected will be anonymized: no participant-identifiable information will be available. The 
anonymization and randomization data will be kept separately.  All data will be backed up weekly on an 
encrypted external hard drive. 
 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
25
Project Management 
 
Study Management 
 
Overall study management responsibility lies with the Principal Investigator.  Three monthly Project 
Update Reports will be circulated to co-investigators. Six monthly reports will be sent to Orbis 
International.  Monthly Project Reports will be prepared by the Principal Investigator (LSHTM).  
 
Advisory Panel   
 
The advisory panel are: 
Amelia Geary, USA 
Professor Nathan Congdon, Northern Ireland 
Dr. Hunter Cherwek, USA 
Dr Glenn Strauss, USA 
Mr John Ferris, UK 
Dr Parikshit Gogate, India 
 
Funding 
 
Orbis International 
 
Medical Registration 
 
No medical registration is necessary for participants or collaborators 
 
Data and safety management 
 
All participant information will be randomized, anonymized and encrypted.  All patient-related surgical 
outcomes data will be anonymized and numerated as per local policy.  No patient identifiable information 
will be made available outside of the hospital or training institution or be made available in any form to 
the PI.  
Ethical Considerations 
 
Ethical Approval & Trial Registration 
 
Ethics approval would be obtained from individual institutional review boards per participating institution.   
Educational ethics are important to consider separately for this study. 
 
 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
26
Participant / Trainee Informed Consent 
 
Each trainee eye surgeon attending the training and involved in qualitative research will be invited to read 
and sign a consent form (Appendix 1). It is important to emphasize that there is no fee for the course and 
all educational materials are given free of charge.  
 
Participant trainees should understand that the course is for their personal educational benefit, and they 
give permission for anonymized data from the study to be published in peer-reviewed literature as part 
of broader research into surgical training techniques. 
 
No personal identifiable information will be included at any stage.   
 
Interviews, opinions, video recordings of assessments, and surgical outcome data of the education and 
training will only be used for academic purposes. 
 
No assessment or report will be given to any of the participant trainees’ colleagues, or surgical or 
educational supervisors.  In other words, this training is as a boost to ‘standard training’, and not a 
replacement: none of the results of this study of training will form a part of the participants’ training 
record. 
 
None of the data collected or reported will be made available to work/training institutions or be used for 
any future job selection.  All and any of the training carries no accreditation, nor official continuous 
professional development (CPD) points. 
 
Trainee participants are free to leave the study at any time. If this is the case for any participant, no effort 
will be made to recover any costs incurred or equipment provided. 
 
The research adheres to the tenets of the Declaration of Helsinki. 
 
 
 
 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
27
Dissemination of Results and Publication Policy 
 
There will be a number of separate aspects of this research to analyze and develop into articles for 
submission to international peer-reviewed journals.   
 
Co-authorship of submitted and published articles will be evaluated as per internationally agreed research 
guidelines:  
 
Authorship credit should be based on: 
 
1. Substantial contributions to conception and design, or acquisition of data, or analysis and interpretation 
of data; 
2. Drafting the article or revising it critically for important intellectual content; and 
3. Final approval of the version to be published. 
 
Authors should meet conditions 1, 2, and 3.  
 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
28
References 
 
REFERENCES 
 
1. Dean WH, Murray NL, Buchan JC, et al. Ophthalmic Simulated Surgical Competency 
Assessment Rubric for manual small-incision cataract surgery. J Cataract Refract Surg 
2019;45(9):1252-57. doi: 10.1016/j.jcrs.2019.04.010 [published Online First: 
2019/09/01] 
2. Flaxman SR, Bourne RRA, Resnikoff S, et al. Global causes of blindness and distance vision 
impairment 1990-2020: a systematic review and meta-analysis. Lancet Glob Health 
2017;5(12):e1221-e34. doi: 10.1016/S2214-109X(17)30393-5 [published Online First: 
2017/10/17] 
3. Dean WH, Gichuhi S, Buchan JC, et al. Intense Simulation-Based Surgical Education for 
Manual Small-Incision Cataract Surgery: The Ophthalmic Learning and Improvement 
Initiative in Cataract Surgery Randomized Clinical Trial in Kenya, Tanzania, Uganda, and 
Zimbabwe. JAMA Ophthalmol 2021;139(1):9-15. doi: 
10.1001/jamaophthalmol.2020.4718 [published Online First: 2020/11/06] 
4. Dean WH, Buchan J, Gichuhi S, et al. Simulation-based surgical education for glaucoma versus 
conventional training alone: the GLAucoma Simulated Surgery (GLASS) trial. A 
multicentre, multicountry, randomised controlled, investigator-masked educational 
intervention efficacy trial in Kenya, South Africa, Tanzania, Uganda and Zimbabwe. Br J 
Ophthalmol 2021 doi: 10.1136/bjophthalmol-2020-318049 [published Online First: 
2021/01/27] 
5. Singh A, Strauss GH. High-fidelity cataract surgery simulation and third world blindness. Surg 
Innov 2015;22(2):189-93. doi: 10.1177/1553350614537120 
6. Broyles JR, Glick P, Hu J, et al. Cataract Blindness and Simulation-Based Training for Cataract 
Surgeons: An Assessment of the HelpMeSee Approach. Santa Monica, CA: RAND 
Corporation, 2012. 
7. Resnikoff S, Lansingh VC, Washburn L, et al. Estimated number of ophthalmologists 
worldwide (International Council of Ophthalmology update): will we meet the needs? Br 
J Ophthalmol 2019 doi: 10.1136/bjophthalmol-2019-314336 [published Online First: 
2019/07/04] 
8. Tabin G, Chen M, Espandar L. Cataract surgery for the developing world. Curr Opin 
Ophthalmol 2008;19(1):55-9. doi: 10.1097/ICU.0b013e3282f154bd 
00055735-200801000-00013 [pii] [published Online First: 2007/12/20] 
9. Sherwin JC, Dean WH, Schaefers I, et al. Outcomes of manual small-incision cataract surgery 
using standard 22 dioptre intraocular lenses at Nkhoma Eye Hospital, Malawi. Int 
Ophthalmol 2012;32(4):341-7. doi: 10.1007/s10792-012-9565-x [published Online First: 
2012/05/05] 
10. Gogate P, Biswas P, Natarajan S, et al. Residency evaluation and adherence design study: 
Young ophthalmologists' perception of their residency programs - Clinical and surgical 
skills. Indian J Ophthalmol 2017;65(6):452-60. doi: 10.4103/ijo.IJO_643_16 [published 
Online First: 2017/06/24] 
11. Guzek JP, Ching A. Small-incision manual extracapsular cataract surgery in Ghana, West 
Africa. J Cataract Refract Surg 2003;29(1):57-64. doi: 10.1016/s0886-3350(02)01440-2 
[published Online First: 2003/01/29] 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
29
12. Ruit S, Tabin G, Chang D, et al. A prospective randomized clinical trial of phacoemulsification 
vs manual sutureless small-incision extracapsular cataract surgery in Nepal. Am J 
Ophthalmol 2007;143(1):32-38. doi: S0002-9394(06)00863-4 [pii] 
10.1016/j.ajo.2006.07.023 [published Online First: 2006/12/26] 
13. Gogate P, Deshpande M, Nirmalan PK. Why do phacoemulsification? Manual small-incision 
cataract surgery is almost as effective, but less expensive. Ophthalmology 
2007;114(5):965-8. doi: S0161-6420(06)01338-8 [pii] 
10.1016/j.ophtha.2006.08.057 [published Online First: 2007/02/14] 
14. Haripriya A, Chang DF, Reena M, et al. Complication rates of phacoemulsification and 
manual small-incision cataract surgery at Aravind Eye Hospital. J Cataract Refract Surg 
2012;38(8):1360-9. doi: 10.1016/j.jcrs.2012.04.025 
15. Privett B, Greenlee E, Rogers G, et al. Construct validity of a surgical simulator as a valid 
model for capsulorhexis training. J Cataract Refract Surg 2010;36(11):1835-8. doi: 
S0886-3350(10)01210-1 [pii] 
10.1016/j.jcrs.2010.05.020 [published Online First: 2010/10/30] 
16. Feudner EM, Engel C, Neuhann IM, et al. Virtual reality training improves wet-lab 
performance of capsulorhexis: results of a randomized, controlled study. Graefes Arch 
Clin Exp Ophthalmol 2009;247(7):955-63. doi: 10.1007/s00417-008-1029-7 
17. Waqar S, Park J, Kersey TL, et al. Assessment of fatigue in intraocular surgery: analysis using 
a virtual reality simulator. Graefes Arch Clin Exp Ophthalmol 2011;249(1):77-81. doi: 
10.1007/s00417-010-1531-6 
18. Jonas JB, Rabethge S, Bender HJ. Computer-assisted training system for pars plana 
vitrectomy. Acta Ophthalmol Scand 2003;81(6):600-4. 
19. Thomsen AS, Kiilgaard JF, Kjaerbo H, et al. Simulation-based certification for cataract 
surgery. Acta Ophthalmol 2015;93(5):416-21. doi: 10.1111/aos.12691 
20. HelpMeSee. HelpMeSee Launches Revolutionary Technology in Response to the Global 
Cataract Crisis 2020 [Available from: https://help-me-see.prezly.com/helpmesee-
launches-revolutionary-technology-in-response-to-the-global-cataract-crisis accessed 11 
November 2020. 
21. Phillips C. Ophthalmic Simulated Surgery 2020 [Available from: 
http://www.phillipsstudio.co.uk accessed 1 February 2020. 
22. Landis JR, Koch GG. The measurement of observer agreement for categorical data. 
Biometrics 1977;33(1):159-74. [published Online First: 1977/03/01] 
23. Dean WH, Buchan J, Admassu F, et al. Ophthalmic simulated surgical competency 
assessment rubric (Sim-OSSCAR) for trabeculectomy. BMJ Open Ophthalmol 
2019;4(1):e000313. doi: 10.1136/bmjophth-2019-000313 [published Online First: 
2019/09/17] 
 
 
 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
30
Appendices 
 
 
Appendix 1 Participant Consent Form  
Appendix 2 Participant Information Sheet 
Appendix 3 Training Intervention 
Appendix 4 MSICS Sim-OSSCAR 
Appendix 5 Acceptability and Utility Survey  
Appendix 6 Proposed Budget 
 
 
 
 
 
 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
31
 
Appendix 1  
 
Participant Consent Form (Orbis-FVR)  
 
 
Validity and effectiveness of a virtual reality simulator for manual small incision cataract surgery: The 
Orbis-FundamentalVR MSICS Simulator.  
 
• 
London School of Hygiene and Tropical Medicine, UK 
• 
Orbis International, New York, USA 
• 
Queen’s University Belfast, Northern Ireland, UK 
• 
FundamentalVR, London, UK 
• 
Chittagong Eye Infirmary and Training Complex, Chittagong, Bangladesh 
• 
Addis Ababa University, department of Ophthalmology, Menelik II Hospital, Addis Ababa, 
Ethiopia 
• 
University of Gondar, Gondar, Ethiopia 
• 
Dr. Shroff’s Charity Eye Hospital (SCEH), New Delhi, India 
• 
H.V. Desai Eye Hospital, Pune, India 
• 
The First Central Hospital, Ulaanbaatar, Mongolia 
• 
Shenyang He Eye Specialist Hospital, China. 
• 
Lumiere Devine Eye Hospital, Lome, Togo 
• 
Emory Eye Center, Atlanta, GA USA 
 
I  ____________________________________________________________________ (name) have been 
invited to participate in a study of surgical training, involving a two day intense training and education 
course for cataract surgery and ongoing assessment for the following one month. I understand there is 
no fee for the course, and all educational materials are given free of charge. I understand that the 
course is for my personal educational benefit.   
 
Study Reference Number:  
 
 
 
 
 
 
Please initial box 
1. I confirm that I have read and understand the participant information sheet 
dated ......….. (version ............) for the Orbis-FVR study.  I have had the opportunity to 
consider the information, ask questions and have had these answered fully. 
 
2. I understand that my participation is voluntary, and I am free to withdraw at any 
time, without giving any reason, without training or legal rights being affected. 
 
3. I give my permission for anonymized data from this course to be published in peer-
reviewed literature as part of broader research into surgical training techniques, 
including the placement of an anonymized data set in a data repository. 
 
4. I understand that no personal identifiable information will be included in any 
published output.   
 
5. I understand that interviews, opinions, or recordings of the education and training 
will only be used for academic purposes. 
 
6. I understand that no formal feedback will be given to any of my colleagues or 
surgical supervisors 
 
 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
32
7. I understand that no data will be made available to work/training institutions or be 
used for any future job selection. 
 
 
8. I agree to anonymized simulation video recording and assessment at baseline, and 
one month of my surgery 
 
 
 
 
 
 
Signed _______________________________________________    Date:  ____________________ 
 
Name  __________________________________________________________ 
 
Countersigned by collaborating investigator    
 
Principle Investigator Dr William H Dean PhD FRCOphth  MEd  MBChB  BSc 
Principle Investigator (Orbis): Amelia Geary  
 
Any queries should be directed in the first instance to the Principal Investigator Dr Will Dean: 
Will.Dean@lshtm.ac.uk 
Phone: UK +44(0)7899 753 953    
 
 
 
Please refer to Participant Information Sheet (Orbis-FVR Version 1.0) 
 
 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
33
Appendix 2  
Participant Information Sheet – Orbis-FVR 
 
Validity and effectiveness of a virtual reality simulator for manual small incision cataract surgery: The 
Orbis-FundamentalVR MSICS Simulator.  
 
Participant Information Sheet (Orbis-FVR Version 1.0) 
 
• 
London School of Hygiene and Tropical Medicine, UK 
• 
Orbis International, New York, USA 
• 
Queen’s University Belfast, Northern Ireland, UK 
• 
FundamentalVR, London, UK 
• 
Chittagong Eye Infirmary and Training Complex, Chittagong, Bangladesh 
• 
Addis Ababa University, department of Ophthalmology, Menelik II Hospital, Addis Ababa, 
Ethiopia 
• 
University of Gondar, Gondar, Ethiopia 
• 
Dr. Shroff’s Charity Eye Hospital (SCEH), New Delhi, India 
• 
H.V. Desai Eye Hospital, Pune, India 
• 
The First Central Hospital, Ulaanbaatar, Mongolia 
• 
Shenyang He Eye Specialist Hospital, China. 
• 
Lumiere Devine Eye Hospital, Lome, Togo 
• 
Emory Eye Center, Atlanta, GA USA 
 
LSHTM Principal Investigator:  Dr William Dean PhD FRCOphth  MEd  MBChB  BSc 
Orbis Principal Investigator:   
Amelia Geary. MSc 
Orbis Principal Investigator:   
Professor Nathan Congdon  PhD MD 
 
Introduction 
 
You are being invited to take part in an educational-intervention research study. Before you decide 
whether or not you will be a participant, it is important for you to understand why this research is being 
done and what it will involve.  
 
Please take time to read the following information carefully. Talk to others about the study, including your 
training program Director, if you wish. Ask us if there is anything that is not clear or if you would like more 
information. 
 
This form is designed to tell you everything you need to think about before you decide whether or not 
you agree to be in the study. It is entirely your choice.  If you decide to take part, you can change your 
mind later on and withdraw from the study. The decision to join or not join the study will not cause you 
to lose any of your usual training opportunities within your Ophthalmology Training Institution course. 
 
You can take a copy of this information sheet, to keep. Do not sign the consent form unless you have had 
a chance to ask questions and get answers that make sense to you. By signing this form, you will not give 
up any legal rights. 
 
Do you have to take part in this study? 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
34
No. You do not have to take part in this study. Even if you do not take part in this study you will still be 
offered exactly the same training as per your training institution and curriculum. 
Study Overview 
 
What is the study about? 
Globally there are an estimated 36 million people who are blind and a further 207 million with significant 
visual impairment. Approximately 80% of blindness is preventable or treatable, and 90% of the burden is 
in Low- and Middle-Income Countries (LMIC). Age-related cataract accounts for about one-third of this 
blindness. Manual small incision cataract surgery (MSICS) is a widely accepted, appropriate and affordable 
procedure with high quality visual outcomes.   
 
It has been acknowledged however that the curriculum-integration of simulation is only in its infancy, as 
with many ophthalmology training programs around the world.  There is no coherent, sustainable, 
standardized and educationally-underpinned regional training program employing simulation.  
Furthermore, there is no robust evidence or significant data testing the efficacy of simulation-based 
surgical education in cataract and glaucoma surgery. 
 
Of the more than 230,000 ophthalmologists in the world, less than half perform cataract surgery.  The 
shortage of expert eye surgeons in low and middle-income settings is well documented in the literature.  
This leads to a number of challenges, including the amount of time is available for training. There is a need 
to develop innovative, efficient, well-evidenced, and cost-effective strategies for ophthalmic training in 
the globally. 
 
This is a prospective, single-masked education-intervention study of intense virtual reality (VR) simulation-
based surgical education. The aim is to investigate whether VR simulation-based surgical education 
improves competence. All participants will receive the educational intervention of ‘two-days intense VR 
simulation-based training’. This training is in addition to the trainees’ normal current standard training, 
and not designed to replace it.  
 
Why have you been chosen?  
You are being invited to join the study because you are an ophthalmologist in training at one of the 
collaborating Institutions, and you may meet all the eligibility criteria. 
 
How many people are taking part in this trial? 
We plan to recruit 60 trainees in total. 
 
Procedures 
 
What will we ask you to do? 
 
Baseline assessment:  
We will ask you some basic questions on cataract and cataract surgery.  We will ask you about your 
previous surgical experience. 
 
Further Baseline assessment:  
We will show you some of the basics of the procedure of MSICS, and the performing of a procedure using 
simulation (artificial eyes).  We will then invite you to perform three simulation MSICS procedures using 
artificial eyes, which we will record (these recordings will be anonymized).  
 
Educational Intervention:  

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
35
Once you are enrolled, you will receive clear instruction on how the timetable will run. The intervention 
is an intense virtual reality training program using the Orbis-FundamentalVR simulator.  
 
Follow-up assessments: 
We will revisit you at your Training Institution at 1 month after your enrolment to the study and training 
intervention.  We will invite you to perform three further simulation MSICS procedures using model eyes 
(which again we will record and anonymize).  
 
What is the educational intervention that is being tested?  
The surgical education that is being is investigated is intense simulation-based surgical training.  This 
involves an intense 2-day course. No patients are involved in this training.  This training is not meant to 
replace standard training, but to augment it.  
 
Benefits 
What benefits are there to taking part in the study? 
You will be offered free VR simulation-based surgical training.   
 
Risks 
What are the risks of taking part? 
There are very low risks associated with participating in this study. You will be away from normal work 
and training for two days. No patients are involved in the assessments or training. 
 
There is however no risk that this training will affect, or reflect on, your current training course marks, 
future employment, or be reported to your training program Director. 
 
What will happen to the assessment recordings, interviews, feedback, and surgical outcomes data I 
give? 
The video recordings will be made using the same blue latex-free gloves for all participants, using the 
same instruments, and the same standard recording equipment.  They will also be anonymized so that 
none of your personal information will be identifiable.  These recordings will be stored on an encrypted 
hard drive.  Interviews will be recorded and transcribed, anonymized, and thematized: again, no personal 
identifiable information will be kept.  Once this data is reported, none of your personal related 
information will be made available. Summarized, anonymized data will be including the placement of an 
anonymized data set in a data repository. 
 
Are there any other alternative educational interventions available? 
There is growing evidence that simulation-based surgical education is a valid way to augment surgical 
training.  It is envisaged that in years to come, there will be further local, national, and regional 
opportunities to engage in this.   
 
Withdrawal from the Study 
You have the right to leave a study at any time without penalty. The researchers and sponsor also have the 
right to stop your participation in this study without your consent if, for example: 
• 
They believe there has been ‘contamination’ between “Intervention” and “Control” individuals 
• 
You were not to agree to any future changes that may be made in the study plan 
 
New Information 
What will we do if we find if one educational intervention is better than the other? 
If we find that intense simulation-based surgical training is better than none, we will publish this finding 
and envisage that it will lead to further funding for such training.  

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
36
 
Payment  
You will not be offered payment for being in this study.  
 
Costs 
There will be no costs to you for participating in this study. You will not be charged for any of the research 
activities. You will not receive any additional payments or per diems for participating, beyond your normal 
stipend or salary from your training unit. 
 
 
Confidentiality  
What will happen to the records/interview, and videos we keep of your simulation operations? 
All the information and videos we collect will be kept confidential. It will be kept securely and only the 
primary investigator, or expert markers will have access to it. A study number rather than your name will 
be used on study records wherever possible. Your name and other facts that might identify you will not 
appear when we present this study or publish its results. No information from this study will be placed 
into your ophthalmology training record.   
 
 
In Case of Complaint 
What if there is a problem? 
Any complaint about the way you have been treated during the study will be addressed. Please use the 
addresses below to contact the study coordinators.   
 
Who funded this study? 
The study is funded through Orbis International.     
 
Who has reviewed the study?  
This study was reviewed by the advisory panel of international cataract, ophthalmic surgical training, and 
education experts in India, UK, USA and South Africa.  
 
 
Who is doing this study? 
The study will be coordinated by Dr Will Dean who is an ophthalmology attending consultant who has a 
PhD from LSHTM and a MEd (Masters in Education) in Surgical Education at Imperial College London; a 
Fellowship of the Royal College of Ophthalmology (UK); over 20 years of experience in ophthalmology and 
training ophthalmologists in globally.  
 
 
Contact Information 
If you have any questions please ask us:  
• 
if you have any questions about this study or your part in it, or 
• 
if you have questions, concerns or complaints about the research 
 
Dr. Will Dean at +44 7899 753 953 or will.dean@lshtm.ac.uk 
 
 
You will be given a copy of the information sheet. 
Thank you for considering taking the time to read this sheet. 

 
 
Orbis-FVR – Study Protocol         
Version 3.0         
April 12,  2021 
Appendix 3. 
Training Intervention 
 
Pre-Learning (one month in advance) 
• 
Surgical video review 
 
Pre-Course Assignment (one week in advance) 
• 
Complete and record three full MSICS cases using the Philips MSICS model eye and following the 
steps of the OSSCAR (Appendix 4). 
• 
Upload the 3 recorded simulated surgical procedures to the Cybersight platform 
 
The Orbis-FVR Virtual Reality MSICS simulator has 19 steps in total.  Currently, 10 steps are animated 
and 9 are interactive, as follows: 
20. Patient Preparation – Animated Step 
21. Conjunctival Peritomy – Animated Step 
22. Paracentesis – Aminated Step 
23. Formation of the AC – Animated Step 
24. Anterior Chamber Maintainer – Animated Step 
25. Marking the Scleral Incision – Interactive Step 
26. Scleral Incision – Interaction Step 
27. Sclerocorneal tunnel – Interactive Step 
28. AC Entry – Interactive Step  
29. CCC – Interactive Step 
30. Hydrodissection – Animated Step 
31. Mobilization of the nucleus – Interactive Step 
32. Releasing the nucleus – Animated Step 
33. Nucleus removal – Interactive Step 
34. Cortex removal – Animated Step 
35. IOL insertion – Interactive Step 
36. IOL Dialling and Implantation – Interactive Step 
37. Inspection of the wound – Animated Step 
38. Closure of the wound – Animated Step 
 
Course Overview 
Participants must attend and successfully complete all sessions.  All simulator tasks are to be completed 
in sequential order, in line with the course outline.  A required score of 85 points for each interactive 
step must be reached reliably three times in a row to advance to the next step in the course. 
 
For each step within the simulation the following is recorded:  
Step Time - The total time spent on a step including ALL attempts. 
Pass / Fail - The interpreted pass or fail result of the last attempt of the step. 
Step score - Each step is scored between 0 and 100 points, based on user performance in completing the 
surgical objective, taking into account any adverse events. 
• 
Adverse events –For each step, if triggered, all of the adverse events are recorded. 
• 
No. of attempts - The number of attempts (inc. resets) of the step. 
 
Course Content 
• 
Each course will combine autonomous learning and live mentorship.   

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
38
• 
Autonomous learning will consist of a series of 15-minute video presentations, which include: 
o Narrated demonstration video of specific VR step(s) 
o PowerPoint presentations 
o Instrument use demonstrations 
• 
Live mentorship will be delivered in two ways: 
o Local mentor who provides in-person mentorship 
o International mentor from technical advisory group provides live tele-mentorship 
 
Content hosting 
• 
All content will be hosted on the Orbis Cybersight Learning Management System platform.  Data 
will be randomized and anonymized by the Cybersight Program manager, who will not be 
involved in any other aspect of the research initiative.   
 
Course delivery 
Each day: 
• 
Local mentor will facilitate introductions in each morning session.  This will cover: 
o An overview of the day’s training plan 
o Outlining the learning objectives for the day’s training plan 
o Introduction of the content covered in the day’s training plan  
• 
Local mentor will also facilitate discussion and answer questions on autonomous content 
viewed in the morning and midday sessions  
• 
Local mentor will provide periodic check ins throughout the day’s session 
• 
International mentor will provide tele-mentorship for the first hour of the 2-5pm session 
• 
Trainees will do self-practice for the remainder of the afternoon session 
• 
Each trainee will complete each interactive step on the VR simulator a minimum of 5 times with 
a reliability gate (3) and minimum score of 85 
• 
Each trainee will complete at least 5 full cases on the VR simulator with a reliability gate (3) and 
minimum score of 85 
 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
39
Course outline  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Attendance to be recorded as follows: 
• 
Confirmation of Three video uploads pre-intervention 
• 
Simulator use and assessment logs 
• 
Attendance records taken per daily session 
• 
Confirmation of Three video uploads post-intervention 
 
 
Day 
 
Morning 
8:00 – 10:30 
Midday 
11:00 – 1:00 
Afternoon 
2:00 – 5:00 
Monday 
 
 
 
 
Introduction 
Orientation to 
simulator. 
Review of animated 
steps 1-5 
Marking the scleral 
Incision  
Scleral Incision  
Sclerocorneal tunnel 
 
Mentored practice 
Self-Practice  
Tuesday 
 
 
 
 
Introduction 
Review  
CCC 
AC entry 
Hydrodissection 
Releasing the Nucleus 
Nucleus Removal  
Cortex Removal  
Mentored practice 
Self-Practice 
Wednesday 
 
 
 
Introduction 
Review 
IOL Insertion  
IOL Dialling and 
implantation  
Inspection of the 
wound 
Closure of the wound 
Mentored practice 
Self-Practice 
Thursday  
Introduction 
Review of entire SICS 
procedure. 
Putting it all together: 
Full Cases 
 
Mentored practice 
Self-Practice 
Friday  
Introduction 
Assessment and 
Scoring  
 
Final Review 
 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
40
Appendix 4.   
MSICS Sim-OSSCAR 
 
 
 
 
 
 
 
 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
41
 
 
 
 
Appendix 5. 
Acceptability and Utility Survey & Questionnaire 
 
 
What is your gender? 
• 
Male 
• 
Female 
• 
Other 
 
Level of experience: 
• 
Novice 
• 
Expert 
 
Likert scale statements   
1. 
I was comfortable using the VR simulator 
2. 
I found the VR simulator user friendly 
3. 
I was able to easily use the VR simulator 
4. 
The VR simulator tutorials were helpful  
5. 
I was able to navigate the VR program with ease 
6. 
The VR simulator helped me gain a better understanding of the MSICS procedure 
7. 
The VR simulator improved my knowledge of the MSICS procedure 
8. 
The VR simulator improved my MSICS skills 
9. 
I found the VR simulator hardware user friendly  
10. I would recommend training on the VR simulator to others 
 
Responses will be evaluated on a 5-point Likert scale anchored at ‘1=strongly disagree’ and ‘5=strongly 
agree’. 
 
User feedback – open ended questions 
• 
What elements of the simulator were most useful? Which steps or resources? 
• 
Which were least useful? 
• 
What change or improvement to the simulator program would you recommend?   
• 
Do you think the VR simulator has a place in cataract surgical training?  
• 
Any other comments?   
 
 
Pre- and Post-Intervention Confidence Ratings: 
1> On a scale of 1-10, how confident are you in your ophthalmic surgical skills? 
2> On a scale of 1-10, how confident are you in MSICS skills? 
3> What has impacted this confidence? [open question] 
 
 

Orbis-FVR – Study Protocol         
Version 1.1         
March 2021 
42
 
Appendix 6. 
Proposed Budget 
 
 
Description 
Unit cost (USD) Quantity  
Total (USD) 
Research Project Costs 
Artificial eyes 
250 
60 
15,000 
Shipping for eyes 
1 
3000 
3,000 
Customs fees for eyes  
14.7 
30 
441 
AV/IT for video recording  
5 
100 
5,000 
Cost of grading (outsourced) 
5 
360 
1,800 
Course build out on CS 
2,500 
1 
2,500 
Cost for data analysis 
1 
5000 
5,000 
Publication costs 
1 
3000 
3,000 
Technical support 
700 
12 
8,400 
Research Project subtotal 
    
44,141 
Country Support Costs 
Country team support Bangladesh 
               8,460  
1 
8,460 
Country team support Ethiopia 
12,840 
1 
12,840 
Country team off support India 
8,063 
1 
8,063 
Country office subtotal 
    
29,363 
VR Kit Shipping Costs 
Custom fees 
3000 
6 
18,000 
Shipping fees  
TBD 
7 
15,000 
Shipping Subtotal  
    
33,000 
TOTAL  
    
106,504