A Clinical Investigation of an Autonomous Phone Conversational Agent for Cataract Surgery Follow-up
Phase: NA
Conditions: Cataract, After Cataract
Sponsor: University of Plymouth

 
 
 
 
 
 
 
 
 
 
A Clinical Investigation of an Autonomous Phone Conversational Agent for Cataract Surgery 
Follow-up   
 
 
 
NCT05213390 
 
 
 
17 May 2021 

A clinical investigation of an autonomous phone
conversational agent for cataract surgery follow-up
Protocol Version
1.0
Date
17 May 2021
Sponsor
Imperial College London
IRAS Number
297548
Chief Investigator
Principal Investigator (Imperial)
Eduardo Normando, MD PhD
Clinical Senior Lecturer
Imperial College London
Honorary Consultant Ophthalmologist
Imperial College Healthcare NHS Trust
Co-Investigator
Edward Meinert, MA MSc MBA MPA PhD CEng FBCS
EUR ING
Associate Professor of eHealth
Centre for Health Technology
University of Plymouth
Honorary Senior Lecturer
Imperial College London
Principal Investigator (Oxford)
Kanmin Xue, MA MB BChir PhD FRCOphth
Wellcome Trust Clinical Research Career Development
Fellow
Honorary Consultant Ophthalmologist
Oxford University Hospitals NHS Foundation Trust
Contacts
e.normando@imperial.ac.uk
edward.meinert@plymouth.ac.uk
kanmin.xue@ouh.nhs.uk

Autonomous telephone follow-up after cataract surgery
Table of Contents
1 STUDY INFORMATION
4
1.1 Full/Long Title of the Study
4
1.2 Short Study Title
4
1.3 Protocol Version Number(s) and Date(s)
4
1.4 Research Reference Numbers
4
2 SIGNATURE PAGE
5
3 KEY STUDY CONTACTS
6
4 ABBREVIATIONS AND KEY TERMS
8
5 STUDY SUMMARY
9
5.1 Funding and Support in Kind
9
5.2 Role of Study Sponsor and Funder
10
5.3 Roles and Responsibilities of Study Management Committees/Groups and Individuals
10
5.4 Protocol Contributors
10
5.5 Keywords
11
6 STUDY FLOW CHART
12
7 INTRODUCTION
14
7.1 Background
14
7.2 Rationale
14
7.3 Theoretical Frameworks
15
7.4 Research Question
15
7.5 Aims and Objectives
16
8 METHODS
17
8.1 Trial Design
17
8.1.1 Timeline
17
8.2 Study Context
17
8.2.1 Participants
17
8.2.2 Setting
17
8.2.3 Intervention(s)
18
8.2.4 Comparator(s)
19
8.2.5 Outcomes
19
8.2.5.1 Primary Outcomes
19
8.2.5.2 Secondary Outcomes
20
8.3 Sample and Recruitment
22
8.3.1 Eligibility Criteria
22
8.3.1.1 Inclusion Criteria
22
8.3.1.2 Exclusion Criteria
22
Version number 1.0 – 17 May 2021
1

Autonomous telephone follow-up after cataract surgery
8.3.1.3 Withdrawing
22
8.3.2 Sampling
22
8.3.2.1 Sample Size
22
8.3.2.2 Sampling Technique
24
8.3.3 Recruitment
25
8.3.3.1 Participant Identification
25
8.3.3.2 Consent
25
8.3.4 Adherence
25
8.4 Data Collection
25
8.4.1 Randomisation
25
8.4.1.1 Sequence Generation
25
8.4.1.2 Allocation Concealment Mechanism
25
8.4.1.3 Implementation
25
8.4.2 Blinding
26
8.4.3 Methods of Data Collection
26
8.4.3.1 Quantitative data collection
26
8.4.3.2 Qualitative data collection
26
8.5 Data Analysis
26
8.5.1 Statistical Methods for Primary Outcome
26
8.5.2 Other Data Analyses
27
8.6 Limitations
27
8.7 Generalisability
27
9 ETHICAL AND REGULATORY CONSIDERATIONS
28
9.1 Assessment and Management of Risk
28
9.2 Research Ethics Approval
28
9.3 Peer Review
28
9.4 Patient and Public Involvement
29
9.4.1 Involving a lay representative on the study monitoring group
29
9.4.2 Involving a dedicated PPI group
29
9.5 Protocol Compliance
30
9.6 Consent
30
9.7 Data Protection and Patient Confidentiality
31
9.8 Stopping Guidelines
31
9.9 Indemnity
31
9.10 Access to DORA’s code
32
9.11 Access to Final Study Data
32
10 DISSEMINATION POLICY
33
10.1 Dissemination Strategy
33
Version number 1.0 – 17 May 2021
2

Autonomous telephone follow-up after cataract surgery
10.2 Authorship Eligibility Guidelines and Use of Professional Writers
33
11 DECLARATIONS
34
11.1 Funding
34
11.2 Protocol Registration
34
11.3 Competing Interests
34
12 REFERENCES
35
13 APPENDICES
38
13.1 Appendix 1. Participant Information Sheet
38
13.2 Appendix 2. Informed Consent Form
43
13.3 Appendix 3. SPIRIT-AI Checklist: Recommended items to address in a protocol and related
documents for clinical trials evaluating AI interventions
45
13.4 Appendix 4. Sample Post-Call Survey
51
13.5 Appendix 5. Topic Guide for Semi-Structured Interviews
55
13.6 Appendix 6. Imperial Historical Recruitment Audit Data
56
13.7 Appendix 7. Oxford Historical Recruitment Audit Data
58
Version number 1.0 – 17 May 2021
3

Autonomous telephone follow-up after cataract surgery
1 STUDY INFORMATION
1.1 Full/Long Title of the Study
A clinical investigation of an autonomous phone conversational agent for cataract surgery follow-up
1.2 Short Study Title
Autonomous telephone follow-up after cataract surgery
1.3 Protocol Version Number(s) and Date(s)
.0
Initial draft
10 November 2020
.1
Pre-IRAS format submission draft
19 January 2021
.2
Updated to reflect feedback from trial team
2 February 2021
.3
Revisions from joint call with trial team
22 February 2021
.4
Minor revisions
23 February 2021
.5
Minor revisions
30 March 2021
.6
Sponsor feedback revisions
27 April 2021
.7
Minor revisions
14 May 2021
1.0
Submission version
17 May 2021
1.4 Research Reference Numbers
IRAS Number:
297548
SPONSORS Number:
21WE6780
NIHR Award Number:
AI_AWARD01852
Version number 1.0 – 17 May 2021
4

Autonomous telephone follow-up after cataract surgery
2 SIGNATURE PAGE
The undersigned confirm that the following protocol has been agreed and accepted and that the
Chief Investigator agrees to conduct the study in compliance with the approved protocol and will
adhere to the principles outlined in the Declaration of Helsinki, the Sponsor’s procedures, and other
regulatory requirements.
I agree to ensure that the confidential information contained in this document will not be used for
any other purpose other than the evaluation or conduct of the investigation without the prior
written consent of the Sponsor.
I also confirm that I will make the findings of the study publicly available through publication or other
dissemination tools without any unnecessary delay and that an honest accurate and transparent
account of the study will be given; and that any discrepancies from the study as planned in this
protocol will be explained.
For and on behalf of the Study Sponsor:
Signature:
......................................................................................................
Date:
....../....../......
Name (please print):
......................................................................................................
Position:
......................................................................................................
Chief Investigator:
Signature:
......................................................................................................
Date:
....../....../......
Name: (please print):
Dr Eduardo Normando
Version number 1.0 – 17 May 2021
5

Autonomous telephone follow-up after cataract surgery
3 KEY STUDY CONTACTS
Table 1. Key Study Contacts
Chief Investigator
Eduardo Normando, MD PhD
Clinical Senior Lecturer
Imperial College London
+44 (0)20 3312 3206
e.normando@imperial.ac.uk
Study Coordinator
Serge Miodragovic
Clinical Trials Manager
Jessica Bonetti
Senior Clinical Trials Coordinator
Gina Margai
Clinical Trials Coordinator
Imperial College Ophthalmology Research Group Clinical Trials Unit
Imperial College Healthcare NHS Trust
Sponsor
Imperial College London
Exhibition Rd, South Kensington, London SW7 2BU
020 7589 5111
Funder(s)
National Institute for Health Research (NIHR) - Funding body
Artificial Intelligence in Health and Care Award
Dr Aayesha Hassan, Programme Manager
Programme Management Office
Grange House
15 Church Street
Twickenham, TW1 3NL
aayesha.hassan@nihr.ac.uk
Ufonia Limited - Grant Holder
Dr Nick de Pennington, Chief Executive Officer
Ufonia Limited
c/o Oxford University Innovation
Buxton Court, 3 West Way
Oxford, OX2 0JB
ndep@ufonia.co
Key Protocol
Contributors
Edward Meinert, MA MSc MBA MPA PhD CEng FBCS EUR ING
Associate Professor of eHealth
Centre for Health Technology
University of Plymouth
Plymouth, PL4 6DN
Honorary Senior Lecturer
Imperial College London
Version number 1.0 – 17 May 2021
6

Autonomous telephone follow-up after cataract surgery
edward.meinert@plymouth.ac.uk
Kanmin Xue, MA MB BChir PhD FRCOphth
Wellcome Trust Clinical Research Career Development Fellow
Nuffield Department of Clinical Neurosciences
University of Oxford
John Radcliffe Hospital
Oxford, OX3 9DU
kanmin.xue@ouh.nhs.uk
Ernest Lim, BSc MBBS
Imperial College Healthcare NHS Trust
Western Eye Hospital
London, United Kingdom, NW1 5QH
ernest.lim1@nhs.net
Aisling Higham, MBBS MSc FRCOphth
Oxford University Hospitals NHS Foundation Trust
John Radcliffe Hospital
Oxford, OX3 9DU
aisling.higham@ouh.nhs.uk
Guy Mole, BSc MBBS MSc
Ufonia Limited
c/o Oxford University Innovation
Buxton Court, 3 West Way
Oxford, OX2 0JB
Oxford University Hospitals NHS Foundation Trust
John Radcliffe Hospital
Oxford, OX3 9DU
gm@ufonia.co
Version number 1.0 – 17 May 2021
7

Autonomous telephone follow-up after cataract surgery
4 ABBREVIATIONS AND KEY TERMS
Table 2. Abbreviations
Abbreviation /
Key Term
Full Phrase / Definition
AHSN
Academic Health Science Network
AI
Artificial Intelligence
BRC
Biomedical Research Centre
CONSORT
Consolidated Standards of Reporting Trials
DPA 2018
Data Protection Act 2018
EHR
Electronic Health Record
FAQ
Frequently Asked Question
GDPR
General Data Protection Regulation
GIRFT
Getting It Right First Time programme
ICORG
Imperial College Ophthalmology Research Group
Imperial
Imperial College Healthcare NHS Trust
NASSS
Non-adoption, Abandonment, Scale-up, Spread, and Sustainability framework
NHS
National Health Service
NOD
National Ophthalmology Database
NPS
Net Promoter Score
Oxford
Oxford University Hospitals NHS Foundation Trust
PERC
Patient Experience Research Centre
PPI
Patient and Public Involvement
SPIRIT-AI
Standard Protocol Items: Recommendations for Interventional Trials - Artificial
Intelligence
SUS
System Usability Scale
TUQ
Telehealth Usability Questionnaire
Ufonia
Ufonia Limited (Company Number: 10692039)
Version number 1.0 – 17 May 2021
8

Autonomous telephone follow-up after cataract surgery
5 STUDY SUMMARY
Table 3. Study Summary
Study Title
A clinical investigation of an autonomous phone conversational
agent for cataract surgery follow-up
Short title
Autonomous telephone follow-up after cataract surgery
Study Design
The study will be a multi-centre, mixed-methods clinical
investigation to develop evidence regarding the feasibility,
acceptability and potential effectiveness of DORA.
Study Participants
The study will include adults (aged 18 years or older) who are on
the waiting list for either their first or second eye cataract surgery
at Oxford or Imperial. To be eligible, the patients must have no
history of significant ocular comorbidities and have a routine
surgery with no intraoperative complications.
Planned Size of Sample
The planned sample size is 591 patients, split mostly equally
between Oxford and Imperial clinical sites (see Table 1 for details
on how this estimate was produced).
Follow up duration
Imperial: patients will receive the standard face-to-face
appointment within a few days of having the autonomous call
(dependent on current Covid-19 restrictions).
Oxford: there will be no face-to-face follow-up for patients at the
Oxford site as this is the standard of care.
At both sites, an additional 3-month follow-up period where
unplanned eye-related clinical visits will be recorded and assessed.
Planned Study Period
The total duration of the study will take up to 3 years, including the
recruitment of patients, intervention refinement period, main
intervention period, and analysis of results.
Research Question/Aim(s)
The aim of the study is to assess the evidence for the safety of
using DORA to deliver autonomous cataract surgery follow-up
assessments. This ability will be evaluated in comparison to an
expert human clinician to assess the research question: Can DORA
identify which patients need clinical follow up after a cataract
operation?
5.1 Funding and Support in Kind
Table 4. Funding Information
FUNDER
FINANCIAL AND NON-FINANCIAL SUPPORT GIVEN
National Institute for Health Research (NIHR)
Dr Aayesha Hassan
£503,524.00
Version number 1.0 – 17 May 2021
9

Autonomous telephone follow-up after cataract surgery
Programme Management Office
Grange House
15 Church Street
Twickenham
TW1 3NL
aayesha.hassan@nihr.ac.uk
5.2 Role of Study Sponsor and Funder
This study is funded by an NIHR Artificial Intelligence in Health and Care Award granted to Ufonia.
Ufonia distributes on behalf of NIHR funding to Imperial College London, the University of Plymouth
and the University of Oxford for this study execution. This funding arrangement is governed by a
consortium agreement which defines Imperial College London as being responsible for the study
design, conduct, data analysis and interpretation, manuscript writing, and dissemination of results in
collaboration with the academic consortium including the University of Plymouth and the University
of Oxford. Whilst Ufonia has contributed and reviewed the study design and will support the trial in
providing technology support, the academic collaboration, led by Imperial College London, is solely
responsible for the study execution and dissemination of results to assure an independent
assessment of Ufonia's technology. The academic consortium maintains a free and independent
unrestricted right of publication of the study's findings as to avoid any conflict of interest.
5.3 Roles and Responsibilities of Study Management Committees/Groups and
Individuals
A study steering board composed of the Chief Investigator, Principal Investigators, Co-Investigator, a
member of the public, Ufonia’s CEO, Ufonia’s Medical Director and an external researcher (from the
study) will meet every two months to review progress against the protocol to provide study
governance and oversight. Reports shall be distributed for review by the study team for action.
Ufonia will appoint a Patient and Public Involvement (PPI) committee. With input from the Imperial
College Patient Experience Research Centre (PERC), the Oxford University Academic Health Science
Network (AHSN) and based on NIHR Involve guidance, a term of reference, confidentiality agreement
and a background assessment form to Ufonia to structure the aims of the PPI group. Membership of
this Ufonia PPI group is voluntary but requires members to be committed to attend meetings and to
respond to emails/correspondence. The initial term of membership is for the duration of the study
from early in the preparation phase (March 2021). Further details about PPI are included in the study
protocol (see section 9.4).
5.4 Protocol Contributors
The sponsor, Imperial College London, controls the final decision regarding any aspect of the study.
Dr Eduardo Normando: Dr Normando defined clinical methods for the study. He will oversee the
clinical investigation and lead the clinical delivery of the study at Imperial.
Dr Edward Meinert: Dr Meinert oversaw the drafting of the study protocol and coordinated
revisions. He will lead the mixed-methods evaluation and the preparation of study findings for
dissemination.
Version number 1.0 – 17 May 2021
10

Autonomous telephone follow-up after cataract surgery
Dr Kanmin Xue: Dr Xue defined clinical methods for the study. He will lead the clinical delivery of the
study at Oxford.
Dr Ernest Lim: Dr Lim contributed to drafting and revising the study protocol and facilitated patient
review and feedback.
Dr Aisling Higham: Dr Higham contributed to drafting and revising the study protocol.
Ms Madison Milne-Ives: Ms Milne-Ives contributed to drafting and revising the study protocol.
Dr Guy Mole: Dr Mole contributed to drafting and revising the study protocol. Dr Mole is the Medical
Director of Ufonia and assures compliance with company clinical protocols.
Dr Nick de Pennington: Dr Pennington conceived of the study topic, contributed to the study design
and is the NIHR grant holder on behalf of Ufonia in his role as Chief Executive Officer.
5.5 Keywords
Artificial Intelligence (MeSH), Natural Language Processing (MeSH), Telemedicine (MeSH), Cataract
(MeSH), Aftercare (MeSH), Speech Recognition Software (MeSH)
Version number 1.0 – 17 May 2021
11

Autonomous telephone follow-up after cataract surgery
6 STUDY FLOW CHART
Patients receiving usual care for cataract surgery typically progress through four stages. Participants
in this study will undergo two additional stages that are specific to the study (stages 4 and 6). There
are six stages that participants of the study will potentially go through. These stages, and their
approximate timeline, are visualised in the flow diagram below (Figure 1).
1.
Stage 1. The patient is referred by a clinician for cataract surgery.
2.
Stage 2. The patient’s suitability for surgery is determined at a pre-assessment appointment
or a one-stop cataract clinic. Patients who meet the eligibility criteria for the study at this
stage will be provided with a participant information sheet (see 13.1 Appendix 1) and, if they
are willing to participate, informed consent will be collected (see 13.2 Appendix 2).
3.
Stage 3. The patient undergoes their scheduled cataract surgery. If the surgery was
uncomplicated, a patient may also consent to participate in the study in the recovery lounge.
4.
Stage 4. If the patient is consented to participate in the study, they will receive a follow-up
call from DORA (supervised by a clinician). The call script used by DORA will be reviewed by
supervising PIs in advance in order to reduce incidence of non-compliance through
misunderstanding.
5.
Stage 5. If the patient had the surgery at Imperial, regardless of whether or not they are
participating in the study, they will have a scheduled follow-up with a clinician.
6.
Stage 6. If the patient consented to participate in the study, they may be randomly selected
to participate in a semi-structured interview, to provide more in-depth feedback about their
experience using DORA.
Figure 1. Participant flow diagram
The patient flow described above reflects the delivery stage of the study (see Figure 2). Before
delivery begins, the first part of the study will be dedicated to obtaining the necessary ethical
approvals, NHS contracts, and technical capabilities to deliver the intervention. These steps are
captured in the project management, technical, and evaluation work packages (see WPs 1, 2, and 4
in Figure 2). Participant recruitment will begin approximately one month before data collection, with
Imperial starting recruitment and delivery two months before Oxford (see WPs 3a and 3b in Figure
2). Prior to the main intervention period, a 4-week intervention refinement period will be conducted
to identify any issues and make adjustments to the intervention execution. The final six months of
the study will see the completion of data collection and will focus on conducting the evaluation, and
reporting results for publication (see WPs 1, 4, and 5 in Figure 2).
Version number 1.0 – 17 May 2021
12

Autonomous telephone follow-up after cataract surgery
Figure 2. High-level study gantt chart
Version number 1.0 – 17 May 2021
13

Autonomous telephone follow-up after cataract surgery
7 INTRODUCTION
7.1 Background
The UK’s ageing population is causing an increased demand for healthcare services that is exceeding
clinical capacity [1]. Demand has been further exacerbated by the Covid-19 pandemic, as the
widespread cancellation of elective care has created a large backlog of clinical work [2]. However, a
large proportion of this clinical work is taken up by highly repetitive and low skill tasks. Therefore,
there is a need to improve efficiency in the delivery of care, and to collect data that can be analysed
to support routine improvement and optimisation, through the automation of routine clinical
interactions.
One area of care where improved efficiency is urgently needed is cataract surgery. Cataract surgery is
already the most common operation in the NHS, with approximately 450,000 procedures conducted
per year [3]. Covid-19 has caused record delays in receiving planned surgeries [4], and average wait
times for cataract surgery were already approximately 2.5 months [5]. The ageing population will also
have a significant impact on the number of patients with cataracts, which is expected to double
between now and 2050 [6]. The cataract pathway is also an ideal case for optimisation because there
is little variability and high levels of patient safety; the most significant postoperative complication
(endophthalmitis) occurs in fewer than 1 in 1,000 cases [7]. To address this clinical need, an
innovative solution is required.
7.2 Rationale
Like most operations, cataract surgery requires a post-operative check to monitor for complications
and assess success. This has historically been performed with a face-to-face visit; prior to the
Covid-19 pandemic, this was standard procedure for 72% of NHS Trusts [8]. However, this
post-operative system has a high operational demand and is not always necessary; a recent
ophthalmology Getting It Right First Time (GIRFT) report stated that a hospital review of cataract
surgery patients is not required, if alternative follow-up arrangements are in place [8]. In the current
context of the Covid-19 pandemic, face-to-face visits also pose a high risk for virus transmission due
to the proximity of patient and clinician. This is reflected in the latest guidance from GIRFT and the
Royal College of Ophthalmologists on restarting cataract surgery during the pandemic which
recommends that patients should not be followed up in hospital after routine uncomplicated
cataract surgery and should instead receive a telephone consultation or review with their local
optometrist [9].
The solution developed to improve clinical efficiency for cataract surgery follow-up is a natural
language, voice telemedicine conversation delivered to patients via telephone call (DORA, Ufonia).
Since Ufonia’s conception in 2016, several grants and partnerships have supported DORA’s
development and pilot testing at Buckinghamshire Healthcare NHS Trust (BHT), Department of
Ophthalmology. For the patient, this is intended to be no different than a regular telemedicine
consultation with a doctor or nurse; it does not require the download of an app, the provision of a
device, or any training. This is important because the populations that consume the majority of
healthcare services (elderly and socio-economically disadvantaged) tend to be relatively more
digitally disenfranchised [10].
The solution potentially has several benefits for patients and healthcare staff, which will be evaluated
in the study [11]. For patients, DORA will provide a reliable, consistent safety-net after surgery.
Patients will be able to ask questions about their recovery from the convenience of home. Reducing
the number of in-person follow-ups will also benefit clinicians by enabling them to spend their time
on higher-value clinical activities, making patients more likely to receive timely care for their initial
Version number 1.0 – 17 May 2021
14

Autonomous telephone follow-up after cataract surgery
cataract surgery or for other conditions. In addition, telephone follow-up reduces the risk of Covid-19
transmission and frees up hospital space to help meet the increasing patient demand. Whilst this
study focuses on cataract surgery follow-up, the underlying platform will be applicable to a wide
range of routine clinical tasks. The need for an effective automated tool is especially great as a ‘new
normal’ of widespread remote clinical care is established in the wake of Covid-19.
7.3 Theoretical Frameworks
Two complementary frameworks were used to support the conception and development of the
study plan by ensuring that a holistic set of variables were included in the evaluation [12,13]. Several
quantitative analyses will be conducted to achieve the study’s objectives; including comparing the
agreement of DORA’s and the supervising clinician’s decisions and using validated questionnaires to
evaluate DORA’s usability [14,15]. Qualitative analysis will be completed by semi-structured
interviews with a subset of users structured according to the Theoretical Framework of Acceptability
[16]. The use of both quantitative and qualitative methods to evaluate usability will enable the data
to be triangulated.
These three frameworks and two scales will form the theoretical basis for the evaluation of DORA:
1.
Assessment of Health Information Technology Interventions in Evidence-Based Medicine
Evaluation Framework [12]. This framework was chosen to help structure the study
evaluation because it includes a holistic set of outcomes, including safety, privacy,
appropriateness, and cost-effectiveness.
2.
Long-term adoption and suitability to further trials will be evaluated using the Non-Adoption,
Abandonment and Challenges to the Scale-up, Spread and Suitability (NASSS) framework
[13]. This framework is important to include because interventions can only add value if they
are successfully adopted and used. This framework emphasises the consideration of the
multiple levels - individuals and systems - that influence adoption and non-adoption.
3.
The Theoretical Framework of Acceptability (TFA) will be used to structure the
semi-structured interview guide [16]. This framework was chosen because it was developed
specifically for health interventions, captures a multi-faceted view of acceptability (with
seven components), and emphasises the importance of considering different time points
when evaluating acceptability [16].
4.
The System Usability Scale (SUS) [14]. The SUS is one of the most widely-adopted measures
of usability and is still recommended for use after several decades of use [17].
5.
The Telehealth Usability Questionnaire (TUQ) [15]. In addition to the SUS, the TUQ will be
used in the study because it was developed recently and focuses specifically on modern
telehealth interactions. It has been demonstrated to be a reliable measure of the quality of
telehealth technologies.
7.4 Research Question
DORA was developed to provide an alternative for conducting post-cataract surgery follow up
assessments to save nurses’ and other clinicians’ time. The key outcome of this study is evaluating
DORA’s ability to accurately identify which patients have postoperative complications which would
justify specialist care, and assessing DORA’s accuracy compared to an expert clinician. To assess this
critical capability, the research question that this study will investigate is: Can DORA identify which
patients need clinical follow up after a cataract operation?
Version number 1.0 – 17 May 2021
15

Autonomous telephone follow-up after cataract surgery
7.5 Aims and Objectives
The purpose of this clinical investigation is to evaluate the safety, usability, acceptability, efficacy, and
cost-effectiveness of Ufonia’s autonomous cataract follow-up call system (DORA) at detecting
patients that require further assessment. The aim is to establish preliminary safety evidence by
examining the level of agreement between DORA and the supervising clinician on symptom and
overall management plan decisions. Therefore, the focus will be on patients undergoing routine
surgery. In order to achieve these aims, there are several objectives (see Figure 3):
1.
To evaluate the level of agreement between DORA and an expert human clinician and
identify any factors that affect that level of agreement;
2.
To establish baseline rates of sensitivity and specificity for DORA’s detection of patients that
have postoperative complications;
3.
To evaluate DORA’s feasibility; i.e. whether DORA can engage with patients enough to
produce sufficient data to perform an accurate assessment;
4.
To evaluate the usability of an autonomous call, in comparison with existing standards of
care, explore patient perceptions about whether DORA is acceptable, appropriate,
satisfactory and evaluate cost-effectiveness of autonomous calls in comparison with existing
standards of care.
Figure 3. Logic diagram of the study
Version number 1.0 – 17 May 2021
16

Autonomous telephone follow-up after cataract surgery
8 METHODS
8.1 Trial Design
Using an implementation science construct, the study will use mixed methods. Quantitative methods
will be examining the inter-rater reliability between DORA and the supervising clinician for
assessments of symptoms and management plan decisions. Usability will also be assessed using
validated, quantitative questionnaires. Qualitative methods using semi-structured interviewing will
also assess DORA’s usability, acceptability, satisfactoriness, and appropriateness to collect more
in-depth feedback and explore DORA’s strengths and weaknesses. The development of the protocol
incorporated the guidelines from the SPIRIT-AI checklists (see 13.3 Appendix 3) [18]. Public Health
England’s guidance summaries for evaluation in health and well-being were also consulted when
developing the protocol [19].
The study will last up to three years: the first year will be focused on evaluation and intervention
refinement, including a 4-week trial of the intervention execution to identify any adjustments
needed; this will be followed by the study implementation and follow-up; and the last several
months will be dedicated to post-evaluation analysis and write-up.
8.1.1 Timeline
The study may last up to three years with DORA calls taking place over a six- to twelve-month period
at both clinical sites to enable sufficient flexibility to meet the recruitment targets. Recruitment will
begin approximately one month before the start of call delivery at both sites, but the start dates for
the two sites may be staggered. Call delivery will last for approximately a year after recruitment. For
both sites, recruitment will stop approximately 6 weeks prior to the end of the call delivery period.
This timeline assumes return to elective surgery; any changes due to COVID-19 will require
amendment. The overall study timeline is summarised in Figure 4. 
Figure 4. Overall call delivery timeline
8.2 Study Context
8.2.1 Participants
Patients who are listed for routine cataract surgery at either Imperial or Oxford NHS Trusts are
eligible for this study.
8.2.2 Setting
This is a multi-centre study incorporating two NHS academic teaching hospitals - Imperial College
Healthcare NHS Trust and Oxford University Hospitals NHS trust.
The management of the intervention (DORA) is accessible through any web-browser and
investigators will be provided with secure log in.
Version number 1.0 – 17 May 2021
17

Autonomous telephone follow-up after cataract surgery
8.2.3 Intervention(s)
DORA uses a variety of AI technologies to deliver the patient follow-up call, including: speech
transcription, natural language understanding, a machine-learning conversation model to enable
contextual conversations, and speech generation. Together, these technologies cover the input,
processing and analysis, and output needed to maintain a natural conversation. DORA is configured
to deliver calls through a telephone connection as a real-time, stand-alone system: the operator
inputs individual patient details to initiate the call and completes a summary in the electronic health
record (EHR) afterwards. The call that patients will receive from DORA will include several
conversational elements:
●
Greeting and introduction
●
Identification of patient
●
Cataract follow-up questions
●
Patient’s queries
●
(Decision)
●
Questions about acceptability
●
Closure of call
The entire conversation will be supervised by a clinician. This clinician will be able to interrupt the
call at any point if the system fails, the patient struggles to interact with it, or DORA does not collect
sufficient information from the patient. The call will proceed in the following stages (visualised in
Figure 5):
1.
Stage 1. Call initiation: the supervising clinician will confirm the suitability of the patient,
input their details, and initiate the call.
2.
Stage 2. Symptom assessment: Once the call has begun, DORA will verify the identity of the
patient and proceed through the cataract follow-up questions. As the patient answers the
questions, both DORA and the supervising clinician will record their independent assessment
about the clinical significance of the symptoms elicited.
3.
Stage 3. Management plan decision: Once all of the follow-up questions have been asked,
DORA and the supervising clinician will independently make and record their decisions about
the management plan for the patient. DORA will then inform the patient of the clinician’s
decision about their management plan (i.e. whether specialist review is required or not).
●
Stage 3a: Optionally, if required, the supervising clinician can ask additional clarifying
questions.
Figure 5. Flow Diagram of DORA’s call with patient
Version number 1.0 – 17 May 2021
18

Autonomous telephone follow-up after cataract surgery
8.2.4 Comparator(s)
Imperial sees all of their post-operative cataract patients at a face-to-face out-patient appointment
with an ophthalmologist or specialist nurse. This is conducted between three to four weeks following
the procedure. Therefore, patients at Imperial will receive their call from DORA within 3 days before
their scheduled appointment (day 25-27 post-surgery). This will enable the results of the DORA call
to be compared with the face-to-face appointment at Imperial to assess whether any complications
were correctly identified and if the management plan remains the same.
For patients at both clinical sites, health records will be examined 90 days after the call to determine
if the patient presented to an eye clinic with any complications from surgery. This comparison of the
follow-up with actual complication rate will enable an assessment of DORA’s safety (see Table 6).
8.2.5 Outcomes
8.2.5.1 Primary Outcomes
Agreement
The primary outcome of the study is to establish preliminary safety evidence by evaluating the level
of agreement between DORA and the expert human clinician on symptom and overall management
plan decisions.
To evaluate agreement, DORA’s ability to classify five key symptoms from the conversation will be
compared with a supervising clinician’s classification of the same five symptoms: redness, pain,
reduced vision, flashing lights, and floaters (see Table 5). For this study, the supervising clinician’s
decision - made based on the information from the call with DORA’s decision masked - is considered
the gold-standard for evaluation. Both DORA and the supervising clinician (masked to each other)
will independently indicate for each symptom, whether the symptom is:
1.
Absent (e.g. no pain)
2.
Present but not clinically significant (e.g. mild gritty sensation)
3.
Present and clinically significant (e.g. deep and persistent pain)
4.
Insufficient information for classification
The agreement between DORA and the clinician on the patient’s management plan will be evaluated
in the same way as the symptoms. DORA and the clinician will independently select one of the
following options for the patient’s management plan:
1.
Passes check (patient continues to next step of their care as planned, e.g., Discharge from
hospital care, add to waiting list for second eye surgery, or continue routine follow-up in
clinic for another ophthalmic condition)
2.
Needs clinical assessment relating to the cataract surgery
Ability to gather sufficient information
To evaluate DORA’s ability to gather enough information for a clinician to decide a patient’s
management plan and the necessity for the clinician to ask clarifying questions will be assessed. If
clarifying points are necessary for the clinician to make a decision, the clinician will ask the necessary
questions, and then document updated assessments for symptoms and overall management (see
Figure 5). Either 12 or 18 data points will be collected from each call depending on if clarifying
questions are asked.
●
Every Conversation
Version number 1.0 – 17 May 2021
19

Autonomous telephone follow-up after cataract surgery
o
5 symptom decisions from DORA + 1 clinical decision from DORA
o
5 symptom decisions from Clinician + 1 management plan from Clinician
●
If clarifying questions asked 
o
Post-clarification 5 symptom decisions from Clinician + 1 management plan from
Clinician
This will allow us to test the hypothesis that clarifying questions affect the clinical management plans
(see Table 5).
Table 5. Primary Outcome Measurement
Primary Outcome
Outcome Measure
Agreement
●
Whether or not the clinician had to interrupt the call to ask clarifying
questions
●
Inter-rater reliability: the degree of agreement between DORA and the
clinician on their assessments of the individual symptoms and the
management plan
8.2.5.2 Secondary Outcomes
There will be several secondary outcome measures to assess the clinical safety, feasibility, usability,
and acceptability of the system (see Table 6). 
Clinical safety
To examine safety, two outcomes will be captured. Firstly, patients at both sites will have a
retrospective case review of both general and ophthalmic electronic health records at 90 days after
their initial operation. Complications will be identified from patient’s medical records related to any
unplanned eye-related clinical episode.
Secondly, all cataract surgery patients at Imperial were historically followed up face-to-face between
3-4 weeks post-surgery in a dedicated post-op clinic. Any complications requiring a change in patient
management at slit lamp review or from attending eye casualty services (as the Oxford site does not
normally perform face-to-face follow-ups) will be compared with the decisions made by DORA and
the clinician. If Imperial changes their method of follow-up due to the COVID-19, assessment for
complications will be done by checking hospital medical records as is done at the Oxford site.
Feasibility
Feasibility will be assessed by examining the proportion of autonomous calls that were completed
without needing any intervention from the supervising clinician. All interruptions or clarifying
questions of the supervised call will be logged; the supervising clinician will be asked to log the
reason for asking clarifying questions. If clarifying questions were asked because the clinician felt that
there was insufficient information for either individual symptoms or to make a management plan,
the clinician’s management plan before and after clarifying questions will be captured to see if
clarifying questions changed the management plan.  
Usability
Usability will be assessed using questions presented to patients following calls delivered by online
and paper surveys distributed to participants. These questions will be drawn from the System
Version number 1.0 – 17 May 2021
20

Autonomous telephone follow-up after cataract surgery
Usability Scale (SUS) and Telehealth Usability Questionnaire (TUQ; see 13.4 Appendix 4 for a sample
survey) [14,15].
Acceptability, satisfaction, and appropriateness
DORA’s acceptability, satisfactoriness, and appropriateness will be assessed during qualitative
interviews with a subset of the patients. These semi-structured interviews will provide an
opportunity for patients to provide more in-depth feedback about what they liked about DORA and
any barriers to use that they experienced during the call (see 13.5 Appendix 5 for a sample topic
guide). The supervising clinician will also be asked to provide feedback about their level of
satisfaction with the system and any safety concerns they might have. 
Patients’ willingness to recommend DORA to others will also be assessed at the end of the call [20].
DORA will calculate a net promoter score (NPS) by asking patients “on a scale of 1 to 10, how likely
would you be to recommend this automated service to a friend or colleague.” DORA will follow up by
asking patients to “tell me why you gave that score.”
Cost-effectiveness
Cost-effectiveness will be examined by collecting and analysing health economic data before and
during the study. The costs of implementing DORA will be compared with historical data about the
costs of the usual standard of care.
Table 6. Secondary Outcome Measurement
Secondary Outcome
Outcome Measure
Clinical Safety
Complications identified from patients’ electronic health records up to 90 days
following cataract surgery
Congruence between complications identified and management planned in
DORA call and face-to-face follow up (Imperial)
Comparison to data from patients attending eye casualty (Oxford)
Feasibility
Proportion of autonomous calls that were completed without needing any
intervention from the supervising clinician
Clinician-reported reasons for asking clarifying questions
Usability
System Usability Scale (SUS) [14,17]
Telehealth Usability Questionnaire (TUQ) [15]
Qualitative feedback from semi-structured interviews
Acceptability
Qualitative feedback from semi-structured interviews
Satisfaction
Qualitative feedback from semi-structured interviews, Net Promoter Score (NPS)
Appropriateness
Qualitative feedback from semi-structured interviews
Cost-effectiveness
Comparison of the costs of implementing DORA and the costs of the usual
standard of care
Identify improvements
Synthesis of all of the outcome measures previously described
Version number 1.0 – 17 May 2021
21

Autonomous telephone follow-up after cataract surgery
8.3 Sample and Recruitment
8.3.1 Eligibility Criteria
8.3.1.1 Inclusion Criteria
●
Willing and able to provide informed consent;
●
Aged 18 years or older;
●
On the waiting list for routine cataract surgery. Cataract surgery as part of a combined
procedure with other ocular surgery will not be included;
●
No history or presence of significant ocular comorbidities that would be expected to alter the
risks of cataract surgery or normal post-operative follow-up schedule. Note that significant
ocular comorbidities do not include stable, chronic, or inactive ocular conditions such as
amblyopia, drop-controlled stable glaucoma or ocular hypertension, previous squint surgery,
inactive macular pathology, previous refractive surgery, or previous vitreoretinal surgery with
stable retina.
8.3.1.2 Exclusion Criteria
●
Individuals with any condition that could preclude the ability to comply with the study or
follow-up procedures;
●
Presence of ocular or systemic uncontrolled disease (unless deemed not clinically significant
by the Investigator and Sponsor);
●
Involved in current research related to this technology or been involved in related research
to this technology prior to recruitment;
●
Cognitive difficulties, hearing impairment or non-English speakers;
●
History of current or severe, unstable or uncontrolled systemic disease (unless deemed not
clinically significant by the Investigator and Sponsor).
8.3.1.3 Withdrawing
●
Participants will be withdrawn from the study if complications were encountered during their
cataract surgery and documented in their medical records. Complications of cataract surgery
may include, but are not limited to, capsular tear/rupture, anterior vitrectomy, zonular
dehiscence, sulcus intraocular lens implant, dropped lens fragments, other concurrent
vitreoretinal procedure;
●
Patients who have undergone unplanned clinical review (e.g. eye casualty attendance or any
planned appointment as a result of deviation from normal intraoperative course, e.g. suture)
or additional ocular procedure prior to the follow-up call will be withdrawn from the study.
8.3.2 Sampling
8.3.2.1 Sample Size
An audit of historical data was conducted for both Imperial and Oxford sites to provide evidence for
an estimate of the study’s sample size (see Appendices 7 and 8, respectively). All types of
pre-assessment appointments throughout October 2020 were audited. October 2020 was used to
project capacity in May-July 2021 on the assumption that recovery from this wave of the Covid-19
pandemic would follow a similar trajectory to recovery from the first wave. At Imperial, three
consultant-led one-stop clinics are expected to be in place in July to mitigate surgery backlog.
Version number 1.0 – 17 May 2021
22

Autonomous telephone follow-up after cataract surgery
Sample size calculations were based on estimates deriving from this audit. As outlined in Table 7,
Imperial currently sees about 7 patients per week in their one-stop clinic and pre-assesses about
another 64 per week, almost 90% of whom are listed for surgery. If 80% of the total patients seen are
estimated to be eligible for the trial, approximately 230 patients could be recruited from this clinic
over the course of the study period. At Oxford, about 60 new patients are seen for pre-assessment
per week, with approximately another 20 phone assessments for second eye are also done, with 90%
being listed for surgery. With an estimated 70% of patients consenting to participate, and an attrition
rate of 15%, 591 patients are expected to be recruited over the course of 26 weeks. An a priori power
calculation was conducted to determine the minimum sample size necessary to achieve a power
greater than 0.8 for a 0.05 level of significance [21]. On the basis of a total population of 1014, a 95%
confidence level, and a 3% margin of error, a sample size of 520 is required for statistical significance.
Table 7. Indicative recruitment targets with assumptions of dropout at each stage. Screening capacity
is set at 2 days due to staffing constraints.
Imperial
Imperial
Oxford
Oxford
One-stop
(Projected 3
clinics per week
from 1/2021)
Pre-assessment
(excluding
one-stop)
Pre-assessment
Phone
pre-assessment
(2nd eye)
Current Numbers Assessed
Mean Patients
assessed per clinic
10
8
12
4
Clinics per week
3
8
5
5
Mean Patients
Assessed per week
30
64
60
20
Eligibility Assumptions
% proceeding to
surgery
80%
90%
90%
90%
% routine cases
80%
80%
80%
80%
% eligible for trial
80%
80%
90%
90%
% ineligible due to
complicated surgery
5%
5%
5%
5%
Screening
Compound daily
eligibility rate
49%
55%
62%
62%
Version number 1.0 – 17 May 2021
23

Autonomous telephone follow-up after cataract surgery
Number of days
screening
2
2
2
2
Patients screened
per week
10
9
15
5
Estimated recruitment
% consenting to
trial
70%
70%
70%
70%
Estimated average
recruited per week
7
6
10
3
Duration of trial
(weeks)
26
26
26
26
Estimated % of
patients dropping
out
15%
15%
15%
15%
Estimated
recruitment over
trial
150
135
229
76
Site Total
286
305
Total
591
8.3.2.2 Sampling Technique
All patients who are willing and eligible will be included in the sample receiving the DORA call. Of this
sample, a subset will be selected to participate in a semi-structured interview to collect qualitative
data about patients’ perceptions of DORA (see 13.5 Appendix 5).
To select this subset, a stratified random sampling technique will be used. Patients will be divided
based on demographic characteristics (gender, age, socio-economic status, and ethnicity) and clinical
location (Oxford or Imperial). From these subgroups, patients will be randomly selected so that the
sample interviewed is representative of the UK population. For example, we will randomly select to
ensure the subset of patients being interviewed is split equally by gender and clinical location.
8.3.3 Recruitment
8.3.3.1 Participant Identification
Recruitment will take place in the pre-assessment sites and post-operative discharge lounges at the
Imperial College Healthcare and Oxford University Hospitals NHS Trusts that conduct cataract
surgeries. Participants will be made aware of the study via leaflets and discussion with healthcare
providers during their pre-assessment and, if they are eligible, will be invited to participate. Patients
Version number 1.0 – 17 May 2021
24

Autonomous telephone follow-up after cataract surgery
in the discharge lounges who have undergone uncomplicated cataract surgery will also be invited to
participate in the study. Screening will continue at both sites until the target enrolment is achieved.
8.3.3.2 Consent
Study information will be shared with cataract surgery patients at their initial visit (and via post or
telephone call for patients who are delayed by Covid-19 or cannot visit in person). Informed consent
will be obtained in written form or verbally at the time of pre-assessment, surgical listing (in-person
or virtually depending on the current Covid-19 guidelines), or in the discharge lounge following
surgery (see 13.2 Appendix 2). Patients who have consented to participate will be given further
information while they are in the discharge lounge to remind them about the call from DORA. This
work will be performed by a dedicated research nurse at each site.
8.3.4 Adherence
The DORA calls will be recorded and the study team will monitor the use of the technology and
assure the system is being used within designed parameters during the study.
8.4 Data Collection
8.4.1 Randomisation
8.4.1.1 Sequence Generation
Demographic data will be collected from all patients in the surveys that will be distributed to assess
DORA’s usability after patients have completed the call (see 13.4 Appendix 4). A computer random
number generator will then be used to randomly select patients for semi-structured interviews from
within each demographic subset collected at post-call surveys.
8.4.1.2 Allocation Concealment Mechanism
A masked computer algorithm will be used to generate the subset of patients for interview and
deliver the outcome to researchers at the time that they are assigning patients to be invited to
interview.
8.4.1.3 Implementation
A computer will generate the random allocation sequence and assign participants to interventions.
Healthcare professionals at the pre-assessment clinics will enrol participants in the study.
8.4.2 Blinding
No blinding of participants will be done in the study, as all eligible patients will receive the same
intervention. The clinicians listening to the call will be blinded to DORA’s decisions when they are
making their own assessments.
8.4.3 Methods of Data Collection
8.4.3.1 Quantitative data collection
The key data to be collected are DORA and the clinician’s decisions. This will include data on
symptoms as well as an overall assessment. Both DORA and the clinician will separately record their
decisions about symptoms, clinical assessment, and management plan. DORA will record the
decisions in the DORA database, which will be accessed by the study team during the evaluation to
compare against the clinician decisions. Usability of the system will be assessed using the SUS and
Version number 1.0 – 17 May 2021
25

Autonomous telephone follow-up after cataract surgery
TUQ questionnaires (see 13.4 Appendix 4). These will be delivered to all participants shortly after
they have had the call with DORA via an online or paper survey (see Figure 6).
8.4.3.2 Qualitative data collection
A subset of participants will also be invited to an interview, so that more in-depth, qualitative
feedback can be collected about the user experience with DORA (see Figure 6). For the interviews, a
semi-structured interview approach will be followed (see 13.5 Appendix 5 for a sample topic guide).
If patients consent, the interviews will be audio recorded and professionally transcribed. If not, the
interviewer will take notes, and provide the notes to the patient at the end of the interview for
verification.
Figure 6. Patient data collection flow diagram
8.5 Data Analysis
8.5.1 Statistical Methods for Primary Outcome
The primary analysis will be the calculation of a kappa statistic of inter-observer (DORA & clinician)
reliability of the decision made. Additionally, the outcome of the assessment will be compared with
the ‘real’ complication rate determined by hospital medical records. This will be established by
identification of any hospital presentation within 90-days after the last call. Given the specialist
nature of ophthalmology services it can be assumed that patients will present through the eye
casualty or clinic services offered by each site. Patients who present to the hospital eye service will
be analysed on a case-by-case basis to ensure that complications that were missed by DORA’s
original assessment and complications that developed later can be differentiated. This analysis will
provide baseline sensitivity and specificity data for use in preparing subsequent evaluations of
efficacy.
8.5.2 Other Data Analyses
The usability and acceptability questions delivered to patients via online or paper surveys after the
completion of the call will be analysed quantitatively based on the scales’ scoring criteria - for
instance, a score above 80 on the System Usability Scale is generally considered to indicate an above
average user experience [17]. The semi-structured interview recordings will be transcribed and
assessed using thematic analysis.
8.6 Limitations
There are a couple limitations with the protocol. First, the number of issues identified during routine
follow-up at the two sites cannot be directly compared. Unlike Imperial, Oxford does not proactively
review patients following uncomplicated cataract surgery, instead relying on patients to self-present
to the eye casualty service. This introduces a potential risk for missed complications if DORA decides
Version number 1.0 – 17 May 2021
26

Autonomous telephone follow-up after cataract surgery
that a clinical review is not needed. However, this risk is minimised by the safety net established by
the expert clinician oversight of the call and is a part of the current standard of care at the Oxford
NHS Trust. The limitation is also mitigated by the examination of patient records after 90 days to
identify any complications that might have been missed.
The second limitation is related to generalisability. The evaluation will only include patients who had
uncomplicated cataract surgeries. This focus on routine care is important for establishing the
feasibility of DORA, but means that DORA’s effectiveness for dealing with post-operative follow-up of
complex patient groups cannot be assessed in this study. The National Ophthalmology Database
(NOD) audit report however shows that the most common complication of posterior capsule rupture
occurs in only 1-2% of patients [3].
Technical limitations of the system also affect generalisability. At present, patients with cognitive
difficulties, hearing impairment or non-English speakers are not able to use DORA and will not be
included in the study. However, this reflects the same limitations as human telemedicine services.
Finally, potential bias is introduced to the results by the use of qualitative interviews because of the
influence of the interviewer on how participants respond (e.g. social desirability bias). This potential
bias will be assessed by examining the balance of positive and negative feedback and by comparing
the qualitative and quantitative data about DORA’s usability.
8.7 Generalisability
Limitations to the generalisability of the study results were noted in the previous section. However, it
is expected that the results will be generalisable to the majority of cataract surgery patients. The vast
majority of cataract surgeries are uncomplicated; only approximately 2.5% of cataract surgeries have
intraoperative complications (which would have resulted in exclusion from this study) [3]. Therefore,
if the results of this study are positive, they will demonstrate DORA’s potential to manage almost all
cataract surgery follow-up assessments.
Version number 1.0 – 17 May 2021
27

Autonomous telephone follow-up after cataract surgery
9 ETHICAL AND REGULATORY CONSIDERATIONS
9.1 Assessment and Management of Risk
Risks will be monitored on a weekly basis via the project management process and mitigated by
Risks/Actions/Issues/Decisions log.
The Covid-19 pandemic has raised potential issues for recruitment for the study. To address this risk,
an ethics submission will enable remote informed consent if restrictions prohibit face-to-face
contact.
Risks regarding the system include it failing to detect patients who require urgent clinical review - this
will be mitigated by having a clinician supervise the call in real-time, able to intervene if needed. The
risk of lack of trust in the system has been mitigated by an extensive co-creation process for DORA
that focused on user-centred design.
The interviews are scheduled to take place for between 20 to 30 minutes in order to mitigate time
risk to participants. The nature of interview questions avoid areas of cultural or psychological
sensitivity and are purely focused on impact of the intervention. Since Brexit, the General Data
Protection Regulation (GDPR) no longer applies to the UK; however, it has been incorporated in UK
domestic law [22]. To control any potential perceived issues in this area, participant confidentiality
will be protected using data protection procedures that are compliant with the Data Protection Act
2018 (DPA 2018) [23]. For more details about data protection and patient confidentiality plans for
this study, see section 9.7.
9.2 Research Ethics Approval
Ethical approval is being sought from the Health Research Authority and the relevant Research Ethics
Committee with this submission [24]. As the IRAS system brings together these assessments, only
one application is needed [24]. The protocol will therefore receive medical device review from the
associated REC.
HRA approval is the ethical approval route used for studies with NHS patients at both Imperial
College London and the University of Oxford [25,26]; additional independent ethical approval will be
sought from the University of Plymouth. At the University of Plymouth, an application for ethical
approval will be submitted through the Plymouth Ethics Online System (PEOS) to the Health Faculty
Research Ethics and Integrity Committees (FREIC) [27].
Imperial College London, as the study sponsor, will ensure that the study has received ethics
approval from a research ethics committee (REC) and has received Health Research Authority (HRA)
approval.
9.3 Peer Review
The study will undergo peer review at Imperial College London, where the Peer Review Office (PRO)
commissions proportionate, independent peer reviews and confirms that the reviews have been
done correctly [28]. The University of Oxford does not require a separate review outside what is
completed by the HRA approval process [29]. An independent peer review will be conducted by the
University of Plymouth.
Version number 1.0 – 17 May 2021
28

Autonomous telephone follow-up after cataract surgery
9.4 Patient and Public Involvement
The Oxford University AHSN and Imperial College London have robust PPI support infrastructures to
ensure we are able to have the necessary support in place to establish our PPI framework for the
research study. Partnerships with both of these institutions will ensure we align our PPI practices as
we move through the research cycle.
At Oxford, PPI will be coordinated by Dr. Sian Rees who is the director of Patient and Public
Involvement at the Oxford AHSN to better understand how to incorporate best-practices into our
co-production processes.
At Imperial, the Imperial College Ophthalmology Research Group (ICORG) has close ties to the NIHR
Biomedical Research Centre (BRC) Patient Experience Research Centre (PERC). Serge Miodragovic,
the clinical research manager at ICORG will advise on the use of Imperial’s PPI framework and the
PERC will be used to structure the public engagement strategy. Furthermore, the Western Eye
Hospital already has an established PPI reference group to advise and feedback on the clinical
aspects of the service, these are a group of expert patients who have an interest in ophthalmic
conditions and a source of potential representatives to join the study monitoring and PPI groups.
9.4.1 Involving a lay representative on the study monitoring group
A lay representative will be appointed to the study monitoring group, additional the patient
representative will be invited to key strategic and public meetings. The lay representative will be
involved in all aspects of the study, specifically, these include, but are not restricted to review of each
milestone and acceptance of deliverables - seeking their views around acceptability of patient-facing
materials, study procedures. and plans for translation to clinical practice.
9.4.2 Involving a dedicated PPI group
The broad aims of the PPI group are to support the governance of patient and public involvement
and engagement approaches and activities within this study and ensure the public voice is present
through all stages of the research cycle. Membership of the PPI group will be open to those both
with a direct interest in telemedicine or eye health, but also any lay member with an interest in our
broader aims. We will include patients and lay members both with and without previous experience
of PPI to make sure we have a variation of skills and expertise.
With input from the Imperial College PERC, we have drafted, based on NIHR Involve guidance, a term
of reference, confidentiality agreement and a background assessment form to allow us to realise the
broad aims of our PPI group. Membership of this Ufonia PPI group is voluntary but requires members
to be committed to attend meetings and to respond to emails/correspondence. The initial term of
membership is for the duration of the study from early in the preparation phase (October 2020). We
aim to have a minimum of four formal members at the inception of the panel and each subsequent
meeting with representation from both London and Oxfordshire. We will seek to make participation
as diverse as possible by selecting members based on 1) demographics 2) levels of PPI experience
and 3) connection to telemedicine or eye health based on the results of our basic background form.
In particular, we are aware that we have to ensure Ufonia’s product is as accessible to as wide a
range of possible demographics as possible, and the first phases of setting up our PPI group will
involve a wide search using networks from both Oxford and Imperial AHSN but also through
INVOLVE’s “People in Research” platforms. Given that this is a cohort who are elderly with potential
forms of disability, we are aware that additional resources may be required to cover the costs of
translators or communication support, or equipment and training to enable participation in
telecommunication.
Version number 1.0 – 17 May 2021
29

Autonomous telephone follow-up after cataract surgery
The panel will aim to convene every four months for a 90 minute formal group discussion, given the
evolving COVID situation, we anticipate at least some of these group discussions to be conducted via
video-conferencing, with more informal discussions happening via email or phone-calls. As the study
spans two sites, we envision using teleconferencing to help enable maximal participation of our
participants and to lower the barriers to involvement. At both sites, a dedicated location for
teleconferencing will be designated to ensure we are not excluding participants less comfortable with
or less willing to use technology.
We anticipate that we will be having an ongoing dialogue with study participants both formally in the
afore-described evaluation phase, and informally via ongoing feedback. As such, we anticipate that
new members may be recruited to the PPI group if there are gaps in experience. The panel may
choose to invite other patients/members of the public involved; other university members and/or
representatives of voluntary or community organisations such as the Royal National Society of the
Blind (RNIB) on a one/off or long term basis. Observers, guests and presenters may also be invited on
a one/off basis.
The meetings will be co-chaired by a lay chairperson who is nominated by all attendees of the
meeting, and a professional chairperson will be a senior Ufonia team member who has received
formal experience in chairing PPI meetings. We have drafted a “role” specification with Imperial
PERC of a chairperson which may be a different individual between meetings depending on topics of
discussion.
Payments will be made in recognition of members’ time based on NIHR Involve guidance on payment
of fees and expenses for our members actively involved [30]. Travel expenses will be reimbursed in
accordance with this policy together with other expenses and travel costs. We have budgeted based
on suggested reimbursement schedules and have produced our estimates following use of the
Involvement Cost Calculator.
9.5 Protocol Compliance
Bi-monthly study governance meetings will be held, alternating between monitoring and clinical
reference group sessions. Monitoring meetings will include all study partners and NIHR/NHS
representatives, and patient representatives. They will review progress against the study plan and
budget, sign-off study deliverables and update the project’s risk register. Clinical reference group
meetings will include senior representatives from both clinical sites who will provide oversight of
progress, approval for go-live and ongoing monitoring of delivery. A group of external academics will
be appointed as a trial monitoring committee who will also perform audits to ensure compliance to
procedures.
Planned deviations or waivers to the protocol are not allowed under the UK regulations on Clinical
Trials and will not be used. Any accidental protocol deviations will be adequately documented on the
relevant forms and reported to the Chief Investigator and Sponsor immediately should they happen.
Immediate action will be taken towards any deviations to avoid serious breach. The Sponsor will be
notified immediately should there be a serious breach of the safety or physical or mental integrity of
the participants of the trial or the scientific value of the trial. The Sponsor will notify the licensing
authority in writing of any serious breach.
9.6 Consent
BERA guidelines have been followed for voluntary informed consent, use of methods, and university
policies in the event there are issues in delivery [31]. Prior to completing informed consent,
participants will be given information that fully describes the process of the study (including the
possibility of being randomly selected to be invited to participate in a semi-structured interview),
Version number 1.0 – 17 May 2021
30

Autonomous telephone follow-up after cataract surgery
including why their participation is necessary, how their data will be used, and who the results will be
reported to. As many patients are understandably concerned about how their data will be used, data
management will be explained in detail as part of the consent process. It will also make clear their
right to withdraw from the study at any time and have their data destroyed. Patients will also be
asked to separately consent for their data to be used to further train the conversational systems.
Declining to share this conversation data will not affect patients’ participation in the study or their
clinical care.
9.7 Data Protection and Patient Confidentiality
The Ufonia system stores patient identifiable data as part of the clinical record. Explicit consent is
obtained from patients to use that data in ongoing development. The solution is in compliance with
the UK Data Protection Act 2018 (DPA 2018) [23], as well as being built to meet specific NHS
regulations. All data movement shall be via encrypted data transfer and will not be stored outside
the EEA. The organisations involved in this study (including each Trusts’ Data Protection Officer and
Caldicott Guardian) will undertake a Data Protection Impact Assessment and, where needed, create
Information Sharing Agreements to ensure compliance with relevant data protection regulation. All
collaborators (Imperial, Oxford, Plymouth, Ufonia) and correlating hospitals (Oxford University
Hospitals, Imperial College Healthcare) will be joint controllers.
During the study implementation, each participant will be given a unique identifier. The primary key
between unique ID and participant will be securely held and given to the participant as a reference
ID. Data will be analysed using the unique IDs; the primary key is only maintained to enable
participants to withdraw their data from the study. If such a request is made before data aggregation
or publication, all of their corresponding data and files shall be destroyed.
Follow-up interview sessions shall be audio recorded by the Academic PI and his Research Associate.
The interview will be transcribed by a transcription service with only reference to the unique
identifier provided in the audio file for transcription (the audio file will be reviewed by the PI to
ensure no identifying information is in the audio recording; if any is provided it will be edited out);
the risk of identification of the interview sessions shall be very low due to this measure being taken.
The original audio recording will be destroyed following transcription.
Only the Academic PI, his Research Associate and clinical staff will have access to research data. The
transcription service will have access only to interview audio transcripts, following the controls
previously mentioned. Records of consent will be kept for three years after the publication of final
study results. Audio recordings will be immediately following transcription.
9.8 Stopping Guidelines
The study monitoring group will review the interim data for consideration of stopping the study early.
Given that the estimated sample size is close to the minimum required for adequate power, the
study will not be stopped early on the basis of efficacy or benefit. If the monitoring committee
identifies any risks that have not been foreseen and mitigated against, the study will be paused while
those risks are assessed.
9.9 Indemnity
Imperial College London holds negligent harm and non-negligent harm insurance policies which
apply to this study. The Imperial College Healthcare NHS Trust and the Oxford University Hospitals
Version number 1.0 – 17 May 2021
31

Autonomous telephone follow-up after cataract surgery
NHS Foundation Trust hold standard NHS Hospital Indemnity and insurance cover with NHS
Resolution for NHS Trusts in England, which apply to this study.
9.10 Access to DORA’s code
At the time of the close of the intervention refinement period, the DORA code base will be stored in
a source code repository and version number recorded. Changes to the platform required to
maintain safety, security or 3rd party software dependencies will be approved by the trial steering
committee and ratified by an external trial monitoring committee. Changes will be validated before
and after the trial by an independent software engineer on Ufonia’s source code repository.
9.11 Access to Final Study Data
Audio recordings, transcriptions, and meta-data about the calls will be securely stored in UK
data-centres with strict role-based access control. The transcription service will only have reference
to the unique IDs and the audio recording will be reviewed by the PI to remove any identifying
information before being shared. Patient identifiable data will not be sold to any other party and will
not be shared with any organisation unless they are a partner in the study and have an appropriate
information sharing agreement in place. Records of consent will be kept for three years after the
publication of final study results, but no other personally identifiable information will be stored
beyond the end of the study.
Version number 1.0 – 17 May 2021
32

Autonomous telephone follow-up after cataract surgery
10 DISSEMINATION POLICY
10.1 Dissemination Strategy
The project's two clinical sites are leading medical centres with strong reputations for high quality
patient care. They have large teams of clinicians, with extensive professional networks and training
programmes that involve junior staff from across the globe. These factors mean that their
participation in this project will ensure its outputs are communicated to a wide audience.
The project’s academic partners are world renowned institutions with a track record of delivery and
publication of high-quality research. Senior clinical academics will ensure the project is conducted
robustly and even at this early stage the outputs are presented and published in high impact
journals.
Alongside its clinical and academic partners, the project will leverage relationships with several
organisation to support dissemination:
●
NHS Clinical Entrepreneur Programme - three of the Ufonia team are members of this
programme. They will work with Tony Young and the programmes networks to ensure
visibility of the project to members of the central NHS leadership team. The network of other
entrepreneurs will also provide a potential source of employees to support wider
deployment.
●
Oxford Foundry - Ufonia is a current member of the competitively selected LEV8 Accelerator
programme. Through this initiative they are given access to senior mentors and advisors
(including technical directors and product managers from firms like Google and venture
capital investors) who can support the delivery and growth of the company to meet a pull for
wider adoption.
●
Oxford University Innovation - as the University of Oxford’s technology transfer office they
hold an investment stake in Ufonia and support its business development, as well as applying
specific expertise to contracting and intellectual property protection and exploitation.
●
DigitalHealth.London - Ufonia have engaged with the team and held discussions with their
navigators in order to ensure the Imperial work can be supported by their extensive network.
10.2 Authorship Eligibility Guidelines and Use of Professional Writers
The International Committee of Medical Journal Editors (ICMJE) guidelines will be used to determine
the study authors [32]. The ICMJE stipulates four criteria that people involved in the study must meet
to be considered authors for this paper:
1.
Having made a substantial contribution to the design or execution of the study;
2.
Having drafted or significantly contributed to the revision of the paper;
3.
Having final approval over submission for publication;
4.
Agreeing to be accountable for the published work.
No professional writers will be employed.
Version number 1.0 – 17 May 2021
33

Autonomous telephone follow-up after cataract surgery
11 DECLARATIONS
11.1 Funding
Ufonia has been supported by grant funding from Innovate UK and the Science and Technology
Facilities Council. The company has received business support from Oxford University Innovation and
the Oxford Foundry. The Department of Ophthalmology at Buckinghamshire Healthcare NHS Trust
has collaborated in the development of Ufonia’s system.
This study is supported by the NIHR AI in Health and Care Award (AI_AWARD01852).
11.2 Protocol Registration
The protocol will be registered through this IRAS submission 297548 and additionally on the
database clinicaltrials.gov (reference number TBD) [33].
11.3 Competing Interests
NdP and GM are all employees of Ufonia, a voice artificial intelligence company. Although these
authors were involved in the drafting and revision of the protocol, the final decision on the
evaluation design lies with the academic consortium led by Imperial College London.
NdP is employed part-time by Oxford University Hospitals NHS Trust and responsible for the
development of innovation activities at the Trust. His work for Ufonia has been approved by his line
manager (Chief Digital and Partnerships Officer) and declared in the Trust's register of interests. None
of the resources NdP is responsible for within the Trust are used to support the project.
GM is employed part-time by Oxford University Hospitals NHS Trust. This employment is as a clinician
so he has not been a decision maker regarding any resources from the Trust that relate to this study.
His work for Ufonia has been approved by his line manager at the Oxford University Hospitals NHS
Trust and declared in the Trust's register of interests.
Version number 1.0 – 17 May 2021
34

Autonomous telephone follow-up after cataract surgery
12 REFERENCES
1.
Oliver D, Foot C, Humphries R. Making our health and care systems fit for an ageing population
[Internet]. The King’s Fund; 2014. Available from:
https://www.kingsfund.org.uk/sites/default/files/field/field_publication_file/making-health-car
e-systems-fit-ageing-population-oliver-foot-humphries-mar14.pdf
2.
Illman J. Exclusive: NHS prepares to cancel elective ops in readiness for covid-19 surge
[Internet]. [cited 2021 Jan 20]. Available from:
https://www.hsj.co.uk/free-for-non-subscribers/exclusive-nhs-prepares-to-cancel-elective-ops-i
n-readiness-for-covid-19-surge/7027110.article
3.
Donachie PHJ, Sparrow JM. National Ophthalmology Database Audit: Year 5 Annual Report
[Internet]. The Royal College of Ophthalmologists; 2020 Aug. Available from:
https://www.nodaudit.org.uk/u/docs/20/hqsrgmurnv/NOD%20Audit%20Full%20Annual%20Rep
ort%202020.pdf
4.
Griffin S. Covid-19: Waiting times in England reach record highs. BMJ 2020 Sep 11;370:m3557.
PMID:32917660
5.
Dayan M, Ward D, Gardner T, Kelly E. How good is the NHS? [Internet]. The Health Foundation,
the Institute for Fiscal Studies, The King’s Fund, and the Nuffield Trust; 2018. Available from:
https://www.nuffieldtrust.org.uk/files/2018-06/the-nhs-at-70-how-good-is-the-nhs.pdf
6.
Cataract Tables [Internet]. National Eye Institute. [cited 2021 Jan 20]. Available from:
https://www.nei.nih.gov/learn-about-eye-health/resources-for-health-educators/eye-health-dat
a-and-statistics/cataract-data-and-statistics/cataract-tables
7.
Ophthalmic Services Guidance: Managing an outbreak of postoperative endophthalmitis
[Internet]. The Royal College of Ophthalmologists; 2016 Jul. Available from:
https://www.rcophth.ac.uk/wp-content/uploads/2016/07/Managing-an-outbreak-of-postoperat
ive-endophthalmitis.pdf
8.
MacEwen C, Davis A, Chang L. Ophthalmology: GIRFT Programme National Specialty Report
[Internet]. Getting It Right First Time (GIRFT); 2019 Dec. Available from:
https://gettingitrightfirsttime.co.uk/wp-content/uploads/2019/12/OphthalmologyReportGIRFT1
9P-FINAL.pdf
9.
Ophthalmic Services Guidance: Restarting and Redesigning of Cataract Pathways in response to
the COVID 19 pandemic [Internet]. The Royal College of Ophthalmologists; 2020 Aug. Available
from:
https://www.rcophth.ac.uk/wp-content/uploads/2020/08/Resumption-of-Cataract-Services-CO
VID-August-2020-2.pdf
10. Fischer SH, David D, Crotty BH, Dierks M, Safran C. Acceptance and use of health information
technology by community-dwelling elders. Int J Med Inform 2014 Sep;83(9):624–635.
PMID:24996581
11. de Pennington N, Mole G, Lim E, Milne-Ives M, Normando E, Xue K, Meinert E. Safety and
acceptability of a natural-language AI assistant to deliver clinical follow-up to cataract surgery
patients: Proposal for a pragmatic evaluation. Accepted for publication in JMIR Research
Protocols. 2021 [cited 2021 Jan 29].
Version number 1.0 – 17 May 2021
35

Autonomous telephone follow-up after cataract surgery
12. Christopoulou SC, Kotsilieris T, Anagnostopoulos I. Assessment of Health Information Technology
Interventions in Evidence-Based Medicine: A Systematic Review by Adopting a Methodological
Evaluation Framework. Healthcare (Basel) [Internet] 2018 Aug 31;6(3). PMID:30200307
13. Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, A’Court C, Hinder S, Fahy N, Procter R,
Shaw S. Beyond Adoption: A New Framework for Theorizing and Evaluating Nonadoption,
Abandonment, and Challenges to the Scale-Up, Spread, and Sustainability of Health and Care
Technologies. J Med Internet Res 2017 Nov 1;19(11):e367. PMID:29092808
14. Brooke J. SUS: A “Quick and Dirty” Usability Scale. In: Jordan P, Thomas B, Weerdmeester W,
editors. Usability Evaluation In Industry Taylor & Francis; 1996. p. 189–194.
15. Parmanto B, Lewis AN Jr, Graham KM, Bertolet MH. Development of the Telehealth Usability
Questionnaire (TUQ). Int J Telerehabil 2016 Jul 1;8(1):3–10. PMID:27563386
16. Sekhon M, Cartwright M, Francis JJ. Acceptability of healthcare interventions: an overview of
reviews and development of a theoretical framework. BMC Health Serv Res BioMed Central;
2017 Jan 26;17(1):1–13.
17. Lewis JR. The System Usability Scale: Past, Present, and Future. International Journal of
Human–Computer Interaction 2018;34(7):577–590.
18. Cruz Rivera S, Liu X, Chan A-W, Denniston AK, Calvert MJ, SPIRIT-AI and CONSORT-AI Working
Group. Guidelines for clinical trial protocols for interventions involving artificial intelligence: the
SPIRIT-AI extension. Lancet Digit Health 2020 Oct;2(10):e549–e560. PMID:33328049
19. Public Health England. Evaluation in health and well being: guidance summaries [Internet].
GOVUK. 2018 [cited 2021 Feb 1]. Available from:
https://www.gov.uk/government/publications/evaluation-in-health-and-wellbeing-guidance-su
mmaries/evaluation-in-health-and-well-being-guidance-summaries
20. Reichheld FF. The One Number You Need to Grow. Harvard Business Review [Internet] 2003 Dec
[cited 2021 Feb 23]; Available from:
https://hbr.org/2003/12/the-one-number-you-need-to-grow
21. Bowling A. Research Methods in Health: Investigating Health and Health Services. McGraw-Hill
Education (UK); 2014. ISBN:9780335262755
22. The UK GDPR. ICO; 2021 Jan 13 [cited 2021 Jan 20]; Available from:
https://ico.org.uk/for-organisations/dp-at-the-end-of-the-transition-period/data-protection-no
w-the-transition-period-has-ended/the-gdpr/
23. Data Protection Act 2018 [Internet]. legislation.gov.uk. Queen’s Printer of Acts of Parliament;
2018 [cited 2021 Jan 20]. Available from:
https://www.legislation.gov.uk/ukpga/2018/12/contents/enacted
24. HRA Approval [Internet]. NHS Health Research Authority. 2020 [cited 2021 Jan 20]. Available
from: https://www.hra.nhs.uk/approvals-amendments/what-approvals-do-i-need/hra-approval/
25. Ethics approval overview [Internet]. Imperial College London. [cited 2021 Jan 20]. Available
from:
http://www.imperial.ac.uk/research-and-innovation/support-for-staff/research-ethics-committe
e/ethics-approval-overview/
Version number 1.0 – 17 May 2021
36

Autonomous telephone follow-up after cataract surgery
26. Where and how to apply for ethical review [Internet]. University of Oxford. [cited 2021 Jan 20].
Available from: https://researchsupport.admin.ox.ac.uk/governance/ethics/apply
27. Research ethics [Internet]. University of Plymouth. [cited 2021 Jan 20]. Available from:
https://www.plymouth.ac.uk/research/governance/research-ethics-policy
28. Peer review [Internet]. Imperial College London. [cited 2021 Jan 20]. Available from:
http://www.imperial.ac.uk/research-and-innovation/support-for-staff/research-office/research-
governance-and-integrity/project-planning/peer-review/
29. Protocol [Internet]. University of Oxford. [cited 2021 Jan 20]. Available from:
https://researchsupport.admin.ox.ac.uk/ctrg/documents/protocol#collapse396691
30. Policy on payment of fees and expenses for members of the public actively involved with
INVOLVE [Internet]. INVOLVE, National Institute of Health Research; 2016 Feb. Available from:
https://www.invo.org.uk/wp-content/uploads/2016/05/INVOLVE-internal-payment-policy-2016-
final-1.pdf
31. Ethical Guidelines for Educational Research [Internet]. British Educational Research Association;
2018. Available from:
https://www.bera.ac.uk/publication/ethical-guidelines-for-educational-research-2018-online
32. Defining the Role of Authors and Contributors [Internet]. International Committee of Medical
Journal Editors. [cited 2021 Jan 20]. Available from:
http://www.icmje.org/recommendations/browse/roles-and-responsibilities/defining-the-role-of
-authors-and-contributors.html
33. Home [Internet]. ClinicalTrials.gov. [cited 2021 Jan 20]. Available from:
https://www.clinicaltrials.gov/
Version number 1.0 – 17 May 2021
37

Autonomous telephone follow-up after cataract surgery
13 APPENDICES
13.1 Appendix 1. Participant Information Sheet
Dr Eduardo Normando, Chief Investigator (Imperial)
e.normando@imperial.ac.uk
Dr Kanmin Xue, Principal Investigator (Oxford)
kanmin.xue@ouh.nhs.uk
Dr Edward Meinert, Co-Investigator (Evaluation Lead)
edward.meinert@plymouth.ac.uk
Autonomous telephone follow-up after cataract surgery
PARTICIPANT INFORMATION SHEET
Integrated Research Application System (IRAS) Reference: 297548
Health Research Authority (HRA) Approval Reference: #insert
Research Ethics Committee (REC) Approval Reference: #insert
University of Plymouth Faculty Research Ethics and Integrity Committee: #insert
We would like to invite you to take part in this research project. You should only take part if you want
to; choosing not to take part will not disadvantage you in any way. Before you decide if you wish to
take part, you need to understand why the research is being done and what taking part will involve.
Please take time to read the following information carefully and talk it through with others if you
wish. Please contact us at the above email if there is anything that is not clear or if you would like
more information.
1.
Why is this research being conducted?
The aim of the study is to provide high quality and safe automated telephone follow-up for patients
after cataract surgery.
2.
Who is conducting the project?
The study is being led by Dr Eduardo Normando in a consortium led by Imperial College London and
including the University of Plymouth and the University of Oxford. The Imperial College Healthcare
NHS Trust and the Oxford University Hospitals NHS Foundation Trust are conducting all clinical
activity.
3.
Why have I been invited to participate?
Increasingly hospitals are providing follow up after routine cataract surgery by telephone conducted
by a member of the clinical team. Ufonia Ltd has developed an automated system called DORA. This
system is a computer which can conduct telephone follow up. The aim of this study is to see if this
system is safe and that DORA makes the same recommendations as a clinician. We are evaluating
this for patients who have surgery on certain operating lists which is why you are being asked if you
would like to take part.
The study is being conducted as a collaboration between The Imperial College Healthcare NHS Trust
and the Oxford University Hospitals NHS Foundation Trust.
Version number 1.0 – 17 May 2021
38

Autonomous telephone follow-up after cataract surgery
4.
Do I have to take part?
No, taking part is entirely voluntary. It is up to you to decide whether or not to take part. If you do
decide to take part, you will be given this information sheet to keep and be asked to sign a consent
form. If you decide to take part, you are still free to withdraw at any time and without giving a
reason.
5.
What will happen to me if I take part in the research?
If you choose to take part, you will be called on a specific day three weeks after your surgery by
DORA the automated service. DORA will ask you some questions about how you are doing after your
surgery and plan the next step in your care as well as what you think of the automated service. This
will be overseen by a clinician and if you are not in agreement with the plan you can ask for this to be
reviewed.
After your call with DORA, you will be provided with a paper or online survey about your experiences
using DORA. This survey will also ask some optional demographics questions. One week after your
call with DORA, you may also be invited to take part in a telephone interview for 40-60 minutes,
where you will be asked questions about your experience with DORA. The interviews will be held
privately between you and a researcher. The sessions shall be recorded, subject to your permission,
and transcribed by a third-party supplier of the University of Plymouth.
6.
Are there any potential risks in taking part?
Potential disadvantages are that this is a new system so has not been demonstrated to be effective -
this is why the system is currently overseen by a clinician.
7.
Are there any benefits to taking part?
This safety check is additional to the usual standard of care at the two sites where the study is being
conducted. The process will be overseen by a clinician and the system is designed to provide a
thorough assessment every time. This study will also benefit future patients through a more
convenient follow-up service.
8.
What if something goes wrong?
If you are harmed by taking part in this research project, there are no special compensation
arrangements. If you are harmed due to someone’s negligence, then you may have grounds for a
legal action. Regardless of this, if you wish to complain, or have any concerns about any aspect of
the way you have been treated during the course of this study then you should immediately inform
the Chief Investigator, Dr Eduardo Normando at e.normando@imperial.ac.uk or 02033123206. The
normal National Health Service complaints mechanisms are also available to you.
9.
How will we use information about you?
Imperial College London is the sponsor for this study. Imperial College London, the University of
Plymouth, The University of Oxford and Ufonia will act as the joint data controllers for this study. This
means that all research partners are responsible for looking after your information, using it properly
and will keep your personal data for:
•
10 years after the study has finished in relation to data subject consent forms.
•
10 years after the study has completed in relation to primary research data.
Version number 1.0 – 17 May 2021
39

Autonomous telephone follow-up after cataract surgery
10. What happens to the data provided?
The information you provide during the study is known as the research data.
Any research data from where you can be identified or is linked to you is known as personal data.
This data will be accessed via your digital interactions with DORA and stored on an EEA hosted server
and on secure university network drives. Audio recordings will be created from interview sessions via
a digital recorder. This data will be directly transferred from the device to a secured drive and
transcribed by a 3rd party. Once the transcription is completed, the original interview audio
interview files will be destroyed.
Other research data (including consent forms) will be stored for ten years after publication or public
release of the research and stored on a secure university network drives at Imperial College and the
University of Plymouth. Dr Normando and Dr Meinert and their research teams will have access to
the research data. Responsible members of Imperial College London, the University of Plymouth and
the University of Oxford may be given access to data for monitoring and/or audit of the research.
All data will be stored on a password-protected network drive within the Imperial College London’s
and University of Plymouth’s network. Access to these files will be limited to the study research
team. Electronic data shall be coded using a unique participant number and primary critical
pseudonymisation (creation of a fictitious name linked to participant numbers) process. Physical
copies of consent forms will be stored in a locked folder at Dr Meinert’s office in the University of
Plymouth at 6 Kirkby Place, Room 2.
We would like your permission to use direct quotes with a fictitious name in any research outputs.
We would like your permission to use pseudonymised data in future studies and to share data with
other researchers. All personal information that could identify you will be removed or changed
before the information is shared with other researchers or results are made public.
If you consent to take part in the research, any information you provide may be inspected and used
by administrators of the study. Each participant will be pseudonymised using a unique identifier to
maintain confidentiality, and all data will be securely stored and managed according to sponsor rules
and expected practices. Raw, un-anonymised audio data will be securely stored separately from the
anonymisation key and deleted when it is no longer needed. The pseudonymised transcripts will be
securely stored according to Imperial College London’s and the University of Plymouth protocols and
regulations.
11. Will the use of my data meet GDPR rules?
GDPR stands for the General Data Protection Regulation. In the UK we follow the GDPR rules and
have a law called the Data Protection Act. All research using patient data must follow UK laws and
rules.
Universities, NHS organisations and companies may use patient data to do research to make health
and care better.
When companies do research to develop new treatments, they need to be able to prove that they
need to use patient data for the research, and that they need to do the research to develop new
treatments. In legal terms this means that they have a ‘legitimate interest’ in using patient data.
Version number 1.0 – 17 May 2021
40

Autonomous telephone follow-up after cataract surgery
Universities and the NHS are funded from taxes and they are expected to do research as part of their
job. They still need to be able to prove that they need to use patient data for the research. In legal
terms this means that they use patient data as part of ‘a task in the public interest’.
If they could do the research without using patient data, they would not be allowed to get your data.
Researchers must show that their research takes account of the views of patients and ordinary
members of the public. They must also show how they protect the privacy of the people who take
part. An NHS research ethics committee checks this before the research starts.
12. Will what I say be kept confidential?
All information provided during interviews will be pseudonymised, kept strictly confidential and not
attributed to you. Participants will be allocated a study ID number and any information collected will
only be seen by the study team. The voice data derived from conversations with DORA will be stored
and used by Ufonia, Ltd, the manufacturer of DORA, for up to 10 years to train and improve its
system. This data will be stored using a unique ID and will not include your name.
13. What will happen to the results of the project?
The results of this evaluation study will be used for the purpose of deciding whether the system is
safe to provide autonomous telephone follow-up after cataract surgery without oversight from a
clinician. We will publish findings from the study in a medical journal.
14. Who has reviewed the project?
The study has been approved by the Health Research Authority, the University of Plymouth, and
Imperial College London.
15. Legal Basis
As universities we use personally identifiable information to conduct research to improve health,
care and services. As publicly funded organisations, we have to ensure that this work is in the public
interest when we use personally identifiable information from people who have agreed to take part
in research. This means that when you agree to take part in a research study, we will use your data
in the ways needed to conduct and analyse the research study.
Health and care research should serve the public interest, which means that we have to demonstrate
that our research serves the interests of society as a whole. We do this by following the UK Policy
Framework for Health and Social Care Research
16. International transfers
There may be a requirement to transfer information to countries outside the European Economic
Area (for example, to a research partner). Where this information contains your personal data,
Imperial College London will ensure that it is transferred in accordance with data protection
legislation. If the data is transferred to a country which is not subject to a European Commission (EC)
adequacy decision in respect of its data protection standards, Imperial College London will enter into
a data sharing agreement with the recipient organisation that incorporates EC approved standard
contractual clauses that safeguard how your personal data is processed.
17. Sharing your information with others
Version number 1.0 – 17 May 2021
41

Autonomous telephone follow-up after cataract surgery
For the purposes referred to in this privacy notice and relying on the bases for processing as set out
above, we will share your personal data with certain third parties.
●
Other Imperial College London employees, agents, contractors and service providers (for
example, suppliers of printing and mailing services, email communication services or web
services, or suppliers who help us carry out any of the activities described above). Our
third-party service providers are required to enter into data processing agreements with us.
We only permit them to process your personal data for specified purposes and in accordance
with our policies.
●
The following Research Collaborators / Partners in the study:
-
The University of Plymouth – The University of Plymouth will access research and
personal data for the purpose of completing an evaluation of DORA.
-
The University of Oxford – The University of Oxford will access research and personal
data for the purpose of completing an evaluation of DORA.
-
Ufonia, Ltd. – Ufonia, Ltd. will access personal data for the processing of conversations
with DORA and using these conversations to train and improve DORA’s ability to conduct
automated telephone follow-up.
18. Complaints
If you wish to raise a complaint on how we have handled your personal data, please contact Imperial
College London’s Data Protection Officer via email at dpo@imperial.ac.uk, via telephone on 020 7594
3502 and/or via post at Imperial College London, Data Protection Officer, Faculty Building Level 4,
London SW7 2AZ.
If you are not satisfied with our response or believe we are processing your personal data in a way
that is not lawful you can complain to the Information Commissioner’s Office (ICO). The ICO does
recommend that you seek to resolve matters with the data controller (us) first before involving the
regulator.
19. What should I do if I want to take part?
If you agree to take part, you will be asked to sign a consent form or provide consent verbally and will
be given a date to expect a call from DORA.
Contact for further information:
●
Dr Eduardo Normando – e.normando@imperial.ac.uk
●
Dr Kanmin Xue – kanmin.xue@ouh.nhs.uk
●
Dr Edward Meinert – edward.meinert@plymouth.ac.uk
Version number 1.0 – 17 May 2021
42

Autonomous telephone follow-up after cataract surgery
Version number 1.0 – 17 May 2021
43

Autonomous telephone follow-up after cataract surgery
13.2 Appendix 2. Informed Consent Form
Study Title: Autonomous telephone follow-up after cataract surgery
*Informed consent in this study will be taken in written form or recorded.
Participant Identification number for this trial: _____________________
Name of Principal Investigators: Dr Eduardo M. Normando, Dr Kanmin Xue
Please initial the boxes.
1. I confirm that I have read the information sheet dated.................... (Version............) for the above study. I
have had the opportunity to consider the information, ask questions and have had these answered
satisfactorily.
2. I understand that my participation is voluntary and that I am free to withdraw at any time without giving any
reason, without my medical care or legal rights being affected.
3. I understand that relevant sections of my medical notes and data collected during the study, may be looked
at by individuals from Imperial College London, from University of Oxford, form University of Plymouth, from
regulatory authorities or from the NHS Trust, where it is relevant to my taking part in this research. I give
permission for these individuals to have access to my records.
4. I consent to being contacted regarding potentially taking part in other research studies.
5. I give/do not give (delete as applicable) consent for information collected about me to be used to support
other research in the future, including those outside of the EEA.
6. I consent to my GP being informed about my participation in the study.
7. I agree to being contacted for and interview regarding the user experience.
8. I consent to being audio recorded.
9. I understand how audio recordings will be used in research outputs.
10. I give permission to be quoted directly in research outputs against a pseudonym.
11. I agree to grant access to my follow-up data as specified in the patient information sheet.
12. I consent to take part in the above study.
ㅡㅡㅡㅡㅡㅡㅡㅡㅡㅡㅡㅡㅡㅡ
ㅡㅡㅡㅡㅡㅡㅡ
ㅡㅡㅡㅡㅡㅡㅡㅡㅡ
Name of Participant
Date
Signature
ㅡㅡㅡㅡㅡㅡㅡㅡㅡㅡㅡㅡㅡㅡ
ㅡㅡㅡㅡㅡㅡㅡ
ㅡㅡㅡㅡㅡㅡㅡㅡㅡ
Name of Person taking consent
Date
Signature
(If different from Principal Investigator)
Version number 1.0 – 17 May 2021
44

Autonomous telephone follow-up after cataract surgery
ㅡㅡㅡㅡㅡㅡㅡㅡㅡㅡㅡㅡㅡㅡ
ㅡㅡㅡㅡㅡㅡㅡ
ㅡㅡㅡㅡㅡㅡㅡㅡㅡ
Principal Investigator
Date
Signature
You will be given a copy of this consent form to keep (if recorded you will be provided a transcript following
consent); one copy will be filed in your medical notes, and one will be filed with the study records within the
research folder at the Study Site.
Version number 1.0 – 17 May 2021
45

Autonomous telephone follow-up after cataract surgery
13.3 Appendix 3. SPIRIT-AI Checklist: Recommended items to address in a protocol
and related documents for clinical trials evaluating AI interventions
Cruz Rivera S, Liu X, Chan A-W, Denniston AK, Calvert MJ, SPIRIT-AI and CONSORT-AI Working Group. Guidelines
for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. Lancet Digit
Health 2020 Oct;2(10):e549–e560. PMID:33328049
Section
SPIRIT 2013 Itema
SPIRIT-AI Item
Page
Nob
Administrative Information
Title
1
Descriptive title identifying the study design,
population, interventions, and, if applicable,
trial acronym
SPIRIT-AI
1(i)
Elaboratio
n
Indicate that the
intervention involves
artificial intelligence /
machine learning and
specify the type of model.
Title
page
SPIRIT-AI
1(ii)
Elaboratio
n
Specify the intended use of
the AI intervention.
Title
page
Trial
registratio
n
2a
Trial identifier and registry name. If not yet
registered, name of intended registry
34 -
TBD
2b
All items from the World Health Organization
Trial Registration Data Set
Title
page, 9
Protocol
version
3
Date and version identifier
Title
page, 4
Funding
4
Sources and types of financial, material, and
other support
34
Roles and
responsibil
ities
5a
Names, affiliations, and roles of protocol
contributors
Title
page,
6-7,
10-11
5b
Name and contact information for the trial
sponsor
Title
page,
6-7, 10
5c
Role of study sponsor and funders, if any, in
study design; collection, management, analysis,
and interpretation of data; writing of the
report; and the decision to submit the report
for publication, including whether they will
have ultimate authority over any of these
activities
10
5d
Composition, roles, and responsibilities of the
coordinating centre, steering committee,
endpoint adjudication committee, data
management team, and other individuals or
10
Version number 1.0 – 17 May 2021
46

Autonomous telephone follow-up after cataract surgery
groups overseeing the trial, if applicable (see
Item 21a for data monitoring committee)
Introduction
Backgroun
d and
rationale
6a
Description of research question and
justification for undertaking the trial, including
summary of relevant studies (published and
unpublished) examining benefits and harms for
each intervention
SPIRIT-AI
6a (i)
Extension
Explain the intended use of
the AI intervention in the
context of the clinical
pathway, including its
purpose and its intended
users (e.g. healthcare
professionals, patients,
public).
14
SPIRIT-AI
6a (ii)
Extension
Describe any pre-existing
evidence for the AI
intervention.
14
6b
Explanation for choice of comparators
19
Objectives
7
Specific objectives or hypotheses
16
Methods: Participants, Interventions and Outcomes
Trial
design
8
Description of trial design including type of trial
(e.g., parallel group, crossover, factorial, single
group), allocation ratio, and framework (e.g.,
superiority, equivalence, noninferiority,
exploratory)
17
Study
setting
9
Description of study settings (e.g., community
clinic, academic hospital) and list of countries
where data will be collected. Reference to
where list of study sites can be obtained
SPIRIT-AI 9
Extension
Describe the onsite and
offsite requirements
needed to integrate the AI
intervention into the trial
setting.
17
Eligibility
criteria
10
Inclusion and exclusion criteria for participants.
If applicable, eligibility criteria for study centres
and individuals who will perform the
interventions (e.g., surgeons, psychotherapists)
SPIRIT-AI
10 (i)
Elaboratio
n
State the inclusion and
exclusion criteria at the
level of participants.
22
SPIRIT-AI
10 (ii)
Extension
State the inclusion and
exclusion criteria at the
level of the input data.
22
Interventio
ns
11a
Interventions for each group with sufficient
detail to allow replication, including how and
when they will be administered
SPIRIT-AI
11a (i)
Extension
State which version of the
AI algorithm will be used.
18
SPIRIT-AI
11a (ii)
Extension
Specify the procedure for
acquiring and selecting the
input data for the AI
intervention.
18-20
SPIRIT-AI
11a (iii)
Extension
Specify the procedure for
assessing and handling poor
18, 20
Version number 1.0 – 17 May 2021
47

Autonomous telephone follow-up after cataract surgery
quality or unavailable input
data.
SPIRIT-AI
11a (iv)
Extension
Specify whether there is
human-AI interaction in the
handling of the input data,
and what level of expertise
is required for users.
18- 20
SPIRIT-AI
11a (v)
Extension
Specify the output of the AI
intervention.
19-20
SPIRIT-AI
11a (vi)
Extension
Explain the procedure for
how the AI intervention’s
output will contribute to
decision-making or other
elements of clinical practice.
18
11b
Criteria for discontinuing or modifying allocated
interventions for a given trial participant (e.g.,
drug dose change in response to harms,
participant request, or improving/worsening
disease)
19-20
11c
Strategies to improve adherence to intervention
protocols, and any procedures for monitoring
adherence (e.g., drug tablet return, laboratory
tests)
25
11d
Relevant concomitant care and interventions
that are permitted or prohibited during the trial
N/A
Outcomes
12
Primary, secondary, and other outcomes,
including the specific measurement variable
(e.g., systolic blood pressure), analysis metric
(e.g., change from baseline, final value, time to
event), method of aggregation (eg, median,
proportion), and time point for each outcome.
Explanation of the clinical relevance of chosen
efficacy and harm outcomes is strongly
recommended
19-22
Participant
timeline
13
Time schedule of enrolment, interventions
(including any run-ins and washouts),
assessments, and visits for participants. A
schematic diagram is highly recommended (see
Figure)
12
(Fig 1),
17
(Fig 4),
26
(Fig 6)
Sample
size
14
Estimated number of participants needed to
achieve study objectives and how it was
determined, including clinical and statistical
assumptions supporting any sample size
calculations
22-24,
App. 7
and 8
Version number 1.0 – 17 May 2021
48

Autonomous telephone follow-up after cataract surgery
Recruitme
nt
15
Strategies for achieving adequate participant
enrolment to reach target sample size
24
Methods: Assignment of Interventions (For Controlled Trials)
Sequence
generation
16A
Method of generating the allocation sequence
(eg, computer-generated random numbers),
and list of any factors for stratification. To
reduce predictability of a random sequence,
details of any planned restriction (eg, blocking)
should be provided in a separate document that
is unavailable to those who enrol participants
or assign interventions
25
Allocation
concealme
nt
mechanis
m
16b
Mechanism of implementing the allocation
sequence (eg, central telephone; sequentially
numbered, opaque, sealed envelopes),
describing any steps to conceal the sequence
until interventions are assigned
25
Implement
ation
16c
Who will generate the allocation sequence,
who will enrol participants, and who will assign
participants to interventions
25
Blinding
(masking)
17a
Who will be blinded after assignment to
interventions (eg, trial participants, care
providers, outcome assessors, data analysts),
and how
25
17b
If blinded, circumstances under which
unblinding is permissible, and procedure for
revealing a participant’s allocated intervention
during the trial
N/A
Methods: Data Collection, Management, And Analysis
Data
collection
methods
18a
Plans for assessment and collection of
outcome, baseline, and other trial data,
including any related processes to promote
data quality (eg, duplicate measurements,
training of assessors) and a description of study
instruments (eg, questionnaires, laboratory
tests) along with their reliability and validity, if
known. Reference to where data collection
forms can be found, if not in the protocol
25-26
18b
Plans to promote participant retention and
complete follow-up, including list of any
outcome data to be collected for participants
who discontinue or deviate from intervention
protocols
29-30
Data
manageme
nt
19
Plans for data entry, coding, security, and
storage, including any related processes to
promote data quality (eg, double data entry;
range checks for data values). Reference to
31
Version number 1.0 – 17 May 2021
49

Autonomous telephone follow-up after cataract surgery
where details of data management procedures
can be found, if not in the protocol
Statistical
methods
20a
Statistical methods for analysing primary and
secondary outcomes. Reference to where other
details of the statistical analysis plan can be
found, if not in the protocol
26-27
20b
Methods for any additional analyses (eg,
subgroup and adjusted analyses)
19-20
20c
Definition of analysis population relating to
protocol non-adherence (eg, as randomised
analysis), and any statistical methods to handle
missing data (eg, multiple imputation)
22-24
Methods: Monitoring
Data
monitoring
21a
Composition of data monitoring committee
(DMC); summary of its role and reporting
structure; statement of whether it is
independent from the sponsor and competing
interests; and reference to where further
details about its charter can be found, if not in
the protocol. Alternatively, an explanation of
why a DMC is not needed
29
21b
Description of any interim analyses and
stopping guidelines, including who will have
access to these interim results and make the
final decision to terminate the trial
31
Harms
22
Plans for collecting, assessing, reporting, and
managing solicited and spontaneously reported
adverse events and other unintended effects of
trial interventions or trial conduct
SPIRIT-AI
22
Extension
Specify any plans to identify
and analyse performance
errors. If there are no plans
for this, explain why not.
19-21
Auditing
23
Frequency and procedures for auditing trial
conduct, if any, and whether the process will be
independent from investigators and the
sponsor
30
Ethics and Dissemination
Research
ethics
approval
24
Plans for seeking research ethics
committee/institutional review board (REC/IRB)
approval
28
Protocol
amendme
nts
25
Plans for communicating important protocol
modifications (eg, changes to eligibility criteria,
outcomes, analyses) to relevant parties (eg,
investigators, REC/IRBs, trial participants, trial
registries, journals, regulators)
30
Consent or
assent
26a
Who will obtain informed consent or assent
from potential trial participants or authorised
surrogates, and how (see Item 32)
25,
30-31
Version number 1.0 – 17 May 2021
50

Autonomous telephone follow-up after cataract surgery
26b
Additional consent provisions for collection and
use of participant data and biological specimens
in ancillary studies, if applicable
N/A
Confidenti
ality
27
How personal information about potential and
enrolled participants will be collected, shared,
and maintained in order to protect
confidentiality before, during, and after the trial
31
Declaratio
n of
interests
28
Financial and other competing interests for
principal investigators for the overall trial and
each study site
34
Access to
data
29
Statement of who will have access to the final
trial dataset, and disclosure of contractual
agreements that limit such access for
investigators
SPIRIT-AI
29
Extension
State whether and how the
AI intervention and/or its
code can be accessed,
including any restrictions to
access or re-use.
32
Ancillary
and
post-trial
care
30
Provisions, if any, for ancillary and post-trial
care, and for compensation to those who suffer
harm from trial participation
N/A
Disseminat
ion policy
31a
Plans for investigators and sponsor to
communicate trial results to participants,
healthcare professionals, the public, and other
relevant groups (eg, via publication, reporting in
results databases, or other data sharing
arrangements), including any publication
restrictions
33
31b
Authorship eligibility guidelines and any
intended use of professional writers
33
31c
Plans, if any, for granting public access to the
full protocol, participant-level dataset, and
statistical code
N/A
Appendices
Informed
consent
materials
32
Model consent form and other related
documentation given to participants and
authorised surrogates
38-41
(App 1
and 2)
Biological
specimens
33
Plans for collection, laboratory evaluation, and
storage of biological specimens for genetic or
molecular analysis in the current trial and for
future use in ancillary studies, if applicable
N/A
aIt is strongly recommended that this checklist be read in conjunction with the SPIRIT 2013
Explanation & Elaboration for important clarification on the items.
b Indicates page numbers to be completed by authors during protocol development.
Version number 1.0 – 17 May 2021
51

Autonomous telephone follow-up after cataract surgery
13.4 Appendix 4. Sample Post-Call Survey
Blank, Grant, OxIS 2019 Questionnaire. All Parts (January 30, 2019). Available at SSRN:
https://ssrn.com/abstract=3522118
Brooke J. Systems Usability Scale. In: Jordan P, Thomas B, Weerdmeester W, editors. Usability Evaluation In
Industry Taylor & Francis; 1996. p. 189–194.
Parmanto B, Lewis AN Jr, Graham KM, Bertolet MH. Development of the Telehealth Usability Questionnaire
(TUQ). Int J Telerehabil 2016 Jul 1;8(1):3–10. PMID:27563386
Demographic Questions
1.
AGE: In what year were you born? 
a.
 (If refuse to answer): Would you mind indicating which of these age bands you are
in: 18-24, 25-34, 35-44, 45-54, 55-64, 65-74, 75-84, 85+
2.
GENDER*: How do you currently describe your gender identity? 
a.
*Note: in the OxIS, gender was assessed ‘by observation’. The wording of this
question is therefore taken from a different source (Hughes, Camden, & Yangchen,
2016)
3.
FAMILY: 
a.
Are you…? Single, Married, Living together with a partner, Divorced/separated,
Widowed
b.
How many adults live in your household (people age 18 or more)? [WRITE IN] 
c.
How many children (people age 17 or less) live in your household? [WRITE IN]
4.
ETHNICITY: To which of these groups do you consider you belong? 
a.
ASIAN: of Indian origin
b.
ASIAN: of Pakistani origin
c.
ASIAN: of Bangladeshi origin 
d.
ASIAN: of Chinese origin 
e.
ASIAN: of any other origin (WRITE IN) ___________________________ 
f.
BLACK: of African origin
g.
BLACK: of Caribbean origin 
h.
BLACK: of other origin (WRITE IN) ___________________________ 
i.
WHITE: of British origin 
j.
WHITE: of Scottish origin
k.
WHITE: of Welsh origin
l.
WHITE: of any other origin (WRITE IN) ___________________________ 
m. OTHER (WRITE IN)
n.
Don’t know 
5.
URBAN: Would you describe the place where you live as a big city, the suburbs or outskirts of
a big city, a small city or town, a country village, or, a farm or home in the country?
a.
A big city
b.
The suburbs or outskirts of a big city
c.
A small city or town 
d.
A country village 
e.
A farm or home in the country 
f.
Other 
g.
Don’t know
Version number 1.0 – 17 May 2021
52

Autonomous telephone follow-up after cataract surgery
6.
INCOME: The incomes of households differ a lot in Britain today. Here is a table showing the
range of incomes that people have. Which option best represents the total income of your
household before tax? Please select one answer only. 
a.
Up to £12.500 per year 
b.
£12.500 up to £20.000 per year 
c.
£20.000 up to £30.000 per year 
d.
£30.000 up to £40.000 per year 
e.
£40.000 up to £50.000 per year 
f.
£50.000 up to £60.000 per year 
g.
£60.000 up to £70.000 per year 
h.
£70.000 up to £80.000 per year 
i.
Over £80.000 per year 
j.
Prefer not to say
7.
EDUCATION: What is the highest educational or vocational qualification that you have or that
you will receive if you complete your next set of exams?
a.
No qualifications 
b.
5 or more GCSE grades A-C 
c.
4 or less GCSE grade A-C 
d.
GCSE grade D-G
e.
5 or more Scottish Standard Grades, grades 1-3 
f.
4 or less Scottish Standard Grades, grades 1-3 
g.
5 or more O Grades, grades 1-3 (Scottish Ordinary Grades) 
h.
4 or less O Grades, grades 1-3 (Scottish Ordinary Grades) 
i.
Scottish Standard Grades, grades 4-7 
j.
Scottish Highers (either SCE or SQC) 
k.
Scottish Certificate Sixth Year Studies 
l.
SVQ level 1 or 2 (Scottish Vocational Qualifications) 
m. SVQ level 3 ( “ ) 
n.
SVQ level 4 ( “ ) 
o.
SVQ level 5 ( “ ) 
p.
CSEs 
q.
5 or more O levels 
r.
4 or less O levels 
s.
GCE A levels or equivalent 
t.
NVQ level 1 or 2
u.
NVQ level 3 or 4 
v.
NVQ level 5 
w. GNVQ Foundation
x.
GNVQ Intermediate 
y.
GNVQ Advanced 
z.
Certificate or Diploma of Higher Education 
aa. HND (Higher National Diploma) 
bb. Bachelor’s degree 
cc. Graduate Certificates and Diploma 
dd. Post-degree professional qualification (eg banking accountancy, architecture, etc.) 
ee. Master’s Degree
Version number 1.0 – 17 May 2021
53

Autonomous telephone follow-up after cataract surgery
ff.
Doctoral Degree
gg. Don’t know
8.
EMPLOYMENT: Which of these descriptions best describes your current situation?
a.
Working Full time (30 hours a week or more)
b.
Working Part time (8-29 hours a week) 
i.
If (a) or (b): Apart from working, do you also study? (yes / no / DK) 
c.
Retired 
d.
Unemployed 
e.
Permanently sick or disabled 
f.
In community or military service 
g.
Undergraduate Student 
h.
Post graduate student
i.
In full time education (not higher degree)
j.
In part time education (not higher degree)
i.
If (g)-(j): And apart from studying, do you also work? (yes / no / DK) 
k.
Doing housework, looking after children or other persons
System Usability Scale
Rated on a 5-point Likert scale, from ‘Strongly Disagree’ (1) to ‘Strongly Agree’ (5)
9.
I think that I would like to use this system frequently.
10. I found the system unnecessarily complex.
11. I thought the system was easy to use.
12. I think that I would need the support of a technical person to be able to use this system.
13. I found the various functions in this system were well integrated.
14. I thought there was too much inconsistency in this system.
15. I would imagine that most people would learn to use this system very quickly.
16. I found the system very awkward to use.
17. I felt very confident using the system.
18. I needed to learn a lot of things before I could get going with this system.
Telehealth Usability Questionnaire
Rated on a 7-point Likert scale, from ‘Disagree’ (1) to ‘Agree’ (7). It has been adapted slightly to fit
the particular context of this study.
19. Telehealth improves my access to healthcare services.
20. Telehealth saves me time traveling to a hospital or specialist clinic.
21. Telehealth provides for my healthcare needs.
22. It was simple to use this system.
23. It was easy to learn to use the system.
24. The way I interact with this system is pleasant.
25. I like using the system.
26. The system is simple and easy to understand.
27. This system is able to do everything I would want it to be able to do.
28. I could easily talk to DORA.
29. I could hear DORA clearly.
30. I felt I was able to express myself effectively.
31. I think the visits provided over the telehealth system are the same as in-person visits.
Version number 1.0 – 17 May 2021
54

Autonomous telephone follow-up after cataract surgery
32. Whenever I made a mistake using the system, I could recover easily and quickly.
33. The system gave error messages that clearly told me how to fix problems.
34. I feel comfortable communicating with DORA.
35. Telehealth is an acceptable way to receive healthcare services.
36. I would use telehealth services again.
37. Overall, I am satisfied with this telehealth system.
Version number 1.0 – 17 May 2021
55

Autonomous telephone follow-up after cataract surgery
13.5 Appendix 5. Topic Guide for Semi-Structured Interviews
The topic guide was developed based on the Theoretical Framework of Acceptability (TFA) [16],
which was created to provide a framework for assessing the multiple facets of acceptability of health
interventions. The TFA has seven components: “1) affective attitude, 2) burden, 3) ethicality, 4)
intervention coherence, 5) opportunity costs, 6) perceived effectiveness, and 7) self-efficacy” [16].
Interview questions
* Before the call
1.
What did you expect the experience of talking to DORA to be like before you received the
call?
2.
How did you feel about talking to DORA before you received the call?
3.
What, if any, concerns did you have before you received the call?
4.
How well did you understand the intervention before you received the call?
5.
What benefits or losses did you think you would experience when you received the call?
6.
How successful did you think DORA would be at delivering the follow-up before you had the
call?
7.
How confident were you that you would be able to interact with DORA before you received
the call?
* During the call
1.
How did you feel about your experience interacting with DORA?
2.
How much effort did interacting with DORA take?
3.
What concerns, if any, did you have while talking with DORA?
4.
How well did you understand how the call worked while you were talking to DORA?
5.
What benefits or losses did you experience while you were talking to DORA?
6.
How much confidence did you have in DORA’s ability to perform the follow-up assessment
while you were talking to DORA?
7.
How comfortable and confident were you in your ability to interact with DORA while you
were on the phone?
* After the call
8.
Looking back on the experience now, how do you feel about using DORA to perform
follow-up appointments?
9.
Looking back on the experience now, how much effort did interacting with DORA take?
10. Looking back on the experience now, what concerns, if any, do you have about DORA?
11. Looking back on the experience now, how well do you understand how DORA worked?
12. Looking back on the experience now, what do you think you gained or lost by having your
follow-up assessment with DORA?
13. Looking back on the experience now, how well do you think DORA performed at conducting
your follow-up assessment?
14. Looking back on the experience now, how confident would you be having another interaction
with DORA?
* General feedback
15. What would you suggest to make the experience of using DORA better?
16. How willing would you be to use DORA again?
17. If you had the choice between no follow-up / a face-to-face appointment (depending on
clinical site) and DORA, which would you choose and why?
18. Is there anything else you’d like to mention?
Version number 1.0 – 17 May 2021
56

Autonomous telephone follow-up after cataract surgery
13.6 Appendix 6. Imperial Historical Recruitment Audit Data
Purpose
To provide real world evidence for assumptions made in modelling of recruitment numbers and
study planning. 
Audit
We audited all bookings in the cataract one-stop-clinic for the month of October 2020. After
discussion with the head of the cataract pathway, it was felt this was the most representative of what
the cataract pathway will likely look like in May-July 2021 assuming there is COVID recovery of a
similar trajectory to the first wave. It is anticipated there will be 3 consultant-led one-stop clinics
running in July to address the backlog. Of note, the general pre-assessment pathways have not been
audited.
Key findings
Clinic capacity and utilisation - All October one-stop data
●
33 patients were booked across 4 clinics
●
8.25 patients on average per clinic 
●
88% (29/33) attended their one-stop appointments
Conversion rate
●
Of all patients booked, 76% proceeded for surgery (25/33)
●
Or, of those attending, 86% proceeded to surgery (25/29)
Demographics
●
Mean age was 71.5
●
This was the first eye for 52% of patients
Inclusion/Exclusion 
●
100% (29/29) of those attending were listed for routine cataract surgery
●
10% (3/29) required a translator (Tamil, Punjabi, Arabic) 
●
1 patient (3%) had dementia and would not have been able to consent 
●
Overall, of all attending patients, 76% (29/33) would have been eligible for trial inclusion. 
Withdrawal 
●
No patients in this cohort had unexpected surgical complications 
●
1 patient (4%) attended AnE before follow up with a corneal abrasion and would have been
withdrawn from DORA per our current protocol.
Follow up
●
95% (24/25) of patients attended post-op follow up
●
21% (7/25) of patients had a documented complication requiring management change at
their post-op followup.
o
6 of these patients were uveitis
▪
4 of which were given maxidex and discharged
▪
1 was given maxidex with phone follow up
Version number 1.0 – 17 May 2021
57

Autonomous telephone follow-up after cataract surgery
▪
1 was given maxidex and listed for 2nd eye.
o
1 patient has post-op cystoid macular oedema and maxidex requiring medical retina
review.
Pathway
●
The mean time between one-stop appointment and surgery was 9.88 days.
o
The range was from 2 days to 62 days
o
16 patients had their operations in under 1 week 
●
Mean time between surgery and first follow up was 24.33 days.
●
A number of patients were followed up under the 21 day point
Comparison with initial estimates
Conclusions
●
Based on these samples our projections were conservative apart from % eligible for trial
●
Based on these projects even 2 clinics per week should allow us to meet a recruitment
number of over 200 at Imperial.
●
Recruiting from pre-op assessment will be required to meet aim of >400 patients
Version number 1.0 – 17 May 2021
58

Autonomous telephone follow-up after cataract surgery
13.7 Appendix 7. Oxford Historical Recruitment Audit Data
All bookings for cataract surgery pre-assessments at the Oxford University Hospitals NHS Foundation
Trust for October were audited to provide evidence for an estimate of the recruitment sample size.
Assuming there is a recovery from the Covid-19 pandemic of a similar trajectory to the first way, this
was determined to be the most representative of what the cataract pathway will likely look like in
May-July 2021. The audit report was produced on 21 January, 2021.
Purpose
To provide real world evidence for assumptions made in modelling of recruitment numbers and
study planning.
Audit
We audited the cataract clinic pre-op assessment clinics in October 2020. The clinic numbers were
reviewed for the entire month, and the first 50 consecutive patients were reviewed in detail. It was
felt this was the most representative of what the cataract pathway will likely look like in May-July
2021 assuming there is COVID recovery of a similar trajectory to the first wave.
Key findings
Clinic capacity and utilisation - all October data
●
160 patients were booked across 21 clinics
●
7.6 patients on average per day
●
96% (50/52) attended their one-stop appointments
●
Only 1/160 was booked as a telephone assessment
Conversion rate (reviewing first 50 consecutive patients)
●
Of all patients booked, 86% proceeded for surgery (45/52)
●
Of all patients that attended, 90% were listed for surgery directly (45/50)
○
2 were asymptomatic and discharged, 3 were referred to clinic and subsequently
listed
○
Including those listed from clinic- 48/50, 96% were listed for surgery.
Demographics
●
Mean age was 76.36
●
17 male: 33 female
●
This was the first eye for 78% of patients
Inclusion/Exclusion
●
96% (46/48) of those attending were listed for routine cataract surgery
●
2% (1/48) required a translator
●
2 patients (4%) had learning difficulties and would not have been able to consent (the
patient with learning difficulties also had surgical complexities)
●
Overall, of all attending patients, 92% (44/48) would have been eligible for trial inclusion.
Version number 1.0 – 17 May 2021
59

Autonomous telephone follow-up after cataract surgery
Pathway
●
To date 30/44 (68%) of the patients listed have had cataract surgery
●
The mean time between one-stop appointment and surgery was 30 days.
○
The range was from 0 days to 79 days
○
3 patients had their operations in under 1 week
Withdrawal
●
6.6% (2/30) patients in this cohort had unexpected surgical complications – 1 iris prolapse
and 1 needing sutures
●
The 2 patients with complications attended a planned clinic appointment and were
discharged.
●
1 patient attended ED with CMO 10 weeks post procedure, 1 called eye casualty with red eye
6 weeks post procedure but did not need review
Follow up
●
Only the 2 patients with unexpected complications had planned clinic appointments
Comparison with initial estimates
Oxford
Oxford
Oxford
Pre-assessment
October 2020
Pre-assessment
(Study projections)
Phone pre-assessment
(2nd eye) (Study
projections)
Current Numbers Assessed
Mean Patients assessed
per clinic
7.6
12
4
Clinics per week
5
5
5
Mean Patients Assessed
per week
39.5
60
20
Eligibility Assumptions
% proceeding to surgery
90%
90%
90%
% routine cases
96%
80%
80%
% eligible for trial
92%
90%
90%
% dropout due to
complicated surgery
6.6%
5%
5%
Screening
Compound daily eligibility
rate
74%
62%
62%
Number of days screening
2
2
2
Patients screened per week 11
15
5
Version number 1.0 – 17 May 2021
60

Autonomous telephone follow-up after cataract surgery
Estimated recruitment
% consenting to trial
90%
90%
90%
Estimated recruited per
week
10
13
4
Duration of trial (weeks)
26
26
26
Estimated recruitment over
trial
264
346
115
Conclusions
●
The spread between F2F and telephone clinics was different to previously predicted, with
only 1 patient in October having a telephone appt booked.
●
Similar rates of patients that are potentially eligible in October 2020 compared to previous
estimates
●
Based on these projections even 2 clinics per week should allow us to meet a recruitment
number of over 200 at Oxford
Version number 1.0 – 17 May 2021
61