Detection and Classification of Diabetic Retinopathy From Posterior Pole Images With A Deep Learning Model
Conditions: Diabetic Retinopathy, Diabetic Eye Problems, Diabetic Macular Edema
Sponsor: Ural Telekomunikasyon Sanayi Ticaret Anonim Sirketi

Research Protocol Name: DETECTION AND CLASSIFICATION OF DIABETIC RETINOPATHY
FROM POSTERIOR POLE IMAGES WITH A DEEP LEARNING MODEL
Protocol Number: EC-2021DR-TR
Research Type / Phase: First Application for Non-CE Marked Medical Device Research
(Excluding Thesis and Academic Studies)
We have read the current version of the Declaration of Helsinki, Good Clinical Practices
(ICU) Guidelines and Good Laboratory Practices (ILU) Guidelines, and that we have read the
current version of our clinical study, "Detection and Classification of Diabetic Retinopathy from
Back Pole Photographs with a Deep Learning Model", which we will carry out with 900 volunteer
patients, by the medical consultant and responsible researchers. We declare and undertake that it
will be carried out in accordance with the approved and authorized protocol, the Declaration of
Helsinki, IKU/ILU. In addition, the researchers named below were informed in detail about the
study by the Principal Investigator (single center) of this study.
Associate Professor. A. Burak BİLGİN
Akdeniz University Faculty of Medicine
Department of Ophthalmology
Dumlupinar Boulevard
Akdeniz University Hospital E Block
07059, Campus, ANTALYA

05.10.2021
Version No:1.0
1.
PROTOCOL REVISION HISTORY
Rev No
History
Reason
00
01.07.2021
First version (v1.0)
01
05.10.2021
Second
version
(v1.0)
-
Moved
the
research starting date forward
2.
RESEARCH INFORMATION
2.1. Supporting Organization
Name: URAL Telecommunication Industry Trade Limited Company
Address: Pınarbaşı Mah. Hurriyet Cad. Akdeniz University Antalya Technopolis R&D 2
Uluğbey Building No:3A/Z14, 07070 Konyaaltı/ANTALYA
Name-surname of the official: Rim KHAZHIN
E-mail address: rim@uraltelekom.com
2.2. Supporting Legal Representative
Only the sponsor has the authority to sign. There is no legal representative.
2.3. Medical Consultants and Associated Medical Qualified Physician
Project supervisor:
Name Surname: Ahmet Burak Bilgin
Title: Assoc. Dr.
Specialization: Eye Diseases-Retinal Diseases
Work Address: Akdeniz University Department of Eye Diseases
E-mail address: gozdrburakbilgin@gmail.com
Phone number: 0546 547 3336
Researcher:
Name Surname: Ramazan Sarı
Title: Prof. Dr.
Specialization: Endocrinology and Metabolic diseases
Work Address: Akdeniz University, Department of Internal Medicine
Phone number: 0242 249 6735
EC-2021DR-TR DETECTION AND CLASSIFICATION OF DIABETIC RETINOPATHY FROM POSTERIOR POLE IMAGES WITH A
DEEP LEARNING MODEL
2

05.10.2021
Version No:1.0
Assistant researcher:
Name Surname: Mehmet Erkan Doğan
Title: Dr. Faculty Member
Specialization: Eye Diseases-Retinal Diseases
Work Address: Akdeniz University Department of Eye Diseases
Phone number: 0242 249 6640
2.4. The centers where the research will be conducted and the areas where data collection is
planned:
Akdeniz University Department of Ophthalmology - Dumlupınar Boulevard Akdeniz University
Hospital 07059, Campus, ANTALYA
Akdeniz University Department of Endocrinology and Metabolism Diseases - Dumlupınar Bulvarı
Akdeniz University Hospital 07059, Campus, ANTALYA
EC-2021DR-TR DETECTION AND CLASSIFICATION OF DIABETIC RETINOPATHY FROM POSTERIOR POLE IMAGES WITH A
DEEP LEARNING MODEL
3

05.10.2021
Version No:1.0
CONTENTS
1. PROTOCOL REVISION HISTORY
2
2. RESEARCH INFORMATION
2
2.1. Supporting Organization
2
2.2. Supporting Legal Representative
2
2.3. Medical Consultants and Associated Medical Qualified Physician
2
2.4. The centers where the research will be conducted and the areas where data collection is
planned:
3
1. ABBREVIATIONS
6
2. LOGIN
6
2.1. Research Design
6
2.2. References and Literature full text
8
3. STUDY SUMMARY
8
3.1. Summary of Clinical Findings
8
3.2. Working Schedule and Duration
9
3.3. Flow Chart of Study
9
3.4. Rationale / Importance and Contribution of the Research
9
3.5. Success Criteria of Study
10
3.6. The benefit of the Work and Possible risks and precautions to be taken against harm:
11
3.7. Type of Research
11
3.8. Research Population
12
3.9. Inclusion Criteria for Volunteers
12
3.10. Exclusion Criteria for Volunteers
12
3.11. Withdrawal of Volunteers from the Study - Exclusion Criteria
13
4. WORKING TREATMENTS
13
4.1. Route of administration, dosage, dosing regimen, and Duration of Treatment of Subjects 13
4.2. Drugs Not Allowed Before And/or After Research
14
4.3. Volunteer Compliance Monitoring Methods
14
5. RELIABILITY STATEMENT
14
5.1. Device Reliability Assessment
14
5.2. Adverse device effect (ADE)
14
5.3. Unexpected Adverse Device Effects (UADE)
14
5.4. Adverse device effect notification - Unexpected Adverse Device Effects notification
14
6. STATISTICS
15
6.1. Analysis
15
6.2. Determination of Sample Size
15
6.3. Measurement and evaluation of effectiveness parameters
16
7. WORK MANAGEMENT
16
EC-2021DR-TR DETECTION AND CLASSIFICATION OF DIABETIC RETINOPATHY FROM POSTERIOR POLE IMAGES WITH A
DEEP LEARNING MODEL
4

05.10.2021
Version No:1.0
7.1. Recording Data
16
7.2. Protocol Changes
16
7.3. Storage of operating data
16
7.4. Sharing and publishing results
16
7.5. Patient Confidentiality
17
8. PERSON AND/OR INSTITUTION WITH DIRECT ACCESS AUTHORITY
17
9. ETHICAL ASSESSMENTS
17
9.1. Examining the Ethical Issues of Study
17
9.2. Be Informed
18
10. FINANCE AND INSURANCE
18
11. PUBLICATION POLICY
18
12. SIGNATURE OF THE APPLICANT
19
Researcher Commitment
20
ANNEX-1 ANALYSIS PLAN
21
EC-2021DR-TR DETECTION AND CLASSIFICATION OF DIABETIC RETINOPATHY FROM POSTERIOR POLE IMAGES WITH A
DEEP LEARNING MODEL
5

05.10.2021
Version No:1.0
1. ABBREVIATIONS
DR: Diabetic retinopathy
mtmDR: more than mild diabetic retinopathy
vtDR: vision-threatening DR - Vision threatening diabetic retinopathy
ML: Machine learning
ORF: Case Report Form
ADE: Adverse Device Event
UADE: Unanticipated Adverse Device Effects
EDC: Electronic Data Capture
2. LOGIN
2.1. Research Design
This study is a prospective safety/efficacy study.
In the European Union, approximately 32.7 million adults were found to have diabetes in 2017. It
is also estimated that 12.8 million people have undiagnosed diabetes. The age-standardized
prevalence of diabetes among adults was 6% on average in EU countries in 2017. This rate is
11.9% for our country, which corresponds to approximately 9.6 million diabetic patients. Diabetes
has high morbidity and there are millions of people who need to be screened for diabetic
retinopathy (DR). The number of ophthalmologists is insufficient corresponding to the high patient
density that needs to be screened.
Diabetic retinopathy is a retinal disease that can often be stopped with an early diagnosis but can
lead to severe vision loss, including permanent blindness, in case of neglect. The diagnosis and
stage of the disease can often be made only through fundus examination or fundus photographs.
Artificially intelligent retinal diagnostic softwares present a solution to the shortage of trained
ophthalmologists, the evaluation of retinal photographs taken by an ophthalmologist in another
center and making a diagnosis (telemedicine - telemedicine); Recently, by using artificial
intelligence algorithms, it has become possible to obtain meaningful information from large
volumes of data (more from scanned patients/fundus photos) in a shorter period of time with
much less human intervention. With the artificial intelligence algorithm in which pathological
lesions are taught by using posterior pole photos of the eye, DR lesions are identified with high
accuracy and reliability, faster, and at a lower cost. This offers an excellent screening advantage
for the early diagnosis and treatment of DR and other retinal diseases.
EC-2021DR-TR DETECTION AND CLASSIFICATION OF DIABETIC RETINOPATHY FROM POSTERIOR POLE IMAGES WITH A
DEEP LEARNING MODEL
6

05.10.2021
Version No:1.0
Methods and tools to be used in the research:
i.
Fundus
photography
with
non-mydriatic
camera
and
classification
of
diabetic
retinopathy with artificial intelligence algorithm
ii.
Evaluation of images by retina specialists and comparison of results for clinical
validation of the system.
Clinical and laboratory tests to be performed:
i.
Fundus photography with a non-mydriatic camera. In this examination, a non- invasive
procedure is applied to the patient. The retinal photographs will be taken with special digital
cameras called 'fundus cameras'. In patients whose non-mydriatic images cannot be
obtained, tropicamide drops will be instilled to dilate the pupil, and then photographs will be
taken.
ii.
Pupil dilation will be achieved by instilling Tropicamide drops in both eyes of the patient,
and then 4 quadrant photographs of both eyes will be taken with a mydriatic fundus
camera.
Among the patients admitted at the Akdeniz University Endocrinology and Metabolic Diseases
Department with a diagnosis of diabetes, those who want to participate in the clinical study will be
referred to the clinical study by the endocrinology polyclinic. The authorized nurse involved in the
clinical study will explain the study to the volunteer and ask for the consent form to be signed, and
consent will be obtained. Fundus images will be taken with a non-mydriatic fundus camera by a
designated technician. Ural Telekom firm, EyeCheckup project and Assoc. Dr. It is planned to
diagnose and classify the images in terms of DR with the artificial intelligence (AI) algorithm
developed with the consultancy of A. Burak Bilgin. Images obtained using eye drops will be
evaluated by the faculty members of the Department of Ophthalmology, Retina Unit, and
compared with AI to determine sensitivity and specificity. In this way, it will be possible to screen
millions of diabetic patients who need to be screened for diabetic retinopathy in much shorter
intervals and with less manpower.
The research will be carried out in a single center at Akdeniz University Faculty of Medicine. The
control group will not be used. The EyeCheckup software used in the clinical study does not have
a CE mark. However, the Fundus cameras used to take retinal images from the patient in the
clinical study are CE-marked and EyeCheckup does not have any direct interaction with the
patient. Since the performance and effectiveness of our EyeCheckup software will be measured
in the study, the study will be conducted as a Non-CE Marked Medical Device Research
(Excluding Thesis and Academic Studies).
The duration of the study is planned to be 12 months, but the study may be terminated if the
artificial intelligence algorithm is validated and the intended number of 900 volunteer patients is
reached in a shorter time. Since fundus images of the patients will be used, follow-up time is not
required. The planned trial time is a maximum of 30 minutes per patient.
There is no other diagnosis/treatment method to be applied together. It is only a study to
diagnose DR.
EC-2021DR-TR DETECTION AND CLASSIFICATION OF DIABETIC RETINOPATHY FROM POSTERIOR POLE IMAGES WITH A
DEEP LEARNING MODEL
7

05.10.2021
Version No:1.0
2.2. References and Literature full text
2.2.1.
Ting DSW, Cheung CY, Lim G, Tan GSW, Quang ND, Gan A, Hamzah H,
Garcia-Franco R, San Yeo IY, Lee SY, Wong EYM, Sabanayagam C, Baskaran M,
Ibrahim F, Tan NC, Finkelstein EA , Lamoureux EL, Wong IY, Bressler NM,
Sivaprasad S, Varma R, Jonas JB, He MG, Cheng CY, Cheung GCM, Aung T, Hsu W,
Lee ML, Wong TY. Development and Validation of a Deep Learning System for
Diabetic Retinopathy and Related Eye Diseases Using Retinal Images From
Multiethnic Populations With Diabetes. JAMA. 2017 Dec 12;318(22):2211-2223. doi:
10.1001/jama.2017.18152.
2.2.2.
Abràmoff MD, Lavin PT, Birch M, Shah N, Folk JC. Pivotal trial of an autonomous
AI-based diagnostic system for detection of diabetic retinopathy in primary care
offices. NPJ Digit Med. 2018 Aug 28;1:39. doi: 10.1038/s41746-018-0040.
3. STUDY SUMMARY
3.1. Summary of Clinical Findings
All documents required by the standard for EyeCheckup software have been created, risk
analysis has been made and precautions have been taken against possible risks.
EyeCheckup clinical trial will be performed in a single center at Akdeniz University Hospital. The
target population is people aged 18 and over, who have been diagnosed with diabetes in the
Endocrinology Clinic of Akdeniz University Hospital and have not been diagnosed with diabetic
retinopathy (DR) before, and whose retinal image can be taken. To better align the analysis
population with the proposed target population, those who do not meet certain additional
predetermined criteria (eg: less than 18 years of age, visually impaired) will be excluded.
Volunteers will be taken to the research area allocated by the hospital at Akdeniz University.
Before any volunteers are recruited, candidates will need to verify that they understand the
research and have given written informed consent. Study; CE marking will be made to measure
the success of Artificial Intelligence software in terms of specificity and sensitivity, which detects
the presence of mild DR (mtmDR) and vision-threatening DR (vtDR) for each eye by taking retinal
images from 3 different fundus cameras and using these images.
The artificial intelligence model was trained with fundus images taken from Akdeniz University
Hospital Ophthalmology Clinic with the approval of the Akdeniz University Ethics Committee.
The publicly available Kaggle Diabetic Retinopathy Detection Competition (EyePACS dataset),
Kaggle APTOS 2019 Blindness Detection Competition dataset, Messidor, Messidor 2, IDRiD
Dataset, and DIARETDB0 - Standard Diabetic Retinopathy Database datasets were used as test
datasets. A retrospective study was performed on a total of 4000 test images (dataset). As a
result of the test studies, more than 92% specificity and 92% sensitivity were obtained. The result
of the most successful algorithm is listed below. The most successful algorithm was built on the
Google ML Platform, and this first prototype will be used as a Non-CE Marked Medical Device
Research (Excluding Thesis and Academic Studies).
EC-2021DR-TR DETECTION AND CLASSIFICATION OF DIABETIC RETINOPATHY FROM POSTERIOR POLE IMAGES WITH A
DEEP LEARNING MODEL
8

05.10.2021
Version No:1.0
Diagnosis
Truth
normal eye
97.19%
Mild/Moderate Diabetic Retinopathy
96.43%
Severe / Proliferative Diabetic Retinopathy
100.00%
Diabetic Macular Edema
100.00%
3.2. Study Schedule and Duration
Baseline (first patient enrollment): 01 November 2021
End (last patient to complete study): 01 November 2022
3.3. Flow Chart of Study
Visit
Scanning
Day 1, Visit 1
Registration/Basic
Day 1, Visit 1
Run Termination
Day 1, Visit 1
Be Informed
x
x
x
Inclusion/Exclusion Criteria
x
x
x
Demography
x
x
x
Medical History (Additional Disease
History)
x
x
x
Study Intervention
x
x
x
Dilated Ophthalmoscopy
x
x
x
Full Case Report Forms (Orfs - E-Orf)
x
x
Adverse Events
x
3.4. Rationale / Importance and Contribution of the Research
EyeCheckup is an automated software instrument designed to analyze digital color photographs
of the ocular fundus to quickly screen for diabetic retinopathy (DR).
The main purpose of the study was to detect the presence of mild diabetic retinopathy or
sight-threatening diabetic retinopathy from retinal fundus images in adult patients who applied to
the endocrinology outpatient clinic due to diabetes, to grade and to determine the sensitivity and
specificity of diabetic retinopathy, and to identify and classify the images with the developed
EC-2021DR-TR DETECTION AND CLASSIFICATION OF DIABETIC RETINOPATHY FROM POSTERIOR POLE IMAGES WITH A
DEEP LEARNING MODEL
9

05.10.2021
Version No:1.0
artificial intelligence algorithm. It is a comparison of diagnosis done by artificially intelligent
algorithms and classification of DR done by doctors.
The primary efficacy endpoint in the study was that EyeCheckup's intended image capture
workflow could be successfully used in conjunction with the EyeCheckup device to diagnose 80%
or
more
of
the
participants
for
more
than
mild
diabetic
retinopathy
(mtmDR)
and
vision-threatening diabetic retinopathy (vtDR) separately with a minimum 90% sensitivity and
90% specificity.
The secondary efficacy endpoint in the study was
Additional non-primary analyzes will also be performed to assess the following general
conditions:
●
Positive Predictive Value of EyeCheckup test for mtmDR and vtDR,
●
Negative Predictive Value EyeCheckup test for mtmDR and vtDR.
●
Positive and negative odds ratios for mtmDR and vtDR.
The study will include approximately 900 people with diabetes, excluding exclusions. They will be
recruited voluntarily through 1 clinical center. Two photographs of the ocular fundus (one optic
disc centered, one macula centered) will be obtained per eye using a non-mydriatic eye fundus
camera from both eyes of the participants. These images will be analyzed by EyeCheckup. Then,
pupil dilation will be achieved by instilling Tropicamide drops in both eyes of the patient and 4
quadrant photographs of both eyes will be taken with a fundus camera. Ophthalmologists will
then evaluate and record the condition of the eyes based on images obtained with pupil dilation.
There is no similar or equivalent product in our country. It is the first locally produced and
developed product. Equivalent FDA-approved and CE-marked products are available abroad.
Similar clinical studies have been conducted with equivalent products abroad. Detection and
classification of diabetic retinopathy is planned in the first version of our product. In the next
stages, we aim to detect more than 20 diseases, including glaucoma, cataract, and age-related
macular degeneration, by developing the product. Most similar products only work in the field of
diabetic retinopathy.
3.5. Success Criteria of Study
Three different doctors will evaluate the volunteer's fundus images taken from fundus cameras
and will make a decision about the presence of DR: Not more than Mild DR, More than mild DR
(mtmDR), and Vision-threatening DR (vtDR). Decisions will be made by majority vote. The
doctors' decision will be Ground Truth for us.
EyeCheckup will decide for both eyes of each volunteer. If it is the same as the doctors' decision,
this decision will be considered correct.
Total correct number / Total number of eyes = Success rate
Our criteria for successful completion of the clinical trial: Success rate > 90%
EC-2021DR-TR DETECTION AND CLASSIFICATION OF DIABETIC RETINOPATHY FROM POSTERIOR POLE IMAGES WITH A
DEEP LEARNING MODEL
10

05.10.2021
Version No:1.0
At the same time, the Specificity and Sensitivity values should be calculated separately for
mtmDR and vtDR and each should be above 90%.
Eye doctors (Normal, mtmDR, vtDR) make their decisions according to the ETDRS guidelines.
Main signs to check for: Microaneurysms, Intreretinal hemorrhages (Intraretinal hemorrhage),
Venous beading, IRMA, Neovascularization, Intravitreal hemorrhage, Preretinal hemorrhage
3.6. The benefit of the Research, Possible risks and precautions to be taken against harm:
Benefits;
According to the 2018 data from the World Health Organization, 285 million people worldwide
have a visual impairment. It is known that more than 80% of all visual disorders can be prevented
or treated. The retinal examination should be performed by an ophthalmologist and/or retinal
specialist in order to prevent retinal-related visual disorders by making an accurate diagnosis, but
people only consult an ophthalmologist when they feel any discomfort. Typically, by the time
symptoms appear, the disease has reached advanced stages and requires a variety of expensive
medical and surgical treatments. Unfortunately, at this stage, the disease often results in severe
and permanent visual impairment and vision loss.
EyeCheckup analyzes the patient's fundus/retinal health with images taken from the fundus
camera. If it sees a problem, it directs the patient to the doctor, if not, it recommends scanning
again after an appropriate period of time recommended within the AAO PPP. Thanks to the early
detection of eye problems, early diagnosis and treatment can prevent vision loss and visual
impairment. The most important aim of EyeCheckup is the early diagnosis of diabetic retinopathy
and thus the prevention of possible vision loss and blindness. Secondary objectives of
EyeCheckup are using less trained manpower and screening for DR in populations without
access to an ophthalmologist.
Possible Risks;
The images used in our study do not pose any risk or harm to the patients. Only fundus images of
the patients will be evaluated. The images will be recorded anonymously, and doctors or the
Artificial Intelligence algorithm will not be able to access the identity and health information of the
patients. There will be no harmful use of personal data or health information.
The study is expected to pose minimal risk to both camera operators and participants, as the
camera systems used have been previously approved by the FDA for ocular fundus photography,
and tropicamide 1% eye drops for pupil dilation is an FDA-approved drug product and have been
shown to have minimal side effects.
3.7. Type of Research
In this study, the reliability and effectiveness of our EyeCheckup software will be measured.
Therefore, the study will be carried out as Medical Device Research (Excluding Thesis and
Academic Studies) Without CE Mark, as determined by TITCK.
EC-2021DR-TR DETECTION AND CLASSIFICATION OF DIABETIC RETINOPATHY FROM POSTERIOR POLE IMAGES WITH A
DEEP LEARNING MODEL
11

05.10.2021
Version No:1.0
3.8. Research Population
All adult patients (male-female) aged 18 years and over who were diagnosed with diabetes but
not diagnosed with diabetic retinopathy by the endocrinology outpatient clinic.
3.9. Inclusion Criteria for Volunteers
Fundus images of 900 volunteer patients with evaluable quality, with or without pathology, who
applied to Akdeniz University Endocrinology and Metabolism Diseases Department with the
diagnosis of diabetes, who were not diagnosed with Diabetic Retinopathy before and whose
images can be taken from the posterior polar region of the eye with a fundus camera, will be
included in the study.
●
Diabetes must have been diagnosed
●
Diabetic Retinopathy must not be diagnosed yet,
●
The participant must understand the study and have signed informed consent,
●
No history of any other retinal vascular disease, cataract, or other diseases that may
affect the appearance of the retina or optic disc (refractive defect and ocular surface
disease are allowed),
●
No history of intraocular surgery other than cataract surgery, ocular laser therapy for any
retinal disease, or ocular injection for diabetic macular edema or proliferative disease,
●
No ambient blur preventing good retina photography
●
Persons aged 18 and over are included in the study.
If at least one of these conditions is not met, the participant is not included in the study.
[one] Inan, UU, F. Ozturk, and SS Ermis, Pharmacologic pupil dilation in diabetic patients. Retina, 2003. 23(2): p. 254-6
3.10. Exclusion Criteria for Volunteers
Fundus images that have corneal or lens opacification that will prevent imaging, and that are not
evaluable due to intravitreal hemorrhage or vitreous will not be included in the study. Patients
followed or treated for DR will not be included in the study.
●
If the volunteer are not diagnosed with diabetes,
●
If the potential participant cannot understand the study or informed consent,
●
If the volunteer has a history of retinal vascular disease other than diabetic retinopathy,
glaucoma, or any other disease that may affect the appearance of the retina or optic
disc.
●
If the volunteer have had previous intraocular surgery including than cataract, laser
intervention on the retina, or previous intraocular intervention for the treatment of
diabetic retinopathy,
●
If there is a permanent visual impairment in one or both eyes,
●
If under the age of 18,
EC-2021DR-TR DETECTION AND CLASSIFICATION OF DIABETIC RETINOPATHY FROM POSTERIOR POLE IMAGES WITH A
DEEP LEARNING MODEL
12

05.10.2021
Version No:1.0
●
If the participant is contraindicated for imaging with the fundus imaging systems used in
the study,
●
Exclude if there is an ambient blur in both eyes severe enough to impede good retinal
photography.
If at least one of the above conditions is present, the participant is not included in the study.
3.11. Withdrawal of Volunteers from the Study - Exclusion Criteria
The volunteer may withdraw from this study at any time without giving any reason, provided that
the principal investigator/doctor is informed. He assumes no liability for his refusal or subsequent
withdrawal from this study, and this does not in any way affect the medical care that the patient
needs now or in the future.
The researcher/doctor or the supporting organization, who is the coordinator of the study, may
exclude the subject from the study without the consent of the subject, due to their negligence in
fulfilling the requirements of the study program or in order to improve the quality of the medical
care received by the patient. Fundus images that have corneal or lens opacification that will
prevent imaging, and that are not evaluable due to intravitreal hemorrhage or vitreous will not be
included in the study. Patients already diagnosed with DR will not be included in the study.
Participants withdrawing from the study for a serious or non-serious ACE should be monitored by
the investigator until the clinical outcome of the ADE is determined. Any participant experiencing
an ADE may withdraw from the study at any time at the discretion of the investigator. ADE(s)
should be noted with appropriate ORFs and participant progress should be monitored until the
ADE is resolved. The medical monitor or project manager should be informed.
4. STUDY TREATMENTS
4.1. Route of administration, dosage, dosing regimen, and Duration of Treatment of Subjects
No intervention will be made to the body. Since the medical device is software, there will be no
drug use and no dosage. Pupil dilatation will be provided by instilling 1-2 drops of Tropicamide
drops into both eyes to enlarge the pupil. Since tropicamide drops are eye drops that are always
used during routine eye examinations, they will not be considered a drug intervention in a clinical
study. The processing time is equal to the growth time of the pupil and the time to obtain 2
images from both eyes from the fundus camera. While it takes about 20-30 minutes for the pupils
to grow, it takes 1-2 minutes for the photos to be taken. These procedures are not the application
of the EyeCheckup product to the patient; It is the process of obtaining retinal images with the
standard method using CE-marked eye drops and a CE-marked fundus camera.
There is no follow-up period, as no follow-up is required for the patient whose fundus image was
taken. Since the device is not used for treatment purposes, there is no treatment period. For the
patient with any finding in their eyes, a referral is made to the doctor. This process is advanced by
the doctor.
EC-2021DR-TR DETECTION AND CLASSIFICATION OF DIABETIC RETINOPATHY FROM POSTERIOR POLE IMAGES WITH A
DEEP LEARNING MODEL
13

05.10.2021
Version No:1.0
4.2. Drugs Not Allowed Before And/or After Research
There are no drugs or treatments that are required or are not authorized prior to the trial. Since
only fundus images of the patient will be used, there is no drug use that will affect image
acquisition. Before image acquisition, only 1-2 drops of Tropicamide will be instilled into the eye.
There is no drug or treatment application after the fundus image is taken.
4.3. Volunteer Compliance Monitoring Methods
Follow-up and treatment of volunteers after fundus photography is not planned. The study will
continue through the photographs obtained from the volunteer.
5. RELIABILITY STATEMENT
5.1. Device Reliability Assessment
Ongoing safety monitoring will be performed by the responsible person who will regularly review
reported adverse effects (AEs) or unexpected adverse device effects (UADEs) from the clinical
trial site and promptly report them to the sponsor.
5.2. Adverse device effect (ADE)
Adverse
device effect (ADE) means an adverse event associated with the use of an
investigational medical device.
All adverse events that are suspected, observed, or voluntarily reported causally related to the
study will be recorded on the adverse event page of the electronic case report form.
5.3. Unexpected Adverse Device Effects (UADE)
Unexpected adverse device effect (UADE) is any serious adverse effect on health or safety, or
any life-threatening problem or death, resulting from or associated with its use, not previously
identified in nature, severity, or extent.
UADEs occurring on the visit will be monitored to adequately assess the participant's safety or
until the event has stabilized. When the case is resolved, a resolution date should be written on
the case report form. All UADEs will be followed up until resolution or until the participant is
medically stable. All adverse events that are suspected, observed or voluntarily reported causally
related to the study will be recorded on the adverse event page of the case report form, then
entered into the electronic data capture (EDC) system.
5.4. Adverse device effect notification - Unexpected Adverse Device Effects notification
“Adverse Device Effect” notification will be made to the Ministry and Ethics Committees in
accordance with the provisions of the relevant legislation.
All recordable ADEs will be entered into the security data system within 7 days of identification.
Recordable UADEs will be entered into the data system within 24 hours of identification. If there
EC-2021DR-TR DETECTION AND CLASSIFICATION OF DIABETIC RETINOPATHY FROM POSTERIOR POLE IMAGES WITH A
DEEP LEARNING MODEL
14

05.10.2021
Version No:1.0
are any technical difficulties, UADE will be notified by telephone or fax communication. This data
will be entered into the electronic data capture (EDC) system as soon as the system is available.
6. STATISTICS
6.1. Analysis
Fundus images of 900 volunteer patients with evaluable quality, with or without pathology, who
applied to Akdeniz University Endocrinology and Metabolism Diseases Department with the
diagnosis of diabetes, were not diagnosed with Diabetic Retinopathy before, and whose images
can be obtained from the posterior polar region of the eye with a fundus camera, will be included
in the study, which will be carried out as a single-center study. The sample size was determined
by Akdeniz University Statistics Consultancy Application and Research Center, taking into
account the incidence of diabetic retinopathy in diabetic patients. Upon confirmation that the AI
algorithm is working successfully, the trial will be terminated when the intended number of 900
patients is reached. The success criterion of the study is specified in Article 3.5.
The classification performance of the AI algorithm will be evaluated with reference to the
diagnosis made by the doctors. Gold standard test results will be obtained based on the
diagnosis from three different doctors. At this stage, the compatibility between the diagnosis from
the doctors will be evaluated according to the Kappa statistics, and the similarity of the decisions
made by the doctors with each other will be examined. The diagnoses of the patients will then be
evaluated together with the results of the AI algorithm, and performance measures of the AI
algorithm (sensitivity, selectivity, positive predictive value, negative predictive value, overall
accuracy, etc.) will be obtained. Whether the performances of the AI algorithm are clinically
acceptable or not will be evaluated with ratio tests (single sample population ratio significance
test, Fisher Exact test, Chi-square tests, etc.) according to the threshold values to be determined
by experts. A data analysis software will be selected among various software (R, SPSS,
TURCOSA, Stata, etc.) to do the statistical analysis in accordance with the purpose. For
statistical analysis, the level of significance will be taken as 0.05. (Annex-1 Analysis Plan)
6.2. Determination of Sample Size
The sample size was calculated by evaluating whether the AI algorithm achieved a success rate
of more than 90%, considering 5% Type-I error, 80% power, and different effect sizes listed
below.
Proportion Proportion
Given H0
Given H1
Target
Actual
Reject H0
Power
N
(P0)
(P1)
Alpha
Alpha
Beta If Z>=This
0,8033
5341
0,9000
0,9100
0,0500
0,0511
0,1967
1,6449
0,8002
2286
0,9000
0,9150
0,0500
0,0520
0,1998
1,6449
0,8015
1255
0,9000
0,9200
0,0500
0,0525
0,1985
1,6449
0,8004
778
0,9000
0,9250
0,0500
0,0530
0,1996
1,6449
0,8012
523
0,9000
0,9300
0,0500
0,0541
0,1988
1,6449
0,8047
376
0,9000
0,9350
0,0500
0,0545
0,1953
1,6449
0,8092
284
0,9000
0,9400
0,0500
0,0539
0,1908
1,6449
0,8076
212
0,9000
0,9450
0,0500
0,0565
0,1924
1,6449
0,8004
164
0,9000
0,9500
0,0500
0,0553
0,1996
1,6449
EC-2021DR-TR DETECTION AND CLASSIFICATION OF DIABETIC RETINOPATHY FROM POSTERIOR POLE IMAGES WITH A
DEEP LEARNING MODEL
15

05.10.2021
Version No:1.0
Considering the situation marked in the table above as an example, if at least 778 people are
included in the study, the case that the AI algorithm will achieve at least 2.5% above the 90%
level can be detected at 80% power and 5% type-1 error level. If the success rate of the AI
algorithm increases at a level of 2.5% or less from 90%, this difference will not be statistically
significant. (Annex-1 Analysis Plan)
6.3. Measurement and evaluation of effectiveness parameters
The Diabetic Retinopathy diagnosis of EyeCheckup and Diabetic Retinopathy diagnosis results
made by specialist doctors will be compared, and evaluation and statistical analysis will be made
by the researchers.
7. STUDY MANAGEMENT
7.1. Recording Data
The site coordinator will ensure that all data obtained during patient visits is quickly entered into
the electronic case report form in accordance with other specific information provided. The
explanation for not entering any required data should be stated on the appropriate page.
The site coordinator and the principal investigator must sign with a digital signature to confirm the
accuracy of the information entered in the electronic case report form, thus declaring that they
take responsibility for the accuracy of all case record data.
7.2. Protocol Changes
If a change to the protocol is required, it will be submitted to the ethics committee for approval,
and the investigator will ensure that the change is referenced and approved in a timely manner.
7.3. Storage of operating data
The investigator must retain all legal/study documents, signed informed consent forms (BGOF),
electronic case report forms, and source documents for ten years after study completion.
As the owner of the data, the coordinating investigator must keep all other documentation of the
study for as long as the product is available. These documents and procedures will include all
written reflections on the protocol and procedures, original copies of case report forms for each
study patient, any raw/source data provided by the investigator, the final report, and inspection
certificates, if available.
It will be guaranteed that the highest possible accuracy is achieved during the processing of the
data. The collected data is compiled and analyzed within the framework of statistical models.
According to this analysis, a Clinical Research Report is prepared.
7.4. Sharing and publishing results
All information obtained as a result of the study should be treated as CONFIDENTIAL. Before the
conclusion of the abstract, article, or visual presentation, it is acceptable that the results of this
study will be shared in scientific meetings or publications if the researchers and the coordinator
EC-2021DR-TR DETECTION AND CLASSIFICATION OF DIABETIC RETINOPATHY FROM POSTERIOR POLE IMAGES WITH A
DEEP LEARNING MODEL
16

05.10.2021
Version No:1.0
researcher mutually agree on the contents and conclusion comments. This agreement also
applies to any changes requested later by reviewers or journal editors.
7.5. Patient Confidentiality
All personal information collected and processed in accordance with the purpose of this study will
be handled by the researcher and their team, with appropriate measures to ensure the
confidentiality of this data, and in accordance with applicable national and/or local laws and
regulations regarding personal data protection.
In our clinical research, the records that will show that the identity of the volunteers are kept
confidential, and not disclosed to the public, and the identity of the volunteers remains
confidential even if the research results are published.
Viewers, pollers, the Ethics Committee, the Ministry of Health, and other relevant health
authorities can directly access the original medical records. By signing the informed voluntary
consent form, only the persons and institutions mentioned are granted access.
8. PERSON AND/OR INSTITUTION WITH DIRECT ACCESS AUTHORITY
There is no person and/or institution with direct access authority. The only authorized institution is
URAL Telekom.
9. ETHICAL ASSESSMENTS
9.1. Examining the Ethical Issues of the Study
Due to its design, this study does not interfere with the diagnosis and treatment approaches of
physicians. The procedures performed within the scope of the study will be carried out in
accordance with the revised Declaration of Helsinki and the ethical rules determined by all
national and international regulations.
Ethics committee and TR Ministry of Health approval letters will be sent to the participating
institution before the study is started.
The formation of Ethics Committees should comply with the relevant Turkish laws, regulations,
and guidelines.
Fidelity: Data will be obtained from observations using scientific methods only. Interpretation and
evaluation of data will be based on scientific methods. Any unobtained result will not be shown as
a research result.
No Harm in Scientific Research: Volunteers will not be harmed in any way during the research
process. Volunteers will be informed about the study and a minimum notification consent form will
be signed.
Responsibilities and Rights: The principal investigator will inform the volunteers about the
research findings and warn them about possible harmful practices.
EC-2021DR-TR DETECTION AND CLASSIFICATION OF DIABETIC RETINOPATHY FROM POSTERIOR POLE IMAGES WITH A
DEEP LEARNING MODEL
17

05.10.2021
Version No:1.0
Authors: The names of all researchers are published in proportion to their contributions. The
names of people who did not contribute effectively to the planning, execution, and preparation of
the research cannot be published in the list of authors.
Citations and Citations: The books, journals, and all kinds of citation sources used in the
publication of the research results will be indicated. Any work and information cannot be used or
published without permission and reference.
9.2. Be Informed
Written informed consent will be obtained from each patient prior to entry into the study, that is,
prior to any screening evaluation or any study-related activity. For patients whose written informed
consent cannot be obtained, the patient's legal guardian or guardian must sign the informed
consent form. This form will be received from each individual in accordance with the
recommendations of the revised Declaration of Helsinki. An independent staff member will
explain the form, purpose, and risks of the study. The patient will then be given sufficient time to
consider the study before deciding whether or not to participate. In all cases, the patient will be
given a copy of the signed and voluntary informed consent form.
10. FINANCE AND INSURANCE
EyeCheckup is a software, it has no contact or interaction with the patient. Our study will be
performed with fundus images taken from 900 volunteer patients diagnosed with diabetes from
CE-marked fundus cameras. The EyeCheckup software will not be tested directly on the patient
and has no contact with the patient. It will be installed on the computer and will evaluate the
photos using artificial intelligence algorithms on the cloud.
Since our EyeCheckup software does not have an EC certificate, all volunteers who will
participate in the research will be insured by Ural Telekom.
11. PUBLICATION POLICY
●
All citations used in the article are referenced.
●
The research result in the article is presented openly and honestly, without fabrication,
false or inappropriate data manipulation.
●
Scientific contribution to the article is guaranteed.
●
Names of people who contributed to the article and consent to be named as co-authors
are included in the article.
●
If deemed necessary, access to the data sets in the article is provided.
●
The author is responsible for the opinions expressed in the article.
EC-2021DR-TR DETECTION AND CLASSIFICATION OF DIABETIC RETINOPATHY FROM POSTERIOR POLE IMAGES WITH A
DEEP LEARNING MODEL
18

05.10.2021
Version No:1.0
12. SIGNATURE OF THE APPLICANT
With this application form, on behalf of the applicant myself;
●
All documents submitted in the application are the same as the originals,
●
The information provided in the application is correct,
●
The research will be carried out in accordance with the protocol, the relevant legislation
and the principles of good clinical practice;
●
The research important change application is not made to more than one of the ethical
committees established within the scope of the relevant Regulation,
●
I certify that the proposed change is suitable for implementation.
Applicant
(The applicant must be an authorized signatory.)
Name and surname:
Date (as day/month/year): …/…/….
Signature:
EC-2021DR-TR DETECTION AND CLASSIFICATION OF DIABETIC RETINOPATHY FROM POSTERIOR POLE IMAGES WITH A
DEEP LEARNING MODEL
19

05.10.2021
Version No:1.0
Researcher Commitment
Research Protocol Name: DETECTION AND CLASSIFICATION OF DIABETIC RETINOPATHY
FROM BACK POLE PHOTOS WITH A DEEP LEARNING MODEL
Protocol Number: EC-2021DR-TR
Version: v1.0
I have carefully read this study protocol and confirm that it contains all the information necessary to
conduct the study. I declare and accept that I will conduct the study in accordance with the protocol
ICH GCP, the Good Clinical Practices Guidelines and the regulations of the applicable competent
authorities.
Name and surname:
Date (as day/month/year): …/…/….
Signature:
EC-2021DR-TR DETECTION AND CLASSIFICATION OF DIABETIC RETINOPATHY FROM POSTERIOR POLE IMAGES WITH A
DEEP LEARNING MODEL
20

05.10.2021
Version No:1.0
ANNEX-1 ANALYSIS PLAN
The diagnosis of the patients will be made by 3 doctors by evaluating the images and a joint
diagnosis decision will be made by combining the diagnosis made by the doctors for each patient.
This diagnosis will be considered the "Gold Standard" for the study. With the developed AI
algorithm, it will be investigated to what extent the diagnosis made by the doctors can be predicted.
The AI algorithm developed for the diagnosis of DR is expected to achieve a success rate of 90%
or more. A success above this level will show that the developed AI algorithm will be usable. For
this purpose, whether the AI algorithm achieves a success rate of 90% or more will be investigated
with the one-way hypothesis pair given below.
𝐻0: 𝑃= 0. 90 𝐻1: 𝑃> 0. 90 
The sample size was calculated by evaluating whether the AI algorithm achieved a success rate of
more than 90%, considering 5% Type-I error, 80% power, and different effect sizes listed below.
Proportion Proportion
Given H0
Given H1
Target
Actual
Reject H0
Power
N
(P0)
(P1)
Alpha
Alpha
Beta If Z>=This
0,8033
5341
0,9000
0,9100
0,0500
0,0511
0,1967
1,6449
0,8002
2286
0,9000
0,9150
0,0500
0,0520
0,1998
1,6449
0,8015
1255
0,9000
0,9200
0,0500
0,0525
0,1985
1,6449
0,8004
778
0,9000
0,9250
0,0500
0,0530
0,1996
1,6449
0,8012
523
0,9000
0,9300
0,0500
0,0541
0,1988
1,6449
0,8047
376
0,9000
0,9350
0,0500
0,0545
0,1953
1,6449
0,8092
284
0,9000
0,9400
0,0500
0,0539
0,1908
1,6449
0,8076
212
0,9000
0,9450
0,0500
0,0565
0,1924
1,6449
0,8004
164
0,9000
0,9500
0,0500
0,0553
0,1996
1,6449
Considering the situation marked in the table above as an example, if at least 778 people are
included in the study, the case that the AI algorithm will achieve at least 2.5% above the 90% level
can be detected at 80% power and 5% type-1 error level. If the success rate of the AI algorithm
increases at a level of 2.5% or from to 90%, this difference will not be statistically significant. From
this point of view, at least how high the AI algorithm will be above the threshold value should be
determined and the sample size should be taken into account over this difference. For example,
the 91% performance of the AI algorithm compared to the threshold value and investigating the
statistical significance of the 1% difference may not be clinically appropriate, however, an increase
of 2% or more may be considered clinically significant. At this point, it is important to determine the
minimum effect size (distance to 90% level) that should be taken into account in order to
EC-2021DR-TR DETECTION AND CLASSIFICATION OF DIABETIC RETINOPATHY FROM POSTERIOR POLE IMAGES WITH A
DEEP LEARNING MODEL
21

05.10.2021
Version No:1.0
demonstrate clinical success (in other words, the AI algorithm performs more than 90%). (If it is
necessary to evaluate the graph given above, the red distribution is the reference point for clinical
success, and the blue distribution is the distribution showing how much the AI algorithm can
exceed this threshold value. The effect size is the distance between the red and blue distributions.)
Statistical Analysis Plan
The diagnostic performance of the AI algorithm will be evaluated with reference to the decisions
made by the doctors. Gold standard test results will be obtained based on the diagnosis provided
by three different doctors. At this stage, the compatibility between the diagnosis from the doctors
will be evaluated according to the Kappa statistics, and the similarity of the diagnosis made by the
doctors with each other will be examined. The diagnoses of the patients will then be evaluated
together with the results of the AI algorithm, and performance measures of the AI algorithm
(sensitivity, selectivity, positive predictive value, negative predictive value, overall accuracy, etc.)
will be obtained. Whether the performances of the AI algorithm are clinically acceptable or not will
be evaluated with ratio tests (single sample population ratio significance test, Fisher Exact test,
Chi-square tests, etc.) according to the threshold values to be determined by experts. A data
analysis software will be selected amongst various softwares (R, SPSS, TURCOSA, Stata, etc.) to
do the statistical analysis in accordance with the purpose. For statistical analysis, the level of
significance will be taken as 0.05.
EC-2021DR-TR DETECTION AND CLASSIFICATION OF DIABETIC RETINOPATHY FROM POSTERIOR POLE IMAGES WITH A
DEEP LEARNING MODEL
22