Computer-based Screening for Diabetic Retinopathy
Condition: Diabetic Retinopathy
MeSH terms: Retinal Diseases; Diabetic Retinopathy
Study type: Observational

Summary: 
      The study is an Observational, Phase 0 designed to establish that the risk for diabetic
      retinopathy assigned by the RiskAnalyzer improves the reading accuracy and consistency of any
      reader and it decreases the inter-reader variability.

      Objectives: Objectives one, two, and three are arranged chronologically and in an increasing
      level of complexity as a three tiered approach to support the primary and secondary endpoint
      of the trial. Objective one is to test fully each system components of the study limited to a
      single site. Objective two is to evaluate the efficacy of the RiskAnalyzer to assign the risk
      of Diabetic Retinopathy in comparison to the gold standard. Objective three is to demonstrate
      that the reader's accuracy in grading images is improved when risk levels assigned by the
      RiskAnalyzer are made available to the reader while performing the grading of the images
      which is the primary endpoint of the trial Methods and Research Design: A network of clinical
      study sites will be established to meet the required number of cases needed as calculated by
      statistical analysis. Male and Female Subjects between the ages of 18-65 who are either
      pre-diabetic or diabetics will be eligible for participation in this study. Subjects will be
      recruited, consented, photographed and their images graded by two trained readers and
      analyzed by the RiskAnalyzer . The risk levels that are obtained from the RiskAnalyzer will
      be compared to the current gold standard practice, manual grading of each case by a reader.
      Data collected during this clinical trial will be reported to the referring physician in the
      form of a retinal screening report completed and signed by a licensed Ophthalmic professional
      and delivered to their attending physician. Risk levels for diabetic retinopathy obtained by
      use of the RiskAnalyzer will not be given to the attending physician under any circumstances
      in order to preserve standard of care for the patient.

      The sensitivity, specificity, receiver operating characteristic (ROC), and data flow process
      of the RiskAnalyzer, retinal image reading system will be analyzed and based on the current
      gold standard of a human reader. This study is a three-aims study, 24 month in length,
      prospective, case-only study of the performance of the RiskAnalyzer. The risk levels obtained
      from the RiskAnalyzer are not made visible, i.e. are not unmasked to either of the two
      readers. In year two, the risk levels obtained from the RiskAnalyzer for half of the studies
      are unmasked to the two readers while grading the image.

      Access to all study data and processes follows a role-based design. The clinical staff will
      have access only to clinical data but not the technical data. The technical team will have
      access to the technical data only but not the clinical data. The study coordinator will have
      access to all data. The use of computers will adhere to the Guidance for Industry
      Computerized Systems Used in Clinical Investigations and applicable sections of 21 CFRs part
      11.
    

Detailed description: 
      Sensitivity of retinal screening for risk stratification of DR using digital fundus
      photographs with and without the assistance of the RiskAnalyzer

      Sensitivity and specificity of retinal screening for risk stratification of DR using digital
      fundus photographs and the RiskAnalyzer

      Clinical Trial Methods and Research Design: A total of 10,000 subjects will be recruited to
      participate in this study. These subjects will be enrolled through two parallel
      collaborators, Extension for Community Healthcare Outcomes (Project ECHO) and CommuniCare
      Health Centers. Each clinic will obtain approximately 50% of the cases per year or an average
      of 50 cases per week. Subjects will be identified from clinics, CommuniCare Health Centers of
      San Antonio, TX and Project ECHO, New Mexico. CommuniCare has two associated sites, the
      Barrio Family Health Center in Southeast San Antonio, TX and the Dr. Frank Bryant Health
      Center also in San Antonio. CommuniCare serves approximately 45,000 subjects a year of which
      it is estimated that 5000 are requiring diabetic care. Project ECHO in conjunction with the
      University of New Mexico collaborates with over 20 rural clinics in New Mexico to provide
      service to underserved areas in New Mexico. A recruitment flyer and consent forms will be
      provided to each site. Clinic staff will recruit subject for the study during scheduled
      visits. Subject will be given a flyer containing information about the study and the informed
      consent document. Subjects who are interested in being enrolled in the study will be referred
      to a photographer within the site for Retinal imaging. Subjects will not receive a payment or
      stipend for participation in this study.

      There are no screening requirements for subjects to participate in this trial. Subjects must
      be diagnosed as a person with pre-diabetes or diabetes by a medical professional. There is no
      requirement or solicitation for information regarding prior or concomitant therapy. There are
      no criteria set forth for subjects to exit or be discontinued from the study once the retinal
      screening is performed. Subjects will not be withdrawn from the study once retinal screening
      has occurred. Once the subject completes retinal screening, the period of subject
      participation is concluded. Images taken from subjects may not be used in the study due to
      inadequate image quality. The number of images found to be inadequate will be tracked and
      reported at the completion of the study. Retinal screening marks both the beginning and the
      end of subject participation in this study.

      For each eye, a minimum of two posterior pole images will be collected. One image will be
      centered on the fovea and the second on the optic disc. A third image will be collected of
      the anterior segment (front of the eye). Subjects will not be pharmacologically dilated.
      Imaging will rely on natural dilation due to adaptation to darkness in the imaging room. More
      than two images per eye will be collected only when the imager (photographer) identifies any
      given image of insufficient quality for reading. Poor quality images are typically due to
      physiological reasons, such as blinking or small pupils and will not be used as part of this
      study. The retinal photographer will attempt up to 3 times to acquire images of a subject's
      eyes in order to complete the imaging protocol. The study coordinator will conduct weekly
      audits of the imaging procedure in order to ensure and document compliance with the study
      protocol.

      The Class II Medical Device that will be used for this study will be a non mydriatic or
      equivalent retinal camera. The retinal camera is intended to be used for taking digital
      images of the retina of the human eye without the use of pharmacological dilation. The
      retinal camera has the following photography modes: RGB color. A Digital Camera is mounted to
      the retinal camera to perform retinal imaging.

      A picture archival and communication system (PACS) is used to receive DICOM images,
      scheduling information and textual reports, organize and store them in an internal format,
      and to make that information available across a network via HIPAA and FDA-cleared secure user
      interfaces. The PACS is comprised of a capture component, a central server component, and a
      viewer component that is used on general purpose computing hardware. Subject information,
      procedure information, and image acquisition data is captured, sent to a server or
      alternatively, it is accepted directly from a DICOM compliant device. After receiving image
      objects and associated data, the server registers the incoming data and images with the
      archive including any associations with prior cases. The cases are presented to users with
      the correct permissions from a work list as controlled by a managing user. The image data is
      transmitted and rendered on the user's workstation using a standard web browser. Once the
      server registers a report, the report is available to the referring physician or client
      service electronically or via fax.

      Immediately after the images and Subject demographics are collected, the photographer will
      upload the data as a case to the PACS via a VPN connection established between the imaging
      workstation and PACS. If the VPN connection cannot be established at the time of imaging, the
      upload will be done at a later time, e.g. end of the clinic day. The data will reside in the
      imaging workstation for up to five days in order to assure upload. The clinical coordinator
      will conduct weekly audits of upload completion. Once in PACS, cases are read by readers
      using a built in reading template that generates a report with their findings. The study will
      not disrupt standard of care to the study subjects. Each site will follow its own standard
      procedures and care. Findings will be reported to the clinic which referred the subject for
      screening. Clinics will be responsible for follow-up care including any necessary
      intervention or therapy.

      Performance of the RiskAnalyzer: The performance of the RiskAnalyzer will be calculated by
      applying the minimally acceptable values for sensitivity (Seo) and specificity (Spo) values
      specified in advance. For purposes of this study, the specified null hypotheses values are
      Seo=0.85 and Spo=0.70. These values are based on data obtained from ophthalmic exams. The
      efficacy of trained readers using retinal photography has been demonstrated to provide the
      most sensitive screening and monitoring test for sight-threatening retinopathy, with
      sensitivities > 80%. Abramoff, et al. discuss the limits of sensitivity that can be achieved
      by any system given the fallibility of ground truth based on multiple human readers. He
      suggests that 0.85 is the limit of sensitivity that one can achieve. The specificity is based
      on the efficiency that we seek in order to make this implementation commercially viable. We
      have designed our study to achieve 0.90 statistical power and 0.99 significance level. If
      confidence limits are based on asymptotic normal distribution theory, then sample sizes can
      be based on the asymptotic variance formulae, as given by Pepe.

      Objective One: Objective one is to test fully the study data flow logistics and information
      transfer interfaces. The testing will be done entirely at one recruitment, enrollment and
      imaging site. The duration of time to complete this objective is three months.

      Materials and Research Design for Objective One: Subjects will be recruited from single site.
      All images will be routed to two readers for grading. If the two readers assign equivalent
      grades to the image, a generated report will be given to the clinic to distribute the
      referring physician. For cases in which the readers assign different grades for the same
      image, the image will be routed to an adjudicator for resolution. The adjudicator will have
      access to the reports generated by each reader, but not the risk levels from the
      RiskAnalyzer. A report from the adjudicator will be generated and reported to the clinic that
      will be responsible for distributing the report to the Subject and for any further follow-up.
      Images will also be de-identified and routed to the RiskAnalyzer. The RiskAnalyzer will
      perform its analysis of the image and generate a risk stratification of either "Low risk" or
      "High risk" for diabetic retinopathy. A report containing the risk levels assigned by the
      RiskAnalyzer will not be distributed to the referring physician, clinic or adjudicator but
      used only for statistical analysis in support of the endpoints of the trial.

      Outcomes for Objective One: Data outcomes will demonstrate the ability to recruit and enroll
      subject into the study at a single site, perform retinal imaging, upload the images to a
      PACS, have the same images graded by two different readers, when necessary routed and graded
      by an adjudicator and the ability to generate a final report. The reading process followed by
      the readers is based on the International Clinical Diabetic Retinopathy Disease Severity
      Scale. Additionally data will demonstrate that images are successfully routed to the
      RiskAnalyzer for processing.

      Objective Two: Objective two is to evaluate the efficacy of the RiskAnalyzer to identify the
      risk of Diabetic Retinopathy in comparison to the gold standard. The duration of time to
      complete this objective is nine months.

      Methods and Research Design for Objective Two: Subjects are recruited from multiple
      CommuniCare and Project ECHO sites. All images are graded by two readers and an adjudicator
      should the grading by the first and second reader not be in agreement. The reading process
      followed by the readers is based on the International Clinical Diabetic Retinopathy Disease
      Severity Scale. A consensus report generated by the readers or the adjudicator indicating
      presence and severity of diabetic retinopathy will be given to the Subject through their
      referring physician and referred to a specialist if necessary by the referring physician. The
      same images graded by the readers will also be routed to the RiskAnalyzer for processing
      which will generate a risk level of either "Low risk" or "High risk" for diabetic
      retinopathy. As the images are routed to the RiskAnalyzer, the images will be de-identified
      and sequestered until evaluated by the RiskAnalyzer. Risk levels obtained by use of the
      RiskAnalyzer will not be given to the Subject under any circumstances. The referring
      physician is also masked from the risk levels assigned by the RiskAnalyzer. Referring
      physicians are not co-located at VisionQuest Biomedical and therefore will not have any
      access to the computers or data storage area for the data.

      Outcomes for Objective Two: Statistical analysis will indicate the accuracy level of the
      RiskAnalyzer as compared to the gold standard. Statistical analysis will use sensitivity,
      specificity, AUC, and a level of agreement using Cohen's kappa coefficients.

      Objective Three: Objective three will address the primary endpoint of the trial which is to
      demonstrate that the reader's accuracy in grading images is enhanced when risk levels from
      the algorithm are made available to the reader while performing the grading of the images.
      The duration of time to complete this objective is 12 months.

      Methods and Research Design for Objective Three: The RiskAnalyzer will be used as a first
      reader. There will continue to be two human readers and an adjudicator. Ground Truth will be
      established by either the agreement of reader one with reader two or by an adjudicator. All
      images will be routed to the RiskAnalyzer which will generate a risk level of either "Low
      risk" or "High risk" for diabetic retinopathy. The cases will be divided into two sets and
      routed to both readers. One set of images will be routed to both readers with the algorithm
      assigned risk levels while the other set of images will be routed to both readers without the
      algorithm assigned risk level. An equivalent number of 'Low risk" and "High risk" cases will
      be selected for each set of images. Ground Truth is established by the two readers assigning
      the same grade to the same image. When different grades are assigned to the same image by two
      different readers, the cases will be routed to an adjudicator for grading and reporting. The
      adjudication will become ground truth. The cases that will be adjudicated will only be those
      that are different categories of risk stages. Adjudication will be based on the clinical
      significance of the disagreement between the readers as follows: cases within severity scales
      0 and 1 will be classified as "Low Risk" cases, while cases within the severity scales 2 and
      above will be classified as "High Risk" cases. Cases needing adjudication will be those
      differing in risk and those within the High Risk Category that differ between scales 2 and 3
      (mild and severe), and 2 and 4 (mild and proliferative Diabetic Retinopathy -PDR).

      Outcomes for Objective Three: The outcomes from objective three will establish that the
      RiskAnalyzer improves the reading accuracy and consistency between readers and decreases bias
      between human readers. Statistical Analysis of level of agreement will be performed on the
      risk assigned to the images by the readers when having and not having the risk levels
      assigned by the algorithm using Cohen's kappa coefficients.

      Expected outcomes: Telemedicine retinal screening in conjunction with risk levels from the
      RiskAnalyzer improve the accuracy of readers when grading images. The use of risk levels
      assigned by the RiskAnalyzer in assisting the readers will decrease the amount of time
      readers dedicate to the grading of each case. This may be a model to support the increase of
      access to retinal screening, support compliance with retinal screening recommendations and
      facilitate a more efficient use of the retinal specialist to evaluate and treat disease
      leading to a more cost efficient method of care.
    

Observational model: Case-Only
Time perspective: Prospective
Sampling method: Probability Sample
Study population: 
        Primary care clinics in the Southwest serving a Diabetic Community
      
Keywords: Diabetes Type One; Diabetes Type Two; PreDiabetes; Retinal Screening; RiskAnalyzer; Retinal Imaging; Algorithm; Diabetic Retinopathy; Microaneurysms; Neovascularization; Intraretinal microvascular abnormalities