Evaluating the Validity of an Eye Gaze Paradigm in Predicting Autism Spectrum Disorder
Condition: Autism Spectrum Disorders
MeSH terms: Autistic Disorder; Autism Spectrum Disorder; Child Development Disorders, Pervasive
Study type: Observational

Summary: 
      The primary purpose of the present study is to evaluate the diagnostic validity of eye
      tracking measurements acquired during viewing of socially-relevant stimuli in predicting ASD
      diagnosis. The secondary purpose was to explore the potential prognostic value of eye
      tracking measures through cross-sectional associations with non-verbal cognitive ability.

      Deficits in eye gaze are a hallmark sign of autism. A large and growing body of research
      supports the ability of eye-tracking based measurements to sensitively discriminate
      individuals with ASD and healthy participants. These investigations have identified that the
      core deficit in autism as disruption of social attention, reflecting an inability to
      appropriately engage and track socially- and emotionally-relevant aspects of the visual
      world. Thus, eye gaze tracking, acquired during viewing of socially-relevant stimuli, may be
      a useful approach to identifying objective markers of ASD. Eye tracking also carries the
      advantages of being less intrusive and expensive than MRI and genetic testing and
      specifically focuses on the core neurobehavioral characteristics of ASD - abnormalities in
      social attention.

      After diagnosis of ASD, key clinical tasks in young children involve determining an accurate
      prognosis and tracking the progress of early interventions. Currently, the only prognostic
      indicators are clinical observations (subjective and expensive) and non-verbal cognitive
      ability testing (difficult to acquire, time-consuming, unavailable in many settings).
      Recently, eye gaze tracking was found to predict functional outcomes. Thus, in addition to
      being an objective marker for ASD, eye tracking measurements have potential to be useful for
      predicting cognitive and functional outcomes. Similarly, the only available methods for
      tracking treatment progress are parental reports (highly subjective), clinical observations
      (subjective and expensive), and cognitive measurements (expensive and unavailable in many
      settings. This study will evaluate, using cross-section data, the potential for eye tracking
      data to serve as a proxy for non-verbal cognitive ability scores in determining prognosis for
      ASD-affected children. Additionally, this study will evaluate the test re-test reliability of
      eye tracking parameters that can potentially be used to track treatment progress.
    

Detailed description: 
      The current study will occur in four phases: pilot testing, development, validation, and
      re-test. In each phase, participants will view a visual attention stimulus with social
      elements (social attention paradigm) while eye tracking measurements are remotely acquired.
      The visual attention paradigm will be refined in the pilot testing but will remain the same
      for development, validation, and re-test phases. The entire process, including calibration
      and viewing of the visual paradigm, will take about 15 minutes. The text below describes each
      phase in detail and the reviews specific methodology for the social attention paradigm and
      eye tracking procedures.

        1. Pilot Phase. Pilot testing of the social attention paradigm used to elicit eye gaze
           measurements to ensure they are sufficiently engaging across the ages of children
           participating. The visual attention paradigm during this phase is expected to be 10-12
           minutes. This phase will also be used to determine optimal strategies for maintaining
           attention throughout the testing period. Initial data may also provide some insights
           into the most discriminating stimulus elements to include in future phases.
           Investigators anticipate recruiting approximately 10 ASD and 10 non-ASD participants in
           this phase. The pilot phase is expected to require 2.5 months of recruitment and 0.5
           months of data reduction and analysis.

        2. Development Phase. Once pilot testing is completed, 30 ASD-affected and 30 non-ASD
           children ages 1.5 to 18 will be recruited. Each child will complete the social attention
           paradigm while their eye gaze is remotely tracked. Eye gaze data is then scored for more
           than 100 parameters and these scores are entered into a database that includes
           diagnostic and cognitive information about each participant. The eye tracking data is
           then analyzed to create the diagnostic and prognostic algorithms that will refine the
           system's diagnostic sensitivity and specificity in the ASD population. The development
           period will require 6 months of data collection, followed by 3 months of scoring and
           algorithm computation.

        3. Validation Phase. Collection of the validation sample will include 60 ASD-affected and
           60 non-ASD affected children ages 1.5-18. The validation phase will use the same visual
           social attention paradigm and methods used in the development sample. The validation
           period will overlap algorithm development and require 8 months of data collection and 1
           month of analysis.

        4. Re-Test Phase. Participants in the re-test phase will include 30 ASD and 30 non-ASD
           children recruited in the first half of the validation phase. These participants will be
           explained in the validation phase that investigators will be re-recruiting them for the
           re-test phase approximately 3-6 months following their initial testing during the
           validation phase.

      The target sample size varies depending on the phase of the study. Below are the target
      sample sizes for each phase:

      Pilot Phase = 10 ASD and 10 Non-ASD Development Phase = 30 ASD and 30 Non-ASD Validation
      Phase = 60 ASD and 60 Non-ASD Re-Test Phase = 30 ASD and 30 Non-ASD (from the validation
      phase) The study population is individuals with autism spectrum disorder, or a clinical
      diagnosis of another developmental or psychiatric disorder (developmental/psychiatric
      controls), or have no specific developmental or psychiatric diagnosis (healthy controls),
      ages 1.5 (18 months) to 18 (120 months) at time of consent.

      Eye gaze data will be collected using a remote eye tracker from Sensori-motoric instruments
      (SMI). Remote eye tracking offers minimal invasiveness to the viewer's field of view and
      collects time-stamped, 3D eye position, and binocular gaze and pupil data at a sampling rate
      of 120 Hz. Eye gaze capture is automatically calibrated to 2/5/9 points and provides position
      accuracy to 0.5° at a 60cm viewing distance. Gaze tracking data, screen recordings, user
      events, and gaze position will be recorded simultaneously. Data will be analyzed with
      emphasis on areas-of- interest and dwell time on specific targets on a second-by-second
      basis. Examples of the types of measures captured include: dwell time to any area of the
      stimulus, dwell time to the face and non-face regions of a human form on the video, fixation
      shifts between stimuli.

      Additional tests and demographic data will be collected from standard of care autism
      diagnostic and behavioral health diagnostic appointments that can be found in the medical
      record.

      Pilot Analyses. In the pilot phase, investigators will compute the effect size (Cohen's d)
      between ASD and non-ASD participants for each of the eye gaze parameters acquired for each of
      the individual stimuli in the social attention paradigm. Stimulus elements eliciting the
      largest discrimination between ASD and non-ASD patients will be retained in the development
      phase.

      ASD Diagnostic Algorithm Analyses. In the development phase, all of the eye gaze measurements
      acquired from the social attention paradigm will be included as predictor variables in a
      random forest analysis. This analysis permits evaluation of the discriminative ability of a
      large number of variables in data sets with a modest number of cases. The variables with
      highest importance scores, indicating good diagnostic discrimination, will be entered into a
      logistic regression analysis with ASD diagnostic status (ASD vs. non-ASD) as the dichotomous
      dependent variable. Significant predictors will be retained and coefficients from the
      retained predictors will serve as the diagnostic algorithm.

      The diagnostic algorithm scores will then be submitted to Receiver Operating Characteristic
      (ROC) curve analyses to provide detailed evaluation of sensitivity and specificity of the
      algorithm in the detection of ASD. Areas under the curve of >.90 are expected, indicating
      strong diagnostic validity.

      Prognostic Algorithm Analyses. To identify a prognostic algorithm, a similar process will be
      conducted with all the available eye tracking measurements as predictors and non-verbal
      cognitive ability scores (dichotomized at <70 and 70 and above) as the dependent variable in
      random forest analyses. Non-verbal cognitive ability scores will be dichotomized based on
      previous data suggesting that individuals with ASD and intellectual disability show worse
      outcome. The predictor variables with highest importance, indicating good diagnostic
      discrimination, will then be entered into a logistic regression analysis with non-verbal
      cognitive ability (<70, >=70) as the dichotomous dependent variable. Significant predictors
      will be retained and coefficients from the retained predictors will serve as the prognostic
      algorithm. The prognostic algorithm scores will then be submitted to Receiver Operating
      Characteristic (ROC) curve analyses to provide detailed evaluation of sensitivity and
      specificity of the algorithm in the detection of non-verbal cognitive disability. Areas under
      the curve of >.80 are expected, indicating good validity in predicting cognitive disability.
      Non-verbal cognitive ability will be the primary focus of these analyses because of its
      documented relationship with outcome in individuals with autism. However, similar analyses
      will be computed for verbal cognitive ability and adaptive function scores. Investigators
      will also conduct the above analyses using continuous measurements as dichotomization may
      unnecessarily deflate validity.

      Treatment Tracking Algorithm Analyses. To identify a treatment tracking algorithm,
      investigators will simply combine the eye tracking measurements identified in the diagnostic
      and prognostic algorithm into a single treatment tracking algorithm. To evaluate test re-test
      reliability of these measurements investigators will use intra-class correlation coefficients
      (model 2 - absolute agreement).
    

Observational model: Case-Control
Time perspective: Cross-Sectional
Sampling method: Non-Probability Sample
Study population: 
        ASD and Non-ASD participants will be recruited following their evaluations in the Center
        for Autism diagnostic clinics as well as from the Social Practice and Instruction Enhances
        Socialization (SPIES) program both of which are part of the Cleveland Clinic Center for
        Autism.
      
Keywords: Autism Spectrum Disorder; Eye Gaze; Remote Eye Gaze Tracking; Autism