Automatic algorithms for structural and functional change analysis from multi-modal, multi-temporal retinal fundus images

Detection and analysis of changes from retinal images is important in clinical practice, quantitative scoring of clinical trials, computer-assisted reading centers, and in medical research. In these applications, it is desirable to develop automatic algorithms to detect the changes and produce simple human understandable descriptions of the changes. In this work, a framework has been developed to analyze changes from longitudinal time series of retinal images. The framework has been specialized to analyze structural and functional changes from retinal images from three modalities: color fundus imaging, dual wavelength imaging, and fluorescein angiography. The framework presented here represents a fully-automated approach for robust detection and classification of changes. The methods are robust to: (i) spatial variations in illumination resulting from instrument limitations and changes both within, and between patient visits; (ii) imaging artifacts such as dust particles; (iii) outliers in the training data; and (iv) segmentation and alignment errors. Robustness to illumination variation is achieved by a novel iterative algorithm to estimate the reflectance of the retina exploiting automatically extracted segmentations of the retinal vasculature, optic disc, fovea, and pathologies. Robustness to dust artifacts is achieved by exploiting their spectral characteristics, enabling application to film-based, as well as digital imaging systems. False changes from alignment errors are minimized by sub-pixel accuracy registration using a 12-parameter transformation that accounts for unknown retinal curvature and camera parameters. Two kinds of changes are addressed in this research: (i) Structural changes, and (ii) Functional changes. Structural changes are associated with onset and progression of disease and/or external intervention. Structural changes considered here are: (i) changes associated with diabetic retinopathy in the non-vascular regions, (ii) changes in the vascular regions; (iii) changes associated with choroidal neo-vascularization in macular degeneration. Functional changes are associated with environmental or disease conditions. In this work, methods are developed to automatically extract changes in the blood oxygen saturation or Systemic HbO2 saturation (SO2) in the blood vessels. For the structural changes in the non-vascular regions, a multi-observer validation on eyes involving non-proliferative and proliferative diabetic retinopathies, indicated a 97% change detection rate, a 3% miss rate, and a 10% false alarm rate. The performance in correctly classifying the changes was 99.3%. A self-consistency metric, and an error factor were developed to measure performance over more than two periods. The average self consistency was 94% and the error factor was 0.06%. For vascular changes; the change analysis algorithms had 82% detection rate and 9% false positive rate. For analysis of structural changes from fluorescein angiograms, the algorithms were found to have a detection rate of 83% and 18% false positive rate. For the functional changes, the sensitivity for detecting saturation change when breathing air vs. pure oxygen was found to be 0.0226 Optical Density Ratio (ODR) units, which is in good agreement with previous manual measurements by the dual-wavelength technique, indicating the validity of the automated method.