Patent Description:
<CIT> relates to a system for detecting retinopathy is provided. The system comprises an image input module configured to receive one or more fundus images, a pre-processing module configured to apply one or more transformations to the one or more received fundus images, a feature extraction module configured to extract one or more features from the one or more transformed images using one or more Convolutional Neural Networks (CNNs) and a prediction module configured to determine stage of retinopathy by classifying the one or more extracted features using pre-stored features, wherein the pre-stored features are extracted from one or more training fundus images by the one or more CNNs and further wherein each pre-stored feature corresponds to a class which is associated with a predetermined stage of retinopathy.

<CIT> relates to systems, methods, and devices for classifying ophthalmic images according to disease type, state, and stage based on weighted-linkage of an ensemble of machine learning models. In some parts, each model is trained on a training data set and tested on a test dataset. In other parts, the models are ranked based on classification performance, and model weights are assigned based on model rank. To classify an ophthalmic image, that image is presented to each model of the ensemble for classification, yielding a probabilistic classification score-of each model. Using the model weights, a weighted-average of the individual model-generated probabilistic scores is computed and used for the classification.

<NPL>, relates to a two step textural feature extraction method, which uses feature learning ability of convolutional neural networks to extract a set of low level primitive filter kernels and then generalizes the discriminative power by forming a histogram based descriptor.

Initial steps in medical diagnosis and treatment include identifying and separating abnormal and normal subjects before referring them to specialists, and identifying a possible disease type and treatment plan or otherwise indicate abnormal scans for further analysis by the specialist. Various computer-based systems currently exist for automating the above identifications. But these each suffer from deficiencies.

For example, various binary classifiers in machine learning systems distinguish one major disease from normalcy. But such classifiers often give unreliable responses to non-target class disease data-that is, inputs including diseases other than those the classifier is trained to detect. Further, because some non-major disease data is rare, it can be difficult to find a sufficient amount of data to train the classifiers. Still further, different eye diseases can carry different types of structural and functional changes to normalcy. Thus, two or more eye diseases may co-exist in one eye. In such cases, it is difficult to accurately identify the disease types without a significantly large amount of disease training data.

Additionally, anomaly detection systems can screen abnormal eyes using normative eye data (data from normal eyes) only as the training data or as a reference-that is, by identifying eye data as abnormal if it does not match the normative data used as a reference or for training. This processes mimics how doctors can intuitively tell if any eye is normal or abnormal. As a pre-screening step, these systems can remind doctors to look into data if the data is screened as abnormal (so all data with abnormal structural changes will be screened out), while systems with multiple binary classifiers only work for the disease categories they have been specifically trained for.

According to the invention, an image processing method according to claim <NUM> is provided. The dependent claims set out particular embodiments of the invention.

Because normative data (also referred to herein as normal data) is far more readily available than pathological data, it is easier to train anomaly detection system networks with large amounts of normative data. Further, pathological data collections can be strongly influenced and/or biased by specific inclusion/exclusion criteria in study protocols, whereas healthy/normal data collections may be less biased. Such training and testing can be performed using reference/normative database data sets or data sets typical of high-volume optometry clinics and/or other screening locations (primary care, chain pharmacies, etc.).

In view of the above, the present disclosure is directed to identifying and separating abnormal (used interchangeably with 'disease' herein) and normal subjects and identifying possible disease types with images (e.g., optical coherence tomography (OCT) images of the eye) based on machine learning technologies. These machine learning technologies can include, for example, convolutional neural networks (CNN) and support vector machines (SVMs) where the training data can include only normative data. For ophthalmologic applications, such diseases may include various retinal pathologies including glaucoma (where severity is often defined by Mean Deviation (MD) score from visual field tests); diabetic retinopathy (DR), and age-related macular degeneration (AMD) (which includes several stages including early (small drusen), intermediate (medium-sized drusen), and late (either or both wet and dry atrophic AMD)). Additional conditions may also include but are not limited to Retinitis pigmentosa (RP), uveitis, ischemias, other retinopathies, and choroidal pathologies.

As suggested above, other image analysis systems focus on fundus images or en face OCT images (with a camera-like X/Y planar view), which requires a large number of volumes/eyes to train because only a single image (or a small number of images) exist per volume/eye. In contrast, the present disclosure describes the ability to use B-scan (taken in the X/Z or Y/Z plane, the Z dimension representing depth) or derived scan views and cross-sectional analysis of 3D scans. 3D scans often have between <NUM> and <NUM> (even <NUM> or more) B-scans (images) per volume, whereas en face-based analysis will often reduce a volume down to one or a small number of images thereby losing much of the available information. Thus, by preserving and utilizing multi-dimensional image data, more training data can be obtained from one eye and of rare diseases for which there is relatively less data. Further, analysis of B-scan or other feature abundant 2D images more closely corresponds with how retinal specialists traditionally have viewed 3D OCT volumes for pathologies.

By using these images/volumes, the systems and methods described herein can recognize structural changes occurring in various forms. For example, the input to a CNN can be B-scan views (full B-scans or smaller sections) where AMD structural changes are seen clearly, or the input can be thickness maps where for example glaucoma detection can be more easily seen with Retinal Nerve Fiber layer (RNFL) thickness maps. In other embodiments, the input can be texture information. A multiresolution system is also described that can accommodate structural changes occurring locally or globally.

As shown in <FIG>, the architecture of the system described herein takes images of a patient's eye <NUM>, measurements <NUM>, patient information <NUM>, and/or other related information <NUM> as inputs to a screener <NUM>. The screener <NUM> then outputs an identification that the input information is either abnormal <NUM> or normal <NUM>. When abnormal <NUM>, the system further identifies and outputs the type of abnormality <NUM> (e.g., diseases such as glaucoma, AMD, diabetic retinopathy, and the like). Here it is noted that, in the case of ophthalmological screening, the input images <NUM> can be any retinal images, such as OCT or fundus images, or multi-modality images; the measurements <NUM> may include those taken or derived from the images, for example, thickness, blood flow measurements or other related measurement such as visual acuity, and intraocular pressure (IOP; which may associated with glaucoma); and the patient and related information <NUM>, <NUM> may include, for example, age, gender, alcohol or drug use, and the like. The screening output identifications <NUM>, <NUM> may be made at a B-scan level and/or volume level.

The above system can be implemented by one or more computers including one or more processors and associated memory. The input images <NUM> and information <NUM>, <NUM>, <NUM> may be provided by an input/output interface device. Particularly, the images may be input directly by an imaging system (such as an OCT imaging system), provided over a network, provided via a transferable memory medium, or the like. In some embodiments the above system may be implemented as part of the imaging system itself. Similarly, the outputs may be provided on an attached or networked display device, stored in memory, or printed on a tangible medium.

<FIG> illustrates an example architecture of an anomaly detection screener <NUM> system and method of the present disclosure. Therein, as noted above, the inputs <NUM> can be OCT B-scans (e.g., cross-sectional images including a depth dimension) and/or measurement maps/grids and/or other processed endpoints such as a multi-resolution OCTA signal. Further, the inputs <NUM> can be from any imaging or scan type, for example OCT scans. The inputs <NUM> may also be of any object, although the present disclosure refers to retinal and other ophthalmologic images. For example, the inputs <NUM> may be <NUM> × <NUM> macular scans centered at fovea, <NUM> × <NUM> disc scans centered at disc center scans, <NUM> × <NUM> wide scans covering fovea and disc center, derived circle scans with different degrees of radii and/or types of eccentricity centered at disc center, derived oval scans with different degrees of radii and/or types of eccentricity centered at fovea, and the like. The anomaly detection process can also be extended to a partial or full 3D volume as well.

The input images <NUM> may first be pre-processed <NUM>. The pre-processing <NUM> can include different forms of processing such as image flattening by internal limiting membrane (ILM) or other boundaries, image/map smoothing and speckle noise removal with Gaussian, Bilateral or other similar filters, image/map intensity normalization, thickness ratios, breaking the input images <NUM> into image patches (portions of an image, e.g., an extracted or cropped section of an image such as a cross-sectional B-scan), image resizing, and the like. When image patches are used, one input image or volume can optionally be broken down or segmented into multiple overlapping or non-overlapping sections by, for example, segmenting the images, extracting the sections, cropping the images, or the like. Each section would then constitute an image patch that can be processed further. In this manner, each full input image <NUM> can effectively provide multiple images for anomaly detection with each section. Pre-processing <NUM> can also include data augmentation by image shifting, flipping, intensity enhancement and other augmentation methods.

Depending on the embodiment, image patches may have different sizes, resolutions, be of different types (e.g., cross-sections or projection images from different planes if taken from a volume), be from different spatial locations, and the like. Put another way, the patches may be the result of any pre-processing performed on a larger image or volume. Where image patches are from distinct spatial locations, they would not have any of the same data; however, in some embodiments patches may be spatially overlapping and thus include common data.

Following pre-processing <NUM>, the processed image inputs are fed to a machine learning system <NUM> (e.g., a deep learning-based model) for processing. The machine learning based model may be deep CNN or deep Convolutional Generative Adversarial Network (DCGAN). Briefly, CNNs and DCGANs comprise multiple convolutional layers that each may include multiple filters. These filters are convolved with input data, such as the pre-processed input images, to extract relevant features. In some embodiments, the filters are configured to look at small patches (e.g., corresponding to the size of the filter) of the image input to the machine learning system, and calculate an activation value for a pixel corresponding to the patch (e.g., a center pixel of the patch). An activation map may then be developed by applying the filter across the entire image input to the machine learning system. By applying a series of convolutional layers, edges, parts, and models of the image input to the machine learning system can be extracted. Specific parameters for the filters of the machine learning system are determined via training with the training data. A complete CNN or DCGAN model includes, but is not limited to, convolution layers (e.g., for feature extracting purposes), pooling layers (e.g., to reduce dimensionality and/or to regularize the learning process to avoid overfitting during training), Batch Normalization (for faster convergence and regularization), activation layers (e.g., to introduce non linearity in to the system), un-pooling/Transposed convolution layers (e.g., to upscale feature maps in DCGAN) and fully connected layers (e.g., for classification/decision making purpose in CNN models). As detailed in the following examples, the machine learning system of the present disclosure may detect anomalies based on a classification technique and/or an image inpainting/reconstruction technique. When both techniques are used together, it is referred herein to as an ensemble anomaly detection technique.

An example of the first approach, classification, is illustrated in <FIG>. Compared with traditional techniques, that illustrated in <FIG> further includes a feature extraction machine learning system whereby the extracted features, rather than raw images, are input to a classifier. Accordingly, the classifier can be trained only with normative data. More particularly, as discussed above, input images <NUM> may first be pre-processed <NUM>. Following pre-processing <NUM>, features are extracted <NUM> from the input images <NUM> based on deep learning models, such as a pre-trained deep learning model (e.g., by using transfer learning) or deep learning models trained from scratch. Transfer learning allows for reuse of a pre-trained model for different tasks, or for using a pre-trained model as a starting point for a new task. Pre-trained models may include CNN models, for example VGG, Inception Network models (including Inception V3), ResNet, AlexNet, and others known to those of ordinary skill in the art. The features extracted are dependent on how the feature extraction machine learning system <NUM> is trained, but may relate to image brightness, edge features, colors, shapes, geographic/physiological regions, and the like. These features are preferably ones that are indicative of an abnormality.

The extracted features are then input to a classifier <NUM> (e.g., a one-class classifier or a binary classifier), which is trained to identify 'outliers' from the normative data used to train the classifier <NUM>. In other words, features that do not appear to be normal (relative to corresponding features from the normative training data) may be identified by the classifier <NUM> as abnormal; and thus the image (or volume) from which the features were extracted, may be considered abnormal. For embodiments where an image is broken into multiple patches, each patch being input to the feature extraction system <NUM>, the outputs thereof may be supplied to multiple classifiers <NUM>, which in some embodiments can be ensembled. Ensemble classifiers operate in parallel and produce an output based on the outputs of each of the component classifiers within the ensemble. For example, the output of an ensemble classifier may be an average of the classifiers making up the ensemble.

In one embodiment, if four patches are taken from an input image, there may be two classifier models <NUM> each trained to process features extracted from two of the patches. Each of these two classifier models <NUM> may itself be an ensemble classifier, with component classifiers trained for each of the two patches input to the ensemble. Put another way, each classifier model <NUM> is trained with images from one or more distinct spatial locations (each corresponding to a different patch). In this way, classification accommodates spatial variance of features in the training data set because different classifiers can be specifically trained for certain spatial locations. Such training and mapping of patches to classifiers <NUM> is based on a type, number, location, size, and/or resolution of the patches. In other words, the classifiers <NUM> may each be trained with data corresponding to a particular patch(es) (e.g., its type, location, size, resolution, and the like). Therefore, particular patches are mapped to particular classifiers and these particular classifiers ensembled to classify the input image.

The classifier <NUM> may be trained by normal samples only. In other words, the classifier <NUM> may be trained with data/images from only normal eyes (representing only normative data). The classifier <NUM> may be of any type, such as a one-class classifier in the form of a one-class support vector machine (SVM), one-class random forest model, or the like. In other embodiments, the classifier <NUM> may be a binary classifier. As shown in the example of <FIG>, where the classifier <NUM> is a one-class classifier, the output <NUM> of the one-class classifier <NUM> is a label and a score of the processed images. The label indicates, for example, "normal" or "abnormal"; and the score indicates a degree or level of that label indication. The sign of the score may also indicate whether the label is normal/abnormal (e.g., negative: abnormal; positive: normal). The output may also include heat maps to indicate which part of the input images contribute (and how much) to the output label and/or score. Where a binary classifier is used, only a label would be output <NUM>.

Depending on the score, a particular abnormality or disease may then be identified. For example, as discussed in further detail below with respect to <FIG>, AMD is associated with scores of greater magnitude than drusen. Accordingly, a further analysis of the label and/or score output by the classifier <NUM> may be performed to identify the particular abnormality, for example, by comparing the scores with abnormal labels to thresholds (e.g., upper and lower bounds) associated with different abnormalities. These thresholds may be generated through a statistical calculation based on a test data set. In one embodiment, a receiver operating characteristic (ROC) curve may be utilized to identify thresholds that provide acceptable true-positive and false-positive rates for a desired application. In other examples, the threshold may be manually identified by a clinician based on a desired application.

Optionally, a dimension reduction step <NUM> may be performed on the extracted features prior to their input to the classifier <NUM>. The dimension reduction may be principal component analysis (PCA), independent component analysis (ICA), or the like.

A particular non-limiting embodiment of the classification approach is illustrated in <FIG>. Therein, image patches <NUM> (e.g., those obtained from pre-processing) are input to an Inception V3 deep learning model <NUM> for feature extraction. The features extracted by the Inception V3 model <NUM>. The extract features are then subject to a PCA analysis <NUM> as part of the optional dimension reduction step. The PCA analyzed extract features are then input to a one-class SVM <NUM>, which outputs the final label and/or score <NUM>.

An example of the second approach, image inpainting (a process of reconstructing portions of an image), is illustrated in <FIG>. As discussed above with respect to <FIG> and <FIG>, input images <NUM> may first be pre-processed <NUM>. This pre-processing <NUM> may be of any of the types of pre-processing already discussed. Regions of the pre-processed inputs are then masked <NUM>, and those masked regions reconstructed (inpainted) <NUM> by a machine learning system. Any region of the image may be randomly masked, for example, any structural region of the eye within the image including retinal layers (e.g., RNFL, Inner Plexiform layer (IPL), Retinal Pigment layer (RPE) and other layers). According to one embodiment, a sliding window is used to mask part of the input image or pre-processed image <NUM>, <NUM>. The window may take any size or shape, for example, as a rectangular window. The machine learning system for reconstructing masked regions can be trained from scratch using images taken from healthy patients. As discussed above, because a single input image can have multiple patches, the machine learning system can be trained to process multiple patches, or multiple machine learning systems can be utilized for the different patches. The reconstructed images, and the pre-proceed images (or the input images <NUM> if not pre-processed) are then compared to measure their similarity <NUM>. The label and/or score are finally determined <NUM> based on the measured similarities <NUM>. In other words, where the machine learning system is trained to reconstruct portions of the image based on images of normal and healthy eyes, a high level of similarity between the unmasked input or pre-processed images <NUM>, <NUM> and the reconstructed image indicate that the masked portion is normal and healthy. In contrast, a low level of similarity indicates that input or pre-processed image <NUM>, <NUM> is abnormal.

A particular example of the image inpainting approach is illustrated in <FIG>. Therein, the input images and/or the pre-processed input images are image patches <NUM>. Regions of these patches are then masked <NUM> and reconstructed <NUM> by a DCGAN model. A DCGAN model comprises generator network that encodes and decodes an input image or patch, and discriminator network that determines whether an image or patch is generated (e.g., by a generator network) or is original to the image or patch. The discriminator network helps the generator network to realistically inpaint/reconstruct images. Following reconstruction <NUM>, the reconstructed image patch is compared with original unmasked image patch using a multiscale structural similarity (MS-SSIM) <NUM> technique to make a similarity measurement <NUM>. If a result of the MS-SSIM measurement is very small or close to zero (indicating little similarity), it can be determined that input image patch <NUM> represents some type of disease/anomaly. As noted above, this score can also imply the degree of severity or the type of the disease, and identify a label of "normal" or "abnormal" by setting threshold value for the similarity measure.

The third approach noted above, ensemble anomaly detection, utilizes both the classification and the inpainting approaches in a single system. In one example, each approach is executed in parallel on a common input to detect an anomaly. The final label and/or score then may be aggregate of the two approaches. For example, in some embodiments the final label may be 'abnormal' if either one or both approaches output a result that would be classified as abnormal (considered an 'or' combination); however, other embodiments may require that both approaches output a result of 'abnormal' for that to be used as the final label (considered an 'and' combination). An example of this approach is illustrated in <FIG>, where a common input image <NUM> is input to both a classification system <NUM> and an inpainting system <NUM>. The one class-classification system <NUM> and the inpainting system <NUM> may be of the type described above. The output of each system <NUM>, <NUM>, is then combined via a logical operator <NUM> to determine the label and/or score <NUM> of the input image <NUM>. The logical operator <NUM> may be an OR operator for systems designed to output a label and/or score as an 'or' combination, or be an AND operator for systems designed to output a label and/or score as an 'and' combination, as described above.

While <FIG> illustrates that a common pre-processing <NUM> may be performed on the input image <NUM>, as an alternative, different pre-processing may be performed for each system <NUM>, <NUM>. Of course, other masking, reconstruction, and similarity measurement techniques may be used and are considered within the scope of the present disclosure. Still further, although <FIG> illustrates the classification system <NUM> and the inpainting system <NUM> in parallel, it is also envisioned that these systems may be executed sequentially in any order.

It is also noted that multiple anomaly detections can be similarly executed in parallel for the same data, but with different resolutions, thereby forming a multi-resolution system. In other words, the above process may be performed multiple times on the same input data (e.g., from the same volume), however each iteration of the processes would use input images of different sizes/dimensions/resolutions. The process may also be performed on different inputs, thereby forming a multivariate system. In the above embodiments, the final output label and/or scores can be compiled from individual B-scan/image patch label/scores in a multi-resolution and multivariable form. With these embodiments, it is possible to identify multiple abnormalities (e.g., medical conditions) in a single patient. For example, some minor structural changes caused by abnormalities (e.g., drusen) may be most effectively identified by analyzing small-sized patches, whereas other abnormalities can be most effectively identified by analyzing larger-sized images. Similarly, different extracted features, or analysis of different physiological regions may be most indicative of different abnormalities.

The above architectures are flexible to accommodate a wide range of input types and can be applied to general disease anomaly detection. Besides the above mentioned traditional 3D scan-based B-scan input(s), the input can be customized towards glaucomatous feature associated images or measurements, for example, derived concentric circle or elliptical scans either around the disc or the fovea, or customized scans at optic nerve head or collections of non-raster scans (i.e., not traditional 3D scans) along nerve fiber bundles or various layer thickness measurement maps such as partial or full retinal nerve fiber layer thickness maps. The steps described herein, such as pre-processing, CNN model, DCGAN model, MS-SSIM and one class classifier, could be similar and applicable to all these situations. It should also be noted that these customizations may apply to pathologies other than glaucoma as well.

<FIG> shows an example of how a volume label and/or score can be derived from individual image or image patch scores. Therein, each image or patch has a label and/or score <NUM> determined according to one of the above-described screener architectures, for example, the score being output by a classifier or a similarity measurement technique. A mean score <NUM> of these images is then determined, the mean score representing a "B-scan level" (or local) score. If there are then X consecutive (e.g., adjacent) B-scans having a score less than a predetermined threshold Th, the volume (or composite set of images) may be given an "abnormal" label <NUM>. The X number of consecutive scans can be any number. For example, in one embodiment, the volume is labeled as abnormal if four or more consecutive B-scans have a score less than a zero threshold. Otherwise the volume is labelled "normal". The score for the volume (or composite set of images) <NUM> is then determined as the minimum value of any <NUM> consecutive B-scan scores. For example, if there are multiple sets of four consecutive B-scans having scores are less than zero, the smallest score among all of the sets may be considered the score for the volume. These volume scores <NUM> and labels <NUM> represent global scores and labels. Of course the above thresholds are not limiting, and any threshold criteria may be considered. Similarly, any statistical calculation(s) (for example, minimum, maximum, mean, median, standard deviations) may be used instead of those noted above. As noted above, in some embodiments the sign of the score may be reversed such that a negative value represents a normality, and a positive value represents an abnormality (thus the comparison for determining a label may be whether the score is greater than, rather than less than, the threshold). Similarly, the score may change in any manner with respect to severity. For example, the score may increase or decrease linearly or exponentially as severity changes.

Still further, the above-described process may be repeated for a common image or volume with a different feature extraction and/or classifier in order to make further pathological determinations. For example, a first iteration of the process may be used to determine whether a particular abnormality exists (e.g. a type of disease) and a further iteration may be used to determine the particular pathology of that type of disease that exists. In the former iteration, the machine learning system (e.g., including feature extraction CNN, the classifier, or the reconstructing model) may be particularly trained to recognize the abnormality, whereas in the latter iteration, the machine learning systems may be trained differently to recognize the particular pathologies.

As discussed above, an output of the system and method described herein may be a label and/or score of an input image(s) or other data set. Additional outputs can include any of the images resulting from the processing and pre-processing, and data accumulated during the processing and pre-processing. For example, outputs may include heat maps (or the corresponding data) indicating which regions of individual images or of image volumes that contributed most to a corresponding score and label determined by the system and method. These outputs may be provided directly to an end user, for example, via a display, or stored for later viewing and/or analysis.

<FIG> illustrate results of a test of the above-described methodology. In particular, the test used two independent data sets representing a normal eye, and one data set representing an eye with a retinal disease. Each set comprised <NUM> × <NUM> 3D OCT volume data. The retinal disease symptoms included, among others, drusen, hard drusen, AMD, epiretinal membrane (ERM), macular edema, retinopathy, and central serous chorioretinopathy (CSC). As these figures merely represent one test, it is noted that different input conditions and types of pathologies, different scan regions (including those with anterior images) could be used.

More particularly regarding <FIG> illustrates a histogram showing the number of B-scans within the tested volume having a particular score indicative of drusen, as determined by a one class SVM classifier; and <FIG> illustrates a histogram for AMD scores. As seen therein, drusen was more prevalent-resulting in more B-scans being identified as showing drusen-and tended to present with scores less than -<NUM>, while AMD tended to produce stronger scores of about -<NUM> to -<NUM>. As a precursor for AMD, the presence of drusen amounts to less severe structural changes and thus a lower score. This is consistent with the above discussion that drusen corresponds to lower scores than AMD, and that a threshold may be applied to differentiate drusen scores from AMD scores and thus differentiate diseases associated with an abnormal label. Such histograms can also be used to identify a level of severity for an entire volume of B-scans, for example, by identifying the most common score or a range of the greatest density of scores. Further, as noted above, the negative value of the scores can be used to label them as abnormal and a greater absolute value of the score indicates a greater severity of structural change, or a particular type of disease.

<FIG> illustrate three example B-scans from this test. <FIG> shows a B-scan <NUM> having a score of -<NUM>, showing AMD in the image. <FIG> shows a B-scan <NUM> with drusen and having a score of -<NUM>. <FIG> shows a B-scan <NUM> also with drusen, but having a lower score of -<NUM>. The middle panels of each of <FIG> illustrate the B-scans overlaid with heat maps <NUM>, <NUM>, <NUM>; and the bottom panels illustrate the heat maps <NUM>, <NUM>, <NUM> by themselves. The heat maps indicate areas of each B-scan contributing the most to the final classification. These heat maps may also be produced in color, to show degrees of contribution (e.g., blue indicating little or no contribution to red indicating the most contribution).

Finally, <FIG> illustrates an ROC analysis for the test based on scoring and labeling at the volume level. ROC analyses can be used for evaluating the quality or performance of tests. For example, an ROC curve is a graphical plot that illustrates the diagnostic ability of a binary classifier system as its discrimination threshold is varied. The ROC curve may be created by plotting a true positive rate (TPR) against the false positive rate (FPR) at various threshold settings. The area under the curve (often referred to as simply the AUC) represents an index of diagnostic ability. The larger the AUC, the better quality and performance of the system. The sensitivity at any point on the curve can be identified as the true positive rate, and the specificity at any point on the curve can be identified as one minus the false positive rate.

In particular, <FIG> shows an ROC curve where the score for the volume was based on a sum of four consecutive individual B-scan scores. <FIG> shows an ROC curve where the score for the volume was based on a mean of all available individual B-scan scores. For these analyses, a positive identification was an identification of a retinal disease by the system and method. For the analysis in <FIG>, the system exhibited a sensitivity of <NUM> for a specificity of <NUM>, and reached a sensitivity of <NUM> at a specificity of <NUM>. This corresponds to an area under the ROC curve of <NUM>. <FIG>, for a mean calculation to determine a volumetric score, showed a sensitivity of <NUM> for a specificity of <NUM> and reached a sensitivity of <NUM> at a specificity of <NUM>. The area under the curve for <FIG> was thus <NUM>.

As suggested above, a system for executing the above-described method is also contemplated within the scope of the present disclosure. Such a system may include a computer having one or more processors (e.g., in the form of an integrated circuit(s), discrete circuitry, or the like) for executing the method, storage (such as a hard disk, memory, RAM, or the like) and an input/output interface (e.g., display, keyboard, mouse, and the like). The storage may be located locally with the computer, or remotely, for example at a centralized database. The system may also be integrated or separate from a system used to capture the images and other input data. For example, the computer may be the same as that used to control an optical coherence tomography system.

Claim 1:
An image processing method comprising:
receiving a first image (<NUM>, <NUM>) of an object;
extracting (<NUM>), with a trained machine learning system, a feature of the first image or of a physiological structure shown in the first image, the first image being an input to the trained machine learning system (<NUM>);
classifying, with a first trained classifier (<NUM>), the first image based on the extracted feature, the extracted feature being an input to the first trained classifier; and
determining a label and/or score of the first image based on the classification;
receiving a second image (<NUM>, <NUM>) of the object;
extracting (<NUM>), with the trained machine learning system, a feature of the second image or of a physiological structure shown in the second image, the second image being an input to the trained machine learning system (<NUM>);
classifying, with a second trained classifier (<NUM>), the second image based on the extracted feature of the second image, the extracted feature of the second image being an input to the second trained classifier; and
determining a label and/or score (<NUM>) of the second image based on the classification,
wherein the first image and the second image are patches, and
wherein training and mapping of patches to classifiers (<NUM>) is based on a type, location, size and/or resolution of the patches, wherein each classifier (<NUM>) is trained with images from one or more distinct spatial locations, types, sizes and/or resolutions, each corresponding to a different patch, wherein
each of the plurality of patches are from a distinct spatial region of a common cross-sectional image of the object, and
the first trained classifier and the second trained classifier are component classifiers of an ensemble classifier, the first trained classifier being trained with training image patches from the spatial region corresponding to the first image, and the second trained classifier being trained with training image patches from the spatial regions corresponding to the second image.