# Automatic Detection of Microaneurysms in OCT Images Using Bag of Features

Authors: Elahe Sadat Kazeminasab; Ramin Almasi; Bijan Shoushtarian; Ehsan Golkar; Hossein Rabbani
Journal: Computational and Mathematical Methods in Medicine
 (2022)

---

## Abstract

Diabetic retinopathy (DR) caused by diabetes occurs as a result of changes in the retinal vessels and causes visual impairment. Microaneurysms (MAs) are the early clinical signs of DR, whose timely diagnosis can help detecting DR in the early stages of its development. It has been observed that MAs are more common in the inner retinal layers compared to the outer retinal layers in eyes suffering from DR. Optical coherence tomography (OCT) is a noninvasive imaging technique that provides a cross-sectional view of the retina, and it has been used in recent years to diagnose many eye diseases. As a result, this paper attempts to identify areas with MA from normal areas of the retina using OCT images. This work is done using the dataset collected from FA and OCT images of 20 patients with DR. In this regard, firstly fluorescein angiography (FA) and OCT images were registered. Then, the MA and normal areas were separated, and the features of each of these areas were extracted using the Bag of Features (BOF) approach with the Speeded-Up Robust Feature (SURF) descriptor. Finally, the classification process was performed using a multilayer perceptron network. For each of the criteria of accuracy, sensitivity, specificity, and precision, the obtained results were 96.33%, 97.33%, 95.4%, and 95.28%, respectively. Utilizing OCT images to detect MAs automatically is a new idea, and the results obtained as preliminary research in this field are promising.


---

## Body

## 1. Introduction

Diabetic retinopathy (DR) is a serious and dangerous disease, and since the prevalence of DR is directly related to the increasing prevalence of diabetes, it is growing rapidly in different societies [1]. DR can damage the retinal blood vessels and ultimately leads to blindness [2]. Microaneurysms (MAs) are the first clinical signs of DR [3], and their premature diagnosis can be very useful for preserving patients’ vision [4]. Retinal MAs are small protrusions of the capillary vessel walls and are most commonly seen in the inner nuclear layer [5–7]. The number of MAs indicates the likelihood of developing a more severe level of DR [8]. The fluorescein angiography (FA) imaging technique has been used for over 50 years as the gold standard technique for retinovascular imaging [9]. Ophthalmologists currently use FA as the primary tool for the detection of MAs in the images of DR patients. Although FA is capable of detecting microvasculature details, it is an invasive and time-consuming method that requires intravenous injection and an expert photographer [10]. The imaging technique known as optical coherence tomography (OCT) was introduced in 1991 [11] and subsequently became an essential tool for clinical imaging [10]. OCT is a near-infrared light interferometry method developed for cross-sectional, noninvasive, high-resolution, and three-dimensional tomography imaging in biological systems. Therefore, the use of OCT images as a new type of retinal imaging for automatic detection of MAs could be a good approach to improve the current routine of MA detection.What is done in this article as our main contribution to the automatic detection of MAs is as follows:(1)
At first, the dataset including FA and OCT images of 20 patients were collected for DR disease. The mentioned dataset contains one FA image and 31 OCT B-scans per patient. It should be noted that the process of automatic detection of MAs with the help of OCT images has not been performed in the manner used in this study. Hence, there is no comprehensive database for this purpose(2)
The next step is the registration of OCT B-scans with corresponding FAs. Since many MAs are well recognizable in FA images, the MA and normal areas can be found with the help of the registration of OCT B-scans and the corresponding FA(3)
Afterward, the characteristics of each of the areas related to the two categories, i.e., MA and normal, have been extracted using the BOF method. It makes sense to use local features to describe MAs after limiting their location in the previous step(4)
In next phase, these characteristics have been used to train the multilayer perceptron network, so that the two regions could be separated. Then, the performance of the classifier is compared with several different classifier techniques(5)
Finally, this trained model is used to automatically detect MAs. In this way, vertical strips are extracted from each OCT image as system input, and they are automatically checked and labeled for MA or normality


## 2. Related Works

Previous articles related to the automatic detection of microaneurysms have commonly used FA, fundus, sometimes optical coherence tomography angiography (OCTA), and OCT imaging techniques.


### 2.1. FA Imaging

MAs in these images are often presented as hyperfluorescent points on FA images, and some MAs are associated with focal fluorescence leakage. Initially, MA counting protocols were developed to monitor the progress of DR during drug trials. Automated techniques were then used for more accurate and faster MA detection [12–16]. Although FA shows a very clear image of the retina and many MAs are detectable in these images, it is not well accepted by patients because of its invasive nature. Therefore, the researchers then used fundus images, which are a less-invasive method than FA to detect MAs.


### 2.2. Fundus Imaging

Most of the previous works have been done to automatically detect MA using fundus images. This process in these images is usually done in two steps: (1) extraction of MA candidates and (2) classification. The first step requires image preprocessing to reduce the noise and improve the contrast. Subsequently, candidate regions for MAs are identified. Blood vessel segmentation algorithms are then used to eliminate the blood vessels from the MA candidates to reduce false positives, since many blood vessels may appear as false positives in the pre-processed images. Secondly, the classification algorithm is applied to the classification of these features for the candidates of MA and the candidate of non-MA [17]. In recent works [18–34], the same procedure has been applied by employing different machine learning, deep learning, and image processing methods. For example, in [35], a deep learning method based on transformation splicing (TS) and multicontext ensemble learning is proposed. TS balances the MA size ratio and reduces blood vessel interference by transforming the pixel distribution of each image and fortifying the characteristics of hard samples. In this way, the model learns the features of the improved image better. In this way, the multicontext ensemble learning model used learns the features of the improved image better. The final scores in this paper for e-ophatha-MA, DiaretDB1, and ROC three public datasets are 0.518, 0.429, and 0.306, respectively. Also, in [36], a method for automatic detection of microaneurysms that divides the original image into small independent images and applies a local Fourier transform on them was proposed. In addition, a statistical feature has been considered to check for normality, and a new feature has been designed to show the difference in every direction. Finally, a group of candidate points are extracted, and some features of these key points are used for random forest classification. The FROC curve score for the proposed method on the e-ophtha and ROC datasets is 0.847 and 0.283, respectively. Also, the AUC on the ROC dataset can reach 0.961. In [37], an efficient local structure awareness-based combination of several features for MA diagnosis was proposed. A local structure property called a ring gradient descriptor (RGD) is used to describe the structural differences between an object and its surroundings. A combination of RGD with salience and texture features was then used for a gradient boosting decision tree (GBDT) for classification. The AUC evaluation results of this method on e-ophtha MA and ROC datasets increased from 0.9615 to 0.9751 and from 0.9066 to 0.9409, respectively. In [38], a screening method using the combined features of shape and texture is presented. A proposed hybrid multikernel support vector machine classifier is also used for classification. According to the checking performed in this paper, the FROC score for ROC, DIARETdB1, and MESSIDOR datasets are 0.503, 0.481, and 0.464, respectively. The authors claim that the use of the proposed algorithm on the AGAR300 dataset works better in terms of FROC, AUC, F1 score, precision, sensitivity, and specificity. In [4], fundus images were segmented using particle swarm optimization. This method combines the membership functions with high similarity data grouping in clusters. The authors state that the accuracy score for the DIARETDB0 database is 99.9 percent. In [39], a hybrid feature embedding approach is proposed using pretrained VGG-19 and Inception-v3 models. After evaluating this method on “e-ophtha” and “DIARETDB1” datasets, accuracy scores 96% and 94% were obtained, respectively.However, processing of fundus images is more challenging than FAs because different objects in fundus images may be confused with diabetes lesions [33]. Furthermore, automatic detection of MAs has always been a problem because of their very small size [32], interference with blood vessels, and apparent contrast differences between MAs [35]. Also, the major limitation of these images is that they provide a two-dimensional view of the 3D retinal texture [40].


### 2.3. OCTA Imaging

It should be noted that OCTA imaging is also a new imaging technique that has become popular in recent years. The 3D visualization provided by OCTA images is useful for analyzing MAs and their spatial orientation [5]. MAs appear as saccular or fusiform in these images [41]. Some previous studies have compared FA and OCTA imaging and concluded that FA detects a higher number of MAs than OCTA [42–44]. Measurement of relative blood speed indicates that blood flow is slower in MAs [44–46]. Studies have also identified MAs without blood flow. As a result, fewer MAs are detected in OCTA images because OCTA is based on blood flow [45], and areas where blood flow is slow may not be seen in these images [46]. Studies show that OCTA is more effective in the diagnosis of MAs than fundus imaging [47]. On the other hand, as described in the text, in OCT images, MAs are seen as a reflection. If these reflections are hyporeflective, they will less likely visible in OCTA images. But if they are hyperreflective, they will often be visible in OCTA images, which indicates the rate of blood flow in these MAs [48].


### 2.4. OCT Imaging

This imaging method is also a new imaging technique that has received a lot of attention today. Automatic detection of ocular diseases using OCT images is still in its infancy because only academic research has been published, and no commercial activity is available [49]. Since OCT images facilitate the evaluation of retinal morphology to microscopic resolution [14], it has been attempted to use these images to detect various retinal malformations such as glaucoma, diabetic macular edema (DME), age-related macular degeneration (AMD), retinal detachment, and diabetic retinopathy [50]. Most previous works have focused on the analysis of OCT images on the problem of dividing the retinal layers, retinal thickness measurement, or segmentation of specific lesions such as cysts [49]. OCT is more desirable than fundus imaging because it provides in-depth and cross-sectional information from the eye [51].In recent years, researchers have investigated the potential of using SD-OCT images to detect MAs and have attempted to identify the apparent features of diabetic MAs in these images. In these studies [52–56], the following results have been achieved for the appearance of MAs:MAs have a relatively circular and capsular structure and are mainly located in the inner nuclear layer (INL) layer and the deeper layers than the INL layer. MAs increase the thickness of the retina, and in some cases, reflectivity points are found in their vicinity.Preliminary studies have shown that SD-OCT is a reliable and promising criterion for further investigation to diagnose MAs and also has a good correlation with the equivalent FA image [57].


## 2.1. FA Imaging

MAs in these images are often presented as hyperfluorescent points on FA images, and some MAs are associated with focal fluorescence leakage. Initially, MA counting protocols were developed to monitor the progress of DR during drug trials. Automated techniques were then used for more accurate and faster MA detection [12–16]. Although FA shows a very clear image of the retina and many MAs are detectable in these images, it is not well accepted by patients because of its invasive nature. Therefore, the researchers then used fundus images, which are a less-invasive method than FA to detect MAs.


## 2.2. Fundus Imaging

Most of the previous works have been done to automatically detect MA using fundus images. This process in these images is usually done in two steps: (1) extraction of MA candidates and (2) classification. The first step requires image preprocessing to reduce the noise and improve the contrast. Subsequently, candidate regions for MAs are identified. Blood vessel segmentation algorithms are then used to eliminate the blood vessels from the MA candidates to reduce false positives, since many blood vessels may appear as false positives in the pre-processed images. Secondly, the classification algorithm is applied to the classification of these features for the candidates of MA and the candidate of non-MA [17]. In recent works [18–34], the same procedure has been applied by employing different machine learning, deep learning, and image processing methods. For example, in [35], a deep learning method based on transformation splicing (TS) and multicontext ensemble learning is proposed. TS balances the MA size ratio and reduces blood vessel interference by transforming the pixel distribution of each image and fortifying the characteristics of hard samples. In this way, the model learns the features of the improved image better. In this way, the multicontext ensemble learning model used learns the features of the improved image better. The final scores in this paper for e-ophatha-MA, DiaretDB1, and ROC three public datasets are 0.518, 0.429, and 0.306, respectively. Also, in [36], a method for automatic detection of microaneurysms that divides the original image into small independent images and applies a local Fourier transform on them was proposed. In addition, a statistical feature has been considered to check for normality, and a new feature has been designed to show the difference in every direction. Finally, a group of candidate points are extracted, and some features of these key points are used for random forest classification. The FROC curve score for the proposed method on the e-ophtha and ROC datasets is 0.847 and 0.283, respectively. Also, the AUC on the ROC dataset can reach 0.961. In [37], an efficient local structure awareness-based combination of several features for MA diagnosis was proposed. A local structure property called a ring gradient descriptor (RGD) is used to describe the structural differences between an object and its surroundings. A combination of RGD with salience and texture features was then used for a gradient boosting decision tree (GBDT) for classification. The AUC evaluation results of this method on e-ophtha MA and ROC datasets increased from 0.9615 to 0.9751 and from 0.9066 to 0.9409, respectively. In [38], a screening method using the combined features of shape and texture is presented. A proposed hybrid multikernel support vector machine classifier is also used for classification. According to the checking performed in this paper, the FROC score for ROC, DIARETdB1, and MESSIDOR datasets are 0.503, 0.481, and 0.464, respectively. The authors claim that the use of the proposed algorithm on the AGAR300 dataset works better in terms of FROC, AUC, F1 score, precision, sensitivity, and specificity. In [4], fundus images were segmented using particle swarm optimization. This method combines the membership functions with high similarity data grouping in clusters. The authors state that the accuracy score for the DIARETDB0 database is 99.9 percent. In [39], a hybrid feature embedding approach is proposed using pretrained VGG-19 and Inception-v3 models. After evaluating this method on “e-ophtha” and “DIARETDB1” datasets, accuracy scores 96% and 94% were obtained, respectively.However, processing of fundus images is more challenging than FAs because different objects in fundus images may be confused with diabetes lesions [33]. Furthermore, automatic detection of MAs has always been a problem because of their very small size [32], interference with blood vessels, and apparent contrast differences between MAs [35]. Also, the major limitation of these images is that they provide a two-dimensional view of the 3D retinal texture [40].


## 2.3. OCTA Imaging

It should be noted that OCTA imaging is also a new imaging technique that has become popular in recent years. The 3D visualization provided by OCTA images is useful for analyzing MAs and their spatial orientation [5]. MAs appear as saccular or fusiform in these images [41]. Some previous studies have compared FA and OCTA imaging and concluded that FA detects a higher number of MAs than OCTA [42–44]. Measurement of relative blood speed indicates that blood flow is slower in MAs [44–46]. Studies have also identified MAs without blood flow. As a result, fewer MAs are detected in OCTA images because OCTA is based on blood flow [45], and areas where blood flow is slow may not be seen in these images [46]. Studies show that OCTA is more effective in the diagnosis of MAs than fundus imaging [47]. On the other hand, as described in the text, in OCT images, MAs are seen as a reflection. If these reflections are hyporeflective, they will less likely visible in OCTA images. But if they are hyperreflective, they will often be visible in OCTA images, which indicates the rate of blood flow in these MAs [48].


## 2.4. OCT Imaging

This imaging method is also a new imaging technique that has received a lot of attention today. Automatic detection of ocular diseases using OCT images is still in its infancy because only academic research has been published, and no commercial activity is available [49]. Since OCT images facilitate the evaluation of retinal morphology to microscopic resolution [14], it has been attempted to use these images to detect various retinal malformations such as glaucoma, diabetic macular edema (DME), age-related macular degeneration (AMD), retinal detachment, and diabetic retinopathy [50]. Most previous works have focused on the analysis of OCT images on the problem of dividing the retinal layers, retinal thickness measurement, or segmentation of specific lesions such as cysts [49]. OCT is more desirable than fundus imaging because it provides in-depth and cross-sectional information from the eye [51].In recent years, researchers have investigated the potential of using SD-OCT images to detect MAs and have attempted to identify the apparent features of diabetic MAs in these images. In these studies [52–56], the following results have been achieved for the appearance of MAs:MAs have a relatively circular and capsular structure and are mainly located in the inner nuclear layer (INL) layer and the deeper layers than the INL layer. MAs increase the thickness of the retina, and in some cases, reflectivity points are found in their vicinity.Preliminary studies have shown that SD-OCT is a reliable and promising criterion for further investigation to diagnose MAs and also has a good correlation with the equivalent FA image [57].


## 3. Methods

The method proposed in this paper for automatic detection of MAs in OCT images is divided into two general stages of training and testing (Figure1). Each of these steps consists of several sections, which will be described in more detail below.Figure 1
The general diagram of the method used in this study.FA and OCT images used in this study were captured using the Spectral HRA2/OCT Heidelberg imaging device. For this study, late frames of FA imaging of 20 patients with DR were used. Equivalent to each FA image, 31 OCT B-scan images were produced by the imaging unit, and each B-scan dimension was496∗768 pixels.


### 3.1. Training

In the training phase, the FA images and their equivalent OCT images are initially considered as inputs of the system. The following are the three steps of system training.


#### 3.1.1. Preprocessing

As it is shown in Figure1, the preprocessor consists of two parts: registration of OCT B-scans with corresponding FAs and in the next step extraction of MA and non-MA areas from B-scan images.(1) Registration. Due to the fact that MAs are well visible in FA images related to diabetic Retinopathy patients, with the help of FA and OCT images registration, areas containing MA can be found on OCT images. Since OCT images provide a cross-sectional and depth view of the retina and the FA images represent a surface view of the retina, the two images have different dimensions and cannot be directly aligned together [58]. As stated, the dataset used in this study was prepared by the Heidelberg device, which enables simultaneous scanning laser ophthalmoscopy (SLO)/OCT imaging. SLO provides real-time images from the surface of the retina which is corresponding to the OCT images. Since these two image groups (SLO and OCT) are made from a unit light source, they are pixel-to-pixel aligned. So, by registration of SLO and FA images, OCT B-scans and FAs will be aligned. For this reason, the multistep correlation-based technique has been used for the registration of FA and OCT images [59]. This algorithm consists of two phases: (1) phase I: general registration (rigid analysis) and (2) phase II: local registration in areas with MA (nonrigid analysis) which are briefly explained below:Phase I: first, the vein network is extracted from each image using the Dijkstra Forest Exploratory Algorithm used in [60], and the correlation of FA and SLO images is calculated using a multistep correlation-based method at different angles and dimensions. Then, the most appropriate affine parameters are identified with the highest correlation as a result of the algorithm and used in subsequent phasesPhase II: a patch-based local registration algorithm with several different accuracies is used to improve the results of the first phase of registration in areas with MA. After this step, MA areas can be identified in B-scan OCT images. The result of the registration and the MA regions identified on the FA image and its equivalent B-scan for an example set of the images used in this study are shown in Figure2Figure 2
The FA image (a) and the B-scan image corresponding to the yellow horizontal line displayed on the FA image (b). The location of MA is shown as a red circle in (a) image and red lines in (b) image.
(a)(b)(2) Crop Area including MA and Normal B-Scan Images. After identifying the MAs in FA images and applying the registration stage, the corresponding areas can be extracted from the B-scan images (the red strip in Figure 2(b)). A range of 30 pixels is considered for each detected MA in the B-scan image, and unnecessary areas are removed from the top and bottom of the region of interest (ROI) to preserve useful information around the retinal layers. Therefore, for each desired area, the vertical bar consists of 170∗30 pixels. Normal areas will similarly be extracted from the B-scan image. Figure 3 shows two areas extracted from the MA and normal areas from a sample B-scan image. In total, 92 MA areas and 110 normal areas were used for this study. Some areas were excluded from the MA category due to other lesions related to other diseases. 70% of data are randomly selected for training, 15% for test, and 15% for validation.Figure 3
Extraction of MA (a) and normal (b) areas from a B-scan image. In both cases, the middle image is an area of 30 pixels extracted from the original image, and in the right image, unnecessary parts are removed from the top and bottom.
(a)(b)


#### 3.1.2. Bag of Features

Feature extraction: in this study, the dense SURF local descriptors have been used to identify and describe image features. For this purpose, the image is batched to a8∗8 grids, and key SURF points are extracted from image patches to describe the distribution of local light intensity. This way, local descriptors will be able to identify and localize key points in the horizontal and vertical directions of image patches. Since these features are invariant to scale and rotation and are resistant to noise [61], small MAs with low contrast will also be detectable. As shown in Figure 4, the key points are extracted from the image batches, and for each key point, the square is aligned with the point direction (the point direction is obtained during the previous steps of the SURF algorithm). This square is then divided into 4×4 subregion. This preserves important spatial information around the point. Then, Haar wavelet responses are obtained for each subregion. dx is the Haar wavelet response in the horizontal direction and dy is the Haar wavelet response in the vertical direction. So, for each part, the sum of dx, ∣dx∣, dy, and ∣dy∣ is considered as the descriptor vector.Figure 4
Point descriptors produced by SURF. On the left image, the key points of a sample image with an8×8 grid are shown, and how the SURF descriptor vector is produced is shown in the middle of the figure.Quantization and building visual vocabulary: In the visual word construction phase, the number of features extracted from the previous stage will be reduced. In this way, the feature vectors created by SURF must be grouped into several clusters by a clustering algorithm. In this study, theK-means++ clustering algorithm with the Euclidean distance criterion is used. The reason for choosing this clustering method is its simplicity. The K-means++ clustering algorithm performs better than the original K-means method by changing the way of the initialization of the clusters’ centers [62]. In other words, visual words are the centers of the clusters.The sum of the events of each visual word (each cluster) for all images in each category is shown in Figure5. These two charts show the difference between the two categories of images for 100 visual words.Figure 5
Sum of events of all visual words in each category. The image (a) is for the MA class, and the image (b) is for the normal class.
(a)(b)


#### 3.1.3. Classification

In visual word-based image classification, a training set of images are given to the classifier in the form of term vectors. In this study, a multilayer perceptron (MLP) network with a hidden layer, and 10 neurons were used for the classification process. As can be seen in Figure6, the use of 10 neurons in the hidden layer is the most appropriate choice. In the experiments performed, the MLP classification method showed better performance than other classification algorithms such as Gaussian SVM, linear SVM, KNN, and Naïve Bayes.Figure 6
The accuracy analysis. The effect of the number of hidden layer neurons of the multilayer perceptron network on the accuracy criterion.(1) Test. In the testing phase, the following steps will be done for each OCT image as input to identify the MAs:
(1)
Extraction of the vertical strips from each B-scan image(2)
Encoding each extracted strip using the dictionary created in the training step. In this way, first, the features of each strip are extracted and then assigned to the nearest centers of the clusters. As a result, a visual word vector is created for the strip(3)
This term vector is given to the MLP network trained in the previous step to assign a label (MA or non-MA) to it


## 3.1. Training

In the training phase, the FA images and their equivalent OCT images are initially considered as inputs of the system. The following are the three steps of system training.


### 3.1.1. Preprocessing

As it is shown in Figure1, the preprocessor consists of two parts: registration of OCT B-scans with corresponding FAs and in the next step extraction of MA and non-MA areas from B-scan images.(1) Registration. Due to the fact that MAs are well visible in FA images related to diabetic Retinopathy patients, with the help of FA and OCT images registration, areas containing MA can be found on OCT images. Since OCT images provide a cross-sectional and depth view of the retina and the FA images represent a surface view of the retina, the two images have different dimensions and cannot be directly aligned together [58]. As stated, the dataset used in this study was prepared by the Heidelberg device, which enables simultaneous scanning laser ophthalmoscopy (SLO)/OCT imaging. SLO provides real-time images from the surface of the retina which is corresponding to the OCT images. Since these two image groups (SLO and OCT) are made from a unit light source, they are pixel-to-pixel aligned. So, by registration of SLO and FA images, OCT B-scans and FAs will be aligned. For this reason, the multistep correlation-based technique has been used for the registration of FA and OCT images [59]. This algorithm consists of two phases: (1) phase I: general registration (rigid analysis) and (2) phase II: local registration in areas with MA (nonrigid analysis) which are briefly explained below:Phase I: first, the vein network is extracted from each image using the Dijkstra Forest Exploratory Algorithm used in [60], and the correlation of FA and SLO images is calculated using a multistep correlation-based method at different angles and dimensions. Then, the most appropriate affine parameters are identified with the highest correlation as a result of the algorithm and used in subsequent phasesPhase II: a patch-based local registration algorithm with several different accuracies is used to improve the results of the first phase of registration in areas with MA. After this step, MA areas can be identified in B-scan OCT images. The result of the registration and the MA regions identified on the FA image and its equivalent B-scan for an example set of the images used in this study are shown in Figure2Figure 2
The FA image (a) and the B-scan image corresponding to the yellow horizontal line displayed on the FA image (b). The location of MA is shown as a red circle in (a) image and red lines in (b) image.
(a)(b)(2) Crop Area including MA and Normal B-Scan Images. After identifying the MAs in FA images and applying the registration stage, the corresponding areas can be extracted from the B-scan images (the red strip in Figure 2(b)). A range of 30 pixels is considered for each detected MA in the B-scan image, and unnecessary areas are removed from the top and bottom of the region of interest (ROI) to preserve useful information around the retinal layers. Therefore, for each desired area, the vertical bar consists of 170∗30 pixels. Normal areas will similarly be extracted from the B-scan image. Figure 3 shows two areas extracted from the MA and normal areas from a sample B-scan image. In total, 92 MA areas and 110 normal areas were used for this study. Some areas were excluded from the MA category due to other lesions related to other diseases. 70% of data are randomly selected for training, 15% for test, and 15% for validation.Figure 3
Extraction of MA (a) and normal (b) areas from a B-scan image. In both cases, the middle image is an area of 30 pixels extracted from the original image, and in the right image, unnecessary parts are removed from the top and bottom.
(a)(b)


### 3.1.2. Bag of Features

Feature extraction: in this study, the dense SURF local descriptors have been used to identify and describe image features. For this purpose, the image is batched to a8∗8 grids, and key SURF points are extracted from image patches to describe the distribution of local light intensity. This way, local descriptors will be able to identify and localize key points in the horizontal and vertical directions of image patches. Since these features are invariant to scale and rotation and are resistant to noise [61], small MAs with low contrast will also be detectable. As shown in Figure 4, the key points are extracted from the image batches, and for each key point, the square is aligned with the point direction (the point direction is obtained during the previous steps of the SURF algorithm). This square is then divided into 4×4 subregion. This preserves important spatial information around the point. Then, Haar wavelet responses are obtained for each subregion. dx is the Haar wavelet response in the horizontal direction and dy is the Haar wavelet response in the vertical direction. So, for each part, the sum of dx, ∣dx∣, dy, and ∣dy∣ is considered as the descriptor vector.Figure 4
Point descriptors produced by SURF. On the left image, the key points of a sample image with an8×8 grid are shown, and how the SURF descriptor vector is produced is shown in the middle of the figure.Quantization and building visual vocabulary: In the visual word construction phase, the number of features extracted from the previous stage will be reduced. In this way, the feature vectors created by SURF must be grouped into several clusters by a clustering algorithm. In this study, theK-means++ clustering algorithm with the Euclidean distance criterion is used. The reason for choosing this clustering method is its simplicity. The K-means++ clustering algorithm performs better than the original K-means method by changing the way of the initialization of the clusters’ centers [62]. In other words, visual words are the centers of the clusters.The sum of the events of each visual word (each cluster) for all images in each category is shown in Figure5. These two charts show the difference between the two categories of images for 100 visual words.Figure 5
Sum of events of all visual words in each category. The image (a) is for the MA class, and the image (b) is for the normal class.
(a)(b)


### 3.1.3. Classification

In visual word-based image classification, a training set of images are given to the classifier in the form of term vectors. In this study, a multilayer perceptron (MLP) network with a hidden layer, and 10 neurons were used for the classification process. As can be seen in Figure6, the use of 10 neurons in the hidden layer is the most appropriate choice. In the experiments performed, the MLP classification method showed better performance than other classification algorithms such as Gaussian SVM, linear SVM, KNN, and Naïve Bayes.Figure 6
The accuracy analysis. The effect of the number of hidden layer neurons of the multilayer perceptron network on the accuracy criterion.(1) Test. In the testing phase, the following steps will be done for each OCT image as input to identify the MAs:
(1)
Extraction of the vertical strips from each B-scan image(2)
Encoding each extracted strip using the dictionary created in the training step. In this way, first, the features of each strip are extracted and then assigned to the nearest centers of the clusters. As a result, a visual word vector is created for the strip(3)
This term vector is given to the MLP network trained in the previous step to assign a label (MA or non-MA) to it


## 3.1.1. Preprocessing

As it is shown in Figure1, the preprocessor consists of two parts: registration of OCT B-scans with corresponding FAs and in the next step extraction of MA and non-MA areas from B-scan images.(1) Registration. Due to the fact that MAs are well visible in FA images related to diabetic Retinopathy patients, with the help of FA and OCT images registration, areas containing MA can be found on OCT images. Since OCT images provide a cross-sectional and depth view of the retina and the FA images represent a surface view of the retina, the two images have different dimensions and cannot be directly aligned together [58]. As stated, the dataset used in this study was prepared by the Heidelberg device, which enables simultaneous scanning laser ophthalmoscopy (SLO)/OCT imaging. SLO provides real-time images from the surface of the retina which is corresponding to the OCT images. Since these two image groups (SLO and OCT) are made from a unit light source, they are pixel-to-pixel aligned. So, by registration of SLO and FA images, OCT B-scans and FAs will be aligned. For this reason, the multistep correlation-based technique has been used for the registration of FA and OCT images [59]. This algorithm consists of two phases: (1) phase I: general registration (rigid analysis) and (2) phase II: local registration in areas with MA (nonrigid analysis) which are briefly explained below:Phase I: first, the vein network is extracted from each image using the Dijkstra Forest Exploratory Algorithm used in [60], and the correlation of FA and SLO images is calculated using a multistep correlation-based method at different angles and dimensions. Then, the most appropriate affine parameters are identified with the highest correlation as a result of the algorithm and used in subsequent phasesPhase II: a patch-based local registration algorithm with several different accuracies is used to improve the results of the first phase of registration in areas with MA. After this step, MA areas can be identified in B-scan OCT images. The result of the registration and the MA regions identified on the FA image and its equivalent B-scan for an example set of the images used in this study are shown in Figure2Figure 2
The FA image (a) and the B-scan image corresponding to the yellow horizontal line displayed on the FA image (b). The location of MA is shown as a red circle in (a) image and red lines in (b) image.
(a)(b)(2) Crop Area including MA and Normal B-Scan Images. After identifying the MAs in FA images and applying the registration stage, the corresponding areas can be extracted from the B-scan images (the red strip in Figure 2(b)). A range of 30 pixels is considered for each detected MA in the B-scan image, and unnecessary areas are removed from the top and bottom of the region of interest (ROI) to preserve useful information around the retinal layers. Therefore, for each desired area, the vertical bar consists of 170∗30 pixels. Normal areas will similarly be extracted from the B-scan image. Figure 3 shows two areas extracted from the MA and normal areas from a sample B-scan image. In total, 92 MA areas and 110 normal areas were used for this study. Some areas were excluded from the MA category due to other lesions related to other diseases. 70% of data are randomly selected for training, 15% for test, and 15% for validation.Figure 3
Extraction of MA (a) and normal (b) areas from a B-scan image. In both cases, the middle image is an area of 30 pixels extracted from the original image, and in the right image, unnecessary parts are removed from the top and bottom.
(a)(b)


## 3.1.2. Bag of Features

Feature extraction: in this study, the dense SURF local descriptors have been used to identify and describe image features. For this purpose, the image is batched to a8∗8 grids, and key SURF points are extracted from image patches to describe the distribution of local light intensity. This way, local descriptors will be able to identify and localize key points in the horizontal and vertical directions of image patches. Since these features are invariant to scale and rotation and are resistant to noise [61], small MAs with low contrast will also be detectable. As shown in Figure 4, the key points are extracted from the image batches, and for each key point, the square is aligned with the point direction (the point direction is obtained during the previous steps of the SURF algorithm). This square is then divided into 4×4 subregion. This preserves important spatial information around the point. Then, Haar wavelet responses are obtained for each subregion. dx is the Haar wavelet response in the horizontal direction and dy is the Haar wavelet response in the vertical direction. So, for each part, the sum of dx, ∣dx∣, dy, and ∣dy∣ is considered as the descriptor vector.Figure 4
Point descriptors produced by SURF. On the left image, the key points of a sample image with an8×8 grid are shown, and how the SURF descriptor vector is produced is shown in the middle of the figure.Quantization and building visual vocabulary: In the visual word construction phase, the number of features extracted from the previous stage will be reduced. In this way, the feature vectors created by SURF must be grouped into several clusters by a clustering algorithm. In this study, theK-means++ clustering algorithm with the Euclidean distance criterion is used. The reason for choosing this clustering method is its simplicity. The K-means++ clustering algorithm performs better than the original K-means method by changing the way of the initialization of the clusters’ centers [62]. In other words, visual words are the centers of the clusters.The sum of the events of each visual word (each cluster) for all images in each category is shown in Figure5. These two charts show the difference between the two categories of images for 100 visual words.Figure 5
Sum of events of all visual words in each category. The image (a) is for the MA class, and the image (b) is for the normal class.
(a)(b)


## 3.1.3. Classification

In visual word-based image classification, a training set of images are given to the classifier in the form of term vectors. In this study, a multilayer perceptron (MLP) network with a hidden layer, and 10 neurons were used for the classification process. As can be seen in Figure6, the use of 10 neurons in the hidden layer is the most appropriate choice. In the experiments performed, the MLP classification method showed better performance than other classification algorithms such as Gaussian SVM, linear SVM, KNN, and Naïve Bayes.Figure 6
The accuracy analysis. The effect of the number of hidden layer neurons of the multilayer perceptron network on the accuracy criterion.(1) Test. In the testing phase, the following steps will be done for each OCT image as input to identify the MAs:
(1)
Extraction of the vertical strips from each B-scan image(2)
Encoding each extracted strip using the dictionary created in the training step. In this way, first, the features of each strip are extracted and then assigned to the nearest centers of the clusters. As a result, a visual word vector is created for the strip(3)
This term vector is given to the MLP network trained in the previous step to assign a label (MA or non-MA) to it


## 4. Results

To evaluate the proposed method, several classification methods were tested in comparison to a multilayer perceptron network. The comparison of these methods for the criteria of accuracy, sensitivity, specificity, and precision are presented in Table1.Table 1
Evaluation of the proposed method.
Classification methodEvaluation criteriaAccuracySensitivitySpecificityPrecision1BOF+MLP96.33%97.33%95.4%95.28%2BOF+Gaussian SVM91.63%88.27%95.02%94.65%3BOF+linear SVM91.13%86.52%95.78%95.40%4BOF+KNN84.19%73.14%95.28%93.95%5BOF+Naïve Bayes87.75%85.65%89.91%89.48%6PCA+MLP92.52%93.93%92.22%92.36%7PCA+Gaussian SVM83.38%85.53%81.28%82.16%8PCA+linear SVM80.94%77.77%84.15%83.19%9PCA+KNN57.32%14.77%99.39%96.32%10PCA+Naïve Bayes76.13%76.14%76.15%76.14%Besides, the impact of the existence of a BOF technique to achieve the desired result was evaluated. So, instead of using BOF, the image features were first extracted using SURF. Then, PCA is applied to reduce a large number of features. Finally, these feature vectors are given as input to different classifiers including MLP, Gaussian SVM, linear SVM, KNN, and Naïve Bayes. The average of the measured criteria for these classifiers is shown in Table1. These results show a decrease in accuracy compared to when using BOF.Also in Figure7, a bar chart is plotted for each criterion to better compare the methods described in Table 1.Figure 7
The charts related to 4 criteria of accuracy, sensitivity, specificity, and precision. The numbers in thex-axis of each chart is corresponding to the proposed methods in Table 1.


## 5. Discussion

Given that diabetes is a rapidly growing disease, the traditional diagnosis of DR is very time-consuming, labor cost, and requires expert people. Therefore, a system for screening and monitoring diabetic patients and automatic diagnosis of DR would be beneficial. Since MAs are the first clinical signs of DR, their diagnosis can help early diagnosis of DR [33].Based on these results, the use of MLP as the classification algorithm in this method yields the best results. Then, the SVM classifier with a Gaussian kernel comes in the second.In the published ratings for DR, the difference between the two levels without DR and mild nonproliferative DR is the presence or absence of MA. Therefore, if there are only MAs in an image and the system fails to detect them, the patient is diagnosed mistakenly without DR, causing the disease to become more severe and elevated to a high level in the next visit. This demonstrates the importance of automatic MA detection algorithms and their accuracy and sensitivity.In the “Introduction” section of this article, the superior applicability of OCT as a screening method and its advantages over fundus photography in DR screening courtesy of its strength in providing in-depth information from the retina have been explored. Therefore, in this article, it was decided to OCT imaging over fundus photography.


## 6. Conclusion

Due to the above reasons, it was decided to use OCT images for the automatic detection of MAs. For this purpose, at first, FA images have registered with OCT B-scans. By doing so, the MAs regions are identified in OCT B-scans and can be analyzed. SURF algorithm has been used to extract the features that can detect precise local information from the MA regions and distinguish areas containing MA and normal well. Visual words were then created using the BOF method. Utilizing the BOF method significantly improves the results. Afterward, a multilayer perceptron network was applied to classify these areas using these visual words. The evaluation criteria reviewed show suitable performance for the method proposed in this study. In this paper, it was shown that in addition to identifying areas with clear signs of MA, even areas that do not have clear signs of MA and are not visible to the human eye can be identified with the help of this method. These areas can be identified with the help of features extracted from OCT images and machine learning algorithms.This research is a preliminary study to show that OCT images can be used to detect MAs. However, this method is not without its caveats. Limitations of this method include the following: lack of suitable and comprehensive data for this purpose and incorrect diagnosis of MAs in some cases due to the presence of lesions caused by other diseases and blood vessels in OCT images.To overcome these limitations, the following solutions can be considered in future works:(1)
Expansion and improvement of datasets with the help of various imaging devices or using GAN networks to generate more data(2)
Using information from the before and next B-scans on top of the extracted features from each B-scan to increase the accuracy and prevent the possible loss of an MA(3)
Expanding the categories from two categories: “MA” and “normal” to four categories: “MA,” “non-MA,” “normal,” and “vessel,” so that the “MA” category only includes areas related to MA without any other lesions and vessels. The “non-MA” category includes areas that do not have MA but may have another lesion or vein. The “normal” category includes areas without any lesions or vessels. Finally, the class of “vessel” refers to the areas that only contain the vessel(4)
In addition to the above and as stated in [63], MAs can also be examined in terms of flow, location, and capillary density. Thus, although OCTA devices are not as common as OCT devices in all medical centers, they may be potentially helpful and worthy of further research for MA analysis



---

*Source: 1233068-2022-07-15.xml*