# A Deep Learning Ensemble Method to Visual Acuity Measurement Using Fundus Images
## 4 Deep Learning-Based Ensemble Method for VA Measurement
### Ensemble Methods  
This section discusses the rationale for the use of Ensemble methods consisting of three machine learning techniques: two deep neural network (DNN) models using transfer learning of Convolution Neural Network (CNN) techniques based on VGG19 [28] and EfficientNet-B7 [29], and Support Vector Machine (SVM).  
We applied each technique of VGG19, EfficientNet-B7, and SVM for the original fundus images for 4-level VA classification, and the accuracy of each model could not exceed about 70%. Even after augmenting fundus images, the accuracy of the VA classifier with individual machine learning models could not be more than about 80%. To identify the cause of low accuracy of VA classification by each ML model, we analyzed confusion matrices in Figure 4 generated from the VGG-19 model.  
Figure 4 shows the confusion matrix that the 4-level VA classifier based on VGG-19 returns when applied for each class of 11 VA levels. When the 4-level VA classifier is applied for Class-0.0 fundus images, the classification accuracy is 99% (Figure 4a). For Class-0.1 fundus images, the classifier makes 84% right decisions (Figure 4b). In the case of Class-0.4, only 53% of Class-0.3 fundus images are correctly classified (Figure 4e).  
Based on our observation, we doubted that one of the main reasons for the misclassifications problem is the quantity unbalance between each class of the original datasets: The number of fundus images in Class 1 is 2501, 5% of the total datasets, while that of Class 4 is 32,575, accounting for 59% of the total datasets. To solve this problem, we propose an ensemble method, as shown in Figure 5. In this approach, a fundus image is classified in a hierarchical way, by different machine learning methods which perform the best classification performance at each classification step.  
In the following, Classes 1 and 2 with a small number of fundus images are labeled as Class A, and Classes 3 and 4 with a large number of fundus images are labeled as Class B. Our method consists of three steps: In Step-1, a given fundus image is classified into either of Class A or Class B. In the following steps, the image classified into Class A is identified into either of Class 1 or Class 2 (Step-2-1), and the image in Class B is identified into either of Class 3 or Class 4 (Step-2-2).  
We use three different ML models: VGG19-based CNN (implemented by Tensorflow), EfficientNet-B7-based CNN (implemented by PyTorch), and SVM-RBF-kernel [30]. Table 5 shows the VA classification accuracy of each ML model when those ML models perform each VA classification of 4-Class, Step-1, Step-2-1, and Step-2-2, respectively. We selected the highest accuracy technique for each stage and completed the entire model of fundus image's VA classification machine learning based on these results.  
In our ensemble method, a fundus image is first identified as either Class A (Classes 1 and 2) or Class B (Classes 3 and 4) by VGG19-based-CNN. The fundus image classified as Class A (Classes 1 and 2) at Step 1 is identified, at Step-2-1, as either Class 1 or Class 2 by the SVM-RBF-Kernel. Similarly, the image identified as Class B (Classes 3 and 4) is further identified as either Class 3 or Class 4 by EfficientNet-B7-based CNN on Step-2-2.  
Figure 4: Confusion matrices from 4-Class VA Classifier’s classification results for 11 classes of fundus images. (**a**) Class-0.0; (**b**) Class-0.1; (**c**) Class-0.2; (**d**) Class-0.3; (**e**) Class-0.4; (**f**) Class-0.5; (**g**) Class-0.6; (**h**) Class-0.7; (**i**) Class-0.8; (**j**) Class-0.9; (**k**) Class-1.0.