# Multi-Classification of Chest X-rays for COVID-19 Diagnosis Using Deep Learning Algorithms
## 3 Methodology
### Classification Stage  
To make the model suitable for feature extraction, we do not include the classification layers at the top. After importing the base model from _Keras_, we have some additional layers in our structure. The global average pooling (GAP) layer is added immediately after the base model to reduce the total number of parameters that may lead the model to overfitting. In the next step, we applied the flatten layer, which converts the two-dimensional feature matrix into a vector. After the flatten process, two dense layers with 1024 and 512 neurons, respectively, were added which use the activation function with a threshold equal to alpha, \(\alpha\), followed by the dropout layer with a value of \(\Gamma\). In the proposed model, we used two activation functions. The first is a LeakyReLU activation  
Figure 2: Detailed structure of the Xception model [39]. Reprinted with permission from [39]. Copyright 2017 IEEE.  
function and the second activation is an exponential linear unit (ELU). Those functions are mathematically described by Equations (2) and (3), respectively:  
\[LeakyReLU(R_{i})=\left\{\begin{array}{ll}\alpha R_{i},&R_{i}<0\\ R_{i},&R_{i}\geq 0\end{array}\right. \tag{2}\]  
\[ELU(R_{i})=\left\{\begin{array}{ll}R_{i}&R_{i}\geq 0\\ \alpha\big{(}e^{R_{i}}-1\big{)}&R_{i}<0\end{array}\right. \tag{3}\]  
where \(R_{i}\) is the input data. The main goal of the dropout layer is to avoid the occurrence of overfitting. An activation layer with an activation function was added between the first dropout layer and the second fully connected layer with a threshold equal to \(\alpha\). This layer allows a small gradient during the training process, and it reduces the losses of the model. The main objective of our research is to study the effect of changing \(\alpha\) values and applying different optimizers on the classification accuracy. We performed various experiments to examine the best performance of the model with minimum losses. The model has a softmax layer at the end to generate the output according to the softmax equation. The softmax activation function is defined by Equation (4):  
\[Z(T_{i})=\frac{e^{T_{i}}}{\sum_{k=1}^{M}e^{T_{k}}} \tag{4}\]  
where \(T_{i}\) represents the input data to the softmax layer from the previous layer and \(M\) is the total number of classes. The softmax converts the scores (\(T_{i}\)) to a normalized probability, such that the input belongs to the \(i\)th class. The denominator in Equation (4) ensures that the output sums to unity (i.e.,\"1\"). In the final step, the categorical cross entropy loss function was applied with the Adam optimizer (other optimization algorithms were also investigated). It uses the predicted class (or softmax output) and the ground truth probabilities of the class to calculate the loss, which is defined by Equation (5):  
\[CLF=-\sum_{n=1}^{S}Y_{n}\cdot\log Y_{n^{*}} \tag{5}\]  
where \(S\) is the total number of scalar values in the output, \(Y_{n}\) is the corresponding target value and \(Y_{n}^{*}\) is the \(n\)th scalar value in the output of the model. The proposed model has a total of 23,485,995 parameters, divided into 23,431,467 and 54,528 trainable and non-trainable parameters, respectively, as shown in Table 1.  
\begin{table}
\begin{tabular}{l c c} \hline \hline
**Layer** & **Out Shape** & **No. of Parameters** \\ \hline Input & (200, 200, 3) & 0 \\ Xception (functional) & (None, 7, 7, 2048) & 20,861,480 \\ Global average pooling2D & (None, 2048) & 0 \\ Flatten & (None, 2048) & 0 \\ Dense & (None, 1024) & 2,098,176 \\ Dropout & (None, 1024) & 0 \\ Activation & (None, 1024) & 0 \\ Dense & (None, 512) & 524,800 \\ Dropout & (None, 512) & 0 \\ Dense & (None, 3) & 1539 \\ Total parameters & & 23,485,995 \\ Trainable parameters & & 23,431,467 \\ Non-trainable parameters & & 54,528 \\ Gamma (dropout layer’s threshold) & & 0.1 \\ Batch size & & 32 \\ No. of Epochs & & 10 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Summary of the parameter settings of the proposed structure.