# Performance Analysis for COVID-19 Diagnosis Using Custom and State-of-the-Art Deep Learning Models
## 3 Methodology
### Experimental Setup
#### 3.3.5 Evaluation Metrics and Tools  
The authors aimed to provide and contribute a thorough and comprehensive assessment of deep learning models on the dataset. The selected different evaluation criteria were precision, recall, F1 score, accuracy, confusion matrix, and region of convergence curves (ROC). Each evaluation criterion is detailed below. \begin{table}
\begin{tabular}{l l c c c c c c} \hline \hline \multicolumn{1}{c}{**Architecture**} & \multicolumn{1}{c}{**Optimizer**} & \multicolumn{1}{c}{**Learning Rate**} & \multicolumn{1}{c}{**Batch Size**} & \multicolumn{1}{c}{**Image Shuffling**} & \multicolumn{1}{c}{**Number of Epochs for**} & \multicolumn{1}{c}{**Number of Epochs**} \\  & & & & **in Batches** & \multicolumn{1}{c}{**Convergence**} & & & \\ \hline Custom-Model & Adam & 1.00 \(\times\) 10\({}^{-3}\) & 32 & Yes & N/A & N/A & 200 \\ MobileNetV2 & Adam & 1.00 \(\times\) 10\({}^{-5}\) & 200 & Yes & 573 & 100 & 1000 \\  & & & & & 126 (Initial & 50 (Initial & \\ Xception & Adam & Initial LR = 1.00 \(\times\) 10\({}^{-3}\), & 32 & Yes & Convergence), & 50 (Initial & \\  & & Final LR = 1.00 \(\times\) 10\({}^{-5}\) & & & 146 (Final & \\  & & & & Convergence) & (Final & \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Hyperparameter Comparison. Precision  
Precision measures the accuracy of the positive diagnosed cases. In the subject context, it tells us how accurate the results are, if a person tests positive with COVID-19. In simple words, it is classified as:  
\[\text{Precision}=\frac{\text{True\,Positive}}{\text{Total\,Classified\,Positive}} \tag{6}\]  
The actual formula for the calculation in this research work is in (7). \[\text{Precision}=\frac{\text{True\,Positive}}{\text{True\,Positive}+\text{ False\,Positive}} \tag{7}\]  
Let \(C_{\downarrow}\) represent the subclasses of the class \(C\), \(C_{\downarrow}^{C}\) represent the positive classes predicted by the classifier, and \(C_{\downarrow}^{d}\) represent the class labels for positive cases in the dataset; precision for a subclass can then be defined as follows:  
\[\text{Precision}_{\downarrow}=\frac{\left|C_{\downarrow}^{C}\,\cap\,C_{ \downarrow}^{d}\right|}{C_{\downarrow}^{C}} \tag{8}\]  
2. Recall  
This tells the percentage of true positive predicted by the actual number of true positives. It can be calculated using the formulae in (9) and (10). \[\text{Recall}=\frac{\text{True\,Positive}}{\text{Total\,Actual\,Positive}} \tag{9}\]  
\[\text{Recall}=\frac{\text{True\,Positive}}{\text{True\,Positive}+\text{ False\,Negative}} \tag{10}\]  
As defined for precision, recall can similarly be defined using the same representations \(C\), \(C_{\downarrow}^{C}\). and \(C_{\downarrow}^{d}\), as shown below. \[\text{Recall}_{\downarrow}=\frac{\left|C_{\downarrow}^{C}\,\cap\,C_{ \downarrow}^{d}\right|}{C_{\downarrow}^{d}} \tag{11}\]  
3. F1 Score  
This shows us the balance between precision and recall. The F1 score for each class is calculated employing the mathematical statement in (12) below. \[\text{F1 Score}_{\downarrow}=2\,\times\,\frac{\text{Precision}_{\downarrow} \times\text{Recall}_{\downarrow}}{\text{Precision}_{\downarrow}+\text{ Recall}_{\downarrow}} \tag{12}\]  
For COVID-19, the F1 score will measure the potency of the model to detect all the COVID-19 cases, to avoid other classes. 4. Accuracy  
This is the percentage of correctly identified classes divided by the total number of cases. For a multiclass problem with classes \(i\), mathematically, it is given in (5). \[\text{Accuracy}=\frac{\text{True\,Positive}_{\text{i}}+\text{True\,Negative} _{\text{i}}}{\text{True\,Positive}_{\text{i}}+\text{True\,Negative}_{ \text{i}}+\text{False\,Negative}_{\text{i}}} \tag{13}\]ROC (Receiver Operating Characteristic) Curve  
The ROC curve shows the performance of the classifier at different threshold settings. By varying threshold settings, TPR (true positive rate or recall or sensitivity) is plotted against the FPR (false positive rate or specificity). AUC (area under the curve) represents the area under the ROC curve. 6.