# Investigating the Performance of FixMatch for COVID-19 Detection in Chest X-rays
## 4 Results and Discussion  
### Comparing FixMatch to Supervised Models  
In order to effectively compare the performance of the FixMatch semi-supervised algorithm to existing supervised CNN approaches, the macro-average F1-scores of each of the models were computed and plotted. The formula for the F1-score can be seen in Equation (3), while the formulas for precision and recall can be seen in Equations (4) and (5). \[\text{F1}-\text{Score}=2\times\frac{(\textit{precision}\times\textit{recall})}{( \textit{ precions}+\textit{recall})} \tag{3}\]  
where we have the following:  
\[\textit{Precision}=\frac{\textit{True}\,\textit{Positives}}{\textit{True}\, \textit{Positive}+\textit{False}\,\textit{Positive}} \tag{4}\]  
\[\textit{Recall}=\frac{\textit{True}\,\textit{Positives}}{\textit{True}\, \textit{Positive}+\textit{False}\,\textit{Negatvie}} \tag{5}\]  
Computing the macro-average F1-score involves computing the F1-scores for each class and then averaging it out such that each class has an equal weight in the overall score. To ensure an equal say from each class, this metric was used in order to compare the F1-scores. The class-wise F1-scores of the FixMatch model with varying amounts of labeled data can be seen in Table 7. As Table 7 shows, the average macro-average F1-score across all tests was about 0.74. However, COVID-19 was detected (from others) with a high rate of over 0.87 and as high as 0.94 and 0.93, using only 5% of the labeled data. Interestingly, the model had the most problem detecting normal against others. This is especially noteworthy because the model is trained on equal amounts of data across the classes, and, therefore, in order to gain a better understanding of the considerable difference in scores, an analysis of the data themselves is carried out in Section 4. \begin{table}
\begin{tabular}{c c} \hline
**Parameter** & **Value** \\ \hline Batch Size & 64 \\ Epochs & 100 \\ Optimizer & Stochastic Gradient Descent \\ Batch Size & 32 \\ Learning Rate & 0.01 \\ \hline \end{tabular}
\end{table} 
Table 6: Semi-supervised training parameters. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Class**} & **FixMatch 80\%** & **FixMatch 60\%** & **FixMatch 40\%** & **FixMatch 20\%** & **FixMatch 10\%** & **FixMatch 5\%** \\  & **Labeled** & **Labeled** & **Labeled** & **Labeled** & **Labeled** & **Labeled** \\ \hline Bacterial & 0.68 & 0.64 & 0.7 & 0.66 & 0.66 & 0.67 \\ COVID-19 & **0.99*** & **0.97** & **0.97** & **0.94** & **0.87** & **0.93** \\ Lung Opacity & 0.79 & 0.8 & 0.8 & 0.76 & 0.76 & 0.79 \\ Normal & 0.67 & 0.59 & 0.68 & 0.5 & 0.55 & 0.38 \\ Viral & 0.74 & 0.69 & 0.74 & 0.74 & 0.74 & 0.64 \\ Macro-Average & 0.77 & 0.74 & 0.78 & 0.72 & 0.71 & 0.68 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Class-wise F1-score of FixMatch. In order to further investigate these results, the precision vs. recall graphs of the tests are shown in Figure 2.  
As the figure shows, in all cases, the COVID-19 class outperformed the remaining classes by considerable margins. As expected, the curves do become steeper as the amount of labeled data is reduced, thereby serving as an indicator for the model's deteriorated performance. It can be noted here that the recall of the COVID-19 class consistently remains high, and this is indicative of a low false negative rate. This is especially important because it means that, while there may be cases where the model may diagnose a healthy individual with COVID-19, it is less likely to fail diagnosing a patient with COVID-19. Considering how contagious the COVID-19 disease is, this is an important characteristic for the model to have and, therefore, proves the model's suitability in being used for COVID-19 detection. Table 8 shows the results of using transfer learning with the various pretrained models. As Table 8 shows, using the complete data, InceptionV3 and DenseNet121 performed well, with F1-scores of 0.98 and 0.97. Other models, such as EfficientNet, B3 did not perform well at all.