# Investigating the Performance of FixMatch for COVID-19 Detection in Chest X-rays
## 4 Results and Discussion
### Comparing FixMatch to Supervised Models
Our analysis of the results revealed a number of interesting trends. The FixMatch algorithm is seen to be competitive to the fully supervised models even at lower amounts of labeled data. Indeed, the model was noted to perform adequately well until trained with only 20% of labeled data (corresponding to only about 80 labeled images per class), after which a drop in performance is seen, as the labeled data are reduced to 5%. It is seen that, even with just 5% of labeled data, the model performed fairly well, reporting an F1-score of 0.68, which is just 0.02 less than that reported by the fully supervised MobileNetV2 model. Figure 2: Precision vs. recall curves. Since our class of interest was that of COVID-19, it is useful to examine the results obtained specifically for that class. In general, it is seen that the COVID-19 class performed exceptionally well as compared to the other classes. Indeed, the FixMatch algorithm has outperformed the best supervised CNN with 80% labeled data. Furthermore, it is seen that, even with 5% of labeled data, the FixMatch model was able to show an F1-score of 0.93, matching that of the Xception CNN model and outperforming the ResNet50 model. These satisfactory results with the COVID-19 class further reinforce the usability of semi-supervised learning in the domain, as well as the adaptability of using this technique in cases where labeled data are limited. It is noteworthy that the architecture of the EfficientNet, which has lately emerged as one of the best performing CNN models, seemed to have completely failed to learn the intricacies of detecting the diseases in the chest X-rays. Three different architectures of increasing size, namely B1, B2 and B3, were implemented, but all of them gave an inadequate performance. In order to be able to compare the performance of our models to the initial work performed by [PERSON] et al. [8], the accuracies obtained from each of the runs were tabulated for the benchmark score of 81% obtained by the authors. The results can be seen in Table 9.  
It can be seen from Table 9 that, while FixMatch did outperform the CNN approaches, it remained slightly (\(<\)3%) inferior to the ensemble approach adopted by the authors, with an accuracy of just over 78% as compared to the 81% obtained by [PERSON] et al. [8]. However, given that the underlying architecture of the FixMatch is a simple ResNet18 CNN model, we see that it is much less computationally intensive as opposed to the multi-model ensemble approach proposed by the authors. Having established the efficacy of semi-supervised learning in the domain of COVID-19 detection by using chest X-rays, a number of tests were conducted in order to measure the impact of an imbalanced dataset on the training of the FixMatch model. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Class** & **InceptionV3** & **DenseNet121** & **Xception** & **ResNet50** & **MobileNetV2** & **EfficientNet** & **EfficientNet** & **B1** & **B2** & **B3** \\ \hline Bacterial & 0.69 & 0.68 & 0.63 & 0.63 & 0.46 & 0 & 0.01 & 0 \\ COVID-19 & **0.98*** & **0.97** & **0.93** & **0.89** & **0.97** & **0.33** & **0.29** & **0.39** \\ Lung Opacity & 0.79 & 0.8 & 0.75 & 0.7 & 0.74 & 0 & 0.51 & 0.04 \\ Normal & 0.61 & 0.63 & 0.59 & 0.68 & 0.63 & 0 & 0.19 & 0.14 \\ Viral & 0.78 & 0.76 & 0.74 & 0.74 & 0.7 & 0 & 0.5 & 0.6 \\ Macro-Average & 0.77 & 0.77 & 0.73 & 0.73 & 0.70 & 0.07 & 0.30 & 0.23 \\ \hline \hline \end{tabular}  
* Bold depicts the highest F1-score.  
\end{table} 
Table 8: Class-wise F1-score for supervised CNN models with entire labeled dataset. \begin{table}
\begin{tabular}{c c} \hline \hline
**Model** & **Accuracy** \\ \hline Ensemble Approach [8] & 81\% \\ InceptionV3 & 78 \\ DenseNet121 & 74 \\ Xception & 78 \\ ResNet-50 & 74 \\ MobileNetV2 & 72 \\ FixMatch with 80\% Labeled Data & 78 \\ FixMatch with 60\% Labeled Data & 78 \\ FixMatch with 20\% Labeled Data & 78 \\ FixMatch with 10\% Labeled Data & 78 \\ FixMatch with 10\% Labeled Data & 78 \\ FixMatch with 5\% Labeled Data & 72 \\ \hline \hline \end{tabular}
\end{table}