# Survey on Recent Trends in Medical Image Classification Using Semi-Supervised Learning
## 4 Survey Results
### Consistency-Based Methods
On the other hand, MT provided lower scores than the supervised baseline at higher percentages of labeled data. Lastly, the Knee MRNet dataset consists of 1370 data points labeled with the presence or absence of abnormality, leading to a single labeled classification task. 80.6% of the data points contain abnormalities. 1130 training scans and 120 testing scans were used, with 20% of the training scans being randomly sampled for the validation set. Similar to the previous experiments, NoTeacher achieved higher scores than both the supervised baseline and the other semi-supervised methods. Additionally, at the labeled data percentage of 50%, NoTeacher managed to achieve a score higher than the supervised baseline at 100% labeled data. Self-supervised Mean Teacher for Semi-supervised (S2 MTS2) [55]: S2 MTS2 is a consistency-based SSL method for chest X-ray classification, which uses the Mean Teacher framework. This method consists of two learning stages. In the first stage, the student-teacher model is pre-trained in a self-supervised manner on the labeled and unlabeled data using JCL [41]. JCL involves learning an infinite number of key-query pairs generated from unlabeled data with the goal of enforcing dependencies between different key-query pairs containing the same query, which leads to more consistent representations in each instance-specific class [41]. Thus, each query \(q_{i}\) paired simultaneously with multiple of its positive keys \(k_{i,m}^{+}\) must return a low loss value. The loss of each \(\left(q_{i},k_{i,m}^{+}\right)\) pair is defined as follows:  
\[\mathcal{L}_{i,m}=log\frac{exp\left[\frac{1}{\tau}q_{i}^{\top}k_{i,m}^{+} \right]}{exp\left[\frac{1}{\tau}q_{i}^{\top}k_{i,m}^{+}\right]+\sum_{j=1}^{K} exp\left[\frac{1}{\tau}q_{i}^{\top}k_{i,j}^{-}\right]} \tag{10}\]  
where \(\tau\) is the temperature hyperparameter, \(k_{i,m}^{+}\) is the \(m\)th positive key of \(q_{i}\) and \(k_{i,j}^{-}\) is the \(j\)th negative key of \(q_{i}\). Thus, the total JCL loss can be calculated as follows:  
\[\ell_{p}\Big{(}\mathcal{D}^{\mathcal{X}},\theta_{2},\theta_{2}^{\prime}\Big{)} =-\frac{1}{\left|\mathcal{D}^{\mathcal{X}}\right|}\frac{1}{M}\sum _{i=1}^{\left|\mathcal{D}^{\mathcal{X}}\right|}\sum_{m=1}^{M}[\mathcal{L}_{i,m}] \tag{11}\]where \(\mathcal{D^{X}}\) is the set of labeled and unlabeled images, and \(M\) is the number of positive keys. In the second stage, the pre-trained student-teacher model is fine-tuned using the original Mean Teacher approach by maintaining an EMA following Equation (1). The datasets used for testing were the Chest X-ray14, ISIC2018 Skin Lesion Analysis, and CheXpert. The CheXpert dataset contains 220,000 images labeled with 14 diseases, wherein each image can have more than one label. The backbone model used for all datasets was DenseNet121 [23], with the two-layer multi-layer perceptron (MLP) projection head being replaced with a three-layer MLP. The Chest X-rat14 dataset was resized to 512 \(\times\) 512, and the ISIC2018 and CheXpert datasets were resized to 224 \(\times\) 244. The data augmentations used were random resize, random crop, random horizontal flipping, and random rotation. The Chest X-ray14 dataset was evaluated on different percentages of labeled data (2%, 5%, 10%, 15%, 20%), resulting in AUC scores of 74.69, 78.96, 79.90, 80.31, 81.06, and 82.51. For the CheXpert dataset, the model achieved AUC scores of 66.15, 67.85, 70.83, 71.37, and 71.58 for 100, 200, 300, 400, and 500 labeled samples, respectively. Lastly, the results obtained on the ISIC2018 dataset with 20% labeled samples were an AUC score of 94.71% and an F1 score of 62.67. A summary of the methods discussed in this section is provided in Table 1. Three consistency-based methods were reviewed: SRC-MT, NoT, and S\({}^{2}\)MTS\({}^{2}\). For each model, the table contains the datasets used for testing as well as the best performance score obtained on each dataset.