# FocalMatch: Mitigating Class Imbalance of Pseudo Labels in Semi-Supervised Learning
## 5 Results  
Our experiments use top-1 classification accuracy as the evaluation metric for all baseline methods and FocalMatch. The result is shown in Table 1. It shows that FocalMatch outperforms all baseline methods on most of the benchmarks. FocalMatch performs particularly well when the number of labeled data is extremely small (i.e., four labels per class). However, FocalMatch does not perform as well as expected on SVHN with 10 labels per class. FixMatch and UDA outperform their accuracy by around 0.2% and 0.5%, respectively. We believe this is due to the simplicity of the SVHN dataset. When the amount of labeled data is extremely small (i.e., four labels per class), the model is not able to produce valid pseudo labels evenly for all classes since the overall prediction confidence is not high enough. This causes a severe class imbalance problem which can be effectively alleviated by FocalMatch. When we increase the number of labeled data in the SVHN experiment (i.e., 10 labels per class), the model is confident enough to generate pseudo labels evenly, and the accuracy of each class is relatively high. Therefore, the loss contribution adjustment of FocalMatch can slow down the learning of the model. This may also explain the reason why FocalMatch achieves significant performance improvement when the number of labeled data is extremely small, whereas the performance improvement of FocalMatch reduces as the number of labeled data increases (i.e., less challenging to classify). FocalMatch has substantially extended the learning ability of FixMatch by resolving the latent class imbalance issue. Our method not only outperforms FixMatch in terms of \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline  & \multicolumn{2}{c}{**CIFAR-10**} & \multicolumn{2}{c}{**CIFAR-100**} & \multicolumn{2}{c}{**SVHN**} \\  & **40 Labels** & **100 Labels** & **400 Labels** & **1000 Labels** & **40 Labels** & **100 Labels** \\ \hline \(\Pi\)-Model & 22.92 \(\pm\) 1.26 & 34.98 \(\pm\) 1.53 & 12.34 \(\pm\) 1.37 & 26.17 \(\pm\) 2.31 & 31.33 \(\pm\) 0.75 & 78.88\(\pm\) 0.32 \\ Mean Teacher & 27.13 \(\pm\) 1.31 & 44.41 \(\pm\) 2.42 & 14.31 \(\pm\) 1.53 & 29.50 \(\pm\) 3.67 & 64.04 \(\pm\) 3.18 & 79.83 \(\pm\) 4.41 \\ MixMatch & 61.64 \(\pm\) 3.47 & 79.24 \(\pm\) 2.63 & 22.96 \(\pm\) 2.16 & 44.62 \(\pm\) 2.47 & 71.4 \(\pm\) 2.63 & 96.09 \(\pm\) 0.29 \\ ReMixMatch & 90.26 \(\pm\) 1.41 & 91.96 \(\pm\) 0.75 & 44.03 \(\pm\) 1.33 & 57.49 \(\pm\) 0.95 & 76.27 \(\pm\) 9.54 & 94.18 \(\pm\) 0.48 \\ UDA & 85.31 \(\pm\) 4.37 & 92.33 \(\pm\) 0.23 & 43.17 \(\pm\) 1.41 & 57.85 \(\pm\) 0.71 & 95.36 \(\pm\) 3.47 & 97.92 \(\pm\) 0.04 \\ FixMatch & 89.94 \(\pm\) 0.34 & 92.87 \(\pm\) 0.17 & 43.38 \(\pm\) 1.09 & 57.99 \(\pm\) 0.69 & 96.79 \(\pm\) 1.42 & 97.71 \(\pm\) 0.15 \\ \hline FocalMatch & 92.29 \(\pm\) 0.27 & 93.09 \(\pm\) 0.15 & 46.02 \(\pm\) 0.86 & 58.70 \(\pm\) 0.31 & 97.37 \(\pm\) 1.25 & 97.53 \(\pm\) 0.09 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Accuracy Comparison in Different Methods. classification accuracy on CIFAR-10, CIFAR100, and SVHN (except when the number of labels per class is 10 on the SVHN dataset) but also speeds up the convergence of the model. We compare the convergence speed of our method and FixMatch in terms of the overall top-1 accuracy and loss in Figure 3a,b. It is obvious that the loss curve of FocalMatch is smoother and converges faster compared to FixMatch. Following the approach described in [44], we also compare the accuracy of FixMatch and FocalMatch for each class in Figure 3c,d. It is observed that there is a large gap between the accuracy of each class in FixMatch, which is due to the class imbalance on the pseudo labels generated in the unsupervised learning part. The total unsupervised loss of FixMatch tends to be dominated by classes with a large number of pseudo labels instead of learning from the overall unlabeled data. This could explain why the accuracy of some classes is appreciably lower than that of other classes or even not improving at all (e.g., class 5).