SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Although many efficient methods for implementing SAM have already been proposed [22, 23, 47, 83], they commonly need extra regularization or gradient decomposition, which are not harmonious to SSL and would complicate our method. In our scenario, FlatMatch has a critical difference from SAM regarding the two propagation steps: The gradients from the first step and second step are not computed on the same batch of data, i.e., they are crossed from labeled data to unlabeled data. Since in SSL, each labeled batch and unlabeled batch are randomly coupled, we are allowed to use the gradient computed from the last batch of labeled data to obtain the cross-sharpness from the current batch of unlabeled data. Even better, we can use exponential moving average (EMA) [30, 44, 59] to stabilize the gradient so that our cross-sharpness can be computed accurately. Next, we conduct extensive experiments to carefully validate our approach.
In this section, we conduct extensive comparisons and analyses to evaluate the proposed FlatMatch method. We first describe the experimental setup and implementation details. Then we compare FlatMatch with many edge-cutting SSL approaches to show the effectiveness of our method. Further, we conduct an ablation study to justify our design of FlatMatch. Finally, we demonstrate the efficiency, stability, and sensitivity of FlatMatch through various analytical studies.
5.1 Experimental Setup and Details
We follow the most common semi-supervised image classification protocols by using CIFAR10/100 [42] SVHN [50], and STL10 [18] datasets where a various number of labeled data are equally sampled from each class. Following Wang et al. [66], We choose Wide ResNet-28-2 [78] for CIFAR10 and SVHN, Wide ResNet-28-8 for CIFAR100, ResNet-37-2 [31] for STL10. All
Table 1: Error rates on CIFAR10/100, SVHN, and STL10 datasets. The fully-supervised results of STL10 are unavailable since we do not have label information for its unlabeled data. The best results are highlighted with Bold and the second-best results are highlighted with underline.
Dataset	CIFAR10	CIFAR100	SVHN	STL10
# Label	10	40	250	4000	400	2500	10000	40	250	1000	40	1000
II-Model [44]	79.18±1.11	74.34±1.76	46.24±1.29	13.13±0.59	86.96±0.80	58.80±0.66	36.65±0.00	67.48±0.95	13.30±1.12	7.16±0.11	74.31±0.85	32.78±0.40
Pseudo Label [46]	80.21±0.55	74.61±0.26	46.49±2.20	15.08±0.19	87.45±0.85	57.74±0.28	36.55±0.24	64.61±5.6	15.59±0.95	9.40±0.32	74.68±0.99	32.64±0.71
VAT [49]	79.81±1.17	74.66±2.12	41.03±1.79	10.51±0.12	85.20±1.40	46.84±0.79	32.14±0.19	74.75±3.38	4.33±0.12	4.11±0.20	74.74±0.38	37.95±1.12
Mean Teacher [59]	76.37±0.44	70.09±1.60	37.46±3.30	8.10±0.21	81.11±1.44	45.17±1.06	31.75±0.23	36.09±3.98	3.45±0.03	3.27±0.05	71.72±1.45	33.90±1.37
MixMatch [8]	65.76±7.06	36.19±6.48	13.63±0.59	6.66±0.26	67.59±0.66	39.76±0.48	27.78±0.29	30.60±8.39	4.56±0.32	3.69±0.37	54.93±0.96	21.70±0.68
ReMixMatch [7]	20.77±7.48	9.88±1.03	6.30±0.05	4.84±0.01	42.75±1.05	26.03±0.35	20.02±0.27	24.04±9.13	6.36±0.22	5.16±0.31	32.12±6.24	6.74±0.14
UDA [74]	34.53±10.69	10.62±3.75	5.16±0.06	4.29±0.07	46.39±1.59	27.73±0.21	22.49±0.23	5.12±4.27	1.92±0.05	1.89±0.01	37.42±8.44	6.64±0.17
FixMatch [57]	24.79±7.65	7.47±0.28	4.86±0.05	4.21±0.08	46.42±0.82	28.03±0.16	22.20±0.12	3.81±1.18	2.02±0.02	1.96±0.03	35.97±4.14	6.25±0.33
Dash [76]	27.28±14.09	8.93±3.11	5.16±0.23	4.36±0.11	44.82±0.96	27.15±0.22	21.88±0.07	2.19±0.18	2.04±0.02	1.97±0.01	34.52±4.30	6.39±0.56
MPL [52]	23.55±6.01	6.62±0.91	5.76±0.24	4.55±0.04	46.26±1.84	27.71±0.19	21.74±0.09	9.33±8.02	2.29±0.04	2.28±0.02	35.76±4.83	6.66±0.00
FlexMatch [79]	13.85±12.04	4.97±0.06	4.98±0.09	4.19±0.01	39.94±1.62	26.49±0.20	21.90±0.15	8.19±3.20	6.59±2.29	6.72±0.30	29.15±4.16	5.77±0.18
SoftMatch [16]	-	4.91±0.12	4.82±0.09	4.04±0.02	37.10±0.77	26.66±0.25	22.03±0.03	2.33±0.25	-	2.01±0.01	21.42±3.48	5.73±0.24
FreeMatch [66]	8.07±4.24	4.90±0.04	4.88±0.18	4.10±0.02	37.98±0.42	26.47±0.20	21.68±0.03	1.97±0.02	1.97±0.01	1.96±0.03	15.56±0.55	5.63±0.15
FlatMatch	15.23±8.67	5.58±2.36	4.22±1.14	3.61±0.49	38.76±1.62	25.38±0.85	19.01±0.43	2.46±0.06	1.43±0.05	1.41±0.04	16.20±4.34	4.82±1.21
FlatMatch-e	15.69±6.35	5.63±1.87	4.53±1.85	3.57±0.50	38.98±1.53	25.62±0.88	19.78±0.89	2.66±0.09	1.47±0.08	1.46±0.07	16.32±4.64	5.03±1.06
FlatMatch (Fix label)	7.36±5.62	4.89±1.24	3.90±1.72	3.61±0.49	36.97±0.95	25.38±0.85	19.01±0.43	2.14±0.05	1.43±0.05	1.41±0.04	14.96±0.67	4.82±1.21
Fully-Supervised	4.62±0.05	19.30±0.09	2.13±0.01	-
SSL approaches are implemented using Pytorch framework, and the computation is based on 12GB NVIDIA 3090 GPU. We compare our method with well-known approaches, including II-model [44], Pseudo Label [46], VAT [49], Mean Teacher [59], MixMatch [8], ReMixMatch [7], UDA [74], FixMatch [57], Dash [76], MPL [52], FlexMatch [79], SoftMatch [16], and FreeMatch [66].
To optimize all methods, we use SGD with a momentum of 0.9 with an initial learning rate of 0.03. We set the batch size as 64 for all datasets. Moreover, we set the weight decay value, pseudo label threshold  $\tau$ , unlabeled batch ratio  $\mu$ , and trade-off for SSL regularization as the same for all compared methods. To implement our FlatMatch, we set the perturbation magnitude  $\rho = 0.1$ , and EMA factor  $\alpha = 0.999$  which is the same for Mean Teacher and other methods that require EMA. Moreover, in line 9 from Alg. 1, we need to separate the gradients regarding labeled data and unlabeled data in one back-propagation step. Such an operation can be done using "functorch.make_functional_withBuffers" in Pytorch, which can efficiently compute the gradients for each sample. For more details, please see the appendix.
5.2 Quantitative Comparison
The main comparison results are shown in Table 1, we can see that our FlatMatch achieves state-of-the-art results on many scenarios. For example, on CIFAR100 with 2500 labels, FlatMatch achieves  $25.38\%$  test errors, surpassing the second-best method, FreeMatch, for  $1.09\%$ ; and on CIFAR100 with 10000 labels, FlatMatch achieves  $19.01\%$  performance, further improving the second-best result, ReMixMatch, for  $1.01\%$ , which even passes the fully-supervised baseline result for  $0.29\%$ . Moreover, FlatMatch reaches extremely low error rates in many settings. For instance, in CIFAR10 with 250 labels and 4000 labels, FlatMatch only has  $4.22\%$  and  $3.61\%$  error rates; and in SVHN with 250 labels and 1000 labels, FlatMatch can also surpass the fully-supervised learning by producing  $1.43\%$  and  $1.41\%$  error rates, respectively. Additionally, FlatMatch breaks the record on the STL10 dataset with 1000 labels by reaching  $4.82\%$  performance.
Moreover, we also show the performance of FlatMatch-e, an efficient variant of FlatMatch, which can also surpass many existing state-of-the-art methods. Such as in CIFAR100 with 10000 labels, FlatMatch-e can beat the second-best method, SoftMatch, with a  $2.04\%$  improvement. More importantly, FlatMatch-e only shows slight performance drops from FlatMatch but significantly reduces the computational burden, which is demonstrated in Section 5.4.
Despite its effective performance, we also noticed that in some scenarios where the labels are extremely limited, such as CIFAR10 with 10 labels, CIFAR100 with 400 labels, etc, our FlatMatch is slightly worse than a recently proposed method FreeMatch [66]. This is because the gradient esti
Figure 3: Gradient angle between cross-entropy loss  $\mathcal{L}_l$  and general SSL loss  $\mathcal{L}_{ssl}$ .
mated by the extremely scarce labeled data cannot align with the general training direction. To intuitively demonstrate this issue, we show the gradient angle between the cross-entropy loss from labeled data and the total SSL loss in Fig. 3. We can see that the gradient angle oscillates stronger as
Algorithm 2 FlatMatch with fixed labels for Stabilizing Cross-Sharpness
Input: Labeled dataset  $\mathcal{D}^l$  , unlabeled dataset  $\mathcal{D}^u$  , model  $\theta$  , number of fixed labels #fix, number of pre-train epochs #pre_train. 1: for  $t\in 0,1,\ldots ,T - 1$  do 2: if  $t\in 0,1,\dots ,\# pre\_ train$  then 3: SSL pre-training by optimizing Eq. 3; 4: if  $t = \# pre\_ train$  then 5: Select Top-#fix confident unlabeled data  $x_{i}^{u}$  and assign fixed labels  $\hat{y}_i\coloneqq \mathrm{argmax}(g_\theta (x_i^u))$  . 6: else 7: Using the augmented labeled data to compute cross-sharpness via Eq. 4; 8: Using the original  $\mathcal{D}^l$  and  $\mathcal{D}^u$  to optimize  $\theta$  via Eq. 5;
the number of labels gets smaller. When the label number reduces to only 10, the angle becomes extremely unstable and could surpass  $90^{\circ}$ . As a result, such a phenomenon could affect the worst-case perturbation computed by FlatMatch, which would introduce negligible noise to the training process.
However, this drawback can be properly solved by slightly augmenting the number of labels with pseudo-labeled unlabeled data. Specifically, before we start minimizing cross-sharpness, we first pre-train the backbone model with a common SSL method, such as FixMatch or FreeMatch, for 16 epochs to increase the confidence on unlabeled data. Then, we choose the most confident, i.e. high softmax probability, unlabeled data to be pseudo-labeled as labeled data. Different from other unlabeled data that are also trained with pseudo labels, the selected unlabeled data are fixed with their labels and act as labeled data when computing the sharpness. In this way, the cross-sharpness would be computed accurately by using both the original labeled data and augmented labeled data. Note that the augmentation with fixed labels is only used in the sharpness computation, when conducting the second propagation step, all unlabeled data are treated similar to general SSL strategy, such process is summarized in Algorithm 2. Under this training strategy, we dub our method as "FlatMatch (Fix label)" and show its result in Table 1. The number of fixed labeled data in CIFAR10, CIFAR100, SVHM, and STL10 is set to 500, 2500, 500, and 1000, respectively. If the number of the original labeled dataset contains enough labels, we do not add more fixed labeled data. We can see that in this scenario, our method has the best performance in all settings including the ones with 1 or 4 labels in each class, which demonstrates that using more labels can largely stabilize sharpness computation and further benefits the performance of FlatMatch, thus achieving superior results on all settings.
To carefully justify the design of our method, we compare FlatMatch with "sharp. on labeled data  $\mathcal{D}^l$  and "sharp. on unlabeled data  $\mathcal{D}^u$  where the former one denotes both the worst-case model  $\hat{\theta}$  and sharpness is computed on labeled data and the latter one denotes  $\hat{\theta}$  and sharpness are calculated on unlabeled data. Additionally, we compute the sharpness on the full dataset, as denoted by "sharp. on unlabeled data  $\mathcal{D}^l\cup \mathcal{D}^u$  . Moreover, we also analyze the effect of EMA smoothing on the performance of FlatMatch-e. Specifically, we compare FlatMatch-e with setting "w/o EMA" that just uses a gradient from the last batch of labeled data to calculate our worst-case model. The ablation study is conducted using CIFAR100 with a varied number of labels, which is shown in Table 2.
First, we can see that both two choices of our method are effective. Particularly, computing sharpness only on labeled data  $\mathcal{D}^l$  shows smaller performance degradation than computing sharpness on unlabeled data  $\mathcal{D}^u$ , and it even shows better results than FixMatch. Hence, we know that the sharpness of labeled data can indeed improve the performance of SSL, only having limited performance because the number of labeled data is too scarce. On the other hand, when sharpness is completely based on unlabeled data, the performance significantly drops by nearly  $10\%$
Table 2: Ablation study on CIFAR100.
Dataset	CIFAR100
# Label	400	2500	10000
sharp. on Dl	42.63±0.34	26.85±0.45	21.79±0.24
sharp. on Du	49.45±2.76	36.30±2.01	27.05±2.98
sharp. on Dl ∪ Du	43.88±1.64	27.26±1.62	23.42±1.77
FlatMatch	38.76±1.62	25.38±0.85	19.01±0.43
w/o EMA	40.64±0.97	29.44±1.56	23.23±1.28
FlatMatch-e	38.98±1.53	25.62±0.88	19.78±0.89
compared to "sharpness on  $\mathcal{D}^l$ . This is because the training process on unlabeled data contains too much noise which causes erroneous gradient computation that would hinder the effectiveness of penalizing sharpness. Furthermore, we find that "w/o EMA" shows slightly inferior performance
Figure 4: Loss landscapes of labeled data and unlabeled data obtained simultaneously from training using FlatMatch on CIFAR10 with 250 labels per class. The first row and second row show the results obtained from epoch 60 and epoch 150, respectively. The first column and second column show the 2D loss contours of labeled data and unlabeled data, respectively, and the last column shows the 1D loss curves.
to FlatMatch-e. Such degradation is consistent with the findings from Liu et al. [47] that the last gradient direction is distinct from the current one. As gradient descent has been conducted in the previous batch, reusing the gradient to perturb the current model might not find the perfect worst-case model on the current labeled data. Based on our observation, using EMA can stabilize the gradient can lead to accurate sharpness calculation.
Figure 5: Parameter sensitivity analysis regarding perturbation magnitude  $\rho$  on CIFAR100.
Figure 6: Stability analysis on gradient norm from training on CIFAR100.
Figure 7: Training efficiency comparison on CIFAR100 with 10000 labels.
5.4 Analytical Study
In this section, we analyze the performance of FlatMatch by considering visualization, parameter sensitivity, training stability, and .efficiency.
Loss visualization: To show that FlatMatch can properly solve the sharp loss problem of labeled data, we train FlatMatch under the same setting as the ones demonstrated in Section 1 and plot the 2D loss contour as well as the 1D loss curve in Fig. 4. We can see that our FlatMatch can produce a much flatter loss landscape than FixMatch does in Fig. 1, where the jagged curve has been eliminated and become very smooth. Therefore, by minimizing cross-sharpness, the generalization performance on labeled data can be largely improved.
Sensitivity analysis: Our FlatMatch requires a hyperparameter  $\rho$  which controls the perturbation magnitude to obtain the worst-case model. To analyze the performance sensitivity of varying  $\rho$ , we show the result on CIFAR100 in Fig. 5. We observe that small  $\rho$  values show little impact on the test performance. However, when  $\rho$  increases to more than 0.25, the performance would largely degrade. Moreover, among three settings with varied label numbers, we find that more numbers labels could enhance the model sensitivity against changing of  $\rho$ . For example, when changing the  $\rho$  from 0.1
to 0.25, the performance difference on 400 labels, 2500 labels, and 10000 labels are  $0.05\%$ ,  $1.2\%$ ,  $3.42\%$ . Generally, the optimal value for  $\rho$  is 0.1.
Training stability: To validate the training stability of FlatMatch, we show the gradient norm which is an important criterion to present gradient variation in Fig. 6. We observe that the gradient norm of FlatMatch is significantly smaller than FreeMatch during the whole training phase. Therefore, minimizing the cross-sharpsness can indeed improve training stability.
Training efficiency: We compare the training time and test accuracy of FlatMatch, FlatMatch-e, FreeMatch, FlexMatch, and FixMatch to validate the efficiency property of our method. As shown in Fig. 7, we find that despite the superior accuracy performance, FlatMatch requires more time for each iteration than the other three methods. However, the efficient variant FlatMatch-e can largely reduce the computational cost without losing too much learning performance. Hence, we can conclude that leveraging EMA for computing cross-sharpness is both effective and efficient.
6 Conclusion, Limitation and Broader Impact
In this paper, we propose a novel SSL method dubbed FlatMatch to address the mismatched generalization performance between labeled data and unlabeled data. By minimizing the proposed cross-sharpness regularization, FlatMatch can leverage the richness of unlabeled data to improve the generalization performance on labeled data. As a result, the supervised knowledge from labeled data can better guide SSL, and achieve improved test performance than most existing methods. Moreover, we propose an efficient implementation to reduce the computation cost. We conduct comprehensive experiments to validate our method regarding effectiveness, sensitivity, stability, and efficiency. Additionally, FlatMatch is slightly limited under barely-supervised learning due to the requirement of enough labeled data. Our method shows that SSL can be further improved by exploring generalization, which could be a potential direction in future research.
Li Shen was partially supported by STI 2030—Major Projects (2021ZD0201405). Zhuo Huang was supported by JD Technology Scholarship for Postgraduate Research in Artificial Intelligence (SC4103). Jun Yu was supported by the Natural Science Foundation of China (62276242), National Aviation Science Foundation (2022Z071078001), CAAI-Huawei MindSpore Open Fund (CAAIXSJLJJ-2021-016B, CAAIXSJLJJ-2022-001A), Anhui Province Key Research and Development Program (202104a05020007), USTC-IAT Application Sci. & Tech. Achievement Cultivation Program (JL06521001Y), and Sci. & Tech. Innovation Special Zone (20-163-14-LZ-001-004-01). Bo Han was supported by the NSFC Young Scientists Fund (62006202), NSFC General Program (62376235), and Guangdong Basic and Applied Basic Research Foundation (2022A1515011652). Tongliang Liu was partially supported by the following Australian Research Council projects: FT220100318, DP220102121, LP220100527, LP220200949, and IC190100031.
[1] Maksym Andriushchenko and Nicolas Flammarion. Towards understanding sharpness-aware minimization. In ICML, pages 639-668. PMLR, 2022.
[2] Yingbin Bai and Tongliang Liu. Me-momentum: Extracting hard confident examples from noisily labeled data. In ICCV, pages 9312-9321, 2021.
[3] Yingbin Bai, Erkun Yang, Bo Han, Yanhua Yang, Jiatong Li, Yinian Mao, Gang Niu, and Tongliang Liu. Understanding and improving early stopping for learning with noisy labels. In NeurIPS, volume 34, pages 24392-24403, 2021.
[4] Yingbin Bai, Erkun Yang, Zhaoqing Wang, Yuxuan Du, Bo Han, Cheng Deng, Dadong Wang, and Tongliang Liu. Rsa: Reducing semantic shift from aggressive augmentations for self-supervised learning. In NeurIPS, volume 35, pages 21128-21141, 2022.
[5] Shai Ben-David, Tyler Lu, and David Pal. Does unlabeled data provably help? worst-case analysis of the sample complexity of semi-supervised learning. In  $COLT$ , pages 33-44, 2008.
[81] Zhiyuan Zhang, Ruixuan Luo, Qi Su, and Xu Sun. Ga-sam: Gradient-strength based adaptive sharpness-aware minimization for improved generalization. arXiv preprint arXiv:2210.06895, 2022.
[82] Yang Zhao, Hao Zhang, and Xiuyuan Hu. Penalizing gradient norm for efficiently improving generalization in deep learning. In ICML, 2022.
[83] Yang Zhao, Hao Zhang, and Xiuyuan Hu. Ss-sam: Stochastic scheduled sharpness-aware minimization for efficiently training deep neural networks. arXiv preprint arXiv:2203.09962, 2022.
[84] Dengyong Zhou, Olivier Bousquet, Thomas N Lal, Jason Weston, and Bernhard Schölkopf. Learning with local and global consistency. In NeurIPS, pages 321-328, 2004.
Supplementary Material for "FlatMatch: Bridging Labeled Data and Unlabeled Data with Cross-Sharpsness for Semi-Supervised Learning"
In this Appendix, we provide additional details and experimental results to complement the proposed method. First, we describe supplementary experimental details in Section A. Then, we provide extra quantitative results, including employing FlatMatch to other SSL methods, comparisons on ImageNet30 and ImageNet [55] datasets, performance on Vision Transformer in Section B. Further, we show more empirical results to qualitatively validate FlatMatch in Section C. Then, we conduct convergence study on test accuracy and training loss curves in Section D. Moreover, we provide extra visualizations to show the loss landscape on later state of training in Section E. Finally, we summarize this paper and make a discussion on prospective research in Section F.
A Supplementary Details
The experimental setting of this paper follows Wang et al. [65]. Specifically, the hyper-parameters are composed of algorithm-dependent parameters and algorithm-independent parameters, which are shown in Table 3 and Table 4, respectively. For algorithm-dependent parameters of FlatMatch, we use the same unlabeled data and labeled data ratio as FreeMatch [66] as well as all other baseline methods to sample data into a mini-batch. The perturbation magnitude  $\alpha$  is based on the results from hyper-parameter sensitivity analysis in the main paper and is chosen as 0.05 for all experiments. For updating the historical gradient using a memory buffer, we use EMA with factor  $\alpha$  to ensemble the gradient result. Moreover, we choose the thresholding strategy from FreeMatch and use an EMA decay. Note that for combining the cross-sharpness regularization from FlatMatch with empirical risk, we find that there is no need to introduce another weight to trade off the two loss functions, hence the weight for cross-sharpness is just set to 1 for all experiments. For algorithm-independent hyperparameters, we have listed the important model setting, optimizer parameters, and data sampling setting as below. Note that all baseline methods follow the implementation of USB [65] and are trained with EMA decay with 0.999 to smooth the parameter updating.
Table 3: Algorithm-dependent hyper-parameters.
Algorithm	FlatMatch
Unlabeled Data to Labeled Data Ratio (CIFAR-10/100, STL-10, SVHN)	7
Unlabeled Data to Labeled Data Ratio (ImageNet30)	1
Perturbation magnitude ρ for all experiments	0.05
EMA factor α for updating gradient	0.999
Thresholding EMA decay for all experiments	0.999
Trade-off weight λX-sharp for cross-sharpness	1
In this section, we conduct additional experiments on CIFAR10 and ImageNet30 [55] datasets to compare the performance between some of the most edge-cutting methods, including FixMatch [57], Dash [76], FlexMatch [79], FreeMatch [66], SoftMatch [16], and our FlatMatch.
Table 4: Algorithm-independent hyper-parameters.
Dataset	CIFAR-10	CIFAR-100	STL-10	SVHN	ImageNet30
Model	WRN-28-2	WRN-28-8	WRN-37-2	WRN-28-2	ResNet-50
Weight decay	5e-4	1e-3	5e-4	5e-4	3e-4
Batch size	64	128
Learning rate	0.03
SGD momentum	0.9
EMA decay	0.999
Table 5: Performance on boosting other SSL methods using FlatMatch with the fixed number of labels.
Dataset	CIFAR10
# label	40	250	4000
FixMatch	7.47±0.28	4.86±0.05	4.21±0.08
FixMatch+FlatMatch	6.50±1.25	4.27±2.15	3.92±1.65
Dash	8.93±3.11	5.16±0.23	4.36±0.11
Dash+FlatMatch	6.73±2.49	4.48±1.56	4.02±1.30
FlexMatch	4.97±0.06	4.98±0.09	4.19±0.01
FlexMatch+FlatMatch	4.47±0.92	4.25±1.37	3.88±0.75
SoftMatch	4.91±0.12	4.82±0.09	4.04±0.02
SoftMatch+FlatMatch	4.89±1.32	3.98±1.14	3.84±0.86
FreeMatch	4.90±0.04	4.88±0.18	4.10±0.02
FlatMatch (Fix label)	4.89±1.24	3.90±1.72	3.61±0.49
B.1 Combining FlatMatch with Other Methods on CIFAR10
We choose CIFAR10 dataset with the number of labeled data varied as 40, 250, and 4000, and apply the FlatMatch methodology to several recently proposed SSL methods to show the effectiveness of the proposed cross-sharpness regularization. The results are shown in Table 5, as we can see that our method can further boost the learning performance of all five methods on all three settings, which proves that the cross-sharpness method is quite universal to SSL approaches and can bring non-trivial performance enhancement. Note that in the 40 labels setting, we compute our cross-sharpness on 500 examples with fixed labels, as demonstrated in Section 5.2 from the main paper.
B.2 Comparing FlatMatch to Other Methods on Large-Scale Datasets and Sophisticated Architecture