DAW: Exploring the Better Weighting Function for Semi-supervised Semantic Segmentation
Authors: Rui Sun
Huayu Mai
Tianzhu Zhang
Feng Wu
Venue: NeurIPS 2023 poster (2023)
Keywords: semi-supervised semantic segmentation
TL;DR: We introduce a semi-supervised learning strategy for semantic segmentation.

Abstract: The critical challenge of semi-supervised semantic segmentation lies in how to fully exploit a large volume of unlabeled data to improve the model’s generalization performance for robust segmentation.  Existing methods tend to employ certain criteria (weighting function) to select pixel-level pseudo labels. However, the trade-off exists between inaccurate yet utilized pseudo-labels, and correct yet discarded pseudo-labels in these methods when handling pseudo-labels without thoughtful consideration of the weighting function, hindering the generalization ability of the model. In this paper, we systematically analyze the trade-off in previous methods when dealing with pseudo-labels. We formally define the trade-off between inaccurate yet utilized pseudo-labels, and correct yet discarded pseudo-labels by explicitly modeling the confidence distribution of correct and inaccurate pseudo-labels, equipped with a unified weighting function. To this end, we propose Distribution-Aware Weighting (DAW) to strive to minimize the negative equivalence impact raised by the trade-off. We find an interesting fact that the optimal solution for the weighting function is a hard step function, with the jump point located at the intersection of the two confidence distributions. Besides, we devise distribution alignment to mitigate the issue of the discrepancy between the prediction distributions of labeled and unlabeled data. Extensive experimental results on multiple benchmarks including mitochondria segmentation demonstrate that DAW performs favorably against state-of-the-art methods.

DAW: Exploring the Better Weighting Function for Semi-supervised Semantic Segmentation
Rui Sun
Huayu Mai!
Tianzhu Zhang 1.2
Feng Wu
Deep Space Exploration Laboratory/School of Information Science and Technology, University Science and Technology of China
Institute of Artificial Intelligence, Hefei Comprehensive National Science Center
(issunrui, mai556)0mail ustc. edu..ddu.ee en, {tzzhang, fengwu)oustc. ous.e.euu..uuu........o.ouuu cn
Abstract
The critical challenge of semi-supervised semantic segmentation lies how to fully exploit a large volume of unlabeled data to improve the model's generalization performance for robust segmentation. Existing methods tend to employ certain criteria (weighting function) to select pixel-level pseudo labels. However, the trade-off exists between inaccurate yet utilized pseudo-labels, and correct yet discarded pscudo-labels in these methods when handling pseudo-labels without thoughtful consideration of the weighting function, hindering the generalization ability of the model. In this paper, we systematically analyze the trade-off in previ- methods when dealing with pseudo-labes labels. We formally define the trade-off between inaccurate yet utilized hheudo-labelss and correct yet discarded pseudo- labels by explicitly modeling the confidence distribution of correct and inaccu- rate pseudo-labels, equipped with a unified weighting function. To this end, we propose Distribution-Aware Weighting (DAW) to strive to minimize the negative equivalence impact raised by the trade-off. We find an interesting fact that the optimal solution for the weighting function hard step function, with the jump point located at the intersection of the two confidence distributions. Besides, we devise distribution alignment mitigate the issue of the discrepancy between the prediction distributions of labeled and unlabeled data. Extensive experimental results on multiple benchmarks including mitochondria segmentation demonstrate that DAW performs favorably against ofate-of fftt-oof-hh--rt the-art methods. Code is available at https://gii thub. com/yut homm/yussuen/D// DDW.
Introduction
Semantic segmentation a fundamental task that has achieved conspicuous achievements credited to the recent advances in deep neural networks [1]. However, its data-driven nature makes it heavily dependent on massive pixel-level annotations, which are laborious and time-consuming to gather. To alleviate the data-hunger issue, considerable works [2-7] have turned their attention to semi-supervised semantic segmentation, which has demonstrated great potential practical applications [8, 9]. Since only limited labeled data is accessible, how to fully exploit a large volume of unlabeled data to improve the model's 's generalization performance for robust segmentation is thus extremely challenging.
In previous literature, labeli-g [10-12] and consistency regularization [13-15] have emerged as mainstream paradigms to leverage unlabeled data for semi-supervised segmentation. In specific,
"quual contribution
Corresponning author
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

(a) Pos & Neg. Dist. (b) E1-E2 Trade
(c) E1
E2
E1+E2
Figure 1: Illustration of our motivation. (a) shows the trade-off between inaccurate yet utilized pseudo- labels, and correct yet discarded pscudo-labels by explicitly modeling the confidence distribution of correct and inaccurate pseudo-labels. (b) illustrates the negative equivalence impact on generalization performance raised by the trade-off. (c) (d) (e) summarize the models inevitably face trade-off when dealing with pseudo-labels. Our method can guarantee the theoretical optimal solution by minimizing the negative impact.
the pseudo-labeling methods train the model on unlabeled samples with pseudo labels derived from the up-to-date model's own predictions. And the consistency regularization methods encourage the model to produce consistent predictions for the same sample with different perturbation views, following the smoothness assumption [16]. Recently, these two paradigms are often intertwined in the form of teacher-student scheme [17-20, 3]. The critical idea involves updating the weights of the teacher model using the exponential moving average (EMA) of the student model, and the teacher model generates corresponding pscudo labels of the perturbed samples to instruct the learning of the student model.
Despite yielding promising results, these methods tend to employ certain criteria (referred to as weighting function) to select pixel-level pseudo labels, considering that the quality of the chosen pseudo-labels determines the upper bound of performance. On the one hand, naive pseudo-labeling methods such as Pseudo-Label [10] recruit all pseudo labels into training, assuming that each pseudo label is equally correct (i.e., weighting function can be regarded as constant function). However, as training progresses, maximizing the utilization of pscudo-labels tends to lead to confirmation bias [2 l], which is a corollary raised by erroneous pscudo-labed (i.c., inaccurate yet utilized pseudo-labels). On the other hand, a series of threshold-based pscudo-labeling methods [22] such as FixMatch [18] attempt to set high threshold (e.g., 0.95) to filter out pixel-level pseudo-labels with low confidence (i.e., the weighting function can be considered as a step function that jumps at 0.95). Although tangling the quality of pseudo-labels can alleviate noise, the strict criteria inevitably lead to the contempt of numerous unconfident yet correct pseudo-labels (i.e., correct yet discarded pseudo- labels), hindering the learning process. To make matters worse, the negative impact inevitably amplified by inbuilt low-data regimes of semi-supervised segmentation, leading to sub-optimal results. As a compromise, AEL [19] ad hoc defines the weighting function as power function, which assigns weights conditioned on the confidence of pscudo-labels, that is, convincing pseudo-labels will be allocated more weights. However, the lack of sophisticated consideration and the arbitrary control of hyperparameters (i.c., tunable power) for the weighting function inevitably compromise its capability. In nutshell, the trade-off exists between inaccurate yet utilized pseudo-labels, and correct yet discarded lseudo-labels in these methods when handling pseudo-labels without thoughtful consideration of the weighting function, hindering the generalization ability of the model. Then, the question naturally arises: How to explore the better weighting function to effectively alleviate the negative impact raised by the trude-off?
In this work, we systematically analyze the trade-off in previous methods that hinder the model's learning when dcaling with pseudo-labels in semi-supervised semantic segmentation. We formally define the trade-off between inaccurate yet utilized pseudo-labels, and correct yet discarded pscudo- labels by explicitly modeling the confidence distribution of correct and inaccurate pseudo-labels, equipped with a unified weighting function. In specific, two Gaussian functions excelling at the maximum entropy property are devised to fit the confidence distribution of correct (positive distri- bution in Figure 1 (a)) and inaccurate (negative distribution in Figure 1 (a)) pseudo-labels using maximum likelihood estimation, respectively. The parameters of the Gaussian functions are updated via the exponential moving average (EMA) in pursuit of perceiving the learning status of the model. Then the trf can be naturally derived by calculating the expectations of inaccurate yet utilized pseudo-labels (depicted as E, in Figure 1 (a)), and correct yet discarded pscudo-label (displayed as

---

Table 1: We analyze the learning process of the mainstream methods for remi-supprvised semantic segmentation systematically and uniformly abstract the criteria they adopt to select pseudo labels as weighting function f'()) conditioned on the confudence p of pseudo-labels. There are inherent distributions g(p) for positive and negative pseudo-labels(Gaussii distribution is employed as an approximation).
Method
Pscudo-Label
Fixmatch
Ours
f(p)
tp (or If (8)
Maa Me ( 1
Wo F' (1)
(0 la
A22
0
a/0 1mp
- (a 1o
1 1-a/0 (1)))
No (1
1-
1 1-
+ Mor
II
+1+
-afe
]]
-n(0(
1+ 1+ -afo(
* 60- o" dy
1. (p) -xx (p))
Ale 60 - - der
C - 331o" (9) - 4000
L ()) -016))
Mllo (o) - o do
>" (a) do00)dde
Note
12 - Alle (2) -a do
Smaller than all of them!
>
Eg in Figure (a)) respectively, on top of the weighting function and the corresponding confidence distribution. Now, we are prepared to propose the Distribution Aware Weighting (DAW) function striving minimize the negative equivalence impact on generalization performance raised by the trade-off, i.e., minimizing E1 + E2. By leveraging functional analysis, we find an interesting fact that the optimal solution for the weighting function is a hard step function, with the jump point located at the intersection of the two confidence distributions. Note that the dedicated weighting function is theoretically guaranteed by reconciling the intrinsic tension between E1 and F22 (see Figure 1 (b)), and is free of setting thresholds manually compared to previous methods. Besides, considering the imbalance issue caused by the discrepancy between the prediction distributions of labeled and unlabeled data, we propose distribution alignment to further unlock the potential of the weighting function and enjoy the synergy. In practice, the weighting function generated by DAW determines the criteria for selecting pseudo-labels to minimize the negative equivalence impact of the trade-off, minimizing E1 + E2 (see Figure (e)), which conducive to model training. In this way, the model improves, benefiting from the effective probe of reliable pseudo-labels. And in turn, the positive distribution will be maximally separated from the negative one, leading to a simultaneous decrease in both E1 and E2 (see Figure 1 (c) and Figure I (d)), which is conducive to the generation of the weighting function.
Extensive experiments on mainstream benchmarks demonstrate that our method performs favorably against state-of-the-art semi-supervised semantic segmentation methods, proving that it can better exploit unlabeled data. Besides, we further validate the robustness of DAW on the electron microscopy mitochondria segmentation task, which involves images with dense foreground objects and cluttered backgrounds, making it more challenging to discriminate the reliability of the pseudo-labels.
2 Distribution-Aware Weighting
In this section, we first formulate the semi-supervised semantic segmentation problem as preliminaries (Section 2.1), and then formally define the trade-off between inaccurate yet utilized pseudo-labels (F11,, and correct yet discarded pseudo-labels (F22) by explicitly modeling the confidence distribution of correct and inaccurate pseudo-labels, equipped with a unified weighting function (Section 2.2).
3

---

Based on the analysis, we propose the intribution-aaare weighting function (DAWF) to avoid performance degradation raised by the trade-off (Section 2.3). Finally, distribution alignment (DA) is devised to alleviate the discrepancies between the confidence distributions of labeled and unlabeled data. (Section 2.4).
2.1 Preliminaries
In semi-supervised semantic segmentation, given a of labeled training images Dr = (x,, Nr 1=1 and large amount of unlabeled images D. {x Ji=1 where N₁ and N₂, denote the number of labeled and unlabeled images, respectively, and N, > N. Let q(x;)) of the j-th pixel in the i-th labeled (or unlabeled) image, and {1, u., C' is the number of categories. Then the supervised loss L. can be formulated as,
N.,
) Rº denotes the prediction
N;
WH
C. 2
Σ Leo g(x,)))))))
M BL-WH
where and H represent the width and height of the input image, lo denotes the standard pixel-wise cross-entropy loss, and y'y denotes the ground-truth label from D.. Considering most methods [22, 18, 19, 10, 3, 23] tend employ certain criteria (weighting function) to attempt to select reliable pseudo labels, we formulate the unsupervised loss L,, as weighted cross-entropy for the convenience of introducing the weighting function (pis),
C.
WH Σ f(pi)) Lee (5 (AA (xx(x)))))
(2)
No
where /A4 denotes weak/strong perturbation to encourage the model to produce consistent predictions, yi, denotes (x(:) ()fit)) i.e., i.e., prediction under the weak perturbation view. And yū argmax (y is the one-hot pseudo-label. (Pij) is weighting function conditioned Pij. and Piy =yax(y") denotes the maximum confidence of the prediction. Then define the overall loss function as La + Lu-
2.2 E1-E2 Trade-off from Unified Weighting Function
We formally define the trade-off between inaccurate yet utilized pseudo-labels, and correct yet discarded pseudo-labels by explicitly modeling the confidence distribution of correct and inaccurate pseudo-labels, equipped with unified weighting function. We instantiate the different biases inherent in the offe of previous methods and reveal their tight connection with the capability of the model. We start by defining the pseudo-label confidence distribution.
Orthogonal to other previous models, we assume that the confidence distribution of correct (a+ (p), positive distribution) and inaccurate (g (p), negative distribution) pseudo-labels follows truncated Gaussian distribution with mean μ+ ht/p/² and standard deviation o+ /o formulated as,
(p-u+)² 2(0+)
g+(p)
V exp
C <
0.
otherwise
where 1/a = +b 1- +
denotes the normalization factor, + the cumulative distribution function of the standard normal distribution. Note that =max(q(x!,)) denotes the maximum confidence of the prediction from labeled data, must meet the condition of p > J Similarly,
1/C-== o+
(p-k 2( 1
gp)
g
(p) =
V2xo-
exp
0,
otherwise
where 1/3 (1-1 --
(1/C==).
+
The reason we choose Gaussian is its valuable maximum
entropy property, please refer to the supplementary material for more details. Then estimate

---

the mean and standard deviation of the positive (p) and negative distribution g (p), respectively. resorting to maximum likelihood estimation,
N Ni
N
N
N+ Σ Ppj - x,,
Pay
(o+)²
(5)
Ni
J=1
where P denotes the prediction confidence, where (1gmax((((x))) ground truth vly. and N is the number of p.j the i-th image. For the negative distribution y (p). the parameters are evaluated in the same way, except that the predictions involved in the calculation are not equal to the ground truth. Note that we only consider predictions from labeled data to evaluate the Gaussian distribution equipped with ground truth, rather than estimating biases raised from unlabeled data with noisy pseudo-labels, Then the parameters of the Gaussian functions are updated via the exponential moving average (EMA) in pursuit of perceiving the learning status of the model in dynamic manner,
the labeled data equals the
E. N. E,
p m + (1-m)pt (0+)² == - m)
(ō+)²
(6)
where unbiased variance is adopted for EMA, At and (at)) )² are initialized as 1/C and 1.0 respectively. similar way also works for the negative distribution g (p).
Then the trade-off can be naturally derived by calculating the expectations of inaccurate yet utilized pseudo-labels (E1), and correct yet discarded pseudo labels (E2) respectively, on top of the weighting function (p) and the corresponding confidence distribution (p)/g* (p).
Definition 3.1 Inaccurate yet utilized pseudo-labele E1.
= [[(] [ f(p) - - (p)dpp
(7)
Definition 3.2 Correct yet discarded pseudo-labels, E2.
E2 = 1- Eg [[(P)] =1 S f(p) (p)dp.
(8)
After formally defining the trade-off between E1 and E2, it is natural measure the impact of negative equivalence effects (i.e., El+E2), considering the trade-off between El and E2, where an increase in one necessitates decrease in the other.
Definition 3.3 Negative equivalence effect of the trade-off, E1+E2.
J f(p) (p) g dp.
E1 + E2 E2
Then, we systematically analyze the trade-off in previous methods tabulated in Table 1. For more detailed derivations, please refer to the supplementary material. (1) For example, naive pseudo- labeling methods such as Pseudo-Label [10] enroll all pseudo labels (E2 0) into training. However, as training progresses, maximizing the utilization of pseudo-labels tends to a confirmation bias raised methods such as FixMatch [18], which attempts to set a high threshold (0.95) to filter out pixel-level pseudo-labels with low confidence (small value of E1 caused by the proximity of t 0.95 to 1). However, the strict criteria inevitably lead to the contempt of numerous unconfident yet correct pseudo-labels (large value of F2 caused by the proximity of / 0.95 1). (3) As compromise, AEL [19] ad hoc defines the weighting function as a power function, which assigns weights conditioned on confidence. That is, convincing pseudo-labels will be allocated more weight. However, the lack of sophisticated consideration and the arbitrary control of hyperparameters (tunable power) for the weighting function inevitably compromise its capability (not guarantecing the lowest negative equivalence effect).
by erroneous pseudo-labels (E1 1). (2) And for threshold-based pscudo-labeling
2.3 Distribution-Aware Weighting Function
Then we seek to explore a better weighting function equipped with the formal trade-off definition, aiming at minimizing the negative equivalence impact raised by the trade-off, that is, minimizing
5

---

E2,
min
E1 F2 = 1+ f()) (p) g gt)r dp,
(10)
0 f(p)
1
By leveraging functional analysis, find an interesting fact that the optimal solution for the weighting function a hard step function, with the jump point located at the intersection of the two confidence distributions, formulated as,
((()= - of 0, 0,
1, t" E<pEL t" --= 48) -B)) 1 (221),
(11)
otherwise
))²² 2 20tor In[(ao (nl((aa)//(30 and p =y;)(y?;) denotes the confidence of the prediction from unla- beled data. Please refer to the supplementary material for more detailed derivations. Note that the dedicated weighting function
where 3, = (o+)² (o-)" =
8q = ( (
(p) is theoretically guaranteed by reconciling the intrinsic tension between E1 and E2 (see Table 1) and is free of setting thresholds manually compared to previous methods.
2.4 Distribution Alignment
Furthermore, considering the imbalance issue caused by the discrepancy between the prediction distributions of labeled and unlabeled data, we propose distribution alignment (DA) to further unlock the potential the distribution-awanl weighting function. In specific, we define the confidence distributions from labeled data and unlabeled data expectations ED, [x({g)] and ED, (q(*))]] ]
respectively. Both these are estimated the form EMA in each batch as the training progresses,
denoted as ED, [x({))) and ED, [q(x);)]- Then we use the ratio between the expectations of
labeled and unlabeled to normalize the each prediction y = (xx))) on unlabeled data, formulated as,
ED, [9(x(,)]
DA(y = Norm
yij
(12)
ED.
[a(x)))
where Norm(-) denotes the normalization operation used to constrain the probabilities to sum up to 1. Then we bring the normalized probability back to Equation 2 calculate the loss weight of each pseudo-label after alignment,
N
WH
C. =
Σ f (maxx(DA(x)))) tce (9 (x) j=1
(13)
N
WH
where 8リ = argmax(DAAx In this way, the distribution-awann aware weighting function rewarded with better generalization, benefiting from more equal learning of labeled and unlabeled data, mitigating the issue of distribution imbalance, and enjoying the synergy. The algorithm flow shown in the supplementary material.
3 Experiments
3.1 Experimental Setup
Datasets: (1) PASCAL VOC 2012 [29] is an object-centric semantic segmentation dataset, containing 21 classes with 1,464 and 1,449 finely annotated images for training and validation, respectively. Some researches [30, 19] augment the original training set (c.g., classic) by incorporating the coarsely annotated images in SBD [31], obtaining a training set (e.g., blender) with 10,582 labeled samples. (2) Cityscapes [32]is urban scene understanding dataset with 2,975 images for training and 500 images for validation.
Implementation Details: For a fair comparison, we follow the common practice and use ResNet [33] as our backbone and DeepLabv3+[34] as the decoder. We set the crop size as 513 x 513 for PASCAL and 801 x 801 for Cityscapes, respectively. For both datasets, we adopt SGD as the optimizer with the same batch size of 16 and different initial learing rate, which set as 0.001 and 0.005 for PASCAL. and Cityscapes. We use the polynomial policy to dynamically decay the learning rate along the whole

---

Table Quantitative results of different SSL methods on Pascal classic and blender set. We report mloU (%) under various partition protocols and show the improvements over Sup.-only baseline. The best is highlighted bold.
Method
Classic IM(366)
1/2(732)
Full(1166)
1/16/662)
Blender 1/8(1323)
LM(2616)
Sup-ouny
44.0
52.3
61.7
66.7
72.9
62.4
68.2
72.3
Pscudo- L
71.8
RoaNet
FixMatch wernorl
60.1
67.3
71.4
73.7
76.9
70.6
73.0
70.1
IMAS were [24]
74.8
76.5
17.0
AugSeg-voe
64.2
72.2
76.2
77.4
78.8
74.7
76.0
77.2
DAW (Ours)
78.6
79.7
76.2
77.6
77.4
+14.6
+11.9
+60.8
+18.8
+.5.1
Sup-oonl
64.8
73.5
Pacadda mn
69:4
73.3
77:2
69.1
73.88
76.7
FixMatch serneay [8]
61.9
73.0
75.5
77.8
79.2
74.3
76.3
76.9
CPS on 22
64.1
67.4
71.7
75.9
74.5
76..
Y7.7
ReoNet-
AEL
77.2
77.6
78.1
MMAS
AugSegrremen [25]
71.1
73.5
78.8
80.3
81.4
77.0
77.3
78.8
OCVC [25]
70.2
74.4
77.4
79.1
80.5
77.2
78.4
73.0
DAW (Ours)
74.8
77.4
79.5
80.6
81.5
78.5
78.9
79.6
A
Table 3: Quantitative results of different SSL methods on Cityscapes. We report mIoU (%) under var- ious partition protocols and show the improvements over Sup--only baseline. The best is highlighted in bold.
RexNet-50 1/8(372)
ResNer-10 101
1/16(186)
1/K(44))
1/2(1-88)
1/16(186)
1/5(72))
1A1(74))
1/2(1488)
Sap-yady
70.2
73.1
78-0
Pscudo- mm [10]
67.2
72.4
74.9
77.4
74:3
76.8
78.6
FixMatch seee [18]
72.6
75.7
76.8
78.2
74.2
76.2
77.2
78.4
summnn [7]
74.0
75.8
76.2
75.8
77.9
79.0
50.3
PUR
SegeSeg [3]
iMAS vannon [24]
74.3
77.4
78.1
79.3
AugSeg cxrsczy [25]
73.7
70.5
78.8
79.5
75.2
77.8
79.6
$0.4
DAW (Ours)
75.2
77.5
79.1
79.5
76.6
78.4
79.8
80.6
+11.9
1.7.3
+6.0
42.9
+10.3
+5.6
1.4.8
12.6
training and assemble the channel dropout perturbation [22] to improve the generalization ability of the model. We train the model for 80 epochs PASCAL and 240 epochs Cityscapes, using 8x NVIDIA GeForce RTX 3090 GPUs.
3.2 Comparison with State-of-the-art Methods
We conduct experiments two popular benchmarks including PASCAL VOC 2012 and Cityscapes and make a fair comparison with SOTA semi-supervised semantic segmentation methods. We consistently observe that our DAW outperforms all other methods under all partition protocols all datasets with different backbones, which strongly proves the effectiveness of our method.
Results PASCAL VOC 2012 Dataset. Table 2 shows the comparison of our method with the SOTA methods on PASCAL classic and blender set. Specifically, on the PASCAL classic set, our method outperforms the supervised-only (Sup.-only) model by 29.7%, 22. 2.1%, 14.7%, 10.9% under the partition protocols of I/16, 1/8, 1/4 and 1/2, respectively with ResNet-101. 101. Our method also significantly outperforms the existing semi-supervised SOTA methods under all data partition protocols. Taking the recently proposed method AugSeg [25] as an example, the performance gain of approach reaches to +4.3% under 1/16 partition protocol with ResNet-50. The same superiority of our method can also be observed on the PASCAL blender set.
Results on Cityscapes Dataset. Table compares DAW with SOTA methods on the Cityscapes dataset. DAW achieves consistent performance gains over the Sup.only baseline, obtaining improve- ments of 11 11.9%, 7.3%, 6.0% and 2.9% under 1/16, 1/8, 1/4 and 1/2 partition protocols with ResNet-50, respectively. We can also see that over protocols, DAW outperforms the SOTA methods, e.g., DAW excels to iMAS [24] by 1.1% under the 1/16 partition with ResNet-101.
7

---

Table 4: Ablation studies of different components. Note that "Fixed" de- notes the result of UniMatch.
Table 5: Ablation studies of different momentum PASCAL classic 92.
None
Fixed
DAWF
mloU
m
mloU
62.7
0.99
68.5
66.9
0.999
68.1
The curve of Pos.&Nep. distribution and t' during training.
Figure 2:
68.0
68.5
0.9999
67.9
Image
Ground Truth
Pscudo-Label
FixMatch
Ours
Figure 3: Qualitative comparison with different methods. Note that significant improvements are marked with yellow boxes.
Qualitative Results. We compare qualitative results of our DAW with different SOTA methods. As shown Figure , DAW also shows more powerful segmentation performance in fine-gr fine-grained details (see the first and second row in Figure 3). With the help of the optimal weighting function, DAW demonstrates superior abilities in most scenarios.
3.3 Ablation Study and Analysis
To look deeper into method, perform series of ablation studies PASCAL classic set under 92 partition protocol with ResNet-50 to analyze each component of our DAW, including the Distribution-A ware Weighting Function (DAWF) and the Distribution Alignment (DA) The baseline method is UniMatch [22].
Effectiveness of Components. In Table 4, "None" denotes there is threshold for pseudo-label during training (i.e., Pseudo-Label [10]) while "Fixed" indicates that fixed threshold is set (i.e., UiMatch [22]). A certain performance lift compared with the baseline can be observed owing to
Ejazc
WoDA
Tuadee Cultaaio
Maiia
Toadoe
del Bolea
$30
Fporh 70
CoTE
Pseudo-Label
Fixmatch
AEL
urs
w WDA
Figure Comparison between distributions different methods different training epochs (e.g., epoch vs. epoch 79).
Figure 5: Viz. of Dist. w&w/oo DA.

---

Table Quantitative results different SSL methods on Lucchi dataset. We report mloU (%) under various partition protocols. The best is highlighted in bold.
Method
1/32(5)
1/16(10)
1/8(20)
Sup.-only
45.7
57.4
61.8
MT [35]
71.8
72.4
75.4
CCT [36]
81.7
85.4
85.8
Ground Truth
CPS [30]
84.5
84.6
85.8
Figure 6: Visualization of Lucchi dataset.
DAW (Ours)
85.9
86.6
87.6
the introduction of Distribution-Aware Weighting Function and Distribution Alignment. (1) The utilization of DAWF brings a 1.1% improvement mloU, demonstrating that the negative impact raised by the E1 + E2 trade-off is effectively alleviated. (2) DA brings further accuracy gains, indicating that the existence of the diserepancy between the distributions of labeled and unlabeled data may cause bottleneck in learning. For better visualization, we employ the unused annotations of "unlabeled data" to calculate the ground-truth distribution on the unlabeled data. And as shown in Figure 5, there is a relatively large gap between the distributions, and DA can effectively relieve it, further unlocking the potential of the weighting function and enjoy the synergy.
Hyperparameter Evaluations. As shown in Table 5, it can be observed that the performance is optimal with m 0.99.
Scalability for Other Scenarios. We further conduct extra experiments on Lucchi [37-42] to evaluate the scalability of our method. Figure 6 shows the image and ground-truth of Lucchi dataset, presenting a common problem in electron microscope images that the instances are very small and scattered. This calls for more reliable supervision in training under semi-supervised setting. As shown in Table 6, DAW outperforms other competitive methods the electron microscopy domain, indicating that our method provide more reliable supervision.
Comparison of Distribution. As shown in Figure 4, the distributions of different methods are almost the same. As the learning goes the discrepancy between the positive and negative distributions of ours becomes larger (simultaneously shown in Figure 2) while the others almost no change. A large discrepancy between the positive and negative distributions means that we filter out many negative samples while recruiting as many positive samples as possible, which is conducive to model training. This is the fundamental reason behind why method outperforms other methods.
Related Work
Semi-Supervised Learning. Semantic segmentation is a fundamental task that has achieved con- spicuous achievements credited to the recent advances in deep neural networks [43-48]. However, its data-driven nature makes it heavily dependent on massive pixel-level annotations, which are laborious and tine-consuming to gather. To alleviate the data- data-hunger issue, considerable works have turned their attention to sume-supee learning. Designing appropriate and effective supervision signals for unlabelled data is the core problem of semi-supervised learning. Previous research can be summarized into two learning schemes: self-training and consistency regularization. Self-training based methods [49, 12, 10, 50] aim to train the model based on the pseudo-labed generated by the up-to-date optimized model for the unlabelled data. Consistency regularization based methods aim to obtain prediction invariance under various perturbations, including input perturbation [51, 52], feature perturbation [53] and network perturbation [54-57]. The recently semi-supervised methods, MixMathch [58] and FixMatch [18] combine these two techniques together and achieve state-ofthhe- art performance. Based on the paradigm of existing semi-supervised learning methods, our method explores a better weighting function for the pseudo-labed scheme during training.
Semi-Supervised Semantic Segmentation. Semi-supervised semantic segmentation aims at pixel- level classification with limited labeled data. Recently, following the paradigm of semi-supervised learning, many semi-supervised semantic segmentation methods also focus on the design of self- training [26, 27, 5] and consistency regularization [59, 60, 53, 61] strategies. U [5] proposes to

---

make sufficient use of unreliable pseudo-labeled data. CCT [53] adopts a feature-level perturbation and enforces consistency among the predictions from different decoders. More recently, SOTA semi-supervised segmentation methods also integrate both technologies for better performance. PseudoSeg [7], AEL [27] and UCC [62] propose to use the pseudo-labele generated from weak augmented images to constrain the predictions of strong augmented images. In this paper, we shed light on semi-supervised semantic segmentation based on pseudo-labeling and strive to explore better strategies for using pseudo-labels.
5
Conclusion
In this paper, we propose DAW to systematically analyze the trade-off in previous methods that hinder the model's learning. We formally define the trade-off between inaccurate yet utilized pseudo-labels and correct yet discarded pseudo-labels by explicitly modeling the confidence distribution of correct and inaccurate pseudo-labels, equipped with unified weighting function. Experiments show the effectiveness.
6 Acknowledgments
This work was partially supported by the National Defense Basic Scientific Research Program (Grant JCK (CKY2020003B02))
References
[1] Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for se- mantic segmentation. Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3431-3440, 2015.
[2] Haiming Xu, Lingqiao Liu, Qiuchen Bian, and Zhen Yang. Semi-supervies semantic seg- mentation with prototype-based consistency regularization. Advances in Neural Information Processing Systems, 35:26007-26020, 2022.
[3] Ying Jin, Jiaqi Wang, and Dahua Lin. Semi-supervised semantic segmentation via gentle teaching assistant. Advances in Neural Information Processing Systems, 35:2803-2811 2022.
[4] Lihe Yang, Wei Zhuo, Lei Qi, Yinghuan Shi, and Yang Gao. St++: Make self-training work better for sumi-supe semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4268-4277, 2022.
[5] Yuchao Wang, Haochen Wang, Yujun Shen, Jingjing Fei, Wei Li, Guoqiang Jin, Liwei Wu, Rui Zhao, and Xinyi Le. Semi-supervised semantic segmentation using unreliable pseudo-labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4248 4257, 2022.
[6] Jie Qin, Jie Wu, Ming Li, Xuefeng Xiao, Min Zheng, and Xingang Wang. Multi caltiigranularity distillation scheme towards lightweight cemi-supervised semantic segmentation. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Pro- ceedings, Part XXX, pages 481-498. Springer, 2022.
[7] Yuliang Zou, Zizhao Zhang, Han Zhang, Chun-Liang Li, Xiao Bian, Jia-Bin Huang, and Tomas Pfister. Pseudoseg: Designing pseudo labels for semantic segmentation. arXiv preprint arXiv:2010.097:3 2020.
[8] Mennatullah Siam. Mostafa Gamal, Moemen Abdel-Razek. Senthil Yogamani, Martin Jagersand. and Hong Zhang. A comparative study of real-time semantic segmentation for autonomous driving. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 587-597, 2018.
[9] Saeid Asgari Taghanaki, Kumar Abhishek, Joseph Paul Cohen, Julien Cohen- and Ghassan Hamarneh. Deep semantic segmentation of natural and medical images: review. Artificial Intelligence Review, 54:137-178, 1021.
10

---

[10] Dong- Hyun Lee et al. Pseudo-label: The simple and efficient sumi-super learning method for deep neural networks. In Workshop on challenges in representation learning. ICML, volume 3, page 896, 2013.
[11] Mamshad Nayeem Rizve, Kevin Duarte, Yogesh S Rawat, and Mubarak Shah. In defense of pseudo-labeling: An uncertainty-awar pseudo-label selection framework for semi-supervised learning. arXiv preprint arXiv:2101.0632 2021.
[12] Eric Arazo, Diego Ortego, Paul Albert, Noel E O' O'Connor, and Kevin McGuinness. Pseudo- labeling and confirmation bias in deep semi-supervised learning. In 2020 International Joint Conference on Neural Networks (LJCNN), pages 1-8. IEEE, 2020.
[13] Mehdi Sajjadi, Mehran Javanmardi, and Tolga Tasdizen. Regularization with stochastic transfor- mations and perturbations for deep semi-supervised learning. Advances in neural information processing systems, 29, 2016.
[14] Philip Bachman, Ouais Alsharif, and Doina Precup. Learning with pseudo Advances in neural information processing systems, 27, 2014.
[15] Samuli Laine and Timo Aila. Temporal ensembling for semi-supervised learning, arXiv preprint arX1v:161. 2016.
[16] Olivier Chapelle, Bernhard Scholkopf, and Alexander Zien. Semi-supervised learning (chapelle, o. et al., eds.; 2006][book reviews]. IEEE Transactions on Neural Networks, 20(3):542-544 2009.
[17] Yidong Wang, Hao Chen, Qiang Heng, Wenxin Hou, Marios Savvides, Takahiro Shinozaki, Bhiksha Raj. Zhen Wu, and Jindong Wang. Freematch: Self-adaptive thresholding for semi- supervised learning. arXiv preprint arXiv:2205.07226. 2022.
[18) Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang. Han Zhang, Colin A Raf- fel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. Fixmatch: Simplifying semi- supervised learning with consistency and confidence. Advances in neural information processing systems, 33:5966608 2020.
[19] Hanzhe Hu, Fangyun Wei, Han Hu, Qiwei Ye, Jinshi Cui, and Liwei Wang. Semi-supervised semantic segmentation via adaptive equalization learning. Advances in Neural Information Processing Systems, 34:22106-22118 18, 2021.
[20] Yuyuan Liu, Yu Tian, Yuanhong Chen, Fengbei Liu, Vasileios Belagiannis, and Gustavo Carneiro. Perturbed and strict mean teachers for semi-supervised semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4258-4267, 2022.
[21] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference on machine learning, pages 1321-1330. PMLR, 2017.
[22] Lihe Yang, Lei Qi, Litong Feng, Wayne Zhang, and Yinghuan Shi. Revisiting weak-to-strong consistency in sumi-supervised semantic segmentation. arXiv preprint arXiv:2208. 200222200.090000 2022.
[23] Hao Chen, Ran Tao, Yue Fan, Yidong Wang. Jindong Wang, Bernt Schiele, Xing Xie, Bhiksha Raj. and Marios Savvides. Softmatch: Addressing the quantity-qualin trade-off in semi- supervised learning. arXiv preprint vrXiv:2301.1092. 2023.
[24] Zhen Zhao, Sifan Long, Jimin Pi, Jingdong Wang, and Luping Zhou. instancesspeciff and model-adaptive supervision for semi-supervised semantic segmentation. arXiv preprint urXiv:2211.11333 2022.
[25] Zhen Zhao, Lihe Yang, Sifan Long, Jimin Pi, Luping Zhou, and Jingdong Wang. Augmentation matters: A simple-yet-effective approach to cemi-supervised semantic segmentation. arXiv preprint 1rXiv:2212.049766 2022.
[26] Xiaokang Chen, Yuhui Yuan, Gang Zeng, and Jingdong Wang. Semi-supervised semantic segmentation with cross pseudo supervision. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 2613-2622, 2021.

---

[27] Hanzhe Hu, Fangyun Wei, Han Hu, Qiwei Ye, Jinshi Cui, and Liwei Wang. Semi-supervised semantic segmentation via adaptive equalization learning. Advances in Neural Information Processing Systems, 34:22106-221 221181 2021.
[28] Zicheng Wang, Zhen Zhao, Luping Zhou, Dong Xu, Xiaoxia Xing, and Xiangyu Kong. Conflict based cross- view consistency for semi-supervised semantic segmentation. arXiv preprint arXiv:2303.01276, 2023.
[29] Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International journal of computer vision, 88:303-338 2010.
[30] Xiaokang Chen, Yuhui Yuan, Gang Zeng, and Jingdong Wang. Semi-supervised semantic segmentation with cross pseudo supervision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2613-2622 2021.
[31] Bharath Hariharan, Pablo Arbeláez, Lubomir Bourdev, Subhransu Maji, and Jitendra Malik. Semantic contours from inverse detectors. In 20/1 international conference on computer vision, pages 991-998. IEEE, 2011.
[32] Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke. Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3213-3223, 2016.
[33] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[34] Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European conference on computer vision (ECCV), pages 801-818, 2018.
2018.
[35] Antti Tarvainen and Harri Valpola. Mean teachers better role models: Weight-aweraged con- sistency targets improve cemi-supervised deep learning results. Advances in neural information processing systems, 30, 2017.
[36] Yassine Ouali, Céline Hudelot, and Myriam Tami. Semi-supervised semantic segmentation with cross consissesst training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12674- 12684, 2020.
[37] Aurélien Lucchi, Kevin Smith, Radhakrishna Achanta, Graham Knott, and Pascal Fua. Supervoxel-based segmentation mitochondria in em image stacks with learned shape features. IEEE transactions medical imaging, 31(2):474 2011.
[38] Huayu Mai, Rui Sun, Tianzhu Zhang, Zhiwei Xiong, and Feng Wu. Dualrel: Semi-supervised mitochondria segmentation from a prototype perspective. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 19617-19626, 2023.
[39] Rui Sun, Naisong Luo, Yuwen Pan, Huayu Mai, Tianzuu Zhang, Zhiwei Xiong, and Feng Wu. Appearance prompt vision transformer for connectome reconstruction. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI-23, pages
1433-1431. International Joint Conferences on Artificial Intelligence Organization, 8 2023. Main Track.
[40] Yuwen Pan, Naisong Luo, Rui Sun, Meng Meng, Tianzhu Zhang, Zhiwei Xiong, and Yongdong Zhang. Adaptive template transformer for mitochondria segmentation in electron microscopy images. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 21474-21484, 2023.
[41] Rui Sun, Huayu Mai, Naisong Luo, Tianzhu Zhang, Zhiwei Xiong, and Feng Wu. Structure- decoupled adaptive part alignment network for domain adaptive mitochondria segmentation. In International Conference on Medical Image Computing and Computer-Assie Assisted Intervention, pages 523-533. Springer, 2023.
12

---

[42] Xiaoyu Liu, Wei Huang, Zhiwei Xiong, Shenglong Zhou, Yueyi Zhang, Xuejin Chen, Zheng-Jum Zha, and Feng Wu. Learning rroreseeprinnntttonon affinity consisteney for sparsely supervised biomedical instance segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 21 2110-2111, 2023.
[43] Rui Sun, Yuan Wang, Huayu Mai, Tianzhu Zhang, and Feng Wu. Alignment before aggregation: trajectory memory retrieval network for video object segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1218-1228 2023.
[44] Yuan Wang, Rui Sun, Zhe Zhang, and Tianzhu Zhang. Adaptive agent transformer for few-shot segmentation. In European Conference on Computer Vision, pages 36-52. Springer, 2022.
[45] Rui Sun, Yihao Li, Tianzhu Zhang, Zhendong Mao, Feng Wu, and Yongdong Zhang. Lesion- aware transformers for diabetic retinopathy grading. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 10938-10447, 2021.
[46] Huayu Mai, Rui Sun, Yuan Wang. Tianzhu Zhang, and Feng Wu. Pay attention to target: Relation-aware temporal consistency for domain adaptive video semantic segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence. 2024.
[47] Yuan Wang, Rui Sun, and Tianzhu Zhang. Rethinking the correlation in few-shot segmentation: A buoys view. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages
2023.
[48] Yuan Wang, Naisong Luo, and Tianzhu Zhang. Foeus on query: Adversarial mining transformer for few-shot segmentation. In Advances in Neural Information Processing Systems, 2023.
[49] Paola Cascante-Boniln Fuwen Tan, Yanjun Qi, and Vicente Ordonez. Curriculum labeling: Revisiting
for semi-supervised learning. In Proceedings the AAAI Conference on Artificial Intelligence, volume 35, pages 6912-6920, 2021.
[50] Bowen Zhang, Yidong Wang, Wenxin Hou, Hao Wu, Jindong Wang, Manabu Okumura, and Takahiro Shinozaki. Flexmatch: Boosting superviid learning with curriculum pseudo labeling. Advances in Neural Information Processing Systems, 34: 1800--8849, 2021.
[51] Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, and Shin Ishii Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE transactions on pattern analysis and machine intelligence, 41(8):1979-199 2018
[52] Qizhe Xie, Zihang Dai, Eduard Hovy, Thang Luong, and Quoe Le. Unsupervised data augmenta- tion for consistency training. Advances in neural information processing systems, 33:6256-6268, 2020.
[53] Yassine Ouali, Céline Hudelot, and Myriam Tami. Semi-supervised semantic segmentation with cross consisoons training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12674- 12684, 2020.
[54] Zhengyang Feng, Qianyu Zhou, Qiqi Gu, Xin Tan, Guangliang Cheng, Xuequan Lu, Jianping Shi, and Lizhuang Ma. Dm Dynamic mutual training for semi-supervised learning. Pattern Recognition, 130: 2022.
[55] Zhanghan Ke, Daoye Wang, Qiong Yan. Jimmy Ren, and Rynson WH Lau. Dual student: Breaking the limits of the teacher in semi- supervise learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6728-6736, 2019.
[56] Hieu Pham, Zihang Dai, Qizhe Xie, and Quoc V Le. Meta pseudo labels. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11557-11568, 2021.
[57] Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight-averaged con- sistency targets improve semi-supervised deep learning results. Advances in neural information processing systems, 30, 2017.
[58] David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: A holistic approach to semi-supervised learning. Advances neural information processing systems, 32, 2019.
13

---

[59] Xin Lai, Zhuotao Tian, Li Jiang, Shu Liu, Hengshuang Zhao, Liwei Wang, and Jiaya Jia. Semi- supervised semantic segmentation with directional context-aware consistency. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 1205-1214, 2021.
[60] Yuanyi Zhong, Bodi Yuan, Hong Wu, Zhiqiang Yuan, Jian Peng, and Yu-Xiong Wang. Pixel contrastive-coosiistent semi-supervised semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7273-7282, 2021.
[61] Zenggui Chen and Zhouhui Lian. Semi-supervised semantic segmentation via prototypical contrastive learning. In Proceedings of the 30th ACM International Conference on Mullimedia, pages 6696-6705, 2022.
[62] Jiashuo Fan, Bin Gao, Huan Jin, and Lihui Jiang. Uce: Uncertainty guided cross- head co-training for semi-supervised semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9947-9956, 2022.
14


Reviews:
[{"review_id": "j4CrY4luQf", "reviewer": "Reviewer_ahfa", "summary": "This paper introduces DAW, a method for semi-supervised semantic segmentation that addresses the trade-off between false-positive and false-negative pseudo-labels. DAW utilizes Gaussian functions to model confidence distributions and derives an optimal weighting function based on their inter-section. It also incorporates distribution alignment to reduce the discrepancy between labeled and unlabeled data. Experimental results demonstrate the superiority of DAW over state-of-the-art methods on two benchmarks.", "questions": "Please refer to the Weaknesses.", "limitations": "Yes.", "rating": 5, "confidence": 1, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The author's viewpoint offers valuable insights into the problem of semi-supervised semantic segmentation.\n- The technical approach proposed in the paper is described adequately.", "weaknesses": "- While the paper demonstrates the efficacy of DAW on certain benchmarks, it would be beneficial to investigate the method's performance on larger datasets such as COCO-Stuff, ADE20K, and COCO to provide a comprehensive evaluation across different scales.\n- Table 1 is challenging to interpret without additional illustration of the symbols used in both the table and figure. Further clarification and explanation are needed to enhance readability.\n- The English writing in the paper requires further improvement. For example:\n    + In line 1, revise \"lies how to fully\" to \"lies in how to fully.\" \n    + In line 2, rephrase \"inaccurate yet utilized pseudo-labels, and correct yet discarded pseudo-labels\" to \"false positive pseudo-labels\" and \"false negative pseudo-labels\" respectively.\n\n---\nPost rebuttal:  \nAfter carefully reviewing the authors' feedback, I have made the decision to maintain my rating, which falls within the realm of borderline acceptance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for taking the time to share your comments and suggestions in the review assessment, as well as for acknowledging the novelty and adequate technical approach. \nWe see that your main concerns are more experiments on a larger dataset and further explanation in Table 1. \nAll your comments are addressed point by point in the following. \n\n**Q1:** While the paper demonstrates the efficacy of DAW on certain benchmarks, it would be beneficial to investigate the method's performance on larger datasets such as COCO-Stuff, ADE20K, and COCO to provide a comprehensive evaluation across different scales.  \n\n**A1:** Thanks for this valuable suggestion, to comprehensively assess the performance of our DAW against other competitors, we conduct experiments on the COCO dataset, which comprises 118k/5k training/validation images with 81 classes and can serve as quite a challenging benchmark. \n\nFor a fair comparison, we adopt exactly the same setting following PseudoSeg [1] and PC$^2$Seg [2]. \nAs tabulated in Table, we can observe that DAW achieves consistent performance gains over the Sup.-only baseline, obtaining improvements of 11.6%, 10%, and 7.8% under 1/128, 1/64, and 1/32 partition protocols, respectively. \nBesides, DAW also has a clear lead of ~4% compared to the recent SOTA method (PC$^2$Seg), which confirms the effectiveness of DAW to minimize the negative equivalence impact raised by the trade-off between inaccurate yet utilized pseudo-labels, and correct yet discarded pseudo-labels in previous methods. \n|     COCO      | 1/128 (925) | 1/64 (1849) | 1/32 (3697) |\n| :-----------: | :---------: | :---------: | :---------: |\n|   Sup.-only   |    33.6     |    37.8     |    42.2     |\n| PseudoSeg [1] |    39.1     |    41.8     |    43.6     |\n| PC$^2$Seg [2] |    40.1     |    43.7     |    46.1     |\n|     Ours      |    45.2     |    47.8     |    50.0     |\nWe will supply the related results in the revision. \n\n**Q2:** Table 1 is challenging to interpret without additional illustrations of the symbols used in both the table and figure. Further clarification and explanation are needed to enhance readability. \n\n**A2:** Sorry for not being able to give you an intuitive understanding of Table 1. \nWe have added additional symbol explanations to help readers better understand the trade-off in previous methods that hinder the model’s learning when dealing with pseudo-labels in semi-supervised semantic segmentation. \nPlease refer to the attached PDF in the **global response** for more details. \nWe will improve the writing of this part. \n\n**Q3:** The English writing in the paper requires further improvement.\n\n**A3:** We sincerely appreciate your patience during the review, and will further improve the writing in the revised version. \n\nHope our response could resolve your concern. Please let us know if you have further questions:)\n\n[1] Pseudoseg: Designing pseudo labels for semantic segmentation. ICLR 2021. \n\n[2] Pixel contrastive-consistent semisupervised semantic segmentation. ICCV 2021. "}, {"review_id": "xhsmfVSO74", "reviewer": "Reviewer_mWrq", "summary": "This paper presents a systematic analysis of the trade-off involved in previous methods that deal with pseudo-labels. The authors define the trade-off between inaccurate yet utilized pseudo-labels and correct yet discarded pseudo-labels by modeling the confidence distribution of both types of pseudo-labels. They propose a unified weighting function called DAW to minimize the negative impact caused by this trade-off. Interestingly, they find that the optimal solution for the weighting function is a hard step function, located at the intersection of the confidence distributions. Additionally, the authors introduce distribution alignment to address the discrepancy between the prediction distributions of labeled and unlabeled data. The experimental results on various benchmarks, including mitochondria segmentation, demonstrate that DAW outperforms state-of-the-art methods.", "questions": "Please refer to the Weaknesses.", "limitations": "The authors have adequately addressed the limitations", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "+ The paper is presented in a clear and concise manner, effectively communicating the motivation and ideas behind the proposed approach. The authors thoroughly analyze the trade-off involved in handling pseudo-labels in previous methods, defining it as the balance between utilizing inaccurate pseudo-labels and discarding correct ones.\n+ The paper provides a rigorous evaluation of the proposed approach through extensive experiments conducted on multiple benchmarks, ensuring a comprehensive assessment of its performance. The results obtained from the experiments showcase the superiority of the proposed approach against state-of-the-art methods.", "weaknesses": "- It is not accurate to state that existingmethods tend to employ certain criteria (weighting function) to select pixel-level pseudo labels. For instance, FlexMatch [40] dynamically adjusts thresholds for different classes at each time step to allow informative unlabeled data and their corresponding pseudo labels to be included, and FreeMatch [17] employs a self-adaptive approach to adjust the confidence threshold based on the learning status of the model.\n- The current state-of-the-art method [a] is not included for comparison in Tables 2-3.\n[a] \"NP-SemiSeg: When Neural Processes meet Semi-Supervised Semantic Segmentation.\" ICML. 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for taking the time to share your comments and suggestions in the review assessment, as well as for acknowledging the thorough analysis, rigorous experiments, and well-structured presentation. \nWe see that your main concern is about the imprecise statement. \nAll your comments are addressed point by point in the following. \n\n**Q1:** It is not accurate to state that existing methods tend to employ certain criteria (weighting function) to select pixel-level pseudo labels. For instance, FlexMatch [40] dynamically adjusts thresholds for different classes at each time step to allow informative unlabeled data and their corresponding pseudo labels to be included, and FreeMatch [17] employs a self-adaptive approach to adjust the confidence threshold based on the learning status of the model. \n\n**A1:** Sorry for any misunderstanding. \nOur intention is not to underscore that we are the first work to design a threshold adjustment strategy conditioned on the model’s learning status. \nInstead, we just want to summarize how mainstream methods in *semi-supervised semantic segmentation task* handle pseudo-labels (although, to our knowledge, most methods employ predefined criteria without considering the model's learning status). \n\nFor the mentioned semi-supervised learning methods including FlexMatch and FreeMatch, we argue that they might be insufficient in terms of adjusting thresholds according to the model’s learning progress ascribed to the lack of sophisticated consideration, thus impeding the training process. \nTo be more specific, FlexMatch dynamically adjusts class-wise thresholds to seek model status awareness, but these class-specific thresholds are still mapped from a *predefined fixed global threshold* (e.g., 0.95). \nFurthermore, FreeMatch attempts to empower the global threshold to perceive the learning status, but it is still constrained by the ad hoc definition of the average prediction confidence as a replacement for the previously fixed threshold. \n\nOrthogonal to these methods, our DAW  formally defines the trade-off between inaccurate yet utilized pseudo-labels, and correct yet discarded pseudo-labels by explicitly modeling the confidence distribution of correct and inaccurate pseudo-labels, and derives the better weighting function (i.e., pseudo-label selection criteria), minimizing the negative equivalence impact raised by the trade-off. \nTo further validate this point, we re-implement these two methods and compared them with our DAW  on the VOC blender set under 1/16 partition protocol, as summarized in the following Table, confirming the effectiveness of DAW. \n|   ResNet-50   | Pascal Blender 1/16 (662) |\n| :-----------: | :-----------------------: |\n| FlexMatch |           72.9            |\n| FreeMatch |           74.1            |\n|     Ours      |           75.8            |\n\nWe will improve the writing of this part and add these discussions in the revised paper. \n\n\n**Q2:** The current state-of-the-art method NP-SemiSeg is not included for comparison in Tables 2-3. \n\n**A2:** NP-Semiseg is a very recent work released later than the NeurIPS paper submission. \nBut thanks for mentioning this related work, we add its comparison with DAW on VOC classic set and will include the full comparison and discussion in the revised paper. \n| Pascal classic ResNet-50 | 1/16 (92) | 1/8 (183) | 1/4 (366) | 1/2 (732) |\n| :----------------------: | :-------: | :-------: | :-------: | :-------: |\n|      NP-SemiSeg      |   65.78   |   72.4    |   75.8    |   77.4    |\n|           Ours           |   71.5    |   72.4    |   76.7    |   78.1    |\n\nWe hope our response can resolve your concern on DAW. Please let us know if you still have any concern regarding the paper:)"}, {"review_id": "lEHhACjMn1", "reviewer": "Reviewer_SG4E", "summary": "This paper discusses the challenge of utilizing unlabeled data to improve the generalization performance of semantic segmentation models. Existing methods use weighting functions to select pseudo-labels, but face a trade-off between accuracy and utilization. The proposed method, DAW, models the confidence distribution of pseudo-labels and uses a unified weighting function to define the trade-off problem. The authors find the optimal solution to be a hard step function and propose distribution alignment to mitigate prediction distribution discrepancies. ", "questions": "What kind of results would we get if we combine the DA method with a manual approach of setting the most appropriate hyperparameters while using the FixMatch method?", "limitations": "The paper does not illustrate any limitations or any potential negative societal impact.", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. A new method called DAW is proposed, which models the confidence distribution of correct and inaccurate pseudo-labels and uses a unified weighting function to define the trade-off problem. This method achieves better results in handling pseudo-labels compared to existing methods.\n2. The hard step function and distribution alignment are used to solve the trade-off problem and prediction distribution discrepancy problem.\n3. The writing and presentation of the paper are well-done.", "weaknesses": "1. The premise for the proposed method to work is to correctly estimate the distribution of correct and incorrect pseudo-labels, which is a difficult problem. Firstly, for unlabeled data, it is very challenging to distinguish which pseudo-labels are correct and which are incorrect because there is no ground truth. Secondly, estimating the distribution based on labeled data may not guarantee the effectiveness of all data because in most scenarios, the proportion of labeled data is low.\n2. The paper uses a hard step function as the weighting function. What are the advantages of using a hard step function over a regular step function with a threshold set as a hyperparameter?\n3. From the results of Table 2, 3, it can be seen that the improvement in the final performance in most scenarios is relatively minimal compared to previous works.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for taking the time to share your comments and suggestions in the review assessment, as well as for acknowledging the novelty and well-done presentation. \nWe see that your main concern is about the effectiveness of DAW. \nAll your comments are addressed point by point in the following. \n\n**Q1:** The premise for the proposed method to work is to correctly estimate the distribution of correct and incorrect\npseudo-labels, which is a difficult problem. \nFirstly, for unlabeled data, it is very challenging to distinguish which pseudo-labels are correct and which are incorrect because there is no ground truth. \nSecondly, estimating the distribution based on labeled data may not guarantee the effectiveness of all data because in most scenarios, the proportion of labeled data is low. \n\n**A1:** We agree with the statement that \"The premise for ..., which is a difficult problem\". \nHowever, this is the exact issue we are striving to alleviate in DAW. \n\nTo distinguish which pseudo-labels are correct and which are incorrect, unlike previous methods such as FixMatch and AEL that fail to utilize the unlabeled data more effectively since they either use a pre-defined fixed threshold or an ad-hoc threshold adjusting scheme, \ncompromising their capability. \nWe formally define the trade-off between inaccurate yet utilized pseudo-labels, and correct yet discarded pseudo-labels by explicitly modeling the confidence distribution of correct and inaccurate pseudo-label, and derive the better weighting function (hard step function with the jump point located at the intersection of the two confidence distributions), minimizing the negative equivalence impact raised by the trade-off, that is, *maximizing the separation of correct and incorrect pseudo-labels*.\n\nFurthermore, considering the discrepancy between the prediction distributions of labeled and unlabeled data, we devise distribution alignment to *narrow the gap and enhance applicability to all data*, in pursuit of further unlocking the potential of the weighting function and enjoying the synergy (Please refer to Figure 4 in the main paper). \n\nExtensive experiments on mainstream benchmarks including electron microscope images demonstrate the effectiveness of DAW, especially when the labeled images are extremely limited (+6.3% gains under 1/16 partition protocol with ResNet-50 on VOC classic set compared to recent SOTA  AugSeg in Table 2). \n\nNote that similar paradigms have been proven effective, where the prediction distribution of labeled data is utilized to fit the unlabeled data. \nFor example, AEL employs labeled data to estimate class-wise confidence and guide the sampling and re-weighting of unlabeled data. We will add these discussions in the revised version for better clarification. \n\n**Q2:** The paper uses a hard step function as the weighting function. What are the advantages of using a hard step function over a regular step function with a threshold set as a hyperparameter? \n\n**A2:** \nIn fact, we do NOT take the hard step function as a prerequisite. Instead, we formally define the trade-off by explicitly modeling the confidence distribution of correct and inaccurate pseudo-labels, and derive the optimal solution for the weighting function is a hard step function (referred to as distribution-aware weighting function, DAWF), with the jump point located at the intersection of the two confidence distributions. \nNote that the dedicated weighting function is theoretically guaranteed, and is free of setting thresholds manually compared to the regular step function with a predefined fixed threshold. \n\nBesides, due to the negative/positive distribution evolving with the model's learning status, the DAWF dynamically adjusts the threshold for maximizing the separation between correct and inaccurate pseudo-labels (see Figure 5 in the main paper). \nThis is distinct from regular step functions that are arbitrarily controlled by hyper-parameters and thus disconnected from the model’s learning process, resulting in inferior performance. \n\nThanks for the suggestion. We further combine Distribution Alignment (DA) with regular step function using different hyperparameters to validate the effectiveness of DAWF. As tabulated in the following Table. \n|         ResNet-50         | Ours (DA + DAWF) | DA + 0.95 | DA + 0.9 | DA + 0.8 | DA + 0.6 | DA + 0 | FixMatch |\n| :-----------------------: | :--------------: | :-------: | :------: | :------: | :------: | :----: | :----------: |\n| Pascal blender 1/16 (662) |       75.8       |   71.0    |   70.2   |   69.7   |   69.8   |  69.5  |     70.6     |\n\nWe have the following additional findings: (1) The introduction of DAWF exceeds all fixed threshold settings under the 1/16 partition protocol, which verifies the effectiveness of DAWF. \n(2) DA is devised to narrow the gap between the prediction distributions of labeled and unlabeled data, in pursuit of further unlocking the potential of the weighting function and enjoying the synergy. Therefore, From the 2nd and 8th columns, we can observe that performance gains from simply adopting the DA on the FixMatch with threshold=0.95 are negligible. \n(3) We observe that entries with low thresholds performed worse than those with high thresholds. We conjecture that this is due to the fact that while low threshold hyperparameters increase the utilization of pseudo-labels, they inevitably recruit noisy pseudo labels into training, leading to confirmation bias, which is a corollary raised by erroneous pseudo-labels.\n\nPlease refer to the **global response** for Rebuttal -- Part 2."}, {"review_id": "QypWjAd44y", "reviewer": "Reviewer_Sjij", "summary": "In this paper, the authors thoroughly analyze the pixel-level pseudo label selection criteria and the trade-off associated with weighting functions in semi-supervised semantic segmentation. The authors propose a novel method called distribution-aware weighting (DAW) based on their analysis and confidence distribution modeling. The authors present sophisticated mathematical derivations and provide extensive experimental results to demonstrate the effectiveness of their approach. This research makes a valuable contribution to the field of semi-supervised semantic segmentation.", "questions": "1. In the confidence distribution modeling approach proposed by the authors, the normalization factor (\\alpha) for the truncated Gaussian distribution modeling the confidence distribution of pseudo-labels is not explicitly specified in the provided information. How is the value of the normalization factor obtained? \n2. The specific implementation details regarding the semi-supervised semantic segmentation methods employed in the paper have not been mentioned. Which kind of semi-supervised semantic segmentation methods are used in your paper? Teacher-Student scheme? Or other kinds? Please point out detailed implementation.\n3. The authors justify their choice of the Gaussian distribution for confidence distribution modeling based on the principle of maximum entropy and provide a detailed mathematical derivation to support their decision. However, it is worth considering if there are alternative explanations to support their opinion, such as experimental observations or more general assumptions.\n", "limitations": "The authors have not stated the limitations of their work.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The authors establish a well-thought-out confidence modeling framework and conduct a comprehensive analysis of the trade-off involved in pseudo-labeling. The mathematical derivations presented in this paper are sophisticated and thoughtful. The theoretical analysis provides valuable insights and guidance for future research.\n2. The theoretical analysis of the trade-off effectively demonstrates the effectiveness of the distribution-aware weighting (DAW) method and explains the reasons behind its superiority over competitor approaches. This mathematical tool not only validates the performance of DAW but also enables subsequent researchers to assess the theoretical performance of semi-supervised learning (SSL) techniques.\n3. In addition to evaluating the effectiveness of this work on commonly used semi-supervised semantic segmentation benchmarks, the authors extend their analysis to include other benchmarks, such as Lucchi. \n", "weaknesses": "1.\tThe notation in L107, L108 and L109 are not well matched. The authors use N_L and N_U in L107, L108, but write this notation as N_l and N_u in the following.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for taking the time to share your comments and suggestions in the review assessment, as well as for acknowledging the well-thought-out design, comprehensive analysis, sophisticated mathematical derivation and valuable insight. \nAll your comments are addressed point by point in the following. \n\n**Q1:** The notation in L107, L108 and L109 are not well matched. The authors use $N_L$ and $N_U$ in L107, L108, but write this notation as $N_l$ and $N_u$ in the following. \n\n**A1:** Thank you for your patient reading and sorry for the clerical error. We mix up the $N_L/N_U$ and $N_l/N_u$ incorrectly. We will carefully check our manuscripts in the revised version. \n\n**Q2:** In the confidence distribution modeling approach proposed by the authors, the normalization factor ($\\alpha$) for the truncated Gaussian distribution modeling the confidence distribution of pseudo-labels is not explicitly specified in the provided information. How is the value of the normalization factor obtained? \n\n**A2:** The effect of the normalization factor is to make the integral from $1/C$ to $1$ of the truncated Gaussian distribution 1 (this is the definition of the probability density function). We specify the value in L132 for $\\alpha$ and L136 for $\\beta$. \n\n**Q3:** The specific implementation details regarding the semi-supervised semantic segmentation methods employed in the paper have not been mentioned. Which kind of semi-supervised semantic segmentation methods are used in your paper? Teacher-Student scheme? Or other kinds? Please point out detailed implementation. \n\n**A3:** Following FixMatch[18], there is only one segmentation model in our method (Deeplabv3+), acting as teacher and student in the meanwhile. \nWe will supply these descriptions in the revision.\n\n**Q4:** The authors justify their choice of the Gaussian distribution for confidence distribution modeling based on the principle of maximum entropy and provide a detailed mathematical derivation to support their decision. However, it is worth considering if there are alternative explanations to support their opinion, such as experimental observations or more general assumptions. \n\n**A4:** Maximum entropy means that we can make the assumption with minimal error expectations without any prior, and is the core property of the Gaussian distribution, which is why the Gaussian distribution is so widely used, even in the Batchnorm. \n\nWe appreciate your positive review. Please let us know if you have further questions:)\n\n**Limitation and Society Impact.** Our method is primarily geared towards enhancing the utilization of unlabeled data, with comparatively lesser attention directed towards the labeled data. \nExploring avenues to effectively harness the labeled data in semi-supervised semantic segmentation is also a promising investigation. \nWithin this paper, we present an approach for semi-supervised semantic segmentation, a pivotal research area in the realm of computer vision, with no apparent negative societal implications known thus far. "}, {"review_id": "9hkQWpXkOA", "reviewer": "Reviewer_UScw", "summary": "This paper studies semi-supervised semantic segmentation, where the key challenge is how to select high-quality pseudo-labels. This paper systematically analyzes the trade-off between correct unreliable predictions and wrong confident predictions and theoretically finds the optimal weighting function. Interestingly, by explicitly modeling the confidence distributions of both correct and wrong predictions, the optimal weighting function is simply the hard step function with the jump point located at the intersection of the estimated two distributions. Experiments under various evaluation protocols demonstrate the efficacy of the proposed method.", "questions": "I do not have any questions. Please refer to the weaknesses section.", "limitations": "The authors **did not** discuss limitations and broader impacts.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. This paper is well-written and very easy to follow.\n2. Sufficient details are provided in the supplementary material.\n3. The idea of minimizing $E_1+E_2$ is quite novel.\n4. The proposed method surpasses state-of-the-art alternatives on various benchmarks, especially when the labeled images are extremely limited (e.g., 1/16 protocol on classic VOC).", "weaknesses": "There are some minor questions.\n1. The detailed operation of the proposed Distribution Alignment is relatively confusing. Does \"$\\cdot$\" in Eq. (12) indicate dot-product? It is better to provide a more detailed explanation of how Eq. (12) operates.\n2. Will more precise estimations contribute to better performances? For instance, will re-computing $(\\mu^{+}, \\sigma^{+})$ and $(\\mu^{-}, \\sigma^{-})$ over the *whole* labeled set after each iteration bring improvements? Or, is there any other technique better than the simple EMA when estimating the parameters of distributions?\n3. I am curious about the performance when applying DAW to other related fields, such as domain adaptive segmentation and semi-supervised classification/detection.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for taking the time to share your comments and suggestions in the review assessment, as well as for acknowledging the good writing, sufficient supplementary material, novel idea and convincing experiment result. \nWe see that your main concern is how to better estimate the distribution. \nAll your comments are addressed point by point in the following. \n\n**Q1:** The detailed operation of the proposed Distribution Alignment is relatively confusing. Does \"$\\cdot$\" in Eq. (12) indicate dot-product? It is better to provide a more detailed explanation of how Eq. (12) operates.\n\n**A1:** Sorry for not being able to give you an intuitive understanding in Eq. (12). The Distribution Alignment is proposed to narrow the discrepancy between the prediction distributions of labeled and unlabeled data to further unlock the potential of the Distribution-Aware Weighting Function. \nSpecifically, we first calculate the class-wise prediction confidence expectation of the model for labeled data and unlabeled data. \nThen an element-wise division operation is done between the expectation of labeled data and unlabeled data, which represents the prediction discrepancy of the model w.r.t labeled data and unlabeled data. \nFinally, we rectify the prediction for unlabeled data in a modulation manner, i.e., \"$\\cdot$\" denotes the element-wise multiplication operation. \nWe will provide a more detailed explanation in the revised version to further improve the readability. \n\n**Q2:** Will more precise estimations contribute to better performances? For instance, will re-computing ($\\mu^+, \\sigma^+$) and ($\\mu^-, \\sigma^-$) over the whole labeled set after each iteration bring improvements? Or, is there any other technique better than the simple EMA when estimating the parameters of distributions? \n\n**A2:** The prerequisite for our method to work well is a good estimation of the inherent positive and negative distribution. \nTheoretically, re-computing the ($\\mu, \\sigma$) over the whole labeled set after each iteration leads to a more precise estimation while bringing the unacceptable cost of training. \nIn fact, the model’s learning status changes not much between two adjacent iterations. \nAdopting EMA is an effective and efficient approach, which is proven by many current works such as FreeMatch [17], SoftMatch [23]. \nAnyhow, exploring better estimation techniques is what we are committed to in the future. \n\n**Q3:** I am curious about the performance when applying DAW to other related fields, such as domain adaptive segmentation and semi-supervised classification/detection. \n\n**A3:** The mainstream approach to these tasks you enumerated also involves pseudo-label selection. \nAnd our method can theoretically minimize the negative equivalence effect brought by the pseudo-label. \nSo we believe that our method can bring performance gains to these tasks. \nHowever, in this paper, we only consider the semi-supervised setting, where the labeled and unlabeled data come from the same domain, and  a simple Distribution Alignment can work well. \nWhile in the domain adaptation task, the labeled and unlabeled data come from different domains, meaning that the distribution discrepancy between the labeled and unlabeled data further increases. \nOnly by effectively alleviating such a domain discrepancy can our DAWF play its greatest role. \nThanks for your detailed suggestion, which is really worth thinking about in future works. \n\nWe appreciate your positive review. Please let us know if you have further questions:)\n\n**Limitation and Society Impact.** Our method is primarily geared towards enhancing the utilization of unlabeled data, with comparatively lesser attention directed towards the labeled data. \nExploring avenues to effectively harness the labeled data in semi-supervised semantic segmentation is also a promising investigation. \nWithin this paper, we present an approach for semi-supervised semantic segmentation, a pivotal research area in the realm of computer vision, with no apparent negative societal implications known thus far. "}]

Author rebuttal:
**To Reviewer SG4E -- Rebuttal Part2:**

**Q3:** From the results of Tables 2 and 3, it can be seen that the improvement in the final performance in most scenarios is relatively minimal compared to previous works.

**A3:** 
Semi-supervised semantic segmentation is a very challenging task, and DAW indeed provides some significant and promising improvement especially when the labeled images are extremely limited as recognized by the other four reviewers (+4.6% gains under 1/16 partition protocol with ResNet-101 on VOC classic set compared to recent SOTA AugSeg in Table 2). 

On the other hand, since the purpose of this paper is to explore the better weighting function to alleviate the negative impact raised by the trade-off when dealing with pseudo-labels, we adopt FixMatch as a baseline following the simplicity principle. 
In fact, the most direct and effective way to improve performance is data augmentation (e.g., AugSeg), which is also a mainstream direction of semi-supervised learning technologies. 
Theoretically, the techniques can also be incorporated into our method to optimize the overall performance. 
To validate our points, we combine data augmentation strategies from AugSeg with the pseudo-label selection criterion from our DAW, observing further gains in overall performance on the VOC dataset. 
|   ResNet-50   | Pascal blender 1/16 (662) |
| :-----------: | :-----------------------: |
|  AugSeg   |           74.7            |
|     Ours      |           75.8            |
| Ours + AugSeg |           76.9            |

**Q4:** What kind of results would we get if we combine the DA method with a manual approach of setting the most
appropriate hyperparameters while using the FixMatch method?

**A4:** Thanks for the suggestion. In the **A2**, we address all your questions about the effectiveness of DAWF. In addition, we will add all these valuable discussions into the revised version or the supplementary material. 


We hope our response can resolve your concern. Please do not hesitate to let us know if you have further questions:)

**Limitation and Society Impact.** Our method is primarily geared towards enhancing the utilization of unlabeled data, with comparatively lesser attention directed towards the labeled data. 
Exploring avenues to effectively harness the labeled data in semi-supervised semantic segmentation is also a promising investigation. 
Within this paper, we present an approach for semi-supervised semantic segmentation, a pivotal research area in the realm of computer vision, with no apparent negative societal implications known thus far. 

Meta-review: After rebuttal, four reviewers land on the positive side and only one reviewer (SG4E) leans to reject. Specifically, Reviewer SG4E finally leaves the initial rating – “Borderline Reject” unchanged and state that the authors solved some of his major problems in the rebuttal. However, neither which major problems are unsolved are clarified nor the justifications for the final rating are provided. Considering that other reviewers all enjoy the insights offered by the theoretical analysis and their major concerns, e.g., the lack of comparison on larger datasets such as COCO, have been addressed in the rebuttal, the AC decides to accept the paper. The authors should include the discussions and new results in the rebuttal into the final version.
Decision: Accept (poster)