All Points Matter: Entropy-Regularized Distribution Alignment for Weakly-supervised 3D Segmentation
Authors: Liyao Tang
Zhe Chen
Shanshan Zhao
Chaoyue Wang
Dacheng Tao
Venue: NeurIPS 2023 poster (2023)
Keywords: point cloud segmentation
weak supervision

Abstract: Pseudo-labels are widely employed in weakly supervised 3D segmentation tasks where only sparse ground-truth labels are available for learning.
Existing methods often rely on empirical label selection strategies, such as confidence thresholding, to generate beneficial pseudo-labels for model training.
This approach may, however, hinder the comprehensive exploitation of unlabeled data points.
We hypothesize that this selective usage arises from the noise in pseudo-labels generated on unlabeled data. The noise in pseudo-labels may result in significant discrepancies between pseudo-labels and model predictions, thus confusing and affecting the model training greatly.
To address this issue, we propose a novel learning strategy to regularize the generated pseudo-labels and effectively narrow the gaps between pseudo-labels and model predictions.
More specifically, our method introduces an Entropy Regularization loss and a Distribution Alignment loss for weakly supervised learning in 3D segmentation tasks, resulting in an ERDA learning strategy.
Interestingly, by using KL distance to formulate the distribution alignment loss, it reduces to a deceptively simple cross-entropy-based loss which optimizes both the pseudo-label generation network and the 3D segmentation network simultaneously.
Despite the simplicity, our method promisingly improves the performance.
We validate the effectiveness through extensive experiments on various baselines and large-scale datasets.
Results show that ERDA effectively enables the effective usage of all unlabeled data points for learning and achieves state-of-the-art performance under different settings.
Remarkably, our method can outperform fully-supervised baselines using only 1\% of true annotations.
Code and model will be made publicly available at https://github.com/LiyaoTang/ERDA.

All Points Matter: Entropy-Regularized Distribution Alignment for Weakly-supervised 3D Segmentation
Liyao Tang', Zhe Chen', Shanshan Zhao', Chaoyue Wang', Dacheng Tao
The University of Sydney, Australia 2 La Trobe University, Australia 1tan96870umi. syddeey sydney. edu...edd.a. aduu.au, au, zhe chen@latrobe edu. chaoyue wangboutlook. com, (sshan. zhao00, dacheng tao/&gmail.com
Abstract
Pseudo-labels are widely employed in weakly supervised 3D segmentation tasks where only sparse ground-truth labels are available for learning. Existing methods often rely on empirical label selection strategies, such as confidence threshold- ing, generate beneficial pseudo-labels for model training. This approach may, however, hinder the comprehensive exploitation of unlabeled data points. We hypothesize that this selective usage arises from the noise in pseudo-labels gen- erated on unlabeled data. The noise in pscudo-labels may result in significant discrepancies between pseudo-labels and model predictions, thus confusing and affecting the model training greatly. To address this issue, we propose a novel learning strategy regularize the generated pseudo-labels and effectively narrow the gaps between pseudo-labels and model predictions. More specifically, our method introduces an Entropy Regularization loss and Distribution Alignment loss for weakly supervised learning in 3D segmentation tasks, resulting in an ERDA learning strategy. Interestingly, by using KI. distance to formulate the distribution alignment loss, it reduces to a deceptively simple cross-entropy- -hased loss which optimizes both the pseudo-label generation network and the 3D segmentation ner- work simultaneously. Despite the simplicity, our method promisingly improves the performance. We validate the effectiveness through extensive experiments on various baselines and large-scale datasets. Results show that ERDA effectively enables the effective usage of all unlabeled data points for learning and achieves state-of-the--e -he-trr- performance under different settings. Remarkably, our method outperform sully-supe baselines using only 1% of true annotations. Code and model will be made publicly available at https://github.com//iyyy www.ooo//Liioooooo www.o///yyyaaaaa//aD.aa.
Introduction
Point cloud semantic segmentation is a crucial task for 3D scene understanding and has various applications [27, 83), such autonomous driving, unmanned aerial vehicles, and augmented reality. Current sfate-of-the-art approaches heavily rely on large-scale and densely annotated 3D datasets, which are costly to obtain [21, 33J. To avoid demanding exhaustive annotation, seakly-supervised point cloud segmentation has emerged as promising alternative. aims to leverage a small set of annotated points while leaving the majority of points unlabeled in large point cloud dataset for learning. Although current weakly supervised methods can offer practical and cost-effective way to perform point cloud segmentation, their performance is still sub- sub-optimal compared to fully-supervised approaches.
During the exploration of weak supervision [85], significant challenge is the insufficient training signals provided by the highly sparse labels. To tackle this issue, pseudo-labeling methods [85, 94, 32] have been proposed, which leverage predictions on unlabeled points as labels to facilitate the learning of the segmentation network. Despite some promising results, these pseudo-label methods have been outperformed by some recent methods based on consistency regularization [47, 87]. We tend to attribute this to the use of label selection on pseudo-labels, such as confidence thresholding, which
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Network Prodiction q
mloU
ERDA
68
one-bol (existing)
fall
Stop Grad
67
other methods
Pscudo-Label Generation
Label Solection
/
H(p. 9)
huseline wi 1% amno.
66
*
buscline w/ full amno.
HybriOCR (47)
65
(a) Existing: Sparse Pscudo-Labels
64
SON
(95]
63
Network Prediction
62-
shang [94]
61
Pscudo-Label Generation
Lp
LER
LDa
60
H
(none) 500 10
10*
all
peeudo labels (per-scene
gradients direction
(b) ERDA: Dense Pseudo-Labels
(c) Performance Comparison
Figure 1. While existing pscudo-labels (a) are limited in the exploitation of unlabeled points, ERDA (b) simultancously optimizes the pscudo-labels p and predictions q taking the same and simple form of cross- entropy. By reducing the noise via entropy regularization and bridging their distributional discrepancies, ERDA produces informative pscudo-labels that neglect the need for label selection. As in (c), it thus enables the model to consistently benefit from more pseudo-labels, surpasses other methods and its fully sulley-sdd baseline, and can be extended to advance the suply-super performance.
could lead to unlabeled points being wasted and under-explored. We hypothesize that the need for label selection arises from the low-confidence pseudo-labels assigned to unlabeled points, which are known for their noises [21] and potential unintended bias [60, 100]. These less reliable and noisy pscudo-labels could contribute to discrepancies between the pseudo-labels and the model predictions, which might confuse and impede the learning process to great extent.
By addressing the above problem for label selection in weakly supervised 3D segmentation, we propose a novel learning-baad based approach in this study. Our method aims to leverage the information from all unlabeled points by mitigating the negative effects of the noisy pseudo-labels and the distributional discrepancy.
Specifically, we introduce two learning objectives for the pseudo-label generation process. Firstly, introduce an entropy regularization (ER) objective reduce the noise and uncertainty in the pseudo-labels. This regularization promotes more informative, reliable, and confident pseudo-labels, which helps alleviate the limitations of noisy and uncertain pseudo-labels. Secondly, we propose a distribution alignment (DA) loss that minimizes statistical distances between pseudo-labels and model predictions. This ensures that the distribution of generated pseudo-labels remains close the distribution of model predictions when regularizing their entropy.
In particular, discover that formulating the distribution alignment loss using KL distance enables a simplification of our method into a cross-entropy-stykk learning objective that optimizes both the pseudo-label generator and the 3D segmentation network simultaneously. This makes method straightforward to implement and apply. By integrating the entropy regularization and distribution alignment, we achieve the ERDA learning strategy, as shown in Fig. 1.
Empirically, we comprehensively experiment with three baselines and different weak supervision settings, including 0.02% (1-point, or lpt), 1%, and 10%. Despite its concise design, our ERDA out- performs existing
methods on large-scale point cloud datasets such as S3DIS [2], ScanNet [16]. and SensatUrban [33]. Notably,
ERDA can surpass the supervse baselines using only 1% labels, demonstrating its significant effectiveness leveraging pseudo-labels. Further- more, validate the scalability of our method by successfully generalizing it more other settings, which illustrates the benefits of utilizing dense pseudo-label supervision with ERDA.
2 Related Work
Point cloud segmentation. Point cloud semantic segmentation aims to assign semantic labels points. The cutting-edge methods are deep-learning-basee and can be classified into projection-basee and point-based approaches. Projection-bass based methods project points to grid-like structures, such as 2D image [84, 39, 12, 4, 45] or 3D voxels [15, 71, 67, 28, 22, 23, 76]. Alternatively, basnt-

---

methods directly operate on 3D points [56, 57]. Recent efforts have focused on novel modules and
backbones to enhance point features, such as 3D convolution [3, 48, 78, 68, 51, 58], attentions [34, 26, 97, 79, 42, 59], graph-based methods [74, 44), and other modules such as sampling [18, 86, 88, 7] and post-procsssing [54, 35, 66]. Although these methods have made significant progress, they rely on large-scale datasets with point-wise annotation and struggle with few labels [85]. To address the demanding requirement of point-wise annotation, our work explores weakly-supervin learning for 3D point cloud segmentation.
Weakly-supervised point cloud segmentation. Compared to weakly-supervisse 2D image seg- mentation [99, 49, 75, 1, 64], weakly-superviser 3D point cloud segmentation is less explored. In general, weakly-superrvsed 3D segmentation task focus on highly sparse labels: only a few scattered points annotated in large point cloud scenes. Xu and Lee [85] first propose use 10x fewer labels to achieve performance on par with a fully-supervised point cloud segmentation model. Later studies have explored more advanced ways exploit different forms of weak supervision [77, 14, 40] and human annotations [53, 69]. Recent methods tend to introduce perturbed self-distillation [95], consistency regularization [85, 62, 80, 81, 43], and leverage self- learning [62, 37, 47, 87] based on contrastive learning [29, 10]. Pseudo-labels are another approach to leverage unlabeled data, with methods such as pre-training networks colorization tasks [94], using iterative training [32], employing separate networks to iterate between learning pseudo-labels and training 3D segmen- tation networks [53], or using super-point graph [44] with graph attentional module to propagate the limited labels over super-points [13]. However, these existing methods often require expensive training due hand-crafted 3D data augmentations [95, 87, 80, 81]. iterative training [53, 32], or additional modules [87, 32], complicating the adaptation of backbone models from fullyysupervised to weakly-supervised learning. contrast, our work aims to achieve effective weakly supervised learning for the 3D segmentation task with straightforward motivations and simple implementation.
Pseudo-label refinement. Pseudo-labeling [46], versatile method for entropy minimization [24], has been extensively studied in various tasks, including semi semi-supervised 2D classification [82, 60]. segmentation [64, 89]. and domain adaptation [98, 73]. To generate high-quality supervision. various label selection strategies have been proposed based on learning status [72. 91. 20], label uncertainty [60, 98, 73, 50], class balancing [100], and data augmentations [64, 89, 100]. Our method is most closely related to the works addressing bias in supervision, where mutual learning [20. 70, 92] and distribution alignment [100, 31, 41] have been discussed. However, these works typically focus on class imbalance [100, 31] and rely on iterative training [70, 20, 92, 41]. label selection [20, 31], and strong data augmentations [100, 31J, which might not be directly applicable to 3D point clouds. For instance, common image augmentations [64] like cropping and resizing may translate to point cloud upsampling [96], which remains an open question in the related research area. Rather than introducing complicated mechanisms, we argue that proper regularization on pseudo-labels and its alignment with model prediction can provide significant benefits using a very concise learning approach designed for the weakly supervised 3D point cloud segmentation task.
Besides, it is shown that the data augmentations and repeated training in mutual learning [70, 38] are important to avoid the feature collapse, i.e., the resulting pseudo-labels being uniform or the same as model predictions. We suspect the cause may originate from the entropy term in their use of raw statistical distance by empirical results, which potentially matches the pseudo-labels to noisy and confusing model prediction, as would be discussed in Sec. 3.2. Moreover, in self-supervised learning based on clustering [5] and distillation [6]. it has also been shown that it would lead to feature collapse if matching to a cluster assignment or teacher output of close-unifone distribution with high entropy, which agrees with the intuition in our ER term.
3 Methodology
3.1 Formulation of ERDA
As previously mentioned, we propose the ERDA approach to alleviate noise in the generated pseudo- labels and reduce the distribution gaps between them and the segmentation network predictions. In general, our ERDA introduces two loss functions, including the entropy regularization loss and the distribution alignment loss for the learning on pseudo-labels. We denote the two loss functions as LER and T.PA, respectively. Then, we have the overall loss of ERDA as follows:
Lp ALER + LDA
(I)
3

---

where the À > O modulates the entropy regularization which is similar to the studies [46, 24].
Before detailing the formulation of LEn and LDA,, we first introduce the notation. While the losses are calculated over all unlabeled points, we focus on one single unlabeled point for ease of discussion. We denote the pseudo-label assigned to this unlabeled point as p and the corresponding segmentation network prediction as Each p and is a ID vector representing the probability over classes.
Entropy Regularization loss. We hypothesize that the quality of pseudo-labels can be hindered by noise, which in turn affects model learning. Specifically, we consider that the pseudo-label could be more susceptible containing noise when it fails to provide confident pseudo-labeling result, which leads to the presence of a high-entropy distribution in p.
To mitigate this, for the p. propose to reduce its noise level by minimizing its Shannon entropy, which also encourages a more informative labeling result [61]. Therefore, we have:
LER H()),
(2)
where Hp)) = E; -; lo and i iterates over the vector. By minimizing the entropy of the pseudo-label as defined above, we promote more confident labeling results to help resist noise in the labeling process'.
Distribution Alignment loss. In addition to the noise in pseudo-labels, we propose that significant discrepancies between the pseudo-labed and the segmentation network predictions could also confuse the learning process and lead to unreliable segmentation results. general, the discrepancies can stem from multiple sources, including the noise-induced unreliability of pseudo-labels, differences between labeled and unlabeled data [100], and variations in pseudo-labeling methods and segmentation methods [92, 20]. Although entropy regularization could mitigate the impact of noise in pseudo- labels, significant discrepancies may still persist between the pscudo-labels and the predictions of the segmentation network. To mitigate this issue, we propose that the pseudo-labels and network can be jointly optimized to narrow such discrepancies, making generated pseudo-labels not diverge too far from the segmentation predictions. Therefore, introduce the distribution alignment loss.
To properly define the distribution alignment loss (LDA), we measure the KL divergence between the pseudo-labed (p) and the segmentation network predictions (q) and aim to minimize this divergence. Specifically, we define the distribution alignment loss as follows:
LDA (L(pl))).
where K [plllq) refers the KL divergence. Using the above formulation has several benefits. For example, the KL. divergence can simplify the overall loss L, into a deceptively simple form that demonstrates desirable properties and also performs better than other distance measurements. More details will be presented in the following sections.
Simplified ERDA. With the LER and LDA formulated above, given that (pll)) (I q) (p) where H (p, q) is the cross entropy between p and we can have a simplified ERDA formulation as:
Lp Hp, g) + (A- (1)))
(4)
In particular, when À = 1, we obtain the final ERDA loss?:
Lp = (p, q) = Σ -pi log gi
(5)
The above simplified ERDA loss describes that the entropy regularization loss and distribution alignment loss can be represented by single cross-entropy- loss that optimizes both and q.
We would like emphasize that Eq. (5) is distinct from the conventional cross-entropy loss. The conventional cross-entropy loss utilizes fixed label and only optimizes the term within the logarithm function, whereas the proposed loss in Eq. (5) optimizes both p and q simultaneously.
I We note that our entropy regularization aims for entropy mininization on pseudo-labels; and we consider noise as the uncertain predictions by the pseudo-lahels, instead incorrect predictions.
We would justify the choice of X in the following Sec. 3.2 well Sec. 4.3

---

Lea
Kl(pllg)
KLig p))
Jlip.ai
MSE(p.g)
#(p.9) (1 A)M(())
H(g H(g) + AP(p)
H
1 2
A)H(D)
Σ in
of +AH())
9 Name
S2
(A 1)p, Σ log
P. Ap. Σ
1 12c Kps
Kzu
2 2
pt Σn Pn
Table 1. The formulation of Lp using different functions to formulate LDA. We study the gradient update 8Lg under different situations. S1: update given confident pseudo-label, p being one-hot with on 8t, i.e. öay
3pk E P.Ps. Pa 1. S2: update given confusing prediction, q being uniform with ς1 qK k More analysis well visualization can be found in the See. 3.2 and the supplementary Appendix
3.2 Delving into the Benefits of ERDA
To formulate the distribution alignment loss, different functions can be employed to measure the differences between p and q. In addition to the KL divergence, there are other distance measurements like mean squared eπrоr (MSE) mutual learning methods [20, 92, 41, 38] have proven the effectiveness of KI. divergence, a detailed comparison of KL divergence against other measurements is currently lacking in the literature. In this section, under the proposed ERDA learning framework, we show by comparison that K (I(p|(q) a better choice and ER is necessary for weakly-supervised 3D segmentation.
Censen-Shannon (JS) divergence for replacement. Although many
To examine the characteristics different distance measurements, including K LLpp q). K Lglll)) (pll))) and M S (p qq). we investigate the form of our ERDA loss Lp and its impact on the learning for pseudo-label generation network given two situations during training.
More formally, we shall assume a total of K classes and define that pseudo-label p p= [p1..... Pk] based on the confidence scores s = [s1, ..., 9k). and that p = softmax(s). Similarly, we have segmentation network prediction q (q,,..., -x] for the same point. We re-write the ERDA loss Lp in various forms and investigate the learning from the perspective of gradient update, as in Tab. 1.
Gradient update given confident pseudo-label p. We first specifically study the case when p is very certain and confident, i.e., p approaching one-hot vector. As Tab. 1, most distances yield the desired zero gradients, which thus retain the information of a confident and reliable p. this situation, however, the K [L [p), rather than K' [p]l(q) in our method, produces non-zero gradients that would actually increase the noise among pseudo-labels during its learning, which is not favorable according to our motivation.
Situation 1:
Situation 2: Gradient update given confusing prediction q. In addition, we are also interested in how different choices of distance and X would impact the learning on pseudo-label the segmentation model produces confusing outputs, i.e., q tends to be uniform. In line with the motivation of ERDA learning, we aim to regularize the pseudo-labels to mitigate potential noise and bias, while discouraging uncertain labels with little information. However, in Tab. 1, most implementations yield non-zero gradient updates to the pseudo-label generation network. This update would make p closer to the confused q. thus increasing the noise and degrading the training performance. Conversely, only K LL(p)) qq can produce a zero gradient when integrated with the entropy regularization with A == 1. That is, only ERDA in Eq. (5) would not update the pscudo-label generation network when q is not reliable, which avoids confusing the p. Furthermore, when q is less noisy but still close to a uniform vector, it is indicated that there is large close-zero plateau on the gradient surface of ERDA, which benefits the learning on p by resisting the influence of noise in q.
In addition to the above cases, the gradients of ERDA in Eq. (5) could be generally regarded as being aware of the noise level and the confidence of both pseudo-label p and the corresponding prediction q. Especially, FRDA produces larger gradient updates on noisy pseudo-labels, while smaller updates on confident and reliable pscudo-labels or given noisy segmentation prediction. Therefore, our formulation demonstrates its superiority in fulfilling our motivation of simultancous noise reduction and distribution alignment, where both Lgn and KL-based LDA are necessary. We provide more empirical studies in ablation (Sec. 4.3) and detailed analysis in the supplementary.
3.3 Implementation Details Pseudo-Labels
In our study, we use a prototypical pseudo-label generation process due to its popularity as well as simplicity [94]. Specifically, prototypes [63] denote the class centroids in the feature space, which
5

---

Network Prediction
Inputs
Features
Projected
Score
Dense Pseudo-Labbe
MLPs
Lp LER + LDa H(p. q)
Projection
Network
Momentum
update
connection w/ grudients
ERDA
connection wr gradicats
w/ Pseudo-Labe Generation
Class Cantroids
grudients direction
Figure 2. Detailed illustration of our ERDA with the prototypical pseudo-label generation process, which is shared for both (a) and (b) in Fig. 1.
calculated based on labeled data, and pseudo-labels are estimated based on the feature distances between unlabeled points and class centroids.
As shown in Fig. 2, we use momentum- based prototypical pseudo-labet generation process due to popularity as well as simplicity [94, 85, 93]. Specifically, prototypes [63] denote the class centroids in the feature space, which are calculated based on labeled data, and pseudo-labels are estimated based on the feature distances between unlabeled points and class centroids. avoid expensive computational costs and compromised representations for each semantic class [94, 87, 47]. momentum update utilized as an approximation for global class centroids.
Based on the momentum-updaated prototypes, we attach an MLP-based projection network to help generate pseudo-label, and learn with our method. Aligned with our motivation, we do introduce thresholding-be based label selection or one-hot conversion [46, 94] to process generated pseudo-labels. More details are in the supplementary.
More formally, we take as input a point cloud X, where the labeled points are X' and the unlabeled points X". For labeled point 2 denote its label by y. The pseudo-label generation process can be described as follows:
CN = NI.
Σ XEXX
go f(x). Cp t- mCp + (1 -1-10 (1-m) m)
Vx E X" 84. d(g f(x), Ck) p softmax(s),
where C2 denotes the global class centroid for the k-th class, NYF is the number of labeled points of the k-th class, g g((())) is the transformation through the backbone network and the projection network g, is the momentum coefficient, and score s. By default, we use 2-layer MLPs for the projection network g and set m 0.999.
cosine similarity for d(., -) generate the
Besides, due to the simplicity of ERDA, are able to follow the setup of the baselines for training, which enables straightforward implementation and easy adaptation on various backbone models with little overhead.
Overall objective. Finally, with ERDA learning in Eq. (5), we maximize the same loss for both labeled and unlabeled points, segmentation task, and pseudo-label generation, where we allow the gradient to back-propase through the (pseudo-)labee llaells. The final loss is given as
L =
Σ Love (q.3) XEX!
+ +a (g. No
Σ Lp(g.p). *
(6)
NI
where L, p) = Loo p)
H(q. p) is typical cross-entropy loss used for point cloud segmentation, No and N are the numbers labeled and unlabeled points, and a is the loss weight.
Experiments
We present the benefits of our proposed ERDA by experimenting with multiple large-scale datasets, including S3DIS [2], ScanNet [16], SensatUrban [33] and Pascal [19]. We also provide ablation studies for better investigation.

---

settings
methods
mloU
ociling
floor
wall
beam
column
window
doer
table
chair
sofa
bookease
board
clutter
PointNet [56]
41.1
88.8
97.3
69.8
0.1
3.9
46.3
10.8
59.0
52.6
5.9
40.3
26.4
33.2
MinkowskiNet [15]
65.4
91.8
98.7
86,2
0.0
34.1
48.9
74.4
58.6
KPConv [68]
65.4
92.6
97.3
81.4
0.0
16.5
54.5
69.5
90.1
80.2
74.6
66.4
63.7
58.1
SQN [32]
63.7
92.8
96.9
81.8
0.0
25.9
50.5
65.9
79.5
85.3
55.7
72.5
65.8
55,9
HybridCR [47]
65.8
93.6
98.1
82.3
0.0
24.4
59.5
66.9
79.6
87.9
67.1
73.0
66.8
55.7
Fully
RandLA Net [34]
64.6
92.4
96.8
80.8
0.0
18.6
57.2
54.1
87.9
79.8
74.5
70.2
66.2
59.3
EERDA
68.3
93.9
98.5
83.4
0.0
28.9
62.6
70.0
89.4
82.7
75.5
69.5
75.3
58.7
Closerl Lookllokk3555 [51]
66.2
94.2
96.1
82.7
0.0
22.2
57.6
70.4
91.2
81.2
75.3
61.7
65.8
60.4
EERDA
69.6
94.5
98.5
85,2
0.0
31.1
57.3
72.2
91.7
83.6
77.6
74.8
75,8
62.1
PT [97]
70.4
94.0
98.5
86.3
0.0
38.0
63.4
74.3
89.1
82.4
74.3
80.2
76.0
59.3
EERDA
72.6
95.8
98.6
86.4
0.0
43.9
61.2
81.3
93.0
84.5
77.7
81.5
74.5
64.9
zhang et ol. [94]
45.8
PSD [95]
48.2
87.9
96.0
62.1
0.0
20.6
49.3
40.9
55.1
61.9
43.9
50.7
27.3
31.1
MIL--Trans
51.4
86.6
93.2
75.0
0.0
29.3
45.3
46.7
60.5
62.3
47.5
33.7
32.2
HybridCR [47]
51.5
85.4
91.9
65.9
0.0
18.0
51.4
34.2
63.8
78.3
52.4
59.6
29.9
39.0
0.02%
LA-Net [:4]
40.6
84.0
94.2
59.0
0.0
40.4
16.9
52.8
51.4
52.2
16.9
27.8
27.0
(lp))
ERDA
48.4
87.3
96.3
61.9
0.0
11.3
45.9
31.7
73.1
65.1
57.8
26.
36.0
36.4
Clnserl [51]
34.6
33.6
40.3
52.4
0..0
21.1
25.4
35.3
48.9
48.9
53.9
23.8
35.3
30.1
EERD
52.0
90.0
96.7
70.2
0.0
21.5
45.8
41.9
76.0
65.5
56.1
51.5
30.6
30,9
PT [97]
2.2
0.0
0.0
24.2
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
ERDA
26.2
86.8
96.9
63.2
0.0
0.0
0.0
15.1
29.6
26.3
0.0
0.0
0.0
22.8
zhang el al. [94]
61.8
91.5
96.9
80.6
0.0
18.2
58.1
47.2
75.8
85.7
65.2
68.9
65.0
50.2
PSD [95]
63.5
92.3
97.7
80.7
0.0
27.8
56.2
62.5
78.7
84.1
63.1
70.4
58.9
53.2
SQN [32]
63.6
92.0
96.4
81.3
0.0
21.4
53.7
77.8
69.9
66.6
52.5
HybridCR [47]
65.3
92.5
93.9
82.6
0.0
24.2
64.4
63.2
78.3
81.7
69.0
74.4
68.2
56.5
1%
RA-Net [34]
59.8
92.3
97.5
77.0
0.1
15.9
48.7
38.0
83.2
78.0
64.9
50.6
ERDA
67.2
94.2
97.5
82.3
0.0
27.3
60.7
68.8
88.0
80.6
76.0
70.5
68.7
58.4
loser Look3DDD
39.9
95.3
98.4
78.7
0.0
44.4
38.1
84.9
79.0
67.8
53.9
34.1
ERDA 68.2
94.0
98.2
83.8
0.0
30.2
56.7
62.7
91.0
80.8
75.4
80.2
74.5
58.3
PT
63.8
94.2
98.2
83.0
0.0
44.2
50.4
88.1
83.0
47.4
64.3
39.0
EEDA
70.4
95.5
98.1
85.5
0.0
30.5
61.7
73.3
90.1
82.6
77.6
80.6
76.0
63.1
Xu and Lee [85]
48.0
90.9
97.3
74.8
0.0
8.4
49.3
27.3
69.0
71.7
16.5
53.2
23.3
42,8
Semi-sup [37]
57.7
zhang et [94]
64.0
SQN [32]
64.7
93.0
97.5
81.5
0.0
28.0
55.8
68.7
80.1
87.7
55.2
72.3
63.9
57.0
10%
RandL.A. Net [34]
61.7
91.7
97.8
79.4
0.0
50.8
45.5
81.3
$7.1
63.8
51.8
ERDA
67.9
94.3
98.4
83.2
0.0
30.5
60.7
67.4
88.8
83.2
74.5
68.8
72.4
60.4
Closerl 31 [51]
55.5
93.0
98.2
73.6
0.0
25.6
36.0
51.1
ERDA
69.1
94.7
98.5
83.2
0.0
28.8
53.8
70.9
91.5
82.5
75.8
82.1
75.3
61.6
[97]
66.0
91.7
98.3
83.7
0.0
48.1
81.9
67.3
57.2
ERDA
71.7
94.6
98.5
86.5
0.0
49.7
61.3
82.4
89.8
84.3
78.0
70.5
74.1
62.4
Table 2. The results obtained the S3DIS datasets Area 5. For all baseline methods, we follow their official instructions in cvaluation. The bold denotes the best performance in each setting.
4.1 Experimental Setup
We choose RandLA-Net [34] and CloserLook3D [51] our primary basclines following previous works. Additionally, while transformer models [17, 52] have revolutionized the field of computer vision well 3D point cloud segmentation [97, 42], none of the existing works have addressed the training of transformer for point cloud segmentation with weak supervision, even though these models are known to be data-hungry [17]. We thus further incorporate the PoineTransformer (PT) [97] as our baseline to study the amount of supervision demanded for effective training of transformer.
For training, we follow the setup of the baselines and set the loss weight comparison, we follow previous works [94, 95, 32] and experiment with different settings, including the 0.02% (Ipt), 1% and 10% settings, where the available labels are randomly sampled according to the r More details are given in the supplementary.
0.1. For a fair
4.2 Performance Comparison
Results on S3DIS. S3DIS [2] is a large-scale point cloud segmentation dataset that covers 6 large indoor areas with 272 rooms and 13 semantic categories. As shown in Tab. 2, ERDA significantly improves over different basclines on all settings and almost all classes. In particular, for confusing classes such as column, window, door, and board, our method provides noticeable and consistent improvements in all weak supervision settings. We also note that PT suffers from severe wett-fitiing and feature collapsing under the supervision of extremely sparse labels of "Ipt" setting; whereas it is alleviated with ERDA, though not achieving satisfactory performance. Such observation agrees with the understanding that transformer is Mata-hungry [17].
3 'Soe super-voxel-best boxel- approaches, such as OTOC [53]. additionally leverage the super-voxel partition from the dataset annotations [32]. We thus treat them different setting and avoid direct comparison.
7

---

Input
Ground Truth
Baseline
ERDA
Improv
Figure 3. We show obvious improvement of our ERDA over baseline (RandLA-Nar on different scenes from S3DIS Area 5. In the office and hallway (top 2), ERDA produces more detailed and complete segmentation for windows and doors, and avoids over-expansion of the board and bookcase on the wall, thanks to the informative pseudo-labels. In more cluttered scenes (bottom 2), ERDA tends to make cleaner predictions by avoiding improper situations such as desk inside clutter and preserving important semantic classes such as columns.
settings
methods
mlol.
settings
methods
mlol
settings
methods
mloU
PointNet [36]
47.6
PoinICNN [48]
45.8
PointNet [56]
23.7
R-ndL-A-Net [34]
70.0
RandL.A-Net [34]
64.5
PointNet++ [57]
32.9
KPConv
70.6
Fully
KPCouv
68.4
RandLA-Net [34]
52.7
HybridCR [47]
70.7
HybridCR [47]
59.9
Fully
KPConv [68]
57.6
Fully
PT [97]
73.5
LloserLookk1122 ERDA
70.4
L.PFormer [36]
63.4
RandLA--Net + ERDA
PointNeXt-
71.0
20pts
CloseLLook + ERDA
[87]
57.0
RandL A. + ERDA
SQN [32]
64.7
34.0
CloserLook31) + ERDA
ERDA
73.7
0.1%
RandLA-Net + ERDA
SQN
62.0
0.1%
Ren ERDA
56.4
zhang cr [94]
65.9
zhing of, [94]
51.1
Table 5. Results on Sensat Urban test.
PSD
68.0
PSD [95]
18
HybridCR [47]
69.2
HybridCR [47]
56.8
RandLA Net + ERDA
69.4
RandLA-Net + ERDA
63.0
micbods [64]
CloserLook3D ERDA
PT + ERDA
73.5
73.5
78.0
Table 3. Results on S3DIS 6-fold.
Table 4. Results on ScanNel test.
Table Results on Pascal dataset.
Impressively, ERDA yields competitive performance against most supervised methods. For instance, with only 1% of labels, it achieves performance better than its stand-alone baselines with full supervision. Such result indicates that the ERDA is more successful than expected in alleviating the lack of training signals, as also demonstrated qualitatively in Fig. 3.
Therefore, we further extend the proposed method to fully-supervised training, iee,, in setting "Fully" in Tab. More specifically, we generate pseudo-labels for all points and regard the ERDA as an auxiliary loss for supervsuee learning. Surprisingly, we observe non-trivial improvements (+3.7 for RandLA-Net and +3.4 for CloserLook3D) and achieve the state-of-the-ar art performance of 72.6 (+2.2) in mloU with PT. We suggest that the improvements are due to the noise-aware learning from ERDA, which gradually reduces the noise during the model learning and demonstrates to generally effective. Moreover, considering that the ground-truth labels could suffer from the problem of label noise [38, 90, 65], we also hypothesize that pseudo-labels from ERDA learning could stabilize fully-supervised learning and provide unexpected benefits.
We also conduct the 6-fold cross-validation, reported in Tab. We find our method achieves leading performance among both weakly and fully-supervised methods, which further validates the effectiveness of our method.
Results on ScanNet. ScanNet [16] is an indoor point cloud dataset that covers 1513 training scenes and 100 test scenes with 20 classes. In addition to the common settings, e.g., 1% labels, it also provides official data efficient settings, such as 20 points, where for each scene there are pre-defined
8

---

DA mloU ent.
LDa \ A
onc-hot
soft
ERDA
baseline
59.8
65.1
66.3
64
63.1
62.8
64.5
+ pscudo-labels
63.3 2.49
K KL(p [q]
66.1
67.2
66.6
500
63.0
62.3
64.1
65.1 2.26
K ((lllp)
66.1
65,9
65.2
lc3
63.3
63.5
65.5
ERDA
66.1 2.44
JS
65.2
65.4
65.1
le4
63.0
62.9
63.6
67.2 2.40
MSE
66.0
66.2
66.1
dense
62.7
62.6
67.2
(a) ERDA improves the results and reduces the entropy (ent.), individually and jointl.
(b) ER and DA provide better results
(c) ERDA consistently benefits the
when taking (plll) with À = 1.
model with more pseudo-labels (k).
Table Ablations on ERDA. If specified, the model is RandLA-Ne trained with ERDA as well as dense pscudo-labels on S3DIS under the 1% setting and reports mloU. Default settings are marked in gray
set of 20 points with the ground truth label. We evaluate on both settings and report the results in Tab. 4. We largely improve the performance under 0.1% and 1% labels. In 20pts setting, we also employ a convolutional baseline (CloserLook33D for a fair comparison. With no modification on the model, we surpass AlL--transfomer [87] that additionally augments the backbone with transformer modules and multi-scale inference. Besides, we apply ERDA to baseline under supey-suee setting and achieve competitive performance. These results also validate the ability of ERDA in providing effective supervision signals.
Results SensatUrban. SensatUrban [33] is urban-scale outdoor point cloud dataset that covers the landscape from three UK cities. In Tab. 5, ERDA surpasses SQN [32] under the same 0.1 % setting well as its fully-supervised baseline, and also largely improves under full supervision. It suggests that our method can be robust to different types of datasets and effectively exploits the limited annotations as well as the unlabeled points.
Generalizing to 2D Pascal. As our ERDA does not make specific assumptions on the 3D data, we explore its potential in generalizing similar 2D settings. Specifically, study important task of semi-supervised segmentation on image [72, 89, 8] and implement our method to the popular baseline, FixMatch [64], which combines the pseudo-labels with weak-to-strong consistency and is shown to benefit from stronger augmentation [89]. We use DeepLabv3+ [9] with ResNet-101 [30].
As in Tab. 6, show that ERDA brings consistent improvement from low to high data regime, despite the existence of strong data augmentation and the very different data as well as setting". thus indicates the strong generalization of our method. We also see that the improvement is less significant than the 3D cases. It might be because 2D data are more structured (e.g., pixels on a 2D grid) and are thus less noisy than the 3D point cloud.
4.3 Ablations and Analysis
We mainly consider the 1% setting and ablates in Tab. to better investigate ERDA and make a more thorough comparison with the current pseudo-label generation paradigm. For more studies on hyper-parameters, please refer to the supplementary.
Individual effectiveness of ER and DA. To validate our initial hypothesis, we study the individual effectiveness of L and T.D.A in Tab. 7a. While the pseudo-labels essentially improve the baseline performance, we remove its label selection and one-hot conversion when adding the ER or DA term. We find that using ER alone can already be superior to the common pseudo-labels and largely reduce the entropy of pseudo-labels (ent.) expected, which verifies that the pseudo-labels are noisy, and reducing these noises could be beneficial. The improvement with the DA term alone is even more significant, indicating that large discrepancy is indeed existing between the pseudo-labels and model prediction and is hindering the model training. Lastly, by combining the two terms, we obtain the ERDA that reaches the best performance but with the entropy of its pseudo-labels larger than ER only and smaller than DA only. thus also verifies that the DA term could be biased to uniform distribution and that the ER term is necessary.
Different choices of ER and DA. Aside from the analysis in Sec. 3.2, we empirically compare the results under different choices of distance for LDA and for Lgn- As in Tab. 7b, the outstanding result justifies the choice of KL(qllp) with = 1. Additionally, all different choices and combina-
'We note that the number in the first row in Tab. 6 denotes the number of labeled images, which is different from the setting of sparse labels in 3D point clouds.

---

tions of ER and DA terms improve over the common pseudo-labels (63.3), which also validates the general motivation for ERDA.
Ablating label selection. We explore in more detail how the model performance is influenced by the amount of exploitation on unlabeled points, as ERDA learning aims to enable full utilization of the unlabeled points. In particular, we consider three pseudo-labels lablls types: common one-hot pseudo-labed soft pseudo-labels (p), and soft pseudo- labels with ERDA learning. To reduce the number of pseudo-labele we select sparse but nigh-confidence pseudo- labels following common top-k strategy [94, 53J with various values of k to study its influence. As Tab. 7c, ERDA learning significantly improves the model performance under all cases, enables the model to consistently benefit from more pseudo-labels, and thus neglects the need for label selection such as top-k strategy, as also revealed in Fig. 1. Besides, using soft pseudo-labels alone can not improve but generally hinders the model performance, as one-hot conversion may also reduce the noise in pscudo-labels, which is also not required with ERDA.
5 Limitation and Future Work
While we mostly focus on weak supervision in this paper, our method also brings improvements under fully-supervised setting. We would then like further explore the effectiveness and relationship of LEn and LDA under full supervision as a future work. Besides, despite promising results, our method, like other weak supervision approaches, assumes complete coverage of semantic classes in the available labels, which may not always hold in real-world cases. Point cloud segmentation with missing or novel classes should be explored as important future direction.
6 Conclusion
In this paper, we study the supervis 3D point cloud semantic segmentation, which imposes the challenge of highly sparse labels. Though pseudo-labels are widely used, label selection is commonly employed to overcome the noise, but it also prevents the full utilization of unlabeled data. By addressing this, we propose new learning scheme on pseudo-labels, ERDA, that reduces the noise, aligns to the model prediction, and thus enables comprehensive exploitation of unlabeled data for effective training. Experimental results show that ERDA outperforms previous methods in various settings and datasets. Notably, it surpasses its fully-supervised baselines and can be further generalized to full supervision as well as 2D images.
Acknowledgement. This project is supported in part by ARC FL-17010011 17, and 19-190100031.
References
[1] Jiwoon Ahn and Suha Kwak. Learning pixel-level semantic aftinity with image-level supervision for weakly supervised semantic segmentation. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4981-4990, 2018.
[2] Iro Armeni, Sasha Sax, Amir R Zamir, and Silvio Savarese. Joint 2d- 3d-semantic data for indoor scene understanding. arXiv preprint orXiv:/702.00) I05, 2017. 2, 6, 7, 15
[3] Alexandre Boulch, Gilles Puy, and Renaud Marlet. Fkaconv: Feature-kernel alignment for point cloud
convolution, 2020.
[4] A. Boulch, B. Le Saux, and N. Audebert. Unstructured point cloud semantic labeling using deep segmentation networks. In Proceedings of the Workshop on 3D Object Retrieval, 3Dor '17, page 17-24, Goslar, DEU, 2017. Eurographics Association.
[5] Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsu- pervised learning of visual features by contrasting cluster assignments. ArXiv, abs/2006.09988 0098822 2020.
[6] Mathilde Caron, Hugo Touvron, Ishan Misra, Herve Jegou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Oct 2021. 3
[7] Chen Chen, Zhe Chen, Jing Zhang, and Dacheng Tao. Sasa: Semantics augmented set abstraction for point-based 3d object detection. In AAAI, 2022.
[8] Hao Chen, Ran Tao, Yue Fan, Yidong Wang. Jindong Wang, Bernt Schiele, Xing Xie, Bhiksha Raj, and Marios Savvides. Softmatch: Addressing the quantity-qualii quality trade-off in semi-supervise learning. 2023.
9
10

---

[9] Liang- Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder- decoder with atrous separable convolution for semantic image segmentation. In European Conference on Computer Vision, 2018.
[10] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. arXiv preprint arXiv:2002.088 05000, 2020. 3, 17
[11]
Ting Chen, Simon Komblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey Hinton. Big self-
supervised models are strong semi-super learners. arXiv preprint arXiv:2006. 10029, 2020. 17
[12] Zhe Chen, Jing Zhang. and Dacheng Tao. Progressive lidar adaptation for road detection. IEEE/CAA Journal of Automatica Sinica, 6:693-702, 05 2019.
[13] Mingmei Cheng, Le Hui, Jin Xie, and Jian Yang. Sspe-net: Semi-supervised semantic 3d point cloud
segmentation network. 2021. 3
[14] Julian Chibane, Francis Engelmann, Tuan Anh Tran, and Gerard Pons-Moll. Box2mask: Weakly supervised 3d semantic instance segmentation using bounding boxcs. Computer Vision ECCV 2022,
pages 681 6819699, 2022.
[15] Christopher Choy, Jun Young Gwak, and Silvio Savarese. spatio-temporal convnets: Minkowski convolutional ncural networks. In Proceedings of the IEEE Conference on Computer Vision and Pottern Recognition, pages 3075-3084, 2019. 2.7
[16] Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proc. Computer Vision and Pattern Recognition (CVPR), IEEE, 2017. 2, 8, 15,
[17] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikow, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghami, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakoh Uszkoreit, and Neil Houlsby. An image is worth 16x16 words:
Transformers for image recognition at scale. ICIR, 2021.
7
[18] Oren Dovrat, Itai Lang, and Shai Avidan. Learning to sample. 2019 IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2019. 3
[19] M. Everingham, S. M. A. Eslami, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The pascal visual object classes challenge: A retrospective. International Journal of Computer Vision, 111(1)998-136,
Jan. 2015. 6
[20] Zhengyang Feng, Qianyu Zhou, Qiqi Gu, Xin Tan, Guangliang Cheng, Xuequan Lu, Jianping Shi, and Lizhuang Ma. Dmt: Dynamic mutual training for semi-supervised learning. Pattern Recognition,
130:108777, Oct 2022. 3,5,
[21] Biao Gao, Yancheng Pan, Chengkun Li, Sibo Geng, and Huijing Zhao. Are we hungry for 3d lidar data for semantic segmentation? survey of datasets and methods. IEEE Transactions on Intelligent Transportation Systems, 2377)66033-081, Jul 2022. 1,2
[22] Benjamin Graham, Martin Engeleke, and Laurens van der Maaten. 3d semantic segmentation with submanifold sparse convolutional networks. 2018 IEEF/CVF Conference on Computer Vision and Pattern
Recognition, Jun 2018. 2
[23] Benjamin Graham and Laurens van der Maaten. Submanifold sparse convolutional networks. ArXiv,
7060170 2017. 2
[24] Yves Grandvalet and Yoshua Bengio. Semi-supervised learning by entropy minimization. L. Saul,
Y. Weiss, and I.. Bottou, editors, Advances in Neural Information Processing Systems, volume 17, MIT
Press, 2004. 3,4
[25] Jean-Bastien Grill, Florian Strub, Florent Altch'e Corentin Tallec, Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Ávila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, Rémi Munos, and Michal Valko. Bootstrap your own latent: A new approach to self-supervised learning. ArXiv, abs/2000.07733s 2020. 17
[26] Meng-Hao Guo, Junxiong Cai, Zheng-Ning Liu, Tai-Jiang Mu, Ralph R. Martin, and Shi-Min Hu. PCT: point cloud transformer. CoRR, abs/2012.09688,, 2020.
[27] Yulan Guo, Hanyun Wang, Qingyong Hu, Hao Liu, Liu, and Mohammed Bennamoun. Deep learning for 3d point clouds: A survey. IEEE transactions on pattern analysis and machine intelligence, 2020.
[28] Lei Han, Tian Zheng, Lan Xu, and Lu Fang. Occuseg: Occupancy
-wwrre aware 3d instance segmentation. 2020 IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2937-2946, 2020.
[29] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation lcarning. arXiv preprint arXiv: 191 1.05722, 2019.
[30] K. He, X. Zhang. Ren, and Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770-778, Los Alamitos, CA. USA, jun 2016. IEEE Computer Society. 9
[31] Ruifei He, Jihan Yang, and Xiaojuan Qi. Re-distributing biased pseudo labels for semi-supervised semantic segmentation: A baseline investigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6930-6940, 2021.
11

---

[32] Qingyong Hu, Bo Yang, Guangchi Fang, Yulan Guo, Ales Leonardis, Niki Trigoni, and Andrew Markham. Sqn: Weakly-supervised semantic segmentation of large-scale 3d point clouds. In European Conference on Computer Vision, 2022. 1, 3. 8,
[33] Qingyong Hu, Bo Yang. Sheikh Khalid, Wen Xiao, Niki Trigoni, and Andrew Markham, Sensaturban: Learning semantics from urban-scale photogrammetrric point clouds. International Journal of Computer Vision, 130(2)311-333,, 2022. 2, 6, 15, 17
[34] Qingyong Hu, Bo Yang, Linhai Xie, Stefano Rosa, Yulan Guo, Zhihua Wang, Niki Trigoni, and An- drew Markham, Randla-net: Efficient semantic segmentation of large-scale point clouds. CoRR,
abs/1911.11236 2019. 8, 15
[35] Zeyu Hu, Mingmin Zhen, Xuyang Bai, Hongbo Fu, and Chiew-Lan Tai. Jsenet: Joint semantic segmenta- tion and edge detection network for 3d point clouds. CoRR, abs/2007. fos//2077006888, 2020. 3
[36] Zhuo Huang, Zhiyou Zhao, Banghuai Li, and Jungong Han. Lepformer: Towards effective 3d point cloud
analysis via local context propagation in transformers. ArXiv, abs/2210.12755 2022.
[37] Li Jiang. Shaoshuai Shi, Zhuotao Tian, Xin Lai, Shu Liu, Chi- Wing Fu, and Jiaya Jia. Guided point contrastive learning for semi-supervised point cloud semantic segmentation. In ICCV, pages 6403 6412,
10 2021. 7
[38] Yi Kun and Wu Jianxin. Probabilistic End-to-end Noise Correction for Learning with Noisy Labels. In
The IEEE Conference on Computer Vision and Puttern Recugnition (CVPR), 2019. 3, 5,
[39] Abhijit Kundu, Xianqi Yin, Alireza Fathi, David A. Ross, Brian Brewington, Thomas A. Funkhouser, and Caroline Pantofaru. Virtual multi-view fusion for 3d semantic segmentation. In ECCV, 2020. 2
[40] Hyeokjun Kweon and Kuk-Jin Yoon. Joint learning of 2d-3d weakly supervised semantic segmentation. Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022. 3
[41] Donghyeon Kwon and Suha Kwak. Semi-supervised semantic segmentation with error localization network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9957-9967, June 2022. 3,5
[42] Xin Lai, Jianhui Liu, Li Jiang, Liwei Wang. Hengshuang Zhao, Shu Liu, Xiaojuan Qi, and Jiaya Jia. Stratified transformer for 3d point cloud segmentation. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8490-8499, 2022. 3,7
[43] Yuxiang Lan, Yachao Zhang, Yanyun Qu, Cong Wang, Chengyang Li, Jia Cai, Yuan Xie, and Zongze Wu. Weakly supervised 3d segmentation via receptive-driven pscudo label consistency and structural consistency, Proceedings of the AAAI Conference on Artificial Intelligence, 37(1): 1230, Jun. 2023. 3
[44] L. Landricu and M. Simonovsky. Large-scale point cloud semantic segmentation with superpoint graphs. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
2018. 3
[45] Felix Järemo Lawin, Martin Danelljan, Patrik Tosteberg. Goutam Bhat, Fahad Shahbaz Khan, and Michael Felsberg. Deep projective 3d semantic segmentation. Lecture Notes in Computer Science, page 95-107, 2017. 2
[46] Dong-Hyum Lee. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. 2013. 3, 3.
[47] Mengtian Li, Yuan Xie, Yunbang Shen, Bo Ke, Ruizhi Qiao, Bo Ren, Shaohui Lin, and Lizhuang Ma. Hybrider: Weakly-supervised 3d point cloud semantic segmentation via hybrid contrastive regularization. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pages
14930-14939, 14939, June 2022. 3, 6, 7, 8, 17
[48] Yangyan Li, Rui Bu, Mingehao Sun, Wei Wu, Xinhan Di, and Baoquan Chen. Pointenn: Convolution on x-transformed points. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 31. Curran Associates,
Inc., 2018. 8
[49] Di Lin, Jifeng Dai, Jiaya Jia, Kaiming He, and Jian Sun. Scribblesup: Scribble-supervised convolutional networks for semantic segmentation. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3159 3167, 2016.
[50] Yuyuan Liu, Yu Tian, Yuanhong Chen, Fengbei Liu, Vasileios Belagiannis, and Gustavo Carneiro. Per- turbed and strict mean teachers for semi-supervised semantic segmentation. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2022. 3
[51] Ze Liu, Han Hu, Yue Cao, Zheng Zhang, and Xin Tong. A closer look at local aggregation operators in point cloud analysis. ECCV, 2020. 3, 7.
[52] 7e Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hicrarchical vision transformer using shifted windows. arXiv preprint arXiv:2103. 14030, 2021. 7
[53] Zhengzhe Liu, Xiaojuan Qi. and Chi Mhig Fu. One thing one click: A self-training approach for weakly supervised 3d semantic segmentation. CoRR, abs/2104.00246, 2021. 3, 7, 10
12

---

[54] Tao Lu, Limin Wang. and Gangshan Wu. Cga-net: Category guided aggregation for point cloud semantic segmentation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 11693-11702. June 2021. [55] A. Milioto, I. Vizzo. J. Behley, and C. Stachniss. Rangenet++: Fast and accurate lidar semantic
segmentation. In 2019 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). pages 4213-4220, 2019.
[56] Charles Ruizhongtai Qi. Hao Su, Kaichun Mo, and Leonidas Guibas. Pointnet: Deep learning on point
sets for 3d classification and segmentation. CoRR, abs/1612.00593, 2016. 7,8
[57] Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J. Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. CoRR, abs/ 0tts/170..000 02013, 2017. 3.8
[58] Guocheng Qian, Yuchen Li, Houwen Peng, Jinjie Mai, Hasan Hammoud, Mohamed Elhoseiny, and Bernard Ghanem. Pointnext: Revisiting pointnet++ with improved training and scaling strategies. In Advances in Neural Information Processing Systems (NeurIPS), 2022, 3, 8
[59] Haibo Qiu, Baosheng Yu, and Dacheng Tao. Collect-and-distriiute transformer for 3d point cloud analysis. arXiv preprint ar 2.ii:23306.0257... 2023. 3
[60] Mamshad Naycem Rizve, Kevin Duarte, Yogesh S Rawat, and Mubarak Shah. In defense of pscudo- labeling: An uncertainty-aware pseudo-label selection framework for semi-supervised learning. In International Conference on Learning Representations, 2021. 2,
[61] Claude Elwood Shannon. A mathematical theory of communication. The Bell System Technical Journal, 27:379-423, 1948. 4
[62] Hanyu Shi, Jiacheng Wei, Ruibo Li, Fayao Liu, and Guosheng Lin. Weakly supervised segmentation on outdoor 4d point clouds with temporal matching and spatial graph propagation. In 2022 IEFF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11830-11839, 2022. [63] Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In Advances
Neural Information Processing Systems, 2017. 5, 6
[64] Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 596-608. Curran Associates, Inc.,
2020. 3, 8, 9, 15
[65] Hwanjun Song, Minseok Kim, Dongmin Park, Yooju Shin, and Jae-Gil Lee. Learning from noisy labels with deep neural networks: survey. IEEE Transactions on Neural Networks and Learning Systems,
pages 1-19, 2022. 8
[66] Liyao Tang. Yibing Zhan, Zhe Chen, Baosheng Yu, and Dacheng Tao. Contrastive boundary learning for point cloud segmentation. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8479-8489, 2022. 3
[67] Lyne Tehapmi, Christopher Choy, Iro Armeni, Jun Young Gwak, and Silvio Savarese. Segeloud: Semantic segmentation of 3d point clouds. 2017 International Conference on 3D Vision (3DV), Oct 2017. 2
[68] Hugues Thomas, Charles R. Qi, Jean- Emmanuel Deschaud, Beatriz Marcotegui, François Goulette, and Lconidas J. Guibas. Kpconv: Flexible and deformable convolution for point clouds. CORR, abs/1904.08889) 2019. 3, 7,8, 15
[69] Ozan Unal, Dengxin Dai, and Lue Van Gool, Scribhle-supervisedd lidar semantic segmentation. ArXiv,
abs/2203.08537, 2022.
[70] Guo- Wang and Jianxin Wu. Repetitive reprediction deep decipher for semi-supervised learning. Proceedings of the AAAI Conference on Artificial Intelligence, 1700477677--6777 Apr. 2020. 3
70-6177,
[71] Peng-Shuai Wang, Yang Liu, Yu-Xiao Guo, Chun-Yt Yu Sun, and Xin Tong. O-CNN: Oetree-based Convolutional Neural Networks for 3D Shape Analysis. ACM Transactions Graphics (SIGGRAPH). 36(4), 2017. 2
[72] Yidong Wang, Hao Chen, Qiang Heng. Wenxin Hou, Yue Fan, Zhen Wu, Jindong Wang, Marios Savvides, Takahiro Shinozaki, Bhiksha Raj. Bernt Schiele, and Xing Xie. Freematch: Self-adaptive thresholding for semi-supervised learning. 2023. 3,9
[73] Yuxi Wang. Junran Peng, and Zhaoxiang Zhang. Uncertainty aware pseudo label refinery for domain adaptive semantic segmentation. In 2021 IEEE/CVF International Conference on Computer Vision
(ICCV), pages 9072-9081, 2021. 3
[74] Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay F. Sarma, Michael M. Bronstein, and Justin M. Solomon.
Dynamic graph CNN for lcarning on point clouds. CoRR, abs/1801 801107829, 2018. 3
[75] Yuchao Wang, Haochen Wang, Yujun Shen, Jingjing Fei, Wei Li, Guoqiang Jin, Liwei Wu, Rui Zhao, and
Xinyi Le. Semi-supervised semantic segmentation using unreliable pscudo-labels. ArXiv, ths/2203.03884.
2022.
[76] Zongji Wang and Feng Lu. Voxsegnet: Volumetric cnns for semantic part segmentation of 3d shapes. IEEE transactions visualization and computer graphics, 2018. 2
13

---

[77] Jiacheng Wei, Guosheng Lin, Kim-HHi Yap. Tzu-Yi Hung, and Lihua Xie. Multi-path region mining for weakly supervised 3d semantic segmentation on point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4384-4393, 2020.
[78] Wenxuan Wu, Zhongang Qi, and Fuxin Li. Pointconv: Deep convolutional networks on 3d point clouds.
CoRR, abs/1811.07244 2018. 3
[79] Xiaoyang Wu, Yixing Lao, Li Jiang, Xihui Liu, and Hengshuang Zhao. Point transformer v2: Grouped
vector attention and partition-based pooling. In NewrlPS, 2022. 3
[80] Zhonghua Wu, Yicheng Wu. Guosheng Lin, and Jianfei Cai. Reliability
consistency regulariza-
tion for weakly-superrised point cloud segmentation, 2023, 3
[81] Zhonghua Wu, Yicheng Wu, Guosheng Lin, Jianfei Cai, and Chen Qian. Dual adaptive transformations
for weakly supervised point cloud segmentation. arXiv preprint arXiv:2207. 09900444 2022.
[82] Qizhe Xie, Minh-Thann Thang Luong, Eduard Hovy, and Quoc V Le. Self-training with noisy student improves imagenet classification. arXiv preprint 10Xiv:9911.042555 2019.
[83] Y. Xie, J. Tian, and X. X. 7hu. Linking points with labels in 3d: A review of point cloud semantic segmentation. IEEE Geoscience and Remote Sensing Magazine, 8(4):38-59, 2020.
[84] Chenfeng Xu, Bichen Wu, Zining Wang, Wei Zhan, Peter Vajda, Kurt Keutzer, and Masayoshi Tomizuka. Squeezesegv3: Spatially-adaptin convolution for efficient point-cloud segmentation. ArXiv, abs/2004.0 00000, 2020. 2
[85] Xun Xu and Gim Hee Lee. Weakly supervised semantic point cloud segmentation: Towards 10x fewer
labels. In CVPR. 2020. 1, 6,
[86] Xu Yan, Chaoda Zheng, Zhen Li, Sheng Wang, and Shuguang Cui. Pointasnl: Robust point clouds processing using nonlocal neural networks with adaptive sampling. CoRR. 100/2003.00492. 2020. [87] Cheng-Kun Yang. Ji-Jia Wu, Kai-Syun Chen, Yung- Yu Chuang, and Yen-Yr Yu Lin. An mil-derived transformer for weakly supervised point cloud segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pages 1 11830-11839, June 2022. 1, 3, 6. 7, 8,
[88] Jiancheng Yang. Qiang Zhang. Bingbing Ni. Linguo Li, Jinxian Liu. Mengdie Zhou, and Qi Tian. Modeling point clouds with self-attention and gumbel subset sampling. CoRR, abs/1904.03375. 2019. 3 [89] Lihe Yang, Lei Qi, Litong Feng. Wayne Zhang, and Yinghuan Shi. Revisiting weak-to-strong consistency in semi-supervised semantic segmentation. In CVPR, 2023. 3.9
[90] Shuquan Ye, Dongdong Chen, Songfang Han, and Jing Liao. Learning with noisy labels for robust point cloud segmentation. International Conference on Computer Vision, 2021.
[91] Bowen Zhang, Yidong Wang, Wenxin Hou, Hao Wu, Jindong Wang, Manahu Okumura, and Takahiro Shinozaki. Flexmatch: Boosting semi- supervise learning with curriculum pseudo labeling. 2021. 3
[92] Pan Zhang, Bo Zhang, Ting Zhang. Dong Chen, and Fang Wen. Robust mutual learning for semi- supervised semantic segmentation, 2021. 3, 4,
[93] Xiaolong Zhang, Zuqiang Su, Xiaolin Hu, Yan Han, and Shuxian Wang. Semisupervised momentum prototype network for gearbox fault diagnosis under limited laheled samples. IEEE Transactions on Industrial Informatics, 18(9):6203- 22116023366662222001212222211111 2022. 6
[94] Yachao Zhang. Zonghao Li, Yuan Xie, Yanyun Qu, Cuihua Li, and Tao Mei. Weakly supervised semantic segmentation for large-scale point cloud. In Proceedings of the AAAI Conference on Artificial Intelligence,
volume 35. pages 3421-3429 2021. 1, 3, 6. 7,8, 10
[95] Yachao Zhang, Yanyun Qu, Yuan Xie, Zonghao Li, Shanshan Zheng, and Cuihua Li. Perturhed self- distillation: Weakly supervised large-scale point cloud semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 55520-15528, 2021. 7,
[96] Yan Zhang. Wenhan Zhao, Bo Sun, Ying Zhang, and Wen Wen. Point cloud upsampling algorithm: A
systematic review. Algorithms, 15(4), 2022. 3
[97] Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip Torr, and Vladlen Koltun. Point transformer, 2021. 3, 7, 8,
15
[98] Zhedong Zheng and Yi Yang. Rectifying pseudo label learning via uncertainty estimation for domain adaptive semantic segmentation. International Journal of Computer Vision, 129(4): 1000:100611111 1120, Jan 2021.
[99] Bolei Zhou, Aditya Khosla, Àgata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2921-2922 2016. 3
[100] Yanning Zhou, Hang Xu, Wei Zhang, Bin Gao, and Pheng-Ann Heng. C3 C3-semiseg: Contrastive semi- supervised segmentation via cross-set learning and dynamic blass-balancing, In 2021 IEEE/CVF Interna- tional Conference on Computer Vision (ICCV), pages 7016-7025, 2021. 2, 3.4
14

---

A Supplementary: Introduction
In this supplementary material, we provide more details regarding implementation details Ap- pendix B, more analysis of ERDA in Appendix C, full experimental results in Appendix D, and studies on parameters in Appendix E.
B Supplementary: Implementation and Training Details
For the RandLA-Net [34] and CloserLook3D [51] baselines, we follow the instructions in their released code for training and evaluation, which are here (RandLA-Net) and here (CloserLookk3D) respectively. Especially, in []]s)],.oo3D((]],,, there are several local aggregation operations and we use the "Pseudo Grid" IKPConv-like one, which provides neat re- immmeemnnaaiinn of the popular KPConv [68] network (rigid version). For point transformer (PT) [97], we follow their paper and the instructions in the code base that claims to have the official code (here). For FixMatch [64], we use the publicly available implementation here.
Our code and pre-traine pre-trained models will be relcased.
C Supplementary: Delving into ERDA with More Analysis
Following the discussion in Sec. 3, we study the impact of entropy regularization as well as different distance measurements from the perspective of gradient updates.
In particular, we study the gradient the score of the i-th class i.e., Si. and denote it 9, 0s: 0p; Given that 8s; ((i=))P: PaPy. have 9, ps E, Pr{ SL, aL, Op: ap, ). As shown in Tab. 8, demonstrate the gradient update A 9i under different situations.
8L,
In addition to the analysis in Sec. 3.2, we find that, when q is certain, Le., approaching a one-hot vector, the update of our choice K (plll) would approach the infinity. We note that this could be hardly encountered since is typically also the output of a softmax function. Instead, we would rather regard it as a benefit because it would generate effective supervision on those model predictions with high certainty, and the problem of gradient explosion could also be prevented by common operations such as gradient clipping.
In Fig. 4, we provide visualization for a more intuitive understanding on the impact of different formulations for LDA well their combination with LÆn- Specifically, consider simplified case of binary classification and visualize their gradient updates when À takes different values. We also visualize the gradient updates of LER- By comparing the gradient updates, observe that only (I(p)lq) with À = 1 can achieve small updates when q is close to uniform (q under the binary case), and that there is close-0 plateau indicated by the sparse contour lines.
Additionally, we also find that, when increasing the X, all distances, except the K LL /q), are modulated to be similar to the updates of having LER alone; whereas K L(pllq) can still produce effective updates, which may indicate that (L(pplq) is more robust to the X.
D Supplementary: Full Results
We provide full results for the experiments reported the main paper. For S3DIS [2], we provide the full results of S3DIS with 6-fold cross-validation in Tab. 10. For ScanNet [16] and SensatUrban [33], we evaluate on their online test servers, which are here and here, and provide the full results Tab. 11 and Tab. 12, respectively.
15

---

Loa
KZiplkl
J.ip. 41
M581p.91
((p-0) (0in-0--- (1- -
Mo - Mog) + Milp)
A/MM())
2 1000
20 stin
pr log M mn
- mn X Joe
1/ Alloo 3
mew mal in Mnii oo Ar En
Sinurions
A -1
6. Jowe
6 Wr
(A- 10s > Po 2t
- Ao > oo mm
" M p kon Kay 1A- go $
-8 a tn ب 20
201
ind
I-n Ap E Aus
E Eaighe los Bitil (A- (ilee))
-Rt -ni I of + An Enle St
0/1
of waw E in E p, Rt
Table The formulation of Lp using differrnt functions to formulate LDA. We present the gradients y 05: and the corresponding update A -: under different situations. Analysis can be found in the Sec. 3.2 and Appendix C.
Uhe
klipliy)
KL(q))p)
]5(p.9) 9)
MSE(p.9)
417
017
027
10
24
(a)
up
A=
"P
p
P
(c) x 2
k0
(d) Gradient updates of Tgn
Figure 4. Contour visualization of the gradient update with binary classes for better understanding. For a clearer view, we red for positive updates and blue for negative updates, the darker indicates larger absolute values and the lighter indicates smaller absolute values.
16

---

77
mIoU
projection
mloU
mloU
0.9
66.19
65.90
0,001
65.25
0.99
66.80
linear
66.55
0.01
66.01
0.999
67.18
2-layer MLPs 67.18
0.1
67.18
0.9999
66.22
3- 3-layer MLPs 66.31
65.95
(a) Momentum update.
(b) Projection network.
(c) Loss weight.
Table Parameter study ERDA. If not specified, the model is RandLA-Ner with ERDA trained with loss weight a 0.1, momentum m 0.999, and 2-layer MLPs as projection networks under 1% setting on S3DIS. Default settings are marked in gray
E Supplementary: Ablations and Parameter Study
We further study the hyper- parametersssssss involved in the implementation of ERDA with the prototypical pseudo-label generation, including loss weight a, momentum coefficient m, and the use projection network. As shown in Tab. 9, the proposed method acquires decent performance (mloUs are all > 65 and mostly > 66) in a wide range of different hyper-paramete settings, compared with its sully-supervised baseline (64.7 mloU) and previous state-of-the-art performance (65.3 mloU by HybridCR [47]).
Additionally, suggest that the projection network could be effective in facilitating the ERDA learning, which can be learned to specialize in the pseudo-label generation task. This could also be related the advances in contrastive learning. Many works [10, 11, 25] suggest that further projection on feature representation can largely boost the performance because such projection decouples the learned features from the pretext task. We share similar motivation in decoupling the features for ERDA learning on the pseudo-label generation task from the features for the segmentation task.
ceiling
Randl N.r
Fully
CloserLook3D
ERDA
73.7
94.1
93.6
63.5
79.2
71.8
79.6
74.8
73.0
72.0
59.5
PT ERDA
76.3
94.9
97.8
96.2
65.4
55.2
64
80.9
84.8
79.3
74.0
74.0
69.3
66.2
X-nol---Ner FXIA
69.4
933
12.3
81.7
601
45.0
6016
7013
65.1
764
71.1
R3.1
63.1
35.0
CloserLookk. ERDA
72.3
94.1
97.5
84.1
62.9
99.2
73.0
71.5
77.0
73.6
71.0
67.7
61.2
PT ERDA
94.9
91.7
85.3
66.7
69.2
73.3
61.7
65.9
62.1
Table 10. The full results of ERDA with different baselines S3DIS 6-fold cross-validation.
Fale
ERDA
70.4
25.9
16.2
310
66.2
S6.3
48.1
613
621
61.
0.3
53.
169
57.1
58.0
75.9
763
6.7
862
84.3
66.5
12.1
722
300
649
461
430
620
12.4
639
563
15.
46.3
56.
832
425
196
ReaEL BROM
650
62.
13.1
655
60.5
9044
400
72.
585
424
9.3
55,0
350
550
57.0
004
75.
61.9
78.
73.3
62.
Table The full results of ERDA with different baselines on ScanNet [16] test sct, obtained from its online
benchmark site by the time of submission.
ScRungs
Retlods
BloL
0%
Grond
Vageatos
Buidiings
Walls
Palking
Roil
Roods
Stneet Puraiize
Cans Footpoth
BAGes 45.1
Water
Fully
96.1
65.
46.5
0.
Table 12. The full results ERDA with different baselines SensatUrban [33] test set, obtained from its online benchmark site by the time of submission.
17


Reviews:
[{"review_id": "b4NvEHeEQE", "reviewer": "Reviewer_aBL8", "summary": "The paper proposes a novel learning strategy to regularize the generated pseudo-labels and effectively narrow the gaps between pseudo-labels and model predictions, which introduces an Entropy Regularization loss and a Distribution Alignment loss for weakly supervised learning in 3D segmentation tasks, resulting in an ERDA learning strategy.", "questions": "See section weakness.", "limitations": "There are some issues not explained clearly,  see Section Weakness.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper solve an interesting problem, and the experimental results support the conclusions. ", "weaknesses": "1.There exists some confusions in Figure 1:\na)For (a), there are generally two ways of generating pseudo-labels based on self-training weakly supervised methods. One way is to input both the sample and its augmented version into two different or shared-weight networks (with gradient updating); the other way is to input them into the student network and the teacher network updated via EMA (with only the student network being updated). I think the author's general description of sparse pseudo-labels in (a) is inadequate, which makes it difficult to establish a connection with (b) and understand the essential differences between them.\nb)I think the author's naming of (b) is inappropriate. It only optimizes the pseudo-labels p and predictions q simultaneously, without reflecting the \"dense\" aspect.\n2.I appreciate the author's theoretical analysis of entropy regularization and distribution alignment, as well as the evaluation of different loss combinations via formula derivation. However, when I saw Line 160 and Table 7, I wondered if when lambda=1, the best result is achieved, and at this point, the ERDA loss simplifies to a single cross-entropy-based loss that optimizes both p and q. My question is, to my knowledge, I don't think only optimizing p for gradient update at the same time can bring such a high performance gain, and I speculate that the diversity of different perturbations as input enables the model to learn the geometric invariance of features, and the ERDA loss is just the icing on the cake.\n3.In Line 207, the author uses a prototypical pseudo-label generation process due to its popularity and simplicity. However, in Table 7(a), the baseline with pseudo-labels already reaches 63.3%. I would like to see the difference between the results of prototypical pseudo-labels and plainest pseudo-labels.\n4.I am a little confused about the author's approach under the setting of “fully”. Generally, the PL strategy is applied to unlabeled data, and then leverage prototypes, perturbation or contrastive learning to improve the model's robustness to the unlabeled data. However, in the fully experiment, the author applies ERDA to labeled data, so how can we talk about pseudo-labeling? There is also no gradient update for p.\n5.Table 1 is difficult to understand, and I think this part is the core of the method. Therefore, the author needs to provide a detailed description of it.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for your time and efforts and we are grateful for your confirmation of the novelty and effectiveness of the proposed method. In the following, we address your concerns carefully.\n\n---\n### Q1. Descriptions in Figure 1:\n\nThanks for your advice.\n\nFor your question **a)**, we would like to mention that the concrete pseudo-label generation pipeline we studied falls in the category of prototypical pseudo-labels, which has been adopted with various modifications in point cloud segmentation [1] and other related fields [2,3]. We will revise our descriptions to make them clearer and more accurate. \n\nRegarding your question **b)**, we agree that the connection between our naming and the 'dense' concept is somewhat implicit.\nIn general, we would like to mention that, with our ERDA learning on pseudo-label generation, the benefits of dense pseudo-labels can be better exploited;\nin contrast, existing methods suffer from noises in dense pseudo-labels and require label selections and thus use only limited sparse pseudo-labels.\nWe thus highlight the ERDA optimization in Fig.1(b), and only hint on the resulting dense pseudo-labels in Fig.1(b) with corresponding performance comparison in Fig.1(c).\n\nWe will revise our figure to improve the clarity of Figure 1 for describing the connection between our method and the 'dense' concept.  \n\n\n---\n### Q2. Reason for high-performance gain.\n\nWe thank you for your acknowledgment of our theoretical analysis.\n\nWe would like to first clarify that, as discussed in our paper, optimizing the pseudo-label generation network in a cross-entropy-like loss has the effects of both reducing the entropy of pseudo-labels and maintaining consistency in label distribution at the same time.\nWe believe the high performance gain is supported by the effective exploitation of ALL unlabeled data due to the more appropriate utilization of pseudo-labels, which has been demonstrated by our experimental results.\nWhile other similar formulations (*eg* when $\\lambda\\neq1$) could also be promising, their improvements are not as significant as our cross-entropy-like ERDA loss, as in Tab.7(b).\n\nSecondly, we would also like to mention that we follow the training of baseline and do not impose any specifically designed perturbation or weak-to-strong consistency.\n\nLastly, we also demonstrated that the performance would be worsened without using our method when generating dense pseudo-labels, as in Fig.1(c) and Tab.7(c).\nThis may also suggest that the diversity of different perturbations would not be helpful and even harmful if not using our ERDA loss. \n\nAs a result, we believe our method is far beyond the icing on the cake. \n\n---\n### Q3. The difference between prototypical pseudo-labels and plainest pseudo-labels.\n\nBy removing the momentum prototype, we further explore the effect of our plainest pseudo-label generation and achieve 62.3 mIoU, which is not significantly different from 63.3 mIoU achieved with momentum prototype in Tab.7(a).\n\nBesides, we may refer to Zhang et al. [1]. Our pseudo-label generation could be closely related to theirs, where they adopt classic prototypical pseudo-labels and can also be viewed as the \"plainest\" prototypical pseudo-labels with no momentum. They achieve 61.8 mIoU. More concretely, in Fig. 1(c), we have included [1] for comparison with our prototypical pseudo-labels (blue), and it does not yield a significant difference.\n\n---\n### Q4. Improvement of ERDA in fully-supervised setting. \n\nWe thank you for pointing this out. We would like to first clarify that, in fully-supervised setting, we generate the pseudo-labels for all points and regard the ERDA loss as an auxiliary loss for fully-supervised learning (so there actually exist updates on p).\n\nWe would also like to mention that the promising improvement brought by ERDA for the fully supervised setting is truly a surprising benefit as we designed the ERDA primarily for improving learning on unlabelled data. Regarding this, we hypothesize that the ERDA for pseudo-labels can help stabilize fully supervised learning. Considering that there could be noises in the ground-truth labels [4,5], pseudo-labels from ERDA learning may provide unexpected benefits.\n\nTo answer this question better, we have also performed more ablations in fully-supervised setting in **Tab.R1** above.\nInterestingly, we find that the distribution alignment (DA) shows more benefits than entropy regularization (ER), but both terms are beneficial.\nWe would like to further explore their relationship under full supervision as a promising future work, but it may be beyond the scope of this paper that focuses on weak supervision.\n\nWe would also include more descriptions and discussion for fully-supervised setting in the paper.\n\n\n---\n### Q5. Better Tab.1 discussion.\n\nWe agree that Tab.1 is at the core of our method and thus dedicate Sec 3.2 for discussion. We also realize that prolonged formulas may hinder readability. In addition to the visualization of Tab.1 we offered in Appendix C, we will also revise the paper to facilitate easier and better understanding. For example, we can include some visualization of Tab.1 in the main paper, as in Appendix C.\n\n---\n### References:\n[1] Zhang et al. Weakly supervised semantic segmentation for large-scale point cloud (AAAI 2021) \\\n[2] Zhang et al. Semisupervised Momentum Prototype Network for Gearbox Fault Diagnosis Under Limited Labeled Samples (TII 2022) \\\n[3] Li et al. MoPro: Webly Supervised Learning with Momentum Prototypes (ICLR 2021) \\\n[4] Ye et al. Learning with Noisy Labels for Robust Point Cloud Segmentation (ICCV 2021) \\\n[5] Song et al. Learning from Noisy Labels with Deep Neural Networks: A Survey (TNNLS 2022)\n"}, {"review_id": "E3Nn18pJ0i", "reviewer": "Reviewer_7SFL", "summary": "This paper considers the task of weakly supervised 3D scene semantic segmentation, where only a limited number of points in each training scene are given labels. Assuming a baseline system that operates within a pseudo-label paradigm, the paper proposes a new set of regularizing loss terms, that aim to (1) reduce pseudo-label entropy and (2) align the distribution of the pseudo-labels and network predictions. Under a default weighting strategy, these terms simplify into cross entropy from the pseudo-labels to the network predictions. Under a wide variety of experimental settings, the paper shows that incorporating this term leads to improved performance for 3d scene semantic segmentation, regardless of the level of supervision. ", "questions": "Perhaps the most surprising result in the paper is that the method improves baselines, even under full label supervision. While I don’t doubt that trend is “real”, as the experimentation seems robust and well-designed, I was not quite satisfied by the explanation given to explain the result on lines 250-254. Is this explanation claiming that the “gt” labels have noise, so using ERD, which is “noise-aware”, can help regulate and remove the noise present in them? If so, this seems like a testable hypothesis (e.g. analyzing differences between pseudo-label predictions and gt label predictions). While I don’t think its required to have a compelte explanation for this phenomenon, the paper should either clarify the explanation here, or simply say that it is unknown why ERDA offers benefits in this paradigm, and to fully understand it would require further study. ", "limitations": "\"Limited\" limitations are given, see weaknesses section as to other potential limitations that should be explored.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "This is a well-written, clear, and compelling paper on an important topic of interest to the community. While the introduced technique is not terribly complex, its benefits are well-justified, and the paper provides substantial analysis to support its inclusion: investigating how gradients from this loss term behave under different prediction settings, and why those gradients align with desirable properties. Further, the paper provides extensive ablation experiments that experimentally support this analysis, and show that all of its components lead to improved performance on the domains under investigation.\n\nThe strongest point for the paper is in its thorough and overwhelmingly positive experimental results. For multiple datasets of 3d scenes (all standard), under multiple levels of supervision, adding this loss to an array of baseline models always improves performance, and outperforms previous state-of-the-art models on competitive benchmarks. Substantial improvements are observed when labels are severely limited, and even under fully supervised settings, including this loss term is helpful. From the presented evidence, it seems likely this term should be widely useful for this task and domain in future work, as it presents robustly strong performance under a myriad of framings and settings. \n", "weaknesses": "My biggest outstanding question is to what extent this technique can offer benefits for other domains? In its formulation, there is nothing specific to 3D scene segmentation, so ostensibly it could be generally useful for other weakly supervised domains that employ pseudo-labels. Some evidence is provided that it can transfer to image segmentation, but it would also be interesting to consider domains like 3D shape segmentation. The initial results (on domains other than 3d scenes) provided by the paper are encouraging, but a more thorough analysis would of course strengthen the paper, and likely dramatically improve the reach/impact of the contribution.\nRelatedly, I would like to see more analysis / discussion about under what situations this term is helpful? Is it always beneficial to include such a term (no matter the domain / task). For instance, I could imagine that when the initial pseudo-labeling mechanism is highly inaccurate, this term might actually be harmful for learning. For 3D scene segmentation, my prior is that pseudo-labeling techniques are largely successful because strong locality cues in this domain can often be used to propagate labels to nearby unlabeled points, with a relative high degree of confidence; so the quality of initial pseudo-labels for 3D scene segmentation might be higher than would be expected for other domains of interest. It would be interesting to consider the effect that the “goodness” of the pseudo-labeling mechanism has on the final model performance, which could potentially be evaluated in a synthetically designed experiment that introduces “corruption” (at varying levels) into the distributions produced by the pseudo-labeling network. A deeper understanding of how the various components of the system interact with the added loss terms would be beneficial, and may give insight into what other domains and systems may benefit from this insights this paper provides. \n\nMinor:\n\nThe formatting of table 2 can be improved. The read highlights are distracting, and largely unneeded as they overstate information. Consider replacing the red text coloring with italics, or better yet, marking only the columns that the baseline does not get improved with the added loss term.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for your acknowledgment of both our theoretical and empirical analysis, as well as the potential broader impact. In the following, we address your concerns carefully.\n\n---\n### Q1. Application to other tasks. \n\nWe also agree that our method could be extended to other tasks, such as the suggested 3D shape segmentation. As in **Tab.R2** above, we find that ERDA also illustrates improvements over the baseline for weakly-supervised shape segmentation.\n\nAlong with experiments on semi-supervised image segmentation, these results may indicate that our method can be extended to different domains and tasks.\nIn the future, we will investigate a more generic formulation of our method with more complete analysis to benefit other tasks like classification and detection. \n\n---\n### Q2: Corruption on pseudo-labels and its relation to model performance.\n\nThanks for the insightful advice for investigating our methods from the perspective of noise, and we provide related experiments in **Tab.R4** above.\n\nWe find that, though corrupted pseudo-labels would affect the model performance, ERDA appears to be relatively robust. Especially, when adding a relatively large noise $\\mathcal N(0,0.1)$ on our cosine distance, ERDA is still able to improve the baseline from 59.8 to 65.55 mIoU. Moreover, from the perspective of the final training loss on the segmentation task, the model training is almost not influenced across different noise levels, which may indicate the strong ability of ERDA learning.\n\n---\n### Q3: Improvement under full supervision and the potential effect of overcoming potential label noise.\n\nWe thank the reviewer for the suggestion to analyze the label noise for a better investigation into the benefits of our method for fully-supervised learning.\n\nWhile we are also surprised to find our method effective under full supervision,\nwe also hypothesize that the ground-truth labels may have noise that would affect the model performance, which has been recognized in general tasks as well as in point cloud datasets [1,2].\n\nMore concretely, we checked the difference between the generated pseudo-labels and the ground-truth labels by estimating the accuracy of pseudo-labels *w.r.t.* ground-truth labels.\nThe results show that even using the top-5 accuracy of pseudo-labels *w.r.t*. ground-truth labels, the accuracy can only reach 98.6 but not 100. Since the level of uncertainty would be reduced by optimizing with our ERDA loss, this phenomenon partially reveals that the ground-truth labels may have noise that would increase uncertainty and thus affect model training to some extent.\nRegarding this, we acknowledge that such divergence between pseudo-labels and ground-truth data is worth further analysis.\n\nBesides, in **Tab.R1** above, we also perform more ablations under full supervision for a deeper understanding. We find that, with full supervision, while both terms are beneficial, the DA term demonstrates more benefits than ER term, and we observe further improvement on performance.\nWe would like to further explore their relationship and how they are effective under full supervision as a promising future work, but it may be beyond the scope of this paper that focuses on weak supervision.\n\nLastly, we would revise to include more discussion regarding fully-supervised setting in the paper.\n\n---\n### Q4: Tab.2 formatting.\nThanks very much for your advice. We will revise our table accordingly. \n\n---\n### References:\n[1] Ye et al. Learning with Noisy Labels for Robust Point Cloud Segmentation (ICCV 2021)\\\n[2] Song et al. Learning from Noisy Labels with Deep Neural Networks: A Survey (TNNLS 2022)\n"}, {"review_id": "ZV0A7sFk9n", "reviewer": "Reviewer_ZBhv", "summary": "This paper proposes a novel learning strategy to regularize the generated pseudo-labels and narrow the gaps between pseudo-labels and model predictions. It introduces an Entropy Regularization loss and a Distribution Alignment loss for weakly supervised learning in 3D segmentation tasks. The approach can better leverage unlabeled data points and achieves state-of-the-art performance under different settings.", "questions": "See above. ", "limitations": "See above. ", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed losses can work with other frameworks to consistently boost their performances. It can potentially achieve better performance with future stronger approaches. \n2. The approach can improve many existing approaches by a significant margin. ", "weaknesses": "I think the motivation and design of the proposed losses needs further analysis. \n1. Entropy Regularization loss: does it filter out some high-frequency predictions? How to balance the effects of filtering high-frequency components and removing noise? \n2. Distribution Alignment loss: \n(a) Does it always improve the performance to encourage the consistency of prediction and pseudo labels? \n(b) Are the pseudo labels derived from network predictions? If so, they should be the similar thing. In what cases they tend to be similar and in what cases they are different? This part seems not very clearly presented. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for your acknowledgment of the performance gain of our method and its potential to improve future stronger baselines. In the following, we address your concerns carefully.\n\n---\n### Q1. Entropy Regularization (ER) on high-frequency predictions.\n\nWe would like to mention that, since the ER works on the soft pseudo-labels at a per-point basis, it would tend to reduce the noise by reducing the occurrence of confusing pseudo-label predictions, *e.g.* predictions with a confidence score around 0.5 in the case of binary classification, rather than spatially smoothing out high-frequency predictions such as areas around scene boundaries and edges.\n\nAlthough the high-frequency predictions might still be influenced by reducing the level of confusion, we believe that these influences are generally positive. This is because we observed that the edge areas predicted by the model trained with ERDA loss become cleaner and more accurate, as in original Fig.2. We will discuss this more in our paper.\n\n---\n### Q2. Questions on Distribution Alignment (DA) term.\n\nRegarding your sub-question **(a)** about whether DA consistently improves, we have performed experiments under different settings to support this, as in Tab.7(a) and Tab.7(b).\n\nIn addition, we have also evaluated the effectiveness of DA under fully-supervised setting.\nAs shown in **Tab.R1** above, we find DA to be beneficial as well. More interestingly, we notice that the DA term shows even more benefits than the ER term, while both terms are both beneficial.\nWe would like to further explore their relationship under full supervision as a promising future work, but it may be beyond the scope of this paper that focuses on weak supervision.\n\nRegarding your sub-question **(b)**, we would like to clarify that the pseudo-labels are generated based on the features and prototypes that are projected by a projection network from the backbone features, as shown in Fig.3 in Appendix B.\nThe pseudo-labels and model predictions are thus not necessarily the same or similar.\nFor example, we offer the visualization of pseudo-labels, model prediction, and ground-truth labels in Fig.7 of Appendix F. We find that pseudo-labels can provide different estimations and diverge from the model predictions on complex and cluttered areas, such as boards on walls.\n\nAdditionally, we also provide a visualization of how pseudo-labels evolve as the training proceeds, as in **Fig.R4** above. Since the KL-Distance (DA term) indicates the similarity between pseudo-labels and model predictions, we find that pseudo-labels gradually learn to be different from the model predictions on cluttered areas, which could regularize the model learning, such as preventing the model from overfitting.\n\nWe hope this could shed some light and we will add more discussion and details regarding the pseudo-label generation as well as its comparison with model prediction in our paper."}, {"review_id": "11maS2RwBJ", "reviewer": "Reviewer_iXj3", "summary": "This paper proposes two losses for point cloud semantic segmentation. The first one is Entropy Regularization (ER) loss, which makes the pseudo-labels have low entropy and thereby be confident (like one-hot vectors). The other one is Distribution Alignment (DA) loss, which is a KL divergence between the pseudo-labels and the predictions. Extensive experiments show that the combination of these two losses is not only beneficial in a weakly supervised setting but also in a fully supervised setting.", "questions": "The results of the experiments on 2D semi-supervised semantic segmentation (SSSS) are interesting. I think the less significant improvement is mainly due to the gap between the weakly-supervised 3D semantic segmentation task and the SSSS task. Maybe the proposed ERDA can make benefit from 2D WSSS (using some points or scribbles).", "limitations": "The limitations are properly mentioned in the paper (section 5).", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. The proposed method is extremely simple, but theoretically sounding and effective in practice.\n2. Extensive experiments strongly support the benefit of the proposed method.\n3. Overall presentation is great. Especially, section 3.2 is notably helpful to understand the working logic behind the proposed method.", "weaknesses": "1. Could the authors provide a visualization of the (1) entropy map, (2) pseudo-label, and (3) KD loss map of the point cloud, and how they change as training proceeds? The figure would be extremely helpful to elucidate how the proposed method works.\n\n2. Actually, it was quite surprising for me that the proposed method brings performance gain even in a fully-supervised setting. I think more discussion is required for this. Also, it would be great if the authors could provide an ablation study of Table 7(a) in a fully-supervised setting. I expect that DA would be much more beneficial compared to ER in this case, but not sure of course.\n\n3. I understand why the authors refrained from comparing with some super-voxel-based approaches for a fair comparison. However, using a super-voxel partition is one of the widely used settings in 3D semantic segmentation. Hence, to clearly demonstrate the superiority of the proposed method, it would be much better if the authors could provide the performance of ERDA using a super-voxel partition.\n\n4. Some related works are missing. \n\nJoint Learning of 2D-3D Weakly Supervised Semantic Segmentation (NeurIPS 2022)\n\nBox2Mask: Weakly Supervised 3D Semantic Instance Segmentation Using Bounding Boxes (ECCV 2022)\n\nWeakly Supervised 3D Segmentation via Receptive-Driven Pseudo Label Consistency and Structural Consistency (AAAI 2023)\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "\nWe sincerely thank you for your positive comments and the overall acknowledgment of our deceptively simple yet effective method. In the following, we address your concerns carefully.\n\n---\n### Q1. Visualization of training process.\n\nWe thank you for your advice on inspecting the training process through visualization.\nFor better understanding, we visualize the ER and DA terms, together with the pseudo-labels in **Fig.R4** above, where we find pseudo-labels can capture meaningful estimation for different semantic classes.\n\nSince KL-Distance could indicate the difference between pseudo-labels and model prediction, we observe that, though pseudo-labels are similar to model prediction in an early stage, pseudo-labels appear to be different from model predictions around complex and cluttered areas as the training proceeds. This may show that the pseudo-labels could capture additional information that helps the model learning.\n\nMoreover, we find pseudo-labels can also provide estimations of different entropy, thus different levels of certainty, for different semantic classes and areas. This may indicate that our pseudo-labels could capture some underlying knowledge of the difficulties of various 3D scenes and classes, which then benefits the model learning.\n\n---\n### Q2. Effects of ERDA on fully-supervised learning.\n\nWe thank you for your constructive suggestion.\nAs shown in **Tab.R1** above, we perform more experiments under the fully-supervised setting for a better investigation.\nIndeed, we find the empirical results follow your expectation that distribution alignment (DA) shows more benefits than entropy regularization (ER), and it indeed further boosts our performance.\nBesides, we may also note that both terms are beneficial compared with the baseline.\n\nRegarding this, we hypothesize that the noise-aware learning of ERDA on pseudo-labels could stabilize fully-supervised learning, considering that ground-truth labels may suffer from the problem of label noise [1,2].\n\nWe would also include more discussion regarding full supervision in the paper.\n\n---\n### Q3. Comparing with super-voxel methods.\n\nFor popular super-voxel methods, such as OTOC [3], during the adaptation of our methods to theirs, we find OTOC requires iterative training, which is both time and resource consuming.\nWe could thus not fully reproduce their results.\n\nInstead, since OTOC is based on a popular voxel-based 3D CNN, Minkowvski Unet. We thus adapt our method to its baseline methods (without iterative training). We find that, when using 1% labels, we improve the baseline from 63.4 to 66.8 mIoU by directly adding the proposed ERDA learning without hyper-parameter searching or further adaptation.\n\nAdditionally, we would like to first note that, our performance have already surpassed the results of the popular super-voxel methods in weakly supervised point cloud segmentation, such as OTOC[3], under a fair comparison when there is no other dataset-level meta-knowledge available, such as in S3DIS dataset.\nFor S3DIS, they provide only \"1pt\" performance, which is 50.1, while ours is 52.0, using a similar 3D CNN baseline.\n\n\n---\n### Q4. Missing related work.\n\nWe thank you for your careful read. We also recognize these works are related and would include them in our paper: \\\n\"Joint 2D-3D\" proposes a novel learning setting to leverage paired 2D-3D data to improve weakly-supervised learning in both 2D and 3D domains.\\\n\"Box2Mask\" proposes to use box annotation as weak supervision for 3D instance segmentation, inspired by hough voting and box clustering.\\\n\"Receptive-Driven\" designs three consistency constraints to enhance their pseudo-labels, by utilizing multi-scale consistency, spatial consistency, and semantic consistency.\n\n---\n### Q5. Extension to 2D weak-supervised semantic segmentation (WSSS).\n\nWe also agree that there could be gaps between the 3D weakly-supervised setting and 2D semi-supervised semantic segmentation (SSSS), especially regarding the form of available labels.\n\nWe thank you for your insights and for pointing out a promising future direction about using point-based or scribble-based supervision to make benefits from 2D WSSS.\nConsidering the similarity in type of supervision, *e.g.* point-based supervision, we would expect our method to be effective as well. We will explore this direction in the future. \n\n---\n### References:\n[1] Ye et al. Learning with Noisy Labels for Robust Point Cloud Segmentation (ICCV 2021)\\\n[2] Song et al. Learning from Noisy Labels with Deep Neural Networks: A Survey (TNNLS 2022)\\\n[3] Liu et al. One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation (CVPR 2021)\n"}, {"review_id": "okg8LiCeQb", "reviewer": "Reviewer_qxVg", "summary": "This submission proposes a semi-supervized pseudo-label-based method for 3D point cloud segmentation method with sparse label annotations. Instead of thresholding on the confidence of pseudo labels, the authors propose to use all unlabelled points and encourage high-confidence pseudo labels by regularizing with their entropy. The authors propose a theoretical analysis of their approach and experiments where they show that their method improves the low-label regime for several backbone and several datasets.", "questions": "Overall, the reviewer believes a very good idea is hidden in the paper; but, it takes a lot of effort to see through the subpar writing, imprecise statements, core ideas hidden in the appendix, and confusing notations. \nThe reviewer also believes the authors focused on the wrong part of their contribution.\nAs of now, the paper is not publishable. But if the authors put the work into improving their clarity and rigour, and focused on the important part of their contribution, this could become an impactful paper. The amount of effort might be too much for a rebuttal. \n\nQ1) Equation (6) reduces to classic cross entropy when all points are labelled, and yet the proposed methods improve the backbones in the fully supervised setting?\n\nS1) Swap the derivation of (6) and the pseudo-label generation between the appendix and the main paper.\n\nS2) Add detailed proofs of Table 1 in the appendix\n\nS3) Remove the notion that lower entropy = less noise", "limitations": "ok", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- the authors propose a theoretical analysis with some insight\n\n- The experimental results are convincing\n\n- The experiments are extensive: 4 datasets and 4 baselines", "weaknesses": "\n- The entire ER+DA analysis leads to using a loss which is the cross-entropy between the pseudo labels and the prediction, which is already sensible and does not need to be seen as a special case of a more general setting that is never explored anyway. The fact that KL(p,q)+H(q)=CE(p,q) was known and did not need two full pages of motivation and equations (some parts are interesting such as the gradient limits, but would be better suited in the appendix). Equation (6) is completely logical and can be used directly.\n\n- On the other hand, the most interesting part of the paper is the pseudo-label generation hidden in Appendix B, which, as far as the reviewer knows, is novel. The reviewer thinks that the non-backpropagable momentum update of the class prototypes used for pseudo labels makes this method work. Without this, if p could be learned along q, then equation (6) would be meaningless as p=q is a trivial solution, and the pseudo labels would not help at all. The paper would need to be rewritten to highlight this hidden mechanism, brushed in the main paper in a single sentence, and at the method's core. \n\n- Equation (6) reduces to classic cross entropy when all points are labelled, and yet the proposed methods improve the backbones in the fully supervised setting?\n\n- The writing is subpar and lacks rigour. This leads to imprecise or even outright false statements at the core of the motivation. For example, the authors state that regularizing by the entropy decreases the \"noise\" in pseudo-labels when it actually encourages confident distributions and does not affect noise. There are many vague and uninformative sentences, some dealing with critical aspects of the paper. Some variables are also not introduced. The article remains overall understandable with the suppmat open in another tab.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for your time and your acknowledgment of our extensive experiments and the provided theoretical analysis. In the following, we address your concerns carefully.\n\n---\n### S1. Motivation and swapping contents.\n\nWe very much appreciate your confirmation of our idea.\nBesides the formulation of ERDA loss, we also agree that the pseudo-label generation scheme is indeed another important component of our method. However, we believe that the motivation for ERDA loss could be more beneficial for the weakly supervised 3D segmentation task and can provide better insights to the community.\n\nRegarding the motivation, we would like to mention that, some existing papers [3,4,5] also adopt prototypes for pseudo-label generation in this 3D segmentation task.\nMoreover, while methods such as [4,6] also utilize momentum-update for pseudo-label generation, they generally fail to utilize dense pseudo-labels due to the low-quality pseudo-labels.\nThis drives us to devise an effective learning scheme of ERDA to tackle the issues of using dense pseudo-labels.\nTherefore, we highlight the design of ERDA in our paper.\n\nRegarding the discussion under general setting, we would also like to mention that, the discussion is related to the mutual learning on pseudo-labeling [1,2], as also discussed in Sec.2 \"pseudo-label refinement\". While different types of loss are proposed in both 2D and 3D domains [1,2,3,5] during the use of pseudo-labels, there is generally a lack of comparison among these losses.\nWe thus motivate the ERDA from a more general setting for better comparison.\n\nLastly, we will revise our paper and will certainly add more discussions about our pseudo-label generation process, such as by including Fig.3, and will simplify our existing descriptions about motivation to avoid redundancy. \n\n---\n### Q1. Improvement for fully supervised learning.\n\nWe would like to first clarify that, in fully-supervised setting, we generate the pseudo-labels for all points and regard the ERDA loss as an auxiliary loss for fully-supervised learning.\n\nWe would also like to mention that the promising improvement brought by ERDA for the fully supervised setting is truly a surprising benefit as we designed the ERDA primarily for improving learning on unlabelled data.\nRegarding this, we hypothesize that the ERDA for pseudo-labels can help stabilize fully supervised learning.\nConsidering that there could be noises in the ground-truth labels [7,8], pseudo-labels from ERDA learning may provide unexpected benefits.\n\nAdditionally, we also provided more ablation studies in fully-supervised setting in **Tab.R1** above.\nInterestingly, we find that the distribution alignment (DA) shows more benefits than entropy regularization (ER), but both terms are beneficial.\nWe would like to further explore their relationship under full supervision as a promising future work, but it may be beyond the scope of this paper that focuses on weak supervision.\n\nLastly, we would revise to include more discussion regarding fully-supervised setting in the paper.\n\n---\n### S2. Detailed proofs of Table 1 \n\nThanks for your advice and we will revise our paper accordingly.\n\n---\n### S3. Notions about entropy and noise.\n\nRegarding the confusion about the connection between \"entropy\" and \"noise\" in our paper, we would like to first clarify that we actually consider 'noise' as uncertain predictions in our paper, which we believe aligns with the definition of entropy [9], while this formulation of \"noise\" does not strictly refer to the general concepts like incorrect predictions. As a result, there may be confusion about connecting \"entropy\" with \"noise\" in our paper. We will revise our paper to better clarify these concepts and refine our original statements.\n\nRegarding symbols, some symbols are not defined when introducing the implementation of pseudo-label generation, as they are moved into the appendix due to the limited space. We will revise and include them together with Fig.3 in the main paper for better clarity.\n\n\n---\n### References:\n[1] Feng et al. DMT: Dynamic mutual training for semi-supervised learning (Pattern Recognition) \\\n[2] Wang et al. Repetitive Reprediction Deep Decipher for Semi-Supervised Learning (AAAI 2020) \\\n[3] Zhang et al. Perturbed Self-Distillation: Weakly Supervised Large-Scale Point Cloud Semantic Segmentation (ICCV 2021) \\\n[4] Xu et al. Weakly Supervised Semantic Point Cloud Segmentation: Towards 10x Fewer Labels (CVPR 2020) \\\n[5] Zhang et al. Weakly supervised semantic segmentation for large-scale point cloud (AAAI 2021) \\\n[6] Zhang et al. Semisupervised Momentum Prototype Network for Gearbox Fault Diagnosis Under Limited Labeled Samples (TII 2022) \\\n[7] Ye et al. Learning with Noisy Labels for Robust Point Cloud Segmentation (ICCV 2021) \\\n[8] Song et al. Learning from Noisy Labels with Deep Neural Networks: A Survey (TNNLS 2022) \\\n[9] Shannon. A Mathematical Theory of Communication (1948)\n"}]

Author rebuttal:
We sincerely thank all reviewers time and effort in providing feedback.

Here, we provide more experiments and visualization for better analysis and understanding of our paper, including the table **R1**, **R2**, **R3**, and Figure **R4** mentioned below.


Meta-review: All reviews are unanimous is recommending acceptance for the paper. aBL8 requests additional clarifications and the ones provided in the author rebuttal for improved descriptions and fully-supervised setting, also as addressed to 7SFL, iXj3 and qxVg, should be included in a revised version. Clarifications on the pseudo-label generation made in response to ZBhv and qxVg should also be included in a revision. Overall, the meta-reviewer agrees with the reviewer consensus that the paper makes interesting contributions to weakly supervised 3D segmentation and may be accepted.
Decision: Accept (poster)