CoSW: Conditional Sample Weighting for Smoke Segmentation with Label Noise
Authors: Lujian Yao
Haitao Zhao
Zhongze Wang
Kaijie Zhao
Jingchao Peng
Venue: NeurIPS 2024 poster (2024)
Keywords: Smoke Recognition; Smoke Segmentation; Industrial Applications
Primary area: machine_vision

Abstract: Smoke segmentation is of great importance in precisely identifying the smoke location, enabling timely fire rescue and gas leak detection. However, due to the visual diversity and blurry edges of the non-grid smoke, noisy labels are almost inevitable in large-scale pixel-level smoke datasets. Noisy labels significantly impact the robustness of the model and may lead to serious accidents. Nevertheless, currently, there are no specific methods for addressing noisy labels in smoke segmentation. Smoke differs from regular objects as its transparency varies, causing inconsistent features in the noisy labels. In this paper, we propose a conditional sample weighting (CoSW). CoSW utilizes a multi-prototype framework, where prototypes serve as prior information to apply different weighting criteria to the different feature clusters. A novel regularized within-prototype entropy (RWE) is introduced to achieve CoSW and stable prototype update. The experiments show that our approach achieves SOTA performance on both real-world and synthetic noisy smoke segmentation datasets.

CoSW: Conditional Sample Weighting for Smoke Segmentation with Label Noise
Lujian Yao
Haitao Zhao" Zhongze Wang Kaijie Zhao Jingchao Peng
School of Information Science and Engineering
East China University of Science and Technology
Shanghai, China
[lujianyao, kawangg,jzhhre pjc) @mail www.steeccct.....oo.o edu.teddu.ccmm cn, haitaozhao@ecust. edu. en
Abstract
Smoke segmentation is of great importance in precisely identifying the smoke location, enabling timely fire rescue and gas leak detection. However, due to the visual diversity and blurry edges of the non-grid smoke, noisy labels are almost inevitable in large-scale pixel-level smoke datasets. Noisy labels significantly impact the robustness of the model and may lead to serious accidents. Nevertheless, currently, there are no specific methods for addressing noisy labels in smoke seg- mentation. Smoke differs from regular objects its transparency varies, causing inconsistent features in the noisy labels. In this paper, we propose a conditional sample weighting (CoSW). CoSW utilizes multi-prone prott-protottpe framework, where prototypes serve as prior information to apply different weighting criteria to the different feature clusters. novel regularized within- within-prototyy prooottpoooee entropy (RWE) is introduced to achieve CoSW and stable prototype update. The experiments show that our approach achieves SOTA performance both callworldd and synthetic noisy smoke segmentation datasets.
Introduction
Smoke segmentation holds significant research value it enables precise localization of smoke. In wildlife, smoke serves as vital indicator of fire, and smoke segmentation allows for rapid identification of the source of fire, facilitating prompt rescue [22] 56]. In industrial production, smoke segmentation can identify gas leakage thereby preventing further spread [16]. There have been numerous methods developed for smoke segmentation, ranging from traditional approaches based on color 10 48 and smoke morphology [
to deep learning techniques that involve expanding the
receptive field 22
54 55).
However, to the best of our knowledge, there is no specific work for smoke segmentation with label noise. Noisy labels are almost inevitable in smoke segmentation. Unlike regular objects with clear and concise edges, which are easy to annotate, smoke annotation poses two main challenges: 1) Smoke edges complex and blurry 53 55), making it hard distinguish smoke and background. 2) Smoke is non-rigid (33, [1] and lacks a fixed shape, making difficult for annotators to become proficient through practice with the same shape.
The noisy labels can have a large impact on the robustness of the model due to their strong memoriza- tion power [I [58). Given that smoke segmentation is related to safety problems, errors stemming from the instability can lead to significant disasters, resulting in extensive casualties and property losses. Therefore, it is crucial to develop robust training mitigate the noisy labels in smoke segmentation.
"Correspondigg author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

P(X)
p(xpp)
Prototype
Noisy
High Weight
Weight
Sample Weighting
Conditional Sample weighting
Cluster
(a] Smake Image
(b) Embeddings
[c) CoSW
(d) Intuition CoSW
Figure 1: (a) presents the noisy labels in smoke segmentation (blue for noisy labels, and red for clean). (b) shows the pixel features extracted by the encoder of SAM [25 Noisy labels are variably distributed, with some located along class boundaries and others spread across different regions within the smoke. Additionally, the features demonstrate a polycentric distribution, making them well-suited for prototype-based modeling. In (c), CoSW assigns sample weights that effectively identify regions with noisy labels and reduce their weights. (d) shows the intuition of CoSW.
However, most existing methods primarily focus noisy labels in classification tasks. Even if few for segmentation, they directly apply classification methods and assume that the noise at the pixel level is also i.i.d. (independent and identically distributed).
methods 60 [
This assumption not realistic in smoke segmentation, primarily due to the issue of variable transparency in smoke. As depicted in Fig. [ this problem is ubiquitous in smoke images, mainly caused by the density, size of smoke particles, and lighting conditions, resulting inconsistent features of noisy labels. Noisy labels in the high transparency regions exhibit similarities to the surrounding clean labels, while in the opaque regions, they differ significantly. Treating both types of noisy labels using the same criteria would reduce the accuracy of identifying noisy labels. Moreover, areas with high transparency are more prone to noisy labels and require refined criteria.
In this paper, we propose a conditional sample weighting (CoSW), which employs different weighting criteria different feature clusters to address the problem of feature inconsistency. As illustrated in Fig. D1. CoSW is built upon multi-prototype framework and regards prototype as prior information for determining the weighting criteria to obtain finer distinctions between samples. Pixel features matching to the same prototype are feature cluster. Prototype learning is intuitive and concise, with its roots tracing back the nearest neighbor algorithm [6]. Importantly, by utilizing multiple prototypes, we can establish polycentric pattern that structures and covers embedding space [4, 62]. including both highly transparent and low transparent features.
Under the framework of multi multi-protottyee CoSW needs to tackle two problems: @ How to determine the sample weight through prototypes. @ How to update prototypes under noisy labels. The most related method is CleanNet (27]. which identifies noisy labels based on the similarity between prototypes and samples. However, it only utilizes single prototype for each class and solely considers individual samples, neglecting the holistic information. As a consequence, it can not dynamically adjust the weighting.
In order to obtain comprehensive information of the samples, a regularized within-prototype entropy (RWE) is proposed to address these problems in unified manner. Entropy can be used for uncertainty measurement and the maximum entropy principle (MaxEnt) allows for the consideration of the entire probability distribution function (PDF) with minimal empirical risk, rather than focusing solely on individual samples. RWE uses prototypes as anchors to build a separate noisy-level evaluation system for each feature cluster. By maximizing RWE, we can consider the information between all pixels that matched the given prototype and obtain adaptive sample weights (for @). Furthermore, by calculating the expectation of weighting samples, stable prototypes can be obtained (for ()).
To demonstrate the robustness of our approach, conduct experiments on both real-woria (Smoke- Seg [53] and SMOKE5K (5I]) and a synthetic (NS-IK) noisy smoke segmentation dataset. Methods from different fields (semantic segmentation, smoke segmentation, segmentation with label noise, and sample weighting) are compared to demonstrate the superiority of our approach.
Our main contributions be summarized as follows:

---

To the best of our knowledge, we are the first to investigate noisy labels in smoke segmenta- tion.
We propose CoSW that serves prototypes prior information apply different weighting criteria to the different feature clusters.
A novel RWE is introduced to implement CoSW, which achieves adaptive sample weighting and stable prototype update in concise and unified way.
Our approach achieves SOTA results on real-world and synthetic noisy smoke datasets.
2 Related Work
2.1 Smoke Segmentation
Traditional smoke segmentation approaches mainly concentrate on extracting color and texture features, including color channel analysis [8 and color enhancement [19]. Additionally, vision-based techniques like morphological operations [1 transmission estimation 30) and region growing [44
smoke segmentation methods tackle various challenges related to smoke diversity and ambiguous edges. These approaches encompass multiple aspects, such as 1) fusion of high and low-level features [49] 54 55). 2) expanding the receptive
have been also widely utilized. Deep-learning-aase
field [22 28), and 3) employing coarse-to-fine strategies
[56] 59]. 4) uncertainty estimation [51].
However, the best of our knowledge, there is work that addresses noisy labels in smoke segmentation.
2.2 Sample Importance Weighting
The sample importance weighting (SIW) assigns low weights to potentially mislabeled samples and high weights to potentially confident samples (20 33 38]. ClcanNet [27] introduces prototype learning for SIW, but it solely relies the similarity between individual samples and prototypes, neglecting the information from other samples. And it employs only one prototype for each class.
There other techniques like Meta-learning 36 37 46), teacher-student architecture (21), itera- tively training [50], and transfer learning (29] for SIW. Nevertheless, these methods are all targeted at image-level classification, failing address the issue of inconsistent features in smoke segmentation with noisy labels.
2.3 Prototype Learning
The pioneer of prototype learning the nearest neighbor algorithm (6]. Building upon this, many nonparametric classification methods have been proposed, including learning vector quantization (LVQ) 26),, and neighborhood component analysis (NCA) [12]. In recent years, the concept of prototype learning has also been incorporated into deep learning as it effectively structures and covers the embedding space using a polycentric pattern. Research fields include supervised [3I], unsupervised [45), and self-supervised learning [4]. In image segmentation, prototype learning also gains significant attention g 43 62 However, prototypes are less investigated in the field of noisy labels, and how to update prototypes in noisy dataset remains an unresolved issue.
2.4 Metric Learning
Metric learning maps raw data to an embedding space where similar features are pulled close while dissimilar ones are pushed away. Metric learning and prototype learning can be naturally linked.
Some cluster-based methods are using one [24] 32 or multiple R4 64
learnable prototypes to
represent the entire class information. They achieve such mapping through specific loss function
such as contrastive loss 14 and triplet loss 35| 42 However, employing metric learning without
regularization can be detrimental under noisy labels 2 58)

---

3 Conditional Sample Weighting (CoSW)
3.1 Intuition and Overview
Previous research on learning with noisy labels primarily targets classification, with scant attention to segmentation, often presuming noise to i..d. However, the noisy labels in smoke segmentation are different due to the variable transparency, resulting in inconsistent features within the noisy labels. The objective of CoSW to apply different weighting criteria to different feature clusters. We achieve CoSW by introducing a regularized within-proot protottpe entropy (RWE). In this section, first review the concepts of entropy and then introduce a within- protiitpp entropy (WE) and its regularized form, RWE. Finally, the specific formulation of CoSW is presented and illustrated by toy demo.
3.2 Preliminary: Entropy and MaxEnt
The concept of entropy originates in the realm of thermodynamics, but Shannon has a broader its meaning to the information theory. Entropy can be utilized for measuring uncertainty. For probability distribution II (T,, ....... TN) of N random variables {z1, 22, ÆN} }, Shannon measures the uncertainty for this distribution by T(I)= E In nT, with the constrain Σ "T 1.
There is an infinity probability distributions satisfying the constraint. While maximum entropy theory (MaxEnt) [18] states that, under given set of constraints, the probability distribution with maximum entropy is the most representative of current knowledge system. Specifically, MaxEnt allows for the consideration of the entire probability distribution with minimal empirical risk, rather than focusing solely on individual samples.
3.3 Within-proto & Between-proto Entropy
Intuition. Only use probability do not take prototype information into account. Hence, derive generalized entropy for prototype information.
Detail. Each class E (W1,42;"" , is represented by K prototypes [p*] k== and thus have QK prototypes in total. We give each pixel feature æ RD likelihood value of and let N uh N and EAK D e EBK N = N. Since Σ o//N* and DAK E w we can define the following entropy based on the Shannon entropy:
Total Entropy:
ΩK N T=
In
(1)
N
N
n=
Within-prototype Entropy (WE):
N
N
T=
In
U NE
(2)
N
Between-prototype Entropy:
ΩK
N
N
TB
In
can be proven that T T., + Tr. Additionally, we also provide the WE derived from different entropies (Burg's 's entropy and Kapur's entropy) the basis.
3.4 Regularized Within-prototype Entropy (RWE)
Intuition. Without adding additional constraints, the within- within-prototype entropy is maximized when all pixel features have the same likelihood value (i.e., o = 1). To tackle the issue of noisy labels, we integrate M-estimation [17 into within-prototype entropy, assigning low weights to noisy labels. The key of original M-estimation is to estimate the mean vector under the noise data. However, in deep learning, obtaining the overall mean of samples becomes challenging due to mini-batch training.

---

Prototype Update with Sample weighting
Regularized
Pixel Features x
Seten #i map -
Within-protolype Entropy
Eruia sxpi-
Feature Extractor
Conditional
Sample Weighting
Pixel-Proto Matching
12
N*
Soot reppl ( plat -x
expl -ille
Embedding Space
Seg Map
Pixol Features
Prototype
X Noley
Calculate Weighted Less
Data Flow
High Weight
Low Weight
Conditional Sample Weighting
Figure 2: Architecture illustration of CoSW for smoke segmentation during the training process.
Nevertheless, we observe resemblance between this idea and the concept prototype learning. where prototypes act as feature representations.
Detail. We substituted the vector with prototypes form the constraints:
( arg min
Σ q( p" N
(4)
where p" represents the corresponding prototype of a and q(-) denotes the penalty function that measures the influence of the residual error maximize the following objective function in conjunction with the within-prototyp entropy:
a p*. By incorporating this constraint, we can
max J(P.V) = RK P.V
N
N
RK
N
N
p 112.
(5)
N
N
N
with the constraint n= N where N denotes the number of pixel features belonging to
N²
w. The penalty function g()) employs the L2 norm. The y E [0, 1] is a regularization parameter which controls the degree of punishment. The function ((P, V) in Eq. 5 is called regularized within-prototype entropy (RWE).
Convergence of the RWE. The Eq. 5 can be enlarged by:
N
J(P.V)
No N
(6)
This proves RWE has an upper bound. convergent.
According to Cauchy's convergence rule, the RWE is
3.5 Conditional Sample Weighting (CoSW)
The CoSW can be obtained by solving Eq. and the likelihood vf is regarded as the weighting of the sample. We can transform the Eq. toto incorporate constraints by Lagrange multipliers and obtain the CoSW ef of each sample and the objective values n² for prototype update:
exp( > p²112)
④ N
(7)
A/k exp(~>/|ze p²1la) )|2)
N Φ exp(->/læt (1)2)
D
(8)
No exp(->/lat [1]
The derivation can be found in the Appendix A
Toy Demo. We provide a toy demo to demonstrate CoSW clearly. Let X1 = (1,1,1,1,5) and
-1, -1, -1, 0} be two feature clusters that match to the prototype pl (1} and P2 = [-1]. respectively. Assume X, contains an obvious noisy label (5) and X₂ contains an
X2
covert noisy label {0). For normal sample weighting approaches (such CleanNet), the weights
5

---

calculated by comparing each sample with the center (f0.5)) of two clusters. The weight of the two noisy labels can be calculated as 0(5) = 0.0282 and P(() 1.5409. respectively. This method assigns an excessively high weight to the noisy label {0]. When we use CoSW, which involves using two prototypes to weigh the importance of each feature separately, we can calculate the (5) = 0.0228
and e 0.4211 through Eq. 7 The noticeable decrease in the weight of the noisy label {0). For prototype updating, in the case X,, if CoSW is not applied and only the mean is used for updating, the new prototype = {1.8}. which differs from the original value of {1}. When CoSW is applicd and the noisy label {5} is given a weight, the new prototype is pi = (1.0067) (ef. Eq. 8 indicating minimal impact on its origin value.
(0)
4 CoSW for Smoke Segmentation
In this section, we introduce how to apply CoSW to smoke segmentation, including pixel-pree -roto matching, loss design, prototype updating, and regularized scatter metric learning. The entire process illustrated in Fig. 2
4.1 Pixel-Prototype Matching
Intuition. For multi-prototype methods, crucial task is to match pixels to prototypes. The simplest approach is adopting the nearest neighbors principle. However, this can lead to a large number of pixels being assigned to the same prototype, while the remaining prototypes are left without any matching pixels. Therefore, we apply additional constraints the original optimization problem to prevent the occurrence of trivial solutions. This part refers to the matching process in ProtoSeg [62 and SwAV [4].
Detail. The goal is to match the pixel features X to one of the prototypes in class w. The matching strategy is denoted as M E KK*,", The M is optimized by minimizing the overall distance between each pixel feature (i.e., X E RRDxN N [ and its matched prototype (i.e. pw E DxK pp ))
min TM(T(M
N s.t. (" M 1 N" 1
(9)
1
K
where 1 K and N denote all-one vectors. M N" is a unique matching constrain which guarantees each pixel feature assigned to only one prototype. M = 1 K is an equipartition constraint [4] which avoids the trivial solution in which all pixels are assigned to single prototype. And C XxN" represents the cost matrix that measures the distance between pixel features and prototypes.
N K
Eq. is is typical transportation problem 39
which can be easily calculated by the iterative Sinkhorn- Knopp algorithm (7. Specific derivation and implementation can be found in the Appendix C
4.2 Sample Weighting and Prototype Update
In the pipeline smoke segmentation, CoSW is employed the loss func- tion. We incorporate CoSW into the basic cross-entropy loss:
(a) w/o CosW
Prototype (Current)
Prototype (Previous)
LOCE
en CE(Si,
(10)
Noisy Label
(b) With
Moving Path
where Un refers to Eq. 7 V. and Vn. respectively represent the predicted value and ground truth for each pixel in a mini-batch. N represents the number of pixels in a mini-batch.
Sample Weight
Effective Scope
Figure 3: Regularized prototype update.
As for prototype update, it is challenging to ensure stability under noisy labels, as shown in Fig. 3 Our objective for prototype updating ² (cf. Eq. g with CoSW. To ensure stable training, also

---

Clean
Dilated
Clean
Dilated
Smoke Image
Eroded Edge-distorted
Smoke Image
Eroded Edge-distorted
Figure 4: Examples of the clean and three corrupted masks in NS-1K.
incorporate the update in a momentum way 15
up + (1- u)p k= 1,2, ,,2,
(II)
where μ E [0, 1] is momentum hyper- ppram-.rrrmeter.
4.3 Regularized Scatter Metric Learning (RSML)
Intuition. In prototype framework, metric learning is widely used and powerful tool 662 that brings similar features closer and pushes dissimilar ones apart, enabling the acquisition of discriminative embedding space. However, due to the presence of noisy labels, metric learning may lead to overfitting. Therefore, we further incorporate the weighting into the scatter matrix to ensure the effectiveness of metric learning. The scatter matrix capture the dispersion information of the samples [61].
Detail. The way implemented it is by incorporating CoSW into the Within-prototyy Scatter Matrix (WSM). The WSM serves representation of the dispersion of samples in relation their corresponding prototypes, allowing for the assessment of the compactness within each prototype. integrate CoSW of (cf. Eq. 7) into WSM for regularization:
AK x
(a
10--p""(ee p²)(et -p"
(12)
For the Between- Between-prototype Scatter Matrix (BSM), we do not apply weighting. The BSM can be employed to quantify the separability between prototypes and features. We employ the conparraettric strategy which the scatter matrix calculated between each pixel feature and its nearest neighbor prototype. The BSM is defined as follows:
N
Σ
(a PNN (Z. (x (NN()"))²
(13)
N
where pN is the nearest neighbor prototype of æn..
Finally, we employ the ratio-trace form 41 to transform the scatter matrices into a numeric value: Cs Tr(S s.) where S Sw. sl and the aI is added to guarantee the reversibility of S.. and we E 10-5 DeepLDA [10 has demonstrated that this trace form can be optimized using gradient descent. Our final objective function is combination of weighted cross-entrone entropy loss LW-CE (ef. Eq. 10 and scatter loss Ls: L = L..CF + aLs, where a is weight hyperparameter.
Experiments
5.1 Experimental Setup
Real-world Noise Setting, Due to the visual diversity and blurry edges of smoke, label noise in the large-scale real-world smoke datasets (SmokeSeg (53] and SMOKESK (51) is ubiquitous. Hence, we conduct experiments on both datasets real-world noise evaluation. To accurately measure the robustness of the model, clean validation set is essential. Therefore, we carefully re-annotate the validation set of both datasets. The distinction between SmokeSeg and Smoke5K resides in the

---

Table I: Quantitive comparison the real-world noisy dataset SmokeSeg. We compare methods from various domains, including semantic segmentation (o), smoke segmentation (f), segmentation with label noise (g), and sample weighting (+). The best: bold, the second: underline.
Total
Small
Medium
Large
Methods
Backbone
Fi
mioU'
Fi
mIoU
Fi
mloU
Fi
mJot'
AFFormer"
AFFormer-B
58.41
47.89
45.98
34.10
64.52
53.20
66.96
$4.15
SeaFormer'
40
SeaFormer-B B
57.58
44.70
41.33
30.35
66.02
53.10
67.16
53.21
Real-time
SogFormer
MiT-BO
60.90
48.53
48.31
35.69
68.86
53.34
68.53
55.80
SC
522
MiT-BO
59.78
47.70
45.76
33.15
68.37
55.68
66.13
53.96
CleanNet
MiT-BO
61.63
47.00
47.16
33.57
66.23
50.91
70.96
56,22
Ours
MiT-BO
62.98
48.62
49.85
36.64
68.81
53.87
20.79
$5.85
D
ResNet-50
65.92
$3.50
54.03
41.07
71.82
58.73
71.87
58.95
OCRNer 57
HRNet-48
64.93
52.45
52.04
39.47
71.04
57.47
70.51
58.19
SegNeXi
[3]
MSCAN-L
66.71
52.37
58.05
44,16
70.41
55.77
72.97
58.42
Tra
51
Re
67.15
53,11
59.02
44.62
71.50
56.99
73.36
58.97
Normal
Ours
ResNet-50
68.49
54.09
61.27
46.28
72.31
58.08
74.78
60.81
SegFormer"
4
MiT-B3
67.70
53.37
57.67
45.21
73.87
60.49
72.06
58.52
Trans- TTans-BMM'
51
MiT-B3
67.68
53,09
60.85
45,87
71.73
57.35
73.51
59.10
SC
52
MiT-B3
69.55
55.04
62.26
48.47
71.41
57.06
72.91
58.23
CleanNet"
27
MIT-B3
70.17
56.94
61.98
48.05
73.06
59.07
74.57
60,93
Ours
MiT-B3
72.32
59.25
64.62
50.86
74.37
61.14
75.52
62.30
Table 2: Comparison on the real-world noisy dataset SMOKESK and synthetic noisy dataset NS-IK. Notation: semantic segmentation (o), smoke segmentation (t), segmentation with label noise (5),
sample weighting (+). The best: bold, the second: underline.
(a) Comparison on SMOKESK.
(b) Comparison on the synthctic noisy dataset NS-
Methods
Fg
mlo
Tran-BVM
31
SC²
S
CleanNet
Ours
OCRNet
72.51
63.00
Noise Ratio
Noise Iniensity
Ai
wnfot/
onfol/
mifot
Fi
mifo
DeeplahV3+
5
73.83
64.08
0%
52.79
39.12
51,94
37.84
52,02
38,24
52,59
38.32
SegNeXI [3
76.44
67.08
Trans BVM
[0
76.23
67.55
High
Low
48.02
34.52
51.81
51.14
37.15
51.21
50.34
36.31
51.77
50.99
36.80
Ours
77.02
67.58
Low
45,40
32.30
49.69
35.89
49.09
35.53
50.18
36,69
SegFormer' 47
78.68
68.29
40%
High
41.09
28.15
45.31
32.16
46.08
33.15
48.34
34,73
Trans BVM
68.97
Lau
42.73
44.12
30.87
46.33
SC 52
79.33
69.40
High
40.57
27.50
41.77
28.44
43.69
29.82
45.86
32.08
CleanNer
80.37
70.23
Ours
81.71
71.24
80%
High
Laws
37.38
39.28
25.58
26.42
39.09
40.52
25.88
27.42
42.96
60.27
20.21
29.37
42.37
44.40
29.89
31.24
unique smoke samples in SmokeSeg, which display high and variable transparency. Consequently, SmokeSeg offers a better assessment of the robustness of the model.
Synthetic Noise Setting (NS-1K). To further investigate the robustness of the model to noise, also create a synthetic noise smoke segmentation dataset called NS-1K. We select 1,000 images from SmokeSeg and carefully re-annotate them to obtain clean labels. Among them, 700 images are used for training, and 300 images are used for validation. Then, we add noise to this dataset in different forms, including eroded, dilated, and digttreedt noise, as shown in Fig. 4 It is noted that edge-distorted is implemented by randomly dilating or eroding the pixels on the boundary with circle. In the experiment, we set two noise parameters: one is the ratio of noise data (20%, 40%, 60%, 80%), and the other is the intensity of the noise data (high and low). The intensity of the noise determined by adjusting the degree of pixel displacement for three noise types.
Implementation Details. We implement our method on MMSegmentation. Standard color jittering, random cropping, and random flipping are adopted for data augmentation during the training stage. All experimental backbones are pre-trained on ImageNet-11 K. We utilize the Adam W optimizer, with learning rate starting at 6e-5 and scheduled according to the polynomial annealing policy. For the SmokeSeg, we crop images to a size of 512 x 512 for training, while for SMOKE5K, we follow the previous methods and resize the images to 480 x 480. For validation, method, and for simplicity, we do not use any data augmentation during the validation.
the whole inference
Evaluation Metric. Following previous literature, employ F1 and mloU for evaluation.
8

---

Table 3: Ablation study of CoSW and metric learning.
(a) Ablation study of CoSW (dataset: SmokeSog).
(b) Ablation on metric learning.
Proto
Sample Weight
Proto Weight
Fi
mIou
Fi
m/ol/
bl
67.70
53.37
Proto
68.17
55.23
68.17 0.47
$5.23
1.86
Triplct Loss
67.80
55.04
70.04 2.34
56.40
3.03
Scatter Loss
69.15
56,30
69.38 168
55.72
2.35
CoSW+Triples Triplet
71.88
58.49
71,39 3,69
57.68
14.31
CoSW+ Scatter
72.32
59.25
GT
Trans-BVM
CleunNet
Ours
Image
1k Iters
5k Iters
20k Iters
Comparison of segmentation results in different scales.
(b) The formation process of CoSW (blue curve for noisy labels, and red for clean the first image column).
Figure 5: The comparison of segmentation results in different scales and the formation of CoSW.
5.2 Comparison on Real-world Label Noise
Tab. and Tab. 2a present the comparative results on the real datasets SmokeSeg and SMOKESK, respectively. We compare our method with methods from different domains, including general semantic segmentation, SOTA method Trans-BVM (51] for smoke segmentation, segmentation with noisy labels method, SC [52], and the sample weighting method, CleanNet [27]. The results demonstrate that our method achieves the best performance in both datasets. Fig. (5 presents the segmentation results of different methods. The Trans-BVM exhibits high miss detection, while CleanNet is prone to false alarms. In contrast, our method demonstrates the best performance.
5.3
Comparison on Synthetie Label Noise
Tab. 2b reports the comparative results of different methods synthetic noisy dataset NS- IS-IK. When the dataset is clean or contains only a small amount of low-level noise, Trans-BVM performs the best, as it is specifically designed for smoke detection. However, as the ratio and intensity of noise increase, the performance
the Trans- BVM rapidly deteriorates. contrast, our method maintains robustness. In the case of maximum noise, our method achieves approximately higher F; compared to Trans-BVM.
5.4 Investigation on CoSW
Quantitive Ablation. To investigate the effect of CoSW, we conduct quantitive ablation, as illustrated in Tab, [3a In the ablation study, the baseline is SegFormer. Tab. Baj 1) replaces the decoder head of SegFormer with prototypes. And there is improvement in model performance, indicating that prototypes inherently provide robustness. In Tab. 22) we incorporate CoSW into samples, resulting in a significant performance improvement. Tab. 3a(3) focus on utilizing CoSW for prototype
updates, as stable updates can also enhance robustness. Finally, in Tab. 3a 4), we combine both approaches, resulting in the best performance achieved.

---

How CoSW Formation. To investigate the reasons behind the effects of CoSW, we visualize the formation process of CoSW, as shown in Fig pp. can be observed that as training progresses. CoSW gradually forms its own confidence area. It reduces the weight of the noisy label parts, and even more so for highly transparent areas, as these regions are more prone to noise. This indicates that with the aid of CoSW, the model has developed its own recognition of smoke, rather than being completely governed by labels. This is likely the reason for its robustness against noisy labels. However, the CoSW requires the assumption that the majority of pixels have clean labels. When the label mask completely noisy, the model is also unable distinguish the noisy la- bels, because it can not learn which features are the common of smoke.
Qualitative Comparation. Fig- illustrates the qualitative comparison among prototype-based sample weighting methods, CleanNet, and our CoSW. Although CleanNet can provide confi- dence area similar to that of CoSW, its area is less precise. Moreover, the weighting area of CleanNet changes more abruptly and lacks speci- ficity. In contrast, CoSW differentiates between transparent and opaque areas: the weighting in transparent areas is gradual, while it is steeper in transparent areas.
Image
CleanNet
CcSW (Ours)
Figure 6: Comparison of sample weighting.
5.5 Further Investigation
Effeet of Regularized Scatter Metric Learning. In Sec. 4.3 we introduce RSML,, designed specifically for metric learning under noisy labels. Tab. 3h demonstrates the effect of RSMI.. It can be observed that using the triplet loss directly in metric learning under noisy labels diminishes the performance of the model. However, incorporating CoSW leads improved results, and the performance is further enhanced when using the scatter matrix as a metric for evaluation.
Different Entropies. Our RWE (cf. Eq. 5 utilizes Shannon entropy as its foundation in this paper. We have also experi- mented with different entropies, such Kapur's entropy and
Table 4: Different entropies.
Fi mIoU
Kapur's Entropy
71.34 58.38
Burg's 's entropy. The results are shown in Tab. The derivation process is provided in the Appendix B
Burg's 's Entropy
69.20 55.72
Shannon's 's Entropy
72.32 59.25
6 Conclusion
Smoke annotation is prone to noisy labels, which can lead to model instability and result in serious accidents. However, existing methods have not addressed the issue of noisy labels in this field. In this paper, introduce conditional sample weighting (CoSW) to address inconsistent noisy labels in smoke segmentation. CoSW utilizes prototypes as prior information and measures each feature cluster with different criteria to re-weight the samples adaptively. Experimental results show that our CoSW achieves the best performance on both real and synthetic noisy smoke segmentation datasets.
The CoSW relies the assumption that the majority of pixels have clean labels. If the degree of noise is very high, the prototype may learn the features of the noise rather than the classes (background and smoke). To determine whether the prototype is clean, is necessary to introduce clean validation, which is not implemented in our work. This is also a direction for further research.
Limitations.
Acknowledgements
This work was supported by the National Natural Science Foundation of China (NSFC) under Grant 62173143.
10

---

References
[1] Görkem Algan and Iikay Ulusoy. Image classification with deep learning in the presence of noisy labels: A survey, Knowledge-Baud Based Systems, 215:106771, 2021.
[2] Devansh Arpit, Stanislaw Jastrzebski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder Kan- wal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, et al. A closer look at memorization in deep networks. In International conference on machine learning, pages 233 242. PMLR, 2017.
[3] Jane Bromley, Isabelle Guyon, Yann LeCun, Eduard Säckinger, and Roopak Shah. Signature verification using a" siamese" time delay neural network. Advances in neural information processing systems, 6, 1993.
[4] Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsu- pervised learning of visual features by contrasting cluster assignments. Advances in neural information processing systems, 33:9913399999 12-9924, 2020.
[5] Liang-Chiie Chich Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European conference compuder vision (ECCV), pages 801-818, 2018.
[6] Thomas Cover and Peter Hart. Nearest neighhor pattern classification. IEEE transactions information theory, 13(1):21-27, 1967.
[7] Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in neural information processing systems, 26, 2013.
[8] Bo Dong, Pichao Wang. and Fan Wang. Head-free free lightweight semantic segmentation with linear trans- former. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 516-524. 2023.
[9] Nanqing Dong and Eric P Xing. Few-shot semantic segmentation with prototype lcarning. In BMVC, volume 3, 2018.
[10] Matthias Dorfer, Rainer Kelz, and Gerhard Widmer. Deep linear discriminant analysis. arXiv preprint 1.Xvv1551..0477. 2015.
[11] Alexander Filonenko, Danilo Cáceres Hernández, and Kang-Hyun Jo. Fast smoke detection for video surveillance using cuda. IEEE Transactions on Industrial Informatics, 14(2):25-733, 2017.
[12] Jacob Goldberger, Geoffrey Hinton, Sam Roweis, and Russ R Salakhutdinov. Neighhourhood components analysis. Advances in neural information processing systems, 17, 2004.
[13] Meng-Hao Guo, Cheng Ze Lu, Qibin Hou, Zhengning Liu, Ming-Ming Cheng, and Shi-Min Hu. Segnext: Rethinking convolutional attention design for semantic segmentation. Advances in Neural Information Processing Systems, 35:1 4051156, 2022.
[14] Raia Hadsell, Sumit Chopra, and Yann LeCun. Dimensionality reduction by learning invariant mapping. In 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR 06). volume 2, pages 1735-1742, IEFF, 2006.
[15] Kaiming Hc, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation lcarning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9729-9738, 2020.
[16] Yen-Chia Hsu, Ting- Hao Kenneth Huang, Ting-Yao Hu, Paul Dille, Sean Prendi, Ryan Hoffman, Anastasia Tsuhlares. Jessica Pachuta, Randy Sargent, and Illah Nourbakhsh. Project rise: Recognizing industrial smoke emissions. Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 14813-14821, 2021.
[17] Peter J Huber. Robust statistics, volume 523. John Wiley & Sons, 2004,
[18] Edwin T Jaynes. Information theory and statistical mechanics. Physical review, 106(4):620, 1957.
[19] Yang Jia, Jie Yuan, Jinjun Wang, Jun Fang, Qixing Zhang, and Yongming Zhang. A saliency-based method for carly smoke detection in vidco sequences. Fire technology, 52:1271 1292, 2016.
[20] Lu Jiang, Deyu Meng, Shoou-1 Yu, Zhenzhong Lan, Shiguang Shan, and Alexander Hauptmann. Self Self-paced learning with diversity, Advances neural information processing systems, 27, 2014.
11

---

[21] Lu Jiang, Zhengyuan Zhou. Thomas Leung, Li-Jia Li, and Li Fei-Fei. Mentomet: Learning data-driven curriculum for very deep neural networks corrupted labels. In International conference on machine learning, pages 2304-2313, PMLR, 2018.
[22] Tao Jing, Qing-Har Hao Meng, and Hui-Rang Hou. Smokeseger: A transformer-enn coupled model for urban scene smoke segmentation. IEEE Transactions Industrial Informatics, 2023.
[23] Tao Jing, Ming Zeng, and Qing-Hao Meng. Smokepose: End-to-end smoke keypoint detection. IEEE Transactions on Circuits and Systems for Video Technology, 2023.
(24) Sungyeon Kim, Dongwon Kim, Minsu Cho, and Suha Kwak. Proxy anchor loss for deep metric learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3238- -3247, 2020.
[25] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan- Lo, et al. Segment anything. arXiv preprint arXiv:2304.02643, 2023.
[26] Teuvo Kohonen. The self-organizing map. Proceedings of the IEEE, 7889))1464-11 1480, 1990.
[27] Kuang-Huei Lee, Xiaodong He, Lei Zhang, and Linjun Yang. Cleannet: Transfer learning for scalable image classifier training with label noise. Proceedings of the IEFE conference on computer vision and pattern recognition, pages 5447-5456, 2018.
[28] Xiuqing Li, Zhenxue Chen, QM Jonathan Wu, and Chengyun Liu. parallel fully convolutional networks for real-time video wildfire smoke detection. IEEE Transactions on Circuits and Systems for Video Technology, 30(1) 10011,9-,, 2018.
[29] Or Litany and Daniel Freedman. Soseleto: unified approach transfer lcarning and training with noisy labels. arXiv preprint arXiv: 08888009922. 2018.
[30] Chengjiang Long, Jianhui Zhao. Shizhong Han. Lu Xiong. Zhiyong Yuan, Jing Huang, and Weiwei Gao. Transmission: a new feature for computer vision based smoke detection. In Artificial Intelligence and Computational Intelligence: International Conference, AICI 2010, Sanya, China, October 23-24, 2010, Proceedings, Part 1 2, pages 389-396. Springer, 2010.
[31] Pascal Mettes. Elise Van der Pol. and Cees Snoek. Hyperspherical prototype networks. Advances in neural information processing systems, 32, 2019.
[32] Yair Movshovitz-Attias, Alexander Toshev, Thomas K. Leung. Sergey loffe, and Saurabh Singh. No fuss distance metric learning using proxies. In Proceedings of the IEEE international conference on computer vision, pages 160-368.22 2017.
[33] Deep Patel and PS Sastry. Adaptive sample selection for robust learning under label noise. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 3932-3942, 2023.
[34] Qi Qian, Lei Shang, Baigui Sun, Juhua Hu, Hao Li, and Rong Jin. Softtriple loss: Deep metric learning without triplct sampling. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6450-6458, 2019.
[35] Florian Schroff. Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 815-823, 2015.
[36] Jun Shu, Qi Xie, Lixuan Yi, Qian Zhao, Sanping Zhou, Zongben Xu, and Deyu Meng. Meta Meta-weight-ne weight- Learning an explicit mapping for sample weighting. Advances neural information processing systems, 32, 2019.
[37] Jun Shu, Xiang Yuan, Deyu Meng, and Zonghen Xu. Cmw-net: Learning a class-aware sample weighting mapping for robust deep learning. IEEE Transactions on Pattern Analysis and Machine Intellience, 2023.
[38] Zeren Sun, Fumin Shen, Dan Huang, Qiong Wang, Xiangbo Shu, Yazhou Yao, and Jinhui Tang. Pnp: Robust learning from noisy labels by probabilistic noise prediction. In Proceedings of the IELE/CVF Conference on Computer Vision and Pattern Recognition, pages 5311-5320, 2022.
[39] Cédric Villani et al. Optimal transport: old and new, volume 338. Springer, 2009.
[40] Qiang Wan, Zilong Huang, Jiachen Lu, YU Gang, and Li Zhang. Seaformer: Squeeze transformer for mobile semantic segmentation. In The Eleventh International Conference on Learning Representations, 2022.
Squeeze-enhanced enhanced axial
12

---

Theories and [41] Huan Wang, Shuicheng Yan, Dong Xu, Xiaoou Tang, and Thomas Huang. Trace ratio vs. ratio trace for dimensionality reduction. In 2007 IEEE Conference Computer Vision and Pattern Recognition, pages 1-8. IEEE, 2007.
[42] Jiang Wang. Yang Song, Thomas Leung, Chuck Rosenberg, Jingbin Wang, James Philbin, Bo Chen, and Ying Wu. Learning fine-grained image similarity with deep ranking. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 13366-133,, 2014.
[43] Kaixin Wang, Jun Hao Liew, Yingtian Zou, Daquan Zhou, and Jiashi Feng. Panct: Few-shot image semantic segmentation with prototype alignment. In proceedings of the IEEE/CVF international conference on computer vision, pages 1197-92066 2019.
[44] Xialli Wang, Aiping Jiang, and Yingli Wang. A segmentation method of smoke in forest-fire image based fbm and region growing. In 2011 Fourth International Workshop on Applications, pages 390-393 IEEE, 2011.
[45] Zhirong Wu, Yuanjun Xiong, Stella Yu, and Dahua Lin. Unsupervised feature learning via non parametric instance discrimination. In Proceddings of the IEEE conference on computer vision and pattern recognition, pages 3733-3742, 2018.
[46] Qiangqiang Xia, Feifei Lee, and Qiu Chen. Tec-net: A two-stage training method with contradictory loss and co-teaching based on meta-learning for learning with noisy labels. Information Sciences, 639: 19008. 2023.
[47] Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M Alvarez, and Ping Luo., Segformer: Simple and efficient design for semantic segmentation with transformers, Advances Neural Information Processing Systems, 14:77777-1009 2021.
[48] Deng Xing, Yu Zhongming, Wang Lin, and Li Jinlan. Smoke image segmentation based on color model. Journal on Innovation and Sustainability RISUS, 6(2):130- 138, 2015.
[49] Gao Xu, Yongming Zhang, Qixing Zhang, Gaohua Lin, Zhong Wang, Yang Jia, and Jinjun Wang. Vidco smoke detection based on deep saliency network. Fire Safety Journal, 105:277-285, 2019.
[50] Cheng Xue, Qi Dou, Xueying Shi, Hao Chen, and Pheng- thn Ann Heng. Robust learning at noisy labeled medical images: Applied to skin lesion classification. In 2019 IEEE 16th International Symposiun on Biomedical Imaging (ISBI 2019), pages 1280-1283. IEEE, 2019.
[S1] Siyuan Yan. Jing Zhang, and Nick Barnes. Transmission- bayesian generative model for smoke segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 3009-3017, 2022.
[52] Jiachen Yao, Yikai Zhang, Songzhu Zheng, Mayank Goswami, Prateek Prasanna, and Chao Chen. Learning to segment from noisy annotations: A spatial correction approach. arXiv preprint arXiv:2308..244 2023.
[53] Lujian Yao, Haitao Zhao, Jingchao Peng, Zhongze Wang, and Kaijie Zhao. Fosp: Focus and separation network for early smoke segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages
2024.
[54] Feiniu Yuan, Zeshu Dong, Lin Zhang, Xue Xia, and Jinting Shi. Cubic-cross convolutional attention and count prior embedding for smoke segmentation. Pattern Recognition, 131: 108902, 2022.
[55] Feiniu Yuan, Lin Zhang, Xue Xia, Qinghua Huang, and Xuelong Li. A gated recurrent network with dual classification assistance for smoke semantic segmentation. IEEE Transactions on Image Processing, 30:4409 4422, 2021.
[56] Feiniu Yuan, Lin Zhang, Xue Xia, Boyang Wan, Qinghua Huang, and Xuelong Li. Deep smoke segmenta- tion. Neurocomputing. 357:248-280 2019.
[57] Yuhui Yuan, Xilin Chen, and Jingdong Wang. Object-contextual representations for semantic segmenta- tion. In Computer Vision ECCY 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part VI 16, pages 173-190 Springer, 2020.
[58] Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 64(3):007-155, 2021.
[59] Jianmei Zhang. Hongqing Zhu, Pengyu Wang, and Xiaofeng Ling. Att squeeze u-net: A lightweight network for forest fire detection and recognition. IEEE Access, 1110858-10777 2021.
13

---

[60] Minqing Zhang, Jiantao Gao, Zhen Lyu. Weibing Zhao. Qin Wang, Weizhen Ding, Sheng Wang, Zhen Li, and Shuguang Cui. Characterizing label emors: Confident learning for noisy-labeled image segmentation. In Medical Image Computing and Computer Assisted Interveniin-MICCAN 2020: 23rd International Conference, Lima, Peru, October 4 4-8, 2020, Proceedings, Part / 23, pages 721-730. Springer, 2020.
[61] Haitao Zhao, Zhihui Lai, Henry Leung, Xianyi Zhang. Linear discriminant analysis. Feature Learning and Understanding: Algorithms and Applications, pages 71-85, 2020.
[62] Tianfei Zhou, Wenguan Wang, Ender Konukoglu, prototype view. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2582 2593, 2022.
Van Gool. Rethinking semantic segmentation:
[63] Haidong Zhu, Jialin Shi, and Ji Wu. Pick-and-learn: Automatic quality evaluation for noisy-labeled image segmentation. In Medical Image Computing and Computer Assisted Intervention-MICC 2019: 22nd International Conference, Shenzhen, China, October 13 17, 2019, Proceedings, Part VI 22, pages 576-584. Springer, 2019.
[64] Yuchua Zhu, Muli Yang, Cheng Deng, and Wei Liu. Fewer is more: A deep graph metric learning perspective using fewer proxies, Advances Neural Information Processing Systems, 33:17792-17803, 2020.
14

---

Appendix
A CoSW Derivation
The optimization problem for CoSW is:
N
N
max ((PV) =-
In(
N
N
(14)
aK
N
N
p 112.
NR
with the constraint Σn= o N where N denotes the number of pixel features belonging to w. The penalty function g(-) employs the L2 norm. The 7 E [0, 1] is regularization parameter which controls the degree of punishment.
We can transform the maximization function to incorporate all constraints by Lagrange multipliers:
RK
N
L(P,V)
In
N
NE
==
N
1
p"11
(15)
N
NE
1).
N
Taking the partial derivative of Eq. uik of each sample and the objective values p for prototype update:
15 with respect to en and p equal to zero, we can obtain the CoSW
aL
In
Ouh
N
No
N
(16)
1
1 NE
x
p"lle
+
0,
AL
N²
N²
2 = 0.
2
(xr - p²)
(17)
8p
N
NZ
For Eq.
16 we have In w /
- 7 a
p* ||| + (N/NN/)
Then
s N
p" l|z) exp(
1).
(18)
N
Since Σ o0
N
we have
expl
1) exp(- p* (12)
(19)
N
Substituting Eq. 19 into Eq.
18 the sample weighting can be obtained:
exp(->/1ar, p=112) N²
(20)
ΣN exp( >|| p* 2|2)
And for Eq.
17 we have
2n=1 1.
-== xt exp(->|æ0 p² 111
(21)
Nt
1= uk
DX exp(->|lxt p²112)
15

---

B
Other Entropies
B.1 Burg's Entropy
For probability distribution 11 (2,, .. TN) of N random variables {I1, 22, ... INJ. Burg's entropy is defined as
2 Σ mo
(22)
Based this, can define the following entropy: Troal-prototype entropy:
N
T
In m /N.
(23)
Within-prototype entropy:
AK
T
T
(24)
where TH En= No / formulated as the Burg's entropy of the similarities in the n-th prototype.
Between-prototype entropy:
AK Nl T. 5EEn In /N. km
(25)
Then, also have T = TB + T. The optimization objective can be formulated as
OK
N
max
(P(P,V)
Σ
In(
N
Un
A
(26)
OK
v
N+
Un No
N
We follow the procedure in Sec. A to solve Eq. 26
(->|<>* p 21))
No
(27)
Σ n=
(~>1 - px (1
5 No
n= x (-> x²11 112)
D
(28)
(->ll=af -²112)
B.2 Kapur's Entropy
For a probability distribution 11 = (T,, AT, TN) of N random variables {x1, I2, EN Kapur's entropy is defined
T K (TI)
In
(1-m))l m.)
(29)
Based on this, we can define
Kapur's entropy as
aK
T
In
N
N
(30)
OK
In
1
N
N
NV
1
16

---

For Shannon's 's entropy and Burg's entropy, we have T = TT + To and T Th = + +Tg.. However, in Kapur's entropy, it is not the case. For a given classification problem, maximizing the within- prototype Kapur's entropy is different from maximizing Kapur's entropy on the whole dataset. This means we cannot simply consider maximizing the entropy of each prototype independently. But for consistency and comparison, we also use the within-class Kapur's entropy instead of the within-class entropy to design the objective function.
Based on Eq. 30 we have the following objective function:
N
N
max J (P.V) P.V
=
Σ
N
N
N
N
N
(1
Un
# No
(31)
N
No.
AK
N
N
p 1122
N
The likelihood value and the objective of the prototype update can be obtained by following the procedure in Sec.
N
(32)
N² 1
expl
pa112 2c))
pd
z
(33)
n exp(-1
pp112 )²)
where D are the solutions of m=
No.
Details of Pixel-Prototype Matching (P2M)
C.1
Derivation of P2M
The P2M is based on the matching process in ProtoSeg 62 and SWAV 4. be solved is
The optimization problem
min Tr(M TC", Me
No K
(34)
s.t. M (0. 13 T K N"
N"
K
where M E R is the matching strategy and C E R K represents the cost matrix that measures the distance between pixel features and prototypes:
C || p." 11, = 1,2,... ... K.j = 1,2, N
(35)
F4.34ji is typical transportation problem 39 and solving it directly using linear programming
time-consuming. One fast approach involves incorporating an entropy regularization term T H (M") facilitate the relaxation of M and acquire an approximate solution
min Tr T CCC) H (M
s.t. M KNN M 1 N"
M NW
N
(36)
where H(M") = = E.y MrlogMM and the 0 is applied to control the smoothness of the matching strategy. We set 0.05. The solution of Eq. c6 be given by:
M Diag(u) exp Diag(e).
(37)
where RKK and E N represent renormalization vectors. They can be calculated by the iterative Sinkhorn-Knope algorithm which efficient on GPU it only relies on matrix multiplications.
17

---

Table 5: Impact of the number of iterations in Sinkhorn Sinkhorn-Knopp algorithm (dataset: SmokeSeg. backbone: MiT-B3).
Sinkhorn Iterations
5
10
Fi mIoU
66.40
72.32
72.22
72.15
72.10
52.78
59.83
59.63
59.38
59.23
C.2 The Number of Sinkhorn Ittrations
We also investigate the impact of the number of normalization steps on model performance, as shown in Tab. [ It can be observed that when iteration=l, the model performance is poor. We think this is due to an insufficient number of steps, which fails to adequately match the pixels and prototypes. As the number of iterations increases, the performance of the model improves and tends stabilize.
C.3 The Number of Prototypes Per Class
Tab. 6 reports the performance of different numbers of prototypes. It can be observed that as the number of prototypes increases from 1 to 3, there is significant improvement. This demonstrates that the polycentric embedding space formed by multiple prototypes indeed enhances the representation of the model. The growth becomes slow when the number exceeds 10.
Table 6: Impact of the number of prototypes (dataset: SmokeSeg, backbone: MīT--B)..
# Prototype Number
3
5
10
20
F1 mlol
68.73
70.85
71.71
72.32
72.43
56.50
58.39
59.23
59.83
59.92
18

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The paper discusses the limitations in the Conclusion section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
19

---

Justification:
This paper provides complete and correct proof in the appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification:
The paper fully discloses all the information necessary reproduce the main experimental results (in main submission and appendix), ensuring that the main claims and conclusions of the paper can be independently verified and validated.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
20

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [No]
Justification:
The paper has not yet been open-sourced for data and code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.or cc/
public/ httee//uideess/ooooo
ss ssionoolicyy for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https?
wttp:ccccppcccccccccccccccc i ies CCodeeoodSuaaee bm ss i oonool icy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification: We provide a thorough description of the details of our experiments in the paper and appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [NA]
Justification: There is no reporting of error bars or statistical significance information.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for
example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
21

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative етrоr rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [No]
Justification: The paper does not provide sufficient information the computer resources, such as the type of comput workers, memory, and time of execution needed to reproduce the experiments. However, the paper compensates for this by providing details the computational complexity of the proposed methods and other SOTA methods.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics httpts://eerrisss..... http://pppppppppppcppceceeea htwppbllliwethcccccuccoooo.oeoe.
Answer: [Yes]
Justification: The research conducted in the paper conforms the NeurlPS Code of Ethies, as outlined in the provided URL. The paper adheres to the ethical practices and guidelines specified the NeurIPS Code of Ethics during the research process.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction).
Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: The paper solely emphasizes the positive societal impacts of the work per- formed, omitting any discussion of potential negative consequences societal drawbacks.
Guidelines:
22

---

The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (c.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e-g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The paper thoroughly acknowledges and properly credits the creators or origi- nal owners of assets, including code, data, and models, used in the research. Additionally, explicitly mentions and respects the licenses and terms of use associated with these assets, ensuring ethical and legal compliance.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
23

---

The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, (papersw sepersswthcode com/ dat asats has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The paper contributes a synthetic noisy dataset NS-1K.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
24

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
25


Reviews:
[{"review_id": "VR5zQEjaqM", "reviewer": "Reviewer_T8tv", "summary": "This work proposes a method for Smoke Segmentation with Label Noise, addressing the issue of noisy labels commonly found in non-grid smoke annotations. This idea is meaningful and reasonable. The main contributions of the paper are the conditional sample weighting (CoSW) and regularized within-prototype entropy (RWE).", "questions": "see Weaknesses", "limitations": "see Weaknesses", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "N/A", "weaknesses": "However, during my recent review of ECCV 2024 papers, I gave positive feedback on a similar paper titled “DSA: Discriminative Scatter Analysis for Early Smoke Segmentation,” which has been accepted by ECCV 2024. That paper applied Scatter Matrices to Smoke Segmentation, optimizing the objective function through the ratio-trace form of (S_w)^-1*(S_b). In my view, the regularized within-prototype entropy (RWE) in this submission is very similar to the method I previously reviewed. Therefore, I am currently unable to give a positive rating for this submission unless the authors can clarify the theoretical differences between the two works.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We understand the reviewer's confusion, but this paper is different from DSA. We would like to clarify the differences between our proposed method and the DSA point-by-point below. \n\n1. The **key difference** is that CoSW is to construct **conditional sample weighting** to address the issue of **noisy labels**, while DSA is to formulate scatter matrices to handle the problem of **hard samples**. We introduce the **information theoretic learning** (ITL) to achieve CoSW, which aims to determine sample weighting through uncertainty estimation.  Furthermore, we experiment with **different entropies**, including **Shannon's, Kapur's, and Burg's entropy**, and provide **complete weighting derivations** based on each entropy in the **Appendix**. **CoSW only adopts scatter analysis in the loss part**, and different from DSA, we further incorporate the **sample weighting** into the scatter matrix, to prevent metric learning from overfitting under noisy labels.\n\n2.  In CoSW, we propose a concept of **regularized within-prototype entropy (RWE)**. RWE can establish independent evaluation criteria for each feature group formed by multiple prototypes. This allows for a more fine-grained determination of the weighting, which is suitable for inconsistent smoke features. \n\n3. For the **prototype update**, CoSW is also different from DSA. DSA directly uses gradient descent, but gradient descent can easily be affected by noisy labels. We adopt a regularized nonparametric prototype update (Fig. 3 in the paper), in which the new prototype is **weighted** by all the features that matched the prototype in the previous iteration.\n\n4. The **prediction head** in DSA and CoSW is different as well. The essence of DSA is to integrate the scatter loss into the original model, still using the classic binary prediction head (2 neurons). But CoSW changes the prediction head to a **prototype-based** way, where the final output neurons are 2K, with K being the number of prototypes per class. \n\n5. In the experiment, we not only test on real datasets but also create a **synthetic noise dataset NS-1K**, which has two noisy hierarchies: the ratio of noisy labels to the total labels and the degree of noise. Under this setting, NS-1K can reflect the performance of various models under different noise levels.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7UkThrdzLK", "reviewer": "Reviewer_bKT9", "summary": "Smoke segmentation is an important problem as it can be directly tied to health and safety. That being said, it is also a difficult problem as annotations for smoke segmentation datasets are noisy, sometimes leading to inconsistent or even poor segmentation performance. The authors address this issue by proposing a prototype-based clustering algorithm using different weighting criteria for feature clusters and prototypes via conditional sample weighting (CoSW) and regularized scatter metric learning (RSML). They also introduce regularize within-prototype entropy (RWE) to update prototypes using adaptive sample weighting. The proposed approach can be attached to existing segmentation models (e.g., SegFormer or MiT-B3) to produce state-of-the-art results on two real smoke segmentation datasets and on their new synthetically-noisy smoke segmentation dataset, NS-1K. This method is not only effective at dealing with noisy labels in smoke segmentation, but its general formulation has the potential to be adapted to other problems with noisy segmentation labels.", "questions": "1. What classes are the authors referring to in L137? Smoke vs background?\n2. Could you give some details on the \"Real-time\" vs \"Normal\" column of Table 1? Also, what is the runtime or FPS of this approach compared to others, for real-time applications?\n3. This is not a limitation or a flaw, but more a question born from curiosity. Has this approach been applied to any other noisy-segmentation problem other than smoke segmentation? The formulation of the approach appears fairly general, meaning that it could potentially have broader use.", "limitations": "1. The authors identify a limitation of metric learning in that it does not perform as well as the baseline (proto) when optimized using the triplet loss with noisy labels. However, using CoSW with the triplet loss (or even better, with the scatter loss) noticeably improves performance, showcasing the significance of CoSW in dealing with noisy labels.\n2. The authors also note that RWE is based on Shannon entropy, which could potentially be inferior to Kapur's or Burg's entropy formulations. The authors address this by performing an experiment (see Table 4) with RWE using each of these entropy formulations, which showed that Shannon entropy resulted in the best performance.\n3. Since clustering approaches are iterative, they can take a performance hit in real-time applications compared to single-pass methods like neural networks. The authors provide some results pertaining to this problem (see Table 1) but don't directly address this issue. That being said, runtime performance is not the focus of this problem or paper, so the limitation is not a significant issue.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. Provided a mathematical derivation or foundation for their proposed clustering method’s prototype and feature weighting (CoSW), the regularized scatter metric learning (RSML), and the prototype update method (RWE).\n2. Improved the quality of two existing real smoke segmentation datasets (SmokeSeg and SMOKE5K) by carefully re-annotating the validation set. The community can only progress if our benchmark datasets are reliable enough to believe the results are meaningful. Improving the reliability of the validation set for these datasets not only improves the proposed work's performance, but helps the community as a whole.\n3. Created a synthetic smoke noise dataset, NS-1K. The authors re-annotated 1000 images from SmokeSeg and added artificial noise to the labels to create a new dataset to target label noise problems with smoke segmentation.\n4. A thorough evaluation showing state-of-the-art performance on two existing benchmarks (SmokeSeg and SMOKE5k) and on their new datasets, NS-1K. The proposed approach surpasses the performance of existing methods in both F1 and mIoU with various backbones and even in real-time. The evaluation is thorough and comprehensive, featuring both quantitative and qualitative results compared with previous state-of-the-art methods. The ablation study is also very detailed and showcases the importance of each contribution to the method’s performance.", "weaknesses": "1. The writing could be more clear at times, in particular in some parts of the mathematical derivation. Overall, the writing is understandable but it could be improved for clarity.\n- L104: What diagrams are the authors referring to?\n- L127: This is somewhat unclear, what do the authors means by varieties?\n- L137: What do classes mean in the context of this problem could be more clear with a simple example in parenthesis.\n- L138 and L196: What is D? The pixel dimension, 3 for RGB?\n\nMinor:\n- Strange grammar/wording: L104-105, L128, L282", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for our paper's positive feedback and constructive suggestions. Here are our\nresponses to the reviewer's comments. We adopt different letters to represent the different parts of the question raised. Where \"W\" represents weakness and \"Q\" represents question.\n\n>W1(1). L104: What diagrams are the authors referring to?\n\nThe “diagrams” here refers to the “research fields”.\n\n> W1(2). L127: This is somewhat unclear, what do the authors mean by varieties?\n\nThe “varieties” here mean “N random variables”.\n\n> W1(3). L137: What do classes mean in the context of this problem could be more clear with a simple example in parenthesis.\n\nOur intuition is to provide a more general expression, where CoSW can also be applied to multi-class tasks.\n\n> W1(4). L138 and L196: What is D? The pixel dimension, 3 for RGB?\n\n“D” represents the feature dimension after the pixels have gone through the encoder-decoder. At the same time, it is also the dimension of the prototype. In our experiments, the value of D is 256.\n\n>W1(5). Strange grammar/wording: L104-105, L128, L282.\n\nThank you for the reviewers' suggestions. We explain the issues below:\n\n- L104-105: The research fields include supervised learning, unsupervised learning, and self-supervised learning.\n\n- L128: Shannon entropy can be used to measure the uncertainty of the distribution $ T(\\Pi) = - \\sum _ {i=1} ^ {N} \\pi_{i} \\ln \\pi_{i} $ (with $ \\sum_{i=1}^{N}\\pi_i=1 $ constrain).\n\n- L282: To investigate the reasons behind the effects of CoSW, we visualize the formation process of CoSW.\n\n\n> Q1. What classes are the authors referring to in L137? Smoke vs background?\n\nYes, the classes are smoke and background.\n\n> Q2. Could you give some details on the \"Real-time\" vs \"Normal\" column of Table 1? Also, what is the runtime or FPS of this approach compared to others, for real-time applications?\n\nThe \"real-time\" refers to using lightweight backbones, such as MiT-B0, AFFormer-B, SeaFormer-B, etc. \"Normal\" uses the regular backbones, such as ResNet-50, MiT-B3, HRNet-48, etc. Here, we also provide a FPS comparison for the different methods.\n\n**Real-time**:\t\t\t\t\t\n| Method       | AFFormer     | SeaFormer    | SegFormer  | SC        | CleanNet  | CoSW      |\n|--------------|--------------|--------------|------------|-----------|-----------|-----------|\n| Backbone     | AFFormer-B   | SeaFormer-B  | MiT-B0     | MiT-B0    | MiT-B0    | MiT-B0    |\n| FPS          | 101.3        | 92.5         | 98.4       | 88.4      | 89.5      | 93.6      |\n\n**Normal**:\n| Method  | DeepLabV3+ | OCRNet  | SegNeXt | Trans-BVM | CoSW   | SegFormer | Trans-BVM | SC     | CMW-Net | CleanNet | CoSW   |\n|-----------|------------|---------|---------|-----------|--------|-----------|-----------|--------|---------|----------|--------|\n| Backbone          | ResNet-50  | HRNet-48| MSCAN-L | ResNet-50 | ResNet-50 | MiT-B3   | MiT-B3   | MiT-B3 | MiT-B3  | MiT-B3   | MiT-B3 |\n| FPS       | 32.4       | 22.2    | 20.1    | 30.3      | 36.0   | 44.1      | 30.7      | 27.5   | 28.5    | 35.4     | 36.7   |\n\n*Input shape: 512x512; NVIDIA RTX 3090Ti GPU\n\n>Q3. This is not a limitation or a flaw, but more a question born from curiosity. Has this approach been applied to any other noisy segmentation problem other than smoke segmentation? The formulation of the approach appears fairly general, meaning that it could potentially have broader use.\n\nWe find that the variable transparency also exists in **skin lesions**, so we **supplement the experiments** on skin lesions images. The dataset used is the ISIC-2017 dataset. The ISIC 2017 dataset is a well-known public benchmark of dermoscopy images for skin cancer detection. It contains 2000 training and 600 test images with corresponding lesion boundary masks. For the noise setting, we refer to the noise generation approach from the NS-1K dataset. Below we list the **experimental results**. The **sample weighting visualization** can be seen in the **global response PDF file Fig. 1**.\n\n| Method | Trans-BVM | SC | CleanNet | CoSW |\n| --- | :---: | :---: | :---: | :---: |\n| Clean | 85.40 | 83.80 | 84.16 | 84.32 |\n| Noise Ratio:60%; Degree High* | 69.23 | 71.98 | 72.80 | 74.57 |\n\n*The specific meaning of the noise setting can be seen in the paper Sec. 5.1 NS-1K part.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "08stCajb0Y", "reviewer": "Reviewer_KX4g", "summary": "In order to solve the problems of complex and blurred edges of non-grid smoke in smoke segmentation, as well as the existence of noisy labels in large-scale pixel-level smoke datasets, this paper proposes a conditional sample weighting (CoSW) method. CoSW uses a multi-prototype framework, in which prototypes are used as prior information and different weighting criteria are used in different feature clusters to solve the problem of feature inconsistency. This paper also introduces a new regularized within-prototype entropy (RWE) to achieve steady-state and stable updating of prototypes.", "questions": "1. The two datasets used in the experiments are SmokeSeg and SMOKE5K. Can you add their information, such as the number of images, noise label types, noise rates, resolutions, etc.?\n2. For experimental results in Table 1 and Table 2, only numerical descriptions are listed. Can you add corresponding explanations and discussions for less than optimal performance?\n3. How to understand \"CoSW is concise and does not require data with clean labels during the training.\" in the Section Abstract? Can this method be applied to completely noisy labels? What mechanism is used to achieve this?", "limitations": "1.In Table 1, the results show that “Large” with δ (smoke pixel ratio in an image) greater than 2.5% and “Medium” with δ between 0.5% and 2.5% have poor performance in real-time, but the reason is not analyzed. Does it mean that CoSW is not very effective in the case of high smoke pixel ratio?\n2.In Section 5.3, this paper only introduces the experimental phenomenon that Trans-BVM performs best at low noise rates and CoSW performs best at high noise rates, but does not explain why. Does this result mean that CoSW has limitations at low noise rate scenarios?", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1.This article's expression and language are mostly accurate, and its structure—which includes an introduction to the problem, a method, experimental results—is well-organized. The paper has a reasonable general organization and order.\n2.This paper proposes a conditional sample weighting (CoSW) to handle smoke segmentation in the presence of noisy labels. CoSW is built on a multi-prototype framework, using prototypes as prior information to determine weight criteria, and weighing each feature cluster with different weight criteria.\n3.This paper also introduces a new regularized within-prototype entropy (RWE) to obtain comprehensive information of samples. \n4.From the results and visualization, the proposed method improves the accuracy of smoke segmentation. The visualization results show that this method can miss less high-transparency smoke than previous methods.", "weaknesses": "1.In section 1, two challenges of current smoke annotation are proposed in the second paragraph: \"1) Smoke edges are complex and blurry, making it hard to distinguish smoke and background. 2) Smoke is non-rigid and lacks a fixed shape, making it difficult for annotators to become proficient through practice with the same shape.\" Nevertheless, the fifth paragraph of this paper's proposal for the CoSW approach omits to clarify how and exactly why CoSW can resolve these two problems from a methodological perspective.\n2.In section 3.4, there are two hyper-parameters in Eq. 5 and Eq. 11: γ and μ. The article's explanation of γ and μ is too simplistic, and there is no extensive explanation of how γ and μ are defined, and how they affect the entire formula in the form of weights.\n3.In section 3.4, the derivation process of how Eq. 5 is produced doesn't reflect in Appendix D, which simply covers the procedure of using Eq. 5 to derive future equations.\n4.Please explain the reason for choosing the value of  \"we set ε= 10−5\" in Section 4.4 and what effects it will have if ε is greater or less than 10−5.\n5.The layout and aesthetics of the figures and tables in this article need to be improved, such as the placement of Table 4 and the setting of the table size and so on.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the constructive comments. Below, we have provided a point-by-point response. The \"W\" for weakness, \"Q\" for question, and \"L\" for limitation.\n\n>W1. Relationship between the two smoke challenges and CoSW.\n\nThe two analyses are to explain **why smoke tends to produce noisy labels**. We then examine the characteristics of the noisy labels in smoke, finding that it has feature inconsistency due to variable transparency. \nTo address this problem, we propose a conditional sample weighting (CoSW). CoSW aims to employ different weighting criteria for the samples within different feature clusters by constructing the regularized within-prototype entropy (RWE).\n\n>W2. Further explanation of $\\mu$ and $\\gamma$. \n\nThe $\\mu=0.999$ is for the momentum coefficient. Here we refer to the setting in MoCo (CVPR 2020). The meaning is how much of the previous content is retained with each update.\nWhere $\\gamma$ represents the degree of the regularization term in the RWE, the larger the $\\gamma$, the more sensitive the RWE is to the distance. We provide experiments with different values of $\\gamma$ in the **global response PDF file Tab. 1**.\n\n\n>W3. About the Eq. 5. \n\nEq. 5 means combining WE (Eq. 4) and the constraint equation Eq. 2. The original objective without the regularization is to build a uniform assignment, but after incorporating the regularization term Eq. 4, the RWE can determine the noise level of the features.\n\n>W4. The explanation of $\\varepsilon$.\n\nThe introduction of $\\varepsilon$ is to prevent the occurrence of singular matrices when inverting the matrix. This is a commonly used technique in LDA. The typical value used is 10e-5. We test 10e-4 or 10e-6 and they also worked for training, but setting it to 0 results in non-invertible matrices during the training.\n\n>W5. About the layout and aesthetics of the figures.\n\nWe will revise the layout and aesthetics of the figures and tables.\n\n>Q1. Can you add details of SmokeSeg and SMOKE5K?\n\nSMOKE5K is a mixed dataset (real + synthetic) and SmokeSeg is an entirely real dataset. The majority of images in SmokeSeg are early smoke. \nThe details of the two datasets are shown below.\n\n||Number of Images|Noise Label Types|Noise Rates*|Training Resolution|\n|---|:---:|:---:|:---:|:---:|\n|SMOKE5K|1,360 real + 4K synthetic |real|8.4%|480x480|\n|SmokeSeg|6,144|real|11.5%|512x512|\n\n*The \"Noise Rates\" here is the ratio of noisy pixel labels to clean pixel labels (estimated by the validation set). \n\n>Q2. Can you add discussions for Tab. 1 and Tab. 2?\n\n**In Tab. 1**:\nOur method has a clear advantage in small smoke. As small smoke mostly represents early smoke, being able to recognize early smoke accurately has significance for carrying out rescue work quickly in the real world.\nIn addition, under the same method, we find that CoSW is more suitable for transformer-based backbones. Comparing Trans-BVM and CoSW, the gap is around 1% on ResNet-50, but the gap widens to over 4% on the MiT-B3. A similar phenomenon can be observed on SMOKE5K in **Tab. 2a**.\n\n**In Tab. 2b**:\nThe tests are divided into two: 1) Noise ratio (the proportion of data added noise); and 2) Noise degree (the strength of the noise) (detailed in **Appendix C**). The impact of noise degree is sometimes greater than the noise ratio (e.g., 40% high vs 60% low).\nSince Trans-BVM is specifically designed for smoke, it achieves better results in low-noise scenarios. As the noise continues to increase, the performance of other methods declines rapidly, but CoSW still maintains the performance. \n\n>Q3. How to understand \"CoSW does not require clean labels during the training.\"? Can this method be applied to complete noise? What mechanism?\n\nMany previous methods in noisy labels require a **clean validation** during the training to guide the model identifying noise. 1) the cost of obtaining clean labels is high. 2) Incorporating validation set into the training makes the pipeline complex.\nThe intuition is that under CoSW, the model can find the **common characteristics** of smoke (i.e. multiple prototypes), and then determine the noise level of each feature based on them. This process is specifically carried out through RWE. The CoSW aims to employ **different weighting criteria** in different feature clusters to address the problem of feature inconsistency. Fig. 5b shows the CoSW formation. \nThe CoSW requires the assumption that the **majority of pixels have clean labels**. When the label mask is completely noisy, the model is also unable to distinguish the noisy labels, because it can not learn which features are the common of smoke. \nTo test the anti-noise ability of CoSW, we design an **extreme experiment** by directly adding different levels of Gaussian noise to the original labels, until it **approaches the complete noise**. The results and examples of noisy images can be seen in the **global response PDF file Tab. 2 and Fig. 2**.\n\n>L1. CoSW is not very effective in high smoke pixel ratio.\n\nSince the noisy labels mainly occur at the edges of the smoke, in small smoke, the impact of the noisy labels is greater, while in large smoke (high smoke pixel ratio), the impact is smaller. Therefore, **the small smoke is more important and meaningful in the real world**, as it can assist in timely rescue. Our CoSW achieves the best performance in small smoke, and Tab. 2b demonstrates that our method has robustness against noise. In the high smoke pixel ratio, the impact of noisy labels is not so great, so the performance of CoSW is not very outstanding, but it is second, with the $F_1$ score **close to the first (< 0.2)**.\n\n\n>L2. CoSW has limitations at low noise scenarios.\n\nThe reason why CoSW is slightly inferior to Trans-BVM under a low noise ratio is that CoSW uses the **basic segmentation model**, while Trans-BVM uses a model **specially designed for smoke**, with a Bayesian generative model and a Transmission module. When the noise rate increases, the performance of CoSW begins to emerge.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6OEBuhgagd", "reviewer": "Reviewer_1bVm", "summary": "This paper tackles the problem of noisy label in smoke segmentation by introducing uncertainty measure in the some area, especially in the boundary of smoke/not smoke. Noisy label can be problematic for training stability. Entropy is used to measure uncertainty. Highly uncertain prototypes and pixels should not contribute too much during training.", "questions": "1. Will the annotation of the re-annotated validation set (the clean validation set) of SmokeSeg and SMOKE5K be released?\n\n2. Related to weakness \\#1: is $\\Omega$ in line 37 should be $\\omega_\\Omega$? Where is $\\mathbf p(\\mathbf x^k_n$ in Eq.(4)?\n\n3. Any insight why different entropies leads to different results?\n\n4. What are the $\\lambda$, $\\mu$, $\\gamma$ values used?", "limitations": "No limitation is discussed as also mentioned in the paper checklist.", "rating": 5, "confidence": 1, "soundness": 2, "presentation": 2, "contribution": 4, "strengths": "1. Experiments using synthetic noise dataset and the ablation show the effectiveness of the proposed method.\n\n2. To the best of my knowledge, it is the first paper to tackle the noisy label problem in smoke segmentation.", "weaknesses": "Overall I find the paper difficult to read. I believe it can be caused by these following reasons:\n\n1. Not all of the notations are clearly defined or some notations are defined very late (e.g. $N_k$ has been mentioned since section 3.3 but its definition is in section 3.4). Not all of vector or matrix have their size defined clearly.\n\n2. No reference to equation/section regarding each components in ablation study in Table 3a.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the constructive comments which helped us improve the quality of our work. In the following, we have provided a point-by-point response to the comments. We adopt different letters to represent the different parts of the question raised. The \"W\" represents \"weakness\" and the \"Q\" represents \"question\".\n\n>W1(1). Not all of the notations are clearly defined or some notations are defined very late (e.g. \nhas been mentioned since Sec. 3.3 but its definition is in Sec 3.4). Not all vectors or matrices have their size defined clearly.\n\nWe thank the reviewer for the suggestions, and we will include a list of symbol definitions in the paper.\n\n>W1(2). About the notation and reference in Tab. 3a.\n\nWe thank the reviewer for the suggestion. \"Proto\" means to change the regular prediction head into a prototype-based one. “Sample Weight” refers to Eq. 7 and the “Proto Update” refers to Eq. 8. We will also include the reference in Tab. 3a. \n\n>Q1. Will the annotation of the re-annotated validation set (the clean validation set) of SmokeSeg and SMOKE5K be released?\n\nYes, we will release the re-annotated validation set of SmokeSeg and SMOKE5K soon.\n\n>Q2. is $\\Omega$ in line 37 should be $ \\omega_{\\Omega}$? Where is $\\pmb{𝑝}(\\pmb{𝑥} _ {𝑛}^{k}) $ in Eq.4?\n\nYes, the $\\Omega$ in line137 should be $\\omega_{\\Omega}$.\nThe $\\pmb{𝑝}(\\pmb{𝑥} _ {𝑛}^{k}) $  in line 155 and 156 should be $\\pmb{p}^k$ in the Eq. 4.\n\n>Q3. Any insight why different entropies lead to different results?\n\nThe reason why different entropies lead to different results can be explained from two perspectives:\n\n**1. From the characteristics of the three entropies**:\n\nKapur's entropy and Burg's entropy are both based on Shannon's entropy.\n\n**Shannon Entropy** ($T(P)$)\n- **Formula**: \n  $ T(P) = -\\sum_{i=1}^{N} p_i \\ln p_i $\n- **Characteristics**:\n  - Shannon entropy is a fundamental concept in information theory, used to measure uncertainty or information content.\n  - The more uniform the probability distribution (i.e., the closer each ($p_i$) is to each other), the higher the Shannon entropy, indicating greater uncertainty.\n  - When an event's probability is 1 (a certain event), Shannon entropy is 0, indicating no uncertainty.\n\n**Burg Entropy** ($T^B(P)$)\n- **Formula**: \n  $ T^B(P) = \\sum_{i=1}^{N} \\ln p_i $\n- **Characteristics**:\n  - Burg entropy differs from Shannon entropy in that it directly sums the logarithms of the probabilities.\n  - Since the logarithm function is 0 when the probability is 1 and approaches negative infinity as the probability approaches 0, Burg entropy can heavily **penalize extremely low-probability events**.\n\n**Kapur Entropy** ($T^K(P)$)\n- **Formula**: \n  $ T^K(P) = -\\sum_{i=1}^{N} p_i \\ln p_i - \\sum_{i=1}^{N} (1-p_i) \\ln (1-p_i) $\n- **Characteristics**:\n  - Kapur entropy is an extension of Shannon entropy, considering not only $p_i$ but also $(1-p_i)$.\n  - This entropy measure is more comprehensive, taking into account the uncertainty associated with both **the occurrence and non-occurrence of each event**.\n  - In extreme cases (i.e., when $ p_i $ is 0 or 1), Kapur entropy still provides a reasonable measure, addressing the limitations of Shannon entropy in these scenarios.\n\nBy comparing these different entropies, we can see that while they all measure the uncertainty of a probability distribution, they have distinct applications and characteristics. Shannon entropy is the most **basic and classic** measure, while Burg entropy and Kapur entropy offer **extensions and complements** for different situations and needs.\n\n**2. From the weighting derived**:\n\nShannon's\n$v_n^k=N^k \\frac{\\exp(-\\gamma || \\pmb{x}_n^k-\\pmb{p}^k ||_2)}{\\sum _ {n=1}^{N^k} \\exp(-\\gamma || \\pmb{x}_n^k-\\pmb{p}^k ||_2)}$\n\nBurg's\n$v_n^k=N^k \\frac{(-\\gamma || \\pmb{x}_n^k-\\pmb{p}^k ||_2)}{\\sum _ {n=1}^{N^k}(-\\gamma || \\pmb{x}_n^k-\\pmb{p}^k ||_2)}$\n\nKapur's\n$ v_n^k=N^k \\frac{1}{1 + \\exp( -|| \\pmb{x}_n^k-\\pmb{p}^k ||_2 - \\lambda_k)} $\n\n(where $ \\lambda _ k $ in Kapur's are the solutions of $ \\sum_{n=1}^{N^k} v_n^k=N^k $)\n\nCompared to Shannon's entropy derived weighting (Eq. 7 in paper), Burg's (Eq. 32 in Appendix) **lacks the exponential term**. However, the exponential term can increase the **sensitivity** of the model to noisy labels. As a result, in Table 4, the performance of Burg's entropy is slightly inferior to the others.\n\nThe relatively poor performance of Kapur's entropy derived weighting (Eq. 37 in Appendix) can be attributed to the fact that we do not have $ T_t^K = T_w^K + T_b^K$ for Kapur's entropy, which is valid on Shannon's entropy and Burg's entropy. Hence, for a given classification problem, maximizing the within-prototype Kapur's entropy is different from maximizing Kapur's entropy on the entire dataset. This means **we cannot simply consider maximizing the entropy of each prototype independently**. But for consistency and comparison, we also use the within-class Kapur's entropy instead of the within-class entropy to design the objective function. This may also be the reason why the performance is not as good as Shannon's entropy.\nThe **derivation** of the different entropy measures can be found in the **Appendix**.\n\n>Q4. What are the $\\lambda$, $\\mu$, $\\gamma$ used?\n\nIn our experiment, we set $\\lambda = 0.6$, $\\mu = 0.999$, and $\\gamma = 0.8$.\nThe $\\gamma$ represents the strength of the regularization term in RWE, we provide the performance of models with different values. The details can be seen in the **global response PDF file Tab. 1**.\n\n| $\\gamma$ | 0 | 0.2 | 0.4 | 0.6 | 0.8 | 1 |\n| --- | --- | --- | --- | --- | --- | --- |\n| $F_1$ | 68.17 | 70.14 | 71.22 | 72.30 | **72.32** | 72.04 |\n| mIoU | 55.23 | 56.42 | 57.60 | 59.77 | **59.83** | 58.58 |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank the reviewers for their careful reading of our paper and help with improving our manuscript. 

We sincerely appreciate that you find our work:
- It is the first paper to tackle the noisy label problem in smoke segmentation (Reviewer 1bVm).
- Create a synthetic smoke noise dataset, NS-1K (Reviewer bKT9, Reviewer 1bVm, Reviewer KX4g). 
- Provide a mathematical derivation or foundation for their proposed method (RWE) (Reviewer bKT9).
- Improving the reliability of the validation set helps the community as a whole (Reviewer bKT9).
- The proposed approach surpasses the performance of existing methods in both $F_1$ and mIoU with various backbones and even in real-time (Reviewer bKT9).
- The ablation study is also very detailed and showcases the importance of each contribution to the method’s performance (Reviewer bKT9).

In the subsequent sections, we aim to address the concerns and questions you raised, offering a comprehensive item-by-item response to each of your comments.

We have provided some **additional experiments** as reviewers suggest. Due to space limitations, we've displayed the results tables and figures in the **global response PDF file** for **Reviewer bKT9 Q3, Reviewer KX4g W2, Q3, and Reviewer 1bVm Q4**.

Meta-review: This paper received divergent reviews. R-T8tv is negative, with novelty concerns with respect to an unpublished paper previously reviewed. R-bKT9 is positive, noting technical novelty, dataset contributions, and strong evaluation. R-KX4g leans negative, primarily concerned with motivation (i.e., how the proposed method addresses the problems introduced in the intro) and clarity (reproducibility of the method). R-1bVm leans positive but also notes clarity issues (and indicates low confidence). The authors submitted an extensive rebuttal which did address some of the reviewers concerns (e.g., differentiating from parallel work, adding details that improve clarity).  During the discussion phase, the reviewers and AC concluded that there were no dual submission concerns and the paper should be evaluated on its own merits.  As the ratings remained split, the AC reviewed the paper in detail, finding the arguments of R-bKT9 the most compelling. The paper tackles an important problem, has novelty in the method formulation, contributes to benchmark datasets which may positively impact the community, and has extensive experiments validating the approach. As such, the AC reached a decision to accept the paper. Please take the reviewer feedback into account when preparing the camera ready version.
Decision: Accept (poster)