Semi-supervised Semantic Segmentation with Prototype-based Consistency Regularization
Anonymous Author(s)
AffiliationAddress email
Abstract
Semi-supervised semantic segmentation requires the model to effectively propagatethe label information from limited annotated images to unlabeled ones.
A challengefor such a per-pixel prediction task is the large intra-class variation, i.e., regions belonging to the same class may exhibit a very different appearance even in the samepicture.
This diversity will make the label propagation hard from pixels to pixels.
To address this problem, we propose a novel approach to regularize the distributionof within-class features to ease label propagation difficulty.
Specifically, our approach encourages the consistency between the prediction from a linear predictorand the output from a prototype-based predictor, which implicitly encourages features from the same pseudo-class to be close to at least one within-class prototypewhile staying far from the other between-class prototypes.
By further incorporating
CutMix operations and a carefully-designed prototype maintenance strategy, wecreate a semi-supervised semantic segmentation algorithm that demonstrates superior performance over the state-of-the-art methods from extensive experimentalevaluation on both Pascal VOC and Cityscapes benchmarks.
Introduction
Semantic segmentation is a fundamental task in computer vision and has been widely used in manyvision applications [32, 2, 29].
Despite the advances, most existing successful semantic segmentationsystems [26, 6, 9, 46] are supervised, which require a large amount of annotated data, a timeconsuming and costly process.
Semi-supervised semantic segmentation [49, 44, 30, 20, 8, 45, 19, 38]is a promising solution to this problem, which only requires a limited number of annotated imagesand aims to learn from both labeled and unlabeled data to improve the segmentation performance.
Recent studies in semi-supervised learning approaches suggest that pseudo-labeling [24, 1, 43] andconsistency-based regularization [23, 3, 40] are two effective schemes to leverage the unlabeleddata.
Those two schemes are often integrated into a teacher-student learning paradigm: the teachermodel generates pseudo labels to train a student model that takes a perturbed input [34].
In such ascheme, and also for most pseudo-labeling-based approaches, the key to success is how to effectivelypropagate labels from the limited annotated images to the unlabeled ones.
A challenge for thesemi-supervised semantic segmentation task is the large intra-class variation, i.e., regions belongingto the same class may exhibit a very different appearance even in the same picture.
This diversity willmake the label propagation hard from pixels to pixels.
In this paper, we propose a novel approach to regularize the distribution of within-class featuresto ease label propagation difficulty.
Our method adopts two segmentation heads (a.k.a, predictors):a standard linear predictor and a prototype-based predictor.
The former has learnable parametersthat could be updated through back-propagation, while the latter relies on a set of prototypes thatare essentially local mean vectors and are calculated through running average.
Our key idea is to encourage the consistency between the prediction from a linear predictor and the output from aprototype-based predictor.
Such a scheme implicitly regularizes the feature representation: featuresfrom the same class must be close to at least one class prototype while staying far from the other classprototypes.
We further incorporate CutMix operation [42] to ensure such consistency is also preservedfor perturbed (mixed) input images, which enhances the robustness of the feature representation.
This gives rise to a new semi-supervised semantic segmentation algorithm that only involves oneextra consistency loss to the state-of-the-art framework and can be readily plugged into other semisupervised semantic segmentation methods.
Despite its simplicity, it has demonstrated remarkableimprovement over the baseline approach and competitive results compared to the state-of-the-artapproaches, as discovered in our experimental study.
Related Work
Semi-supervised Learning has made great progress in recent years due to its economic learningphilosophy [48].
The success of most of the semi-supervised learning researches can attributeto the following two learning schemes: pseudo-labeling and consistency regularization.
Pseudolabeling based methods [24, 5, 1, 43] propose to train the model on unlabeled samples with pseudolabels generated from the up-to-date optimized model.
While consistency regularization basedmethods [23, 35, 37, 3, 40] build upon the smoothness assumption [27] and encourage the model toperform consistent on the same example with different perturbations.
The recently proposed semisupervised method FixMatch [34] successfully combine these two techniques together to producethe state-of-the-art classification performance.
Our approach draws on the successful experience ofgeneral semi-supervised learning and applies it to the semi-supervised semantic segmentation task.
Semi-supervised Semantic Segmentation benefits from the development of general semi-supervisedlearning and various kinds of semi-supervised semantic segmentation algorithms have been proposed.
For example, PseudoSeg method [49] utilizes the Grad-CAM [31] trick to calibrate the generatedpseudo-labels for semantic segmentation network training.
While CPS [8] builds two parallel networks to generate cross pseudo labels for each each.
CutMix-Seg method [13] introduces the CutMixaugmentation into semantic segmentation to construct consistency constraints on unlabeled samples.
Alternatively, CCT [30] chooses to insert perturbations into the manifold feature representation toenforce a consistent prediction.
And U 2 PL [38] proposes to make sufficient use of unreliable pseudosupervisions.
Meanwhile, considering the class-imbalance problem of semi-supervised semanticsegmentation, several researches [19, 18, 14] have been published.
Our approach is inspired by theobservation that large intra-class variation hinders the label information propagation from pixelsto pixels in semi-supervised semantic segmentation and we propose a prototype-based consistencyregularization method to alleviate this problem which is novel for related literature.
Prototype-based Learning has been well studied in the machine learning area [16].
The nearestneighbors algorithm
[11] is one of the earliest works to explore the use of prototypes.
Recently,researchers have successfully used prototype-based learning to solve various problems, e.g., theprototypical networks [33] for few-shot learning and prototype-based classifier for semantic segmentation [46].
Our work further introduces the prototype-based learning into the semi-supervisedproblem and proves its effectiveness.
Our Approach
In this section, we first give an overview of our approach and then introduce the core concept ofprototype-based consistency regularization for semi-supervised semantic segmentation.
Finally, weintroduce how the prototype is constructed and maintained throughout the learning process.
3.1 Preliminary
Problem setting: Given a set of labeled training images D l = { ( I li , Y li ) } N l i =1 and a set of unlabeledimages D u
= { I ui } N u
i =1 , where N u ≫ N l , semi-supervised semantic segmentation aims to learn asegmentation model from both the labeled and unlabeled images.
We use ˜ Y denote the segmentationoutput and ˜ Y [ a, b ] indicates the output at the ( a, b ) coordinate.
[Table]
[Figure]
of propagating label information from pixels to pixels.
This simple modification brings a significant improvement.
Overview: the overall structure of the proposed method is shown in Figure1, our approach is builton top of the popular student-teacher framework for semi-supervised learning [35, 34, 47, 28, 43].
During the training procedure, the teacher model prediction will be selectively used as pseudo-labelsfor supervising the student model.
In other words, the back-propagation is performed on the studentmodel only.
More specifically, the parameters of the teacher network are the exponential movingaverage of the student network parameters [35].
Following the common practice [34], we also adoptthe weak-strong augmentation paradigm by feeding the teacher model weakly-augmented images andthe student strongly-augmented images.
In the context of image segmentation, we take the normaldata augmentation (i.e., random crop and random horizontal flip of the input image) as the weakaugmentation and CutMix [42] as the strong data augmentation.
The key difference between our method and existing methods [13, 30, 41, 8, 38] is the use ofboth a linear predictor (in both teacher and student models) and a prototype-based predictor (inthe student model only).
As will be explained in the following section, the prediction from theteacher model’s linear predictor will be used to create pseudo labels to supervise the training of theprototype-based predictor of student model.
This process acts as a regularization that could benefitthe label information propagation.
3.2 Prototype-based Predictor for Semantic Segmentation
Prototype-based classifier is a long-standing technique in machine learning [21, 4].
From its earlyform of the nearest neighbour classifier or the nearest mean classifier to prototypical networks in thefew-shot learning literature [33], its idea of using prototypes instead of a parameterized classifier hasbeen widely adopted in many fields.
Very recently, prototype-based variety has been introduced intothe semantic segmentation task [46] and has been proved to be effective under a fully-supervisedsetting.
Formally, prototype-based classifier/predictors make the prediction by comparing test sampleswith a set of prototypes.
The prototype can be a sample feature or the average of a set of samplefeatures of the same class.
Without loss of generality, we denote the prototype set as P = { ( p i , y i ) } ,with p i indicate the prototype and y i is its associated class.
Note that the number of prototypes couldbe larger than the number of classes.
In other words, one class can have multiple prototypes formodelling its diversity.
More formally, with the prototype set, the classification decision can be madeby using
[Equation]
where sim ( · , · ) represents the similarity metric function, e.g., cosine distance.
˜
y means the classassignment for the test data x .
The posterior probability of assigning a sample to the c -th class can also be estimated in prototype-based classifier via:
[Equation]
where T is the temperature parameter and can be empirically set.
Note that Eq.
2 essentially uses themaximal similarity between a sample and prototypes of a class as the similarity between a sampleand a class.
3.3 Consistency Between Linear Predictor and Prototype-based Predictor
Although both prototype-based classifiers and linear classifiers can be used for semantic segmentation [46], they have quite different characteristics due to the nature of their decision-making process.
Specifically, linear classifiers could allocate learnable parameters 1 for each class, while prototypebased classifiers solely rely on a good feature representation such that samples from the same classwill be close to at least one within-class prototypes while stay far from prototypes from other classes.
Consequently, linear classifiers could leverage the learnable parameter to focus more on discriminative dimensions of a feature representation while suppressing irrelevant feature dimensions, i.e., byassigning a higher or lower weight to different dimensions.
In contrast, prototype-based classifierscannot leverage that and tend to require more discriminative feature representations.
The different characteristics of prototype-based and linear classifiers motivate us to design a loss toencourage the consistency of their predictions on unlabeled data to regularize the feature representation.
Our key insight is that a good feature should support either type of classifier to make correctpredictions.
In addition to using two different types of classifiers, we also incorporate the CutMix [42]strategy to enhance the above consistency regularization.
CutMix augmentation is a popular ingredient in many state-of-the-art semi-supervised semantic segmentation methods [8, 25, 38].
Specially,we first perform weak augmentation, e.g., random flip and crop operations, to the input images of theteacher model and obtain the pseudo-labels from the linear classifier.
Next, we perform the CutMixoperation by mixing two unlabeled images mix ( I i , I j ) and their associated prediction mix ( ˜ Y i , ˜ Y j ) .
The mixed image mix ( I i , I j ) is fed to the student model and the output from the prototype-basedclassifier is then enforced to fit the pseudo-labels generated from mix ( ˜ Y i , ˜ Y j ) .
Algorithm details: As a semi-supervised segmentation algorithm, we apply different loss functionsfor labeled images and unlabeled images.
For a batch of labeled images { ( I li , Y li ) } B l i =1 ∈ D l , we train both the linear predictor and theprototype-based predictor.
The linear classifier { w i } Ci =1 can produce a posterior probability estimation p linears ( Y [ a, b ]
= c
| I li )
[Equation]
where F li [ a, b ] =
f
( A 0 ( I li )) means the feature extracted at location ( a, b ) by first performing weakdata augmentation A 0 to I li and then feed it to the feature extractor f .
Meanwhile, the posteriorprobability of prototype-based predictor p prototypes ( Y [ a, b ] = c
| I li ) can also be estimated via Eq.2.
We use cosine similarity for sim ( · , · ) and empirically set the temperature hyperparameter Tto 0 .
Based on the ground truth label Y li , the student model will be optimized by the gradientback-propagated from the two predictors simultaneously
[Equation]
where
[Equation]
For a batch of unlabeled images { I ui } B u
i =1 ∈ D u , we first use the teacher model to estimate theirposterior probability
[Equation]
where { w ′ i } Ci =1 means the linear classifier weights of the teacher model and F ui [ a, b ] =f (cid:0)
A
( I ui ) (cid:1)
[ a, b ] denotes the extracted feature representation of pixel ( a, b ) from a weakly augmentedunlabeled images.
Then, the class corresponding to the maximal posterior probability is the predictedclass of a pixel in the given unlabeled sample, that is, ˜ Y ui
[ a, b ] = arg max c p lineart ( Y [ a, b ] =
c | I ui ) .
If p lineart ( ˜ Y [ a, b ] | I ui ) ≥ τ , where τ is a confidence threshold which is empirically set to 0 . 8 in ourstudy, ˜ Y [ a, b ] will be used as pseudo-labels to train the student model.
Meanwhile, for the student model we perform CutMix [42] operation among weakly augmentedunlabeled samples in the same batch to create an new image (essentially, the created mix-image canbe considered as a strongly-augmented image), i.e., ˆ I uij = mix (cid:0)
A
( I ui ) , A
( I uj ) (cid:1) s.t., { i, j } ∈ B u ,and their corresponding mixed prediction
ˆ Y uij =
mix ( ˜ Y ui , ˜ Y uj ) .
Therefore, the student model canlearn from the unlabeled samples through the following training objectives
[Equation]
where
[Equation]
where p linears ( Y [ a, b ] | ˆ I uij ) and p prototypes ( Y [ a, b ] | ˆ I uij ) are posterior probability predictions from linearclassifier and prototype-based classifier of student model respectively.
Note that we use the studentteacher training for both the linear predictor and the prototype predictor, as shown in L linearu and
L prototypeu respectively.
Understand L prototypeu in Eq.
In order to better understand the proposed regularization loss term
L prototypeu , we can consider the following significantly-simplified version of our method by omittingthe CutMix operation: now let’s imagine at a certain point of the training process, the learned featurerepresentation can successfully support the linear classifier in making a correct prediction for somepixels.
This means there are at least some discriminative feature dimensions that can distinguishclasses.
Without loss of generality, let’s assume the feature vector for each pixel consists of two partsx =
[ x d , x c ] , where x d is the discriminative part while x c is a less discriminative part, e.g., featuresshared by many classes.
Linear classifiers can assign lower weights to x c to suppress its impact,however, the impact of x c cannot be avoided by using prototype-based classifiers.
Thus from thesupervision of the linear classifier, the training objective of optimizing the prototype-based classifiercould further suppress the generation of x c .
Geometrically, this also encourages the features from thesame class gather around a finite set of prototypes and being apart from prototypes of other classes.
In this way, the (pseudo) class label can propagate more easily from pixel to pixel, which in turnbenefits the learning of the linear classifier.
3.4 Prototype Initialization and Update
Prototype initialization: The prototype-based classifier does not have learnable classifier parametersbut relies on a set of good prototypes.
Thus it is vitally important to carefully devise strategies toinitialize and maintain the pool of prototypes.
To initialize the prototypes, we first use the given labeled samples to train the semantic segmentationnetwork (with a linear predictor) in a fully-supervised way for several epochs.
Then we extract pixelwise feature representation for each class with the trained segmentation network.
With the in-classpixel-wise feature representations, we propose to perform clustering on them to find out internalsub-classes, and the initial micro-prototypes will be obtained by averaging the feature representationswithin the same subclass.
Please find the detailed algorithm table in the supplementary material.
Prototype update: In our approach, the prototypes are dynamically updated from the featuresextracted from the labeled images and those from unlabeled samples during the semi-supervisedlearning process.
When a labeled image is sampled, we assign each pixel to a prototype based on two conditions: (1)the assigned prototype p k should belong to the same class as the pixel.
(2) p k should be the mostsimilar prototype among all other prototypes in the same class.
Once the assignment is done, weupdate p k via
[Equation]
) where F [ a, b ] is the feature representation for the pixel at ( a, b ) .
α is a hyper-parameter controllingthe prototype update speed.
We set α = 0 .
99 throughout our experiment.
For unlabeled images, the ground-truth class label for each pixel is unavailable, thus we use pseudolabel instead.
Recall that the pseudo-label is generated when the prediction confidence is higher thana threshold.
Thus, not every pixel will be used to update the prototype.
Also, since prototype-based classifier is only used for images after the CutMix [42] operation.
In ourimplementation, we use features extracted from the CutMix images to update the prototype ratherthan the original images.
Empirically we find this could slightly improve the performance.
Experiments
4.1 Experimental Setup
Our experiment setting follows the recently proposed state-of-the-art work U 2 PL [38] including theevaluation datasets, semantic segmentation networks and training schedules for a fair comparison 2 .
Our code will be released after the anonymity period.
Some experimental details are listed as follows
Datasets: PASCAL VOC 2012 [12] is designed for visual object class recognition.
It containstwenty foreground object classes and one background class.
The standard partition of the datasetfor training/validation/testing are 1,464/1,449/1,556 images, respectively.
In the semi-supervisedsemantic segmentation literature, some researches [8, 19, 41, 38] also include the augmented set [15]for model training.
This augmented set contains 9,118 images with coarse annotations.
In theliterature [38], two ways of selecting the labeled data are considered: the classic and the blendersetting.
The former selects labeled data from the original 1,464 candidate labeled images while thelatter selects among all the 10,582 images.
We evaluate our method on both settings.
Cityscapes [10] is an urban scene understanding benchmark.
The initial 30 semantic classes arere-mapped into 19 classes for the semantic segmentation task.
The training, validation and testing setincludes 2,975, 500 and 1,525 finely annotated images respectively.
For both of these two datasets,four kinds of label partitions are considered: 1/16, 1/8, 1/4 and 1/2.
In this paper, we compare allmethods under the identical released label splits from U 2 PL [38] for a fair comparison.
Evaluation: We use single scale cropping for the evaluation of PASCAL VOC 2012 and slide windowevaluation for Cityscapes for its high resolution.
The mean of Intersection over Union (mIoU) isadopted as the evaluation metric.
All numbers reported in this paper are measured on the validationset of these two datasets.
Methods: We compare our approach with several peer-reviewed semi-supervised segmentationalgorithms: Mean Teacher (NeurIPS 2017) [35], CutMix-Seg (BMVC 2020)
[13], PseudoSeg (ICLR2020) [49], CCT (CVPR 2020) [30], GCT (ECCV 2020) [20], CPS (CVPR 2021) [8], PC 2 Seg(ICCV2021) [45], AEL (NeurIPS 2021)
[19] and U 2 PL (CVPR 2022)
Meanwhile, performance ofsupervised only on labeled data is also reported for a reference baseline.
To make a fair comparison,we conduct all experiments based on the same codebase released by the authors of U 2 PL [38].
Implementation Details: Following the common practice, we use ResNet-101 [17] pre-trained on
ImageNet [22] as our backbone and DeepLabv3+ [7] as the decoder.
We take the default segmentationhead as the pixel-level linear classifier.
The feature representations for constructing the prototypes of
[Table] [Table]
AEL [19] 77.
U 2 PL [38] 77.
Ours 78.
our approach are extracted from the output of ASPP module [6].
Our experiments were run on 8 *
NVIDIA Tesla V100 GPUs (memory is 32G/GPU).
For both datasets, we adopt stochastic gradient descent (SGD) as the optimizer and set batch size to16 for model optimization.
While other training details are slightly different, e.g., PASCAL VOC2012 is trained with initial learning rate 1 .
× 10 − 3 , weight decay 1 .
× 10 − 4 and 80 training epochs;while Cityscapes is trained with initial learning rate 1 .
× 10 − 2 , weight decay 5 .
× 10 − 4 and 200training epochs.
Meanwhile, we use the polynomial policy to dynamically decay the learning ratealong the whole training: lr = lr init · (1 − iter totaliter ) 0 . 8 .
4.2 Comparison with State-of-the-Arts
Results on PASCAL VOC 2012 Dataset [12]: Table 1 and Table 2 report the comparison resultson PASCAL VOC 2012 validation set under different label quality settings.
First, the results in
Table 1 are obtained under the classic setting and our approach achieves consistent performanceimprovements over the compared methods.
Specifically, our method outperforms the Supervised
Only baseline by a large margin especially for the fewer data settings, e.g., +24.29% for 1/16 and+19.79% for 1/8 setting respectively.
Meanwhile, our approach also successfully beats other semisupervised methods.
Taking the recently proposed state-of-the-art method U 2 PL [38] as an example,the performance gain of our approach reaches to +5.56% and +3.50% mIoU improvements under1/8 and 1/4 label partitions, respectively.
Table 2 presents comparison results on the blender setting.
It is clear that our proposed method stillachieves overall significant improvement over all other baselines.
For example, our method excelsto the Supervised Only baseline over 10% mIoU on the 1/16 split.
Compared with previous wellperformed algorithms, e.g., AEL [19] and U 2 PL [38], our approach yields superior segmentationperformance, e.g., +1.39% , +1.70%
and +1.48% on 1/16, 1/8 and 1/4 label partitions respectively.
Results on Cityscapes Dataset [10]: Table 3 provides comparison results of our method againstseveral existing algorithms on Cityscapes validation set.
Compared to Supervised Only baseline, our
[Table]
method achieves a great performance improvement due to the make use of unlabeled data, e.g., under the 1/16 label partition, our approach surpasses Supervised Only baseline by 7.67% .
Then, comparedto the simple Mean Teacher [35] baseline, our approach also performs better in all cases.
Furthermore,our approach is superior than the state-of-the-art algorithm U 2 PL [38], e.g., Ours excels to U 2 PL by3.11% , 1.94% and 1.93% under the 1/16, 1/8 and 1/4 label partition, respectively.
Note that our method performs slightly worse than AEL [19] on the 1/16 label partition, it is becausethe class imbalance issue is more severe on this partition, and the AEL method, which is speciallydesigned for handling the class imbalance problem, thus gains greater improvement.
Since thepurpose of this paper is to explore the new consistency loss to alleviate intra-class variation for thesemi-supervised semantic segmentation task, we do not explicitly consider measures to handle thelabel imbalance issue.
Theoretically, the techniques for solving label imbalance issues can also beincorporated into our method for optimizing the overall performance.
4.3 Ablation Study
To investigate how our approach works on the semi-supervised semantic segmentation task, weconduct ablation studies on the classic PASCAL VOC 2012 setting under 1/16 and 1/8 partitionprotocols from the following perspective views:
Effectiveness of different components : Table 4 presents ablation studies of several variants of ourapproach based on the usage of different components.
The variant ④ , which uses of all components, isthe default setting of our approach and is presented here for a reference.
The variant ① only containsa linear predictor and the prototype-based predictor is omitted.
It is clear that the performance of thisvariant drops a lot compared to our approach and this proves that the prototype-based predictor playsa key role in our approach.
On the contrary, the variant ② only maintains a prototype-based predictor
[Figure]
linearpred
[Table]
and dynamically updates the prototypes during the training.
The corresponding results are shown tobe the worst among all the compared variants in Table 4.
We postulate the potential reason is that theprototype-based predictor itself is not good enough to generate high quality pseudo-labels withoutthe help of the linear classifier under the limited labeled data setting and thus cannot fully leveragethe large amount of unlabeled samples.
The variant ③ ablates the necessity of prototype update inour approach and the performance gap between this variant and variant ④ shows that our approachwill benefit from the prototype update procedure and produce overall best performance.
Distribution of feature representation :
The core idea of introducing prototype-based predictor inour approach is to utilize the prototype-based consistency regularization for alleviating the strongintra-class variation problem in semi-supervised semantic segmentation.
Therefore, we are interestedin the influence of our method on feature distribution.
Figure 2 presents the feature distribution ofvarious methods for some classes of Pascal VOC 2012.
We can find that our method tends to producemore compact feature distributions than other compared methods for every semantic class and suchcompact feature will ease the label information propagation from pixels to pixels and thus weakenthe influence of intra-class variation.
Number of prototype : For the prototype-based classifier, the number of prototype is not restrictedto be equal to the number of classes.
In our approach, we construct multiple prototypes for eachsemantic class to handle the intra-class variation problem of semi-supervised semantic segmentationtask.
In order to explore the influence of the number of prototypes on our method, we conduct ablationstudies on our approach with different number of prototypes.
As the results shown in Figure 3, theperformance is tend to be saturate when the prototype number reaches to 4 for each semantic class.
Therefore, we empirically take this number as the default value of our approach.
Limitations
One underlying assumption about our approach is that we mainly consider convolutional based semantic segmentation networks.
Recently transformer-based algorithms [9, 39] are being investigatedfor semantic segmentation that are not explored in this paper and is left for future work.
Conclusion
Semi-supervised semantic segmentation aims to propagate label information from pixels to pixelseffectively, but the large intra-class variation hinders the propagation ability.
In this paper, weintroduce a prototype-based predictor into our semi-supervised semantic segmentation network andpropose a novel prototype-based consistency loss to regularize the intra-class feature representationto be more compact.
Experimental results show that our method successfully achieves superiorperformance than other approaches.
Impacts and Ethics
This paper proposes a method for semi-supervised semantic segmentation which is a fundamentalresearch topic in computer vision area and no potential negative societal impacts are known up tonow.
In terms of ethics, we do not see immediate concerns for the models we introduce and to thebest of our knowledge no datasets were used that have known ethical issues.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] See Section
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] See Section
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes] See Section
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[No] The code will
be released after the anonymity period and the data used in our paper are open source
datasets.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] See Sections 4.1 and 4.3.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)?
[No] All compared algorithms are experimented on the same [Table] random seed for a fair comparison in our paper.
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See Sections 4.14.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] See Sections 4.
(b) Did you mention the license of the assets?
[Yes] See Sections 4.
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
[Table]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[No] Data used in this paper has been open source for research purpose.
[Table]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
[Table]
information or offensive content?
[No] Data used in this paper has been open source
for research purpose.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]