Expansion and Shrinkage of Localization for Weakly-Supervised Semantic Segmentation
Anonymous Author(s)
AffiliationAddress
Abstract
Generating precise class-aware pseudo ground-truths, a.k.a , class activation maps(CAMs), is essential for weakly-supervised semantic segmentation.
The original
CAM method usually produces incomplete and inaccurate localization maps.
Totackle with this issue, this paper proposes an Expansion and Shrinkage schemebased on the offset learning in the deformable convolution, to sequentially improvethe recall and precision of the located object in the two respective stages.
Inthe Expansion stage, an offset learning branch in a deformable convolution layer,referred as “expansion sampler” seeks for sampling increasingly less discriminativeobject regions ， driven by an inverse supervision signal that maximizes image-levelclassification loss.
The located more complete object in the Expansion stage is thengradually narrowed down to the final object region during the Shrinkage stage.
Inthe Shrinkage stage, the offset learning branch of another deformable convolutionlayer, referred as “shrinkage sampler”, is introduced to exclude the false positivebackground regions attended in the Expansion stage to improve the precision ofthe localization maps.
We conduct various experiments on PASCAL VOC 2012and MS COCO 2014 to well demonstrate the superiority of our method over otherstate-of-the-art methods for weakly-supervised semantic segmentation.
Introduction
Image semantic segmentation is the task of pixel-level semantic label allocation for recognizingobjects in an image.
The development of Deep Neural Networks (DNNs) has promoted the rapiddevelopment of the semantic segmentation task [6, 58, 19] in recent years.
However, training sucha fully-supervised semantic segmentation model requires large numbers of pixel-wise annotations.
Preparing such a segmentation dataset needs considerable human-labor and resources.
Recently,researchers have studied weakly-supervised semantic segmentation (WSSS) methods to alleviatethe issue of high dependence on accurate pixel-level human annotations for training semanticsegmentation models under cheap supervision.
Weak supervision takes the forms of image-level [51,2, 48, 50, 20], scribbles [33, 46] or bounding box [10, 26, 30, 43].
In this paper, we focus on WSSSmethod based on image-level labels only, because it is the cheapest and most popular option ofweak supervision annotation which only provides information on the existence of the target objectcategories.
Most WSSS methods utilize class labels [51, 2, 48, 50, 20] to generate pseudo ground-truths fortraining a segmentation model obtained from a trained classification network with CAM [59] or
Grad-CAM [40] method.
However, image-level labels can not provide specific object position andboundary information for supervising the network training, resulting in that these localization mapsidentify only local regions of a target object that are the most discriminative ones for the classificationprediction.
Therefore, with the incomplete and inaccurate pseudo ground-truths, training a fully- supervised semantic segmentation network to reach a decent segmentation performance is challenging.
Existing WSSS methods usually attempt to gradually seek out more less discriminative object regionsstarting from the very small and local discriminative regions [5, 2, 1].
Differ from the existing works,in this paper, we attack the partial localization issue of the CAM method with a novel deformabletransformation operation.
We empirically observe that the classification models can re-discover morediscriminative regions when we fix a trained classifier and equip the network with more “sampling”freedom to attend to other less discriminative features.
This inspires us to explore a proper wayto improve the quality of the initial localization maps via a new training pipeline, Expansion and
Shrinkage , shown in Figure 1.
[Figure]
The Expansion stage aims to recover the entire object as much as possible, by sampling the exteriorobject regions beyond the most discriminative ones,to improve the recall of the located object regions.
We introduce a deformable convolution layer after the image-level classification backbone, whoseoffset learning branch serves as a sampler seekingfor sampling increasingly less discriminative objectregions ， driven by an inverse image-level supervisionsignal.
We call this newly embedded deformable convolution layer “expansion sampler” (ES).
During theinverse optimization process, the backbone is frozento provide fixed pixel-wise features obtained in the image-level classification training to be sampledby the offset learning branch in the ES.
In this way, the inverse supervision target solely enforcesthe offset learning in the ES branch to optimize its sampling strategy to gradually attend to the lessdiscriminative regions, given that the pixel-wise features cannot be changed.
Hence, the image-levelloss maximization allows the network to pay more attention to the less discriminative regions viadeformation transformation achieved by ES in the inverse optimization, which are easily ignoredduring the normal image-level classification task.
Having obtained the high- recall object region after the Expansion stage, we propose a Shrinkagestage to exclude the false positive regions and thus further enhance the precision of the located objectregions.
The Shrinkage stage remains the same network architecture as in the Expansion stage exceptthat an extra deformable convolution layer, referred as “shrinkage sampler” (SS), is introduced tonarrow down the object region from the high- recall one.
However, we observe a feature activationbias issue, i.e. , the initial most discriminative parts are more highlighted in the feature map after the
ES in the Expansion stage while the newly attended regions have much weaker feature activation.
Such activation bias serves as prior knowledge which encourages the later shrinkage to converge tothe same discriminative parts as the initial highlighted regions in the original CAM.
To alleviate suchan issue, we propose a feature clipping strategy after the ES in the Expansion stage of training tonormalize pixel-wise feature values, allowing each pixel to have a relatively fair chance to be selectedby the SS in the Shrinkage stage.
Similarly, in the training of Shrinkage stage, all the layers beforethe SS are fixed to provide stable pixel-wise features and only the offset learning branch in the SSis updated to sample the true positive pixels, optimized by the standard image-level classificationsupervision.
The main contributions of this study are summarized as follows.
First, this paper proposes an
Expansion and Shrinkage scheme to sequentially improve the recall and precision of the locatedobject in the two respective stages, leading to high-quality CAM which can be used as strong pseudoground truth masks for WSSS.
Second, both the Expansion and Shrinkage stages are realized bycarefully applying deformable convolution combined with two contrary training signals.
To avoidthe repeated convergence to the initial discriminative parts, a feature clipping method is applied toalleviate the activation bias of these regions.
Third, our approach significantly improves the quality ofthe initial localization maps, exhibiting a superior performance on the PASCAL VOC 2012 and MS
COCO 2014 datasets for WSSS.
Related Work2.1 Weakly-Supervised Semantic Segmentation
Weakly-supervised semantic segmentation pipeline [25, 20] with image-level labels only mostlyconsists of two steps: pseudo ground-truths generation and segmentation model training [2, 1, 7, 51,
Erasure methods [50, 18, 42] applied various iterative erasing strategies to prevent theclassification network from focusing only on the most discriminative parts of objects by feeding theerased image or feature maps to the model.
MDC [51], layerCAM [23] and FickleNet [28] aggregateddifferent contexts of a target object by considering multiple attribution maps from different dilatedconvolutions or the different layers of the DCNNs.
Some works utilized diverse image contextsto explore cross-image semantic similarities and differences [15, 45].
CONTA [56] analyzed thecausalities among images, contexts and class labels and used intervention to remove the confoundingbias in the classification network.
Recently, Anti-Adv [29] utilized an anti-adversarial manipulationmethod to expand the most discriminative regions in the initial CAMs to other non-discriminativeregions.
RIB [27] used the information bottleneck principle to interpret the partial localization issueof a trained classifier and remove the last double-sided saturation activation layer to alleviate thisphenomenon.
However, the generated location maps obtained by the classifier cannot reveal the entireobject areas with accurate boundaries, the initial CAM seeds obtained using the methods above werefurther refined by a subsequent refinement network [2, 1, 7].
In this paper, we also follow this pipelineand our contribution is to propose a new training pipeline to generate high-quality localization maps.
Different from MDC [51] using multi-dilated convolutions to combine multiple contexts for betterfeature mining, our method uses deformation transformation for less discriminative feature discovery.
2.2 Deformation Modeling
We refer to deformation modeling as learning geometric transformations in 2D image space withoutregarding to 3D. One popular way to attack deformation modeling is to craft certain geometricinvariances into networks.
However, to achieve this usually needs specific designs for certain kinds ofdeformation, such as offset shifts, rotation, reflection and scaling [41, 3, 24, 9, 52, 12].
Another lineof this work on deformation modeling task learns to recompose data by either semi-parameterized orcompletely free-form sampling in image space.
STN [21] learnt 2D affine transformations to constructfeature alignment.
Deformable Convolutions [11, 60] applied learnable offset shifts for better featurelearning in free-from transformations.
In WSSS community, applying deformation modeling is stillless explored.
In this paper, we utilize deformation transformation to act as a feature “sampling” tore-discover other non-discriminative regions, instead of better feature representation learning.
Proposed Method
Weakly-supervised semantic segmentation methods use given class labels to produce a pixel-levellocalization map from a classification model using CAM [59] or Grad-CAM [40].
We first give somefundamental introduction to localization maps generation with CAM [59] in Section 3.1.
Then, wepresent the whole framework of our method, Expansion and Shrinkage with Offset Learning(ESOL) , to obtain high-quality localization maps covering more complete and accurate target objectparts in Section 3.2 and Section 3.3, respectively.
We then explain how we train the final semanticsegmentation model with generated localization maps in Section 3.4.
3.1 Prerequisties
We first present the way to generate localization maps via the CAM [59].
A class activation map ofa target object focuses on the regions of an image by a trained classification network for a specificcategory prediction.
The CAM is based on a DCNN with a global average pooling (GAP) before itsfinal classification layer, which is trained by a sigmoid cross-entropy loss function, formulated asfollows:
[Equation]
135 where C is the total number of training classes, i represents the i th training sample in the mini-batch,y
[ i ] is the ground truth label of i
th class with the value of either 0 or 1 while ˆ y
[ i ] is the modelprediction.
The localization maps is realized by considering the class-specific contribution of eachchannel of the last feature map, before the GAP layer, to the final classification prediction.
Given atrained classifier network parameterized by θ = { θ f , w } where f (: , θ f ) is the feature extractor, andw denotes the weight of the final classification layer.
For some class c , the localization map is thencomputed from an input image x as follows:
[Figure]
Expansion
Shrinkage
[Figure]
[Equation]
143 where max( · ) is the maximization over the spatial locations for normalization.
The above method canonly locate the most discriminative regions and fails in locating other less discriminative regions thatare semantically meaningful as well.
In the following work, we elaborate our method by presenting anew training pipeline to capture high-quality object localization maps.
3.2 Expansion
As mentioned above, the localization maps generated by a commonly trained classifier usuallystruggle with the partial localization issue of the target objects, since the image-level labels cannotprovide detailed position or boundary information.
To alleviate this phenomenon, we devise a newtraining pipeline, Expansion, to firstly recover the entire object regions as much as possible
so as toimprove the recall of the located object regions.
Then, we further introduce another training scheme,
Shrinkage, to exclude false positive regions, e.g., background regions, to enhance the precision of thelocated regions.
A commonly trained classifier usually considers only the local regions that make the most contributions to final classification prediction.
Differ from other works, we first enforce the network to seekout the entire target object regions via our proposed Expansion scheme with an offset learning branchin a deformable convolution layer, as shown at the top of Figure 2.
A trained classifier is utilized toprepare our Expansion scheme firstly providing the regular convolutional weights in a deformableconvolution layer, that can capture the most discriminative feature activation, e.g., conv _ 1 or conv _ 2 .
Given an input image x , the classification network θ = { θ f , w } locates the coarse object regions.
Fora specific convolution layer, conv _ 1 with weights w 1 , the output feature maps F 1 is computed foreach location p 0 as follows:
[Equation]
where R defines a specific kernel size with dilation 1 and p n enumerates the locations in R .
For
Expansion scheme, a embedded deformable convolution layer is introduced after the feature extractorf (: , θ f ) .
The new feature maps F ′ 1 is then computed as:
[Equation] [Figure]
168 where the regular grid R is augmented with offset fields { ∆ p 1 n | n = 1 , ..., N } , N denotes the numberof offset points, e.g., 1 × 1 or 3 × 3 .
The learned ∆ p 1 n obtained from the offset learning branch inthe deformable convolution layer further acts as “expansion sampler” to sample the exterior objectregions gradually beyond the most discriminative ones.
This training scheme utilizes an image-levelloss maximization to enforce the network seek for increasingly less discriminative object regions,given unchanged pixel-wise features that is implemented by detaching the loss back-propagation tothe backbone.
The training loss function L expansion then becomes:
[Equation]
176 where α controls the loss weight for updating and L (ˆ y, y ) is the corresponding multi-label classification loss function mentioned in Eq 1.
3.3 Shrinkage
After the Expansion stage, though most possible target object regions are sampled, the backgroundsare also included.
This would inevitably cause imprecise localization maps generation, which resultsin poor quality pseudo ground-truths and hampers the final segmentation performance.
To enhancethe precision of such high-recall regions, Shrinkage scheme is proposed to exclude the false positiveregions of the localization maps.
As shown at the bottom of Figure 2, the Shrinkage stage remains thesame network architecture as in Expansion stage, except that an extra deformable convolution layeris implemented to narrow down the high-recall regions.
Specifically, the model weights obtainedfrom the Expansion are used to initialize the Shrinkage model and a loss minimization is adopted totrain the network, including a multi-label classification loss and an area loss.
Area regularization isadopted to constraint the size of the localization maps to ensure that the irrelevant backgrounds areexcluded in the localization map P k :
[Equation]
191 where S c = 1 HW (cid:80)
Hh =1 (cid:80) Ww =1 P k ( h, w ) , C is the total class category numbers of the dataset, H and
W denotes the height and width of the localization maps, respectively.
We empirically observe that the feature activation bias issue is obvious, the initial most discriminativeparts got more highlighted after ES in the Expansion stage while the newly attended ones haveweaker feature activation.
This activation bias would encourage the later shrinkage to seek for thesame discriminative parts as the initial activation parts, even more local.
To address such an issue, afeature clipping strategy is then proposed after the ES in the Expansion stage to normalize pixel-wisefeatures, providing relatively fair chances for the pixels to be selected by SS in the Shrinkage stage.
The feature clipping strategy is formulated as follows:
[Equation] [Table]
201 where x i is the input feature maps over the spatial dimension, max( · ) is applied to obtain the maximalvalues along the spatial dimension as well.
We present examples of PASCAL VOC 2012 [13] training images for better demonstration shownin Figure 3.
During the Expansion stage, the image-level loss maximization enforces the offsetlearning branch attend on other less discriminative regions, indicating that the ES indeed serves as asampler locating the entire object regions, thanks to the deformation transformation in the deformableconvolution layer.
3.4 Pseudo Ground-truth Generation
For example, the body or legs of the cat or cow shownin Figure 3 is successfully activated by our Expansion scheme, and the overall recall of foregrounds issignificantly improved among diverse hard-thresholdsettings.
We obtain the final localization map M by our final Shrinkage trained model via CAM [59] method.
Since a CAM is computed from down-sampled intermediate feature maps produced by a classifier,it has to be up-sampled to match the size of theoriginal image.
Thus, it tends to localize the target object coarsely and cannot cover the entireregions with exact boundary.
Many WSSS methods [4, 47, 28, 49, 56] produce pseudo ground-truthsby extending their initial CAM seeds using anotherseed refinement methods [1, 2, 7].
Similarly, we obtain our final pseudo ground-truths using IRN [1], a state-of-the-art refinement method, to refine thecoarse map M for generating better segmentation model supervision.
And Figure 4 illustrates thefinal segmentation results on VOC2012 val set.
Experiments
4.1 Experimental Setup
Dataset and Evaluation metric: Experiments are conducted on two publicly available datasets,
PASCAL VOC 2012 [13] and MS COCO 2014 [34].
The Pascal VOC 2012 dataset contains 20foreground categories and the background.
It has three sets, the training, validation, and test set, eachcontaining 1464, 1449 and 1456 images, respectively.
Following most previous works [4, 47, 28, 49,56], we also adopt the augmented training set [16] to yield totally 10582 training images.
The MS
COCO 2014 dataset has 80 foreground categories, including approximately 82K training images and4K validation images.
We evaluate our method on 1449 validation images and 1456 test images from [Figure] the PASCAL VOC 2012 dataset and on 40504 validation images from the MS COCO 2014 datasets.
The mean intersection-over-union (mIoU) [37] is used as the evaluation metric.
Implementation Details: We implement CAM [59] by following the procedure from Ahn et al.
[1],implemented with the PyTorch framework [39] on 12G Nvidia XPs.
We adopt the ResNet-50 [17] asbackbone for the classification model.
To prepare the regular convolutional weights for the deformableconvolution layers, we opt to follow the common settings to train a classifier for our method, exceptthat we add two additional convolution layers ( 3 × 3 or 1 × 1 ) that would be used to perform thedeformable transformation.
For Expansion stage, we train the network 6610 iterations for PASCAL
VOC 2012 and 51730 iterations for MS COCO 2014.
To carefully train the model with a lossmaximization, we set a relatively small learning rate, 0.01 and 0.001 for PASCAL VOC 2012 and MS
COCO 2014, while the controlling parameter α is set to 0.001.
Noting that the loss gradients are notallowed to back-propagate to the backbone, as we want to provide pixel-wise unchanged feature mapsfor ES and SS.
For Shrinkage stage, the network is initialized from the Expansion model weights.
The learning rate is set to be 0.1 and 0.02, and the training iteration is set to be 6610 and 51730 for
PASCAL VOC 2012 and MS COCO 2014, respectively.
The threshold value of the hand-craft featureclipping strategy is 0.15.
The γ and µ are set to be 1.0.
To generate reliable initial localization maps,the scale ratio of multi-scale CAM is { 0 . 5 , 1 . 0 , 1 . 5 , 2 . 0 } .
During testing, DenceCRF [26] is used aspost-processing to refine the generated localization maps.
For the final semantic segmentation, weuse the PyTorch implementation of DeepLab-v2-ResNet101 provided by [38].
4.2 Weakly-supervised Semantic Segmentation
4.2.1 Quality of the initial localization maps and pseudo ground-truths
[Table]
PASCAL VOC 2012 dataset: In Table 1, wereport the mIoU values of the initial localization maps (seed) and pseudo ground-truthsmasks produced by our method, and other recent WSSS techniques.
Following common practice [49, 2, 1, 5], we perform a range of hardthreshold values to distinguish the foregroundand the background regions in localization maps
M to determine the best initial seeds result.
Ourinitial seeds improves 5.2% mIoU over the original CAM seeds (48.4 to 53.6), a baseline for comparison, and outperform those simultaneously fromthe other methods. Noting that our initial seeds are superior to those of CDA [44] or SC-CAM [5],
[Table] [Table]
which applied a complicated context decoupling augmentation on the network training or adoptedsub-category exploration to enhance the feature representation via an iterative clustering method.
After obtaining the initial seeds based on CAM [59], we then apply a post-processing method basedon Conditional Random Field (CRF) [26] for pixel-level refinement on the results from the methodproposed by SC-CAM [5],CDA [44], IRN [1], and our method.
We can observe that applying CRFpost-processing improves all the initial seeds over 5% mIoU. When the seeds produced by our methodis then refined with CRF, it obtains 7.8% mIoU better than that from the original CAM (53.6 to61.4), and consequently outperforms all the recent competitive methods by a large margin.
We thencompare the pseudo ground-truths masks generated after seed refinement with other methods.
For afair comparison, we compute our pseudo ground-truths masks using both seed refinements, PSA [2]or IRN [1].
Table 1 illustrates the masks results from our method yield 66.4% mIoU with PSA [2]and 68.7% mIoU with IRN [1], respectively.
Figure 5 visualizes initial localization maps adopted different components or training phases for the
PASCAL VOC 2012 dataset.
And the visualizations demonstrate the effectiveness of the proposed
Expansion and Shrinkage approach, sequentially balancing the recall and precision of the initiallocalization maps.
More examples are shown in the Appendix.4.2.2 Performance of the Weakly-supervised Semantic Segmentation
PASCAL VOC 2012 dataset: Table 3 shows the segmentation performance (mIoU) on PASCAL
VOC 2012 validation and test set.
We illustrate the results predicted by our method and other recentlyproposed WSSS methods, which use either bounding box or image-level labels.
All the segmentationresults in Table 3 were implemented by a ResNet-based backbone [17].
Our proposed method achieves69.9% and 69.3% mIoU values for the validation and test set on the PASCAL VOC 2012 dataset,outperforming all the WSSS methods that utilize image-level class labels only.
In particular, our method surpasses CONTA [56], a state-of-the-art WSSS competitors recently,obtaining 66.1% mIoU. CONTA adopted SEAM [49], which is applied with WiderResNet-based [53]backbone that is known to be more powerful than IRN [1] based on ResNet-based.
When it wasimplemented with IRN [1] for a fair comparison with our method, its segmentation performance gotonly 65.3%, which is inferior to ours by 3.9% mIoU.
Recently, saliency map cues are introduced to supervise the network training for better localizationperformance, since the offline saliency maps provide detail foreground boundary priors.
In Table 4,we also compare our method with other methods using additional salient object supervision.
Wecombine our final pseudo ground-truths masks with saliency maps produced by Li et al.
[32] or Yaoet al.
The pixels that are considered as foreground are identified as background, or regardedas background are identified as foreground, we simply set them to be 255 on these pseudo groundtruths maps.
Because the semantic segmentation training would ignore these pixels for cross-entropyloss.
We can see that our method achieves 71.1% and 70.4% mIoU values for PASCAL VOC 2012validation and test set, respectively, consistently outperforming all other methods under the salientobject supervision.
[Figure]
MS COCO 2014 dataset: Table 2 illustrates the segmentation performance on MS COCO 2014compared with other methods.
Our method achieves 42.6% in terms of the mIoU values on validationset, surpassing 1.2% over the IRN [1], regarded as our baseline and outperforming the other recentcompetitive methods [8, 56, 49, 1] by a large margin.
In particular, we reproduce different resultsof IRN [1] with CONTA [56], in which we achieve 41.4% mIoU values.
Hence, we compare therelative improvements for comparison: CONTA reaches a 0.8% mIoU improvement compared with
IRN (32.6 to 33.4), while our method achieves 1.2% mIoU improvement (41.4 to 42.6).
4.3 Ablation Studies
In this section, we conduct various ablation experiments on PASCAL VOC 2012 dataset to validatethe effectiveness of each component or training scheme of our method.
4.3.1 Expansion Training Analysis
Loss Maximization Controller α : To analyze the influence of the loss maximization controller α onthe
Expanding training sensitivity, we conduct a range from 0.001 to 0.1 for α as shown in Figure 6(a).
We found that too small or large α values degrade to attend on backgrounds extremely, showing alower foreground recall ratio.
We choose appropriate value: α = 0 .
01 , which balances the recall andprecision of the localization maps.
Some samples are visualized in the Appendix.
Feature Clipping Strategy Effect: In Expansion stage, we tend to provide high-recall foregroundswith a relatively fair chance to be sampled by Shrinkage.
The feature clipping strategy is introducedafter the ES and we study the impact of hand-craft threshold settings in Figure 6 (b).
4.3.2 Shrinkage Training Analysis
Impact of the Loss Functions: Although Expansion training brings high-recall foreground regions,backgrounds cannot be ignored.
A classification loss and an area loss are used to optimize the networkto select true foreground pixels via a loss minimization.
We provide the ablative study in Figure 6 (c)to demonstrate the impact of each one and find that both of them are useful for the network training,constraint the size of the localization maps to ensure that the irrelevant backgrounds are excluded inthe localization maps P k .
Loss Minimization Controller γ and µ : The sensitivity of these two hyper-parameters are performedshown in Figure 6 (d).
It is observed that the performances of our approach are stable with thevariation of γ (from 0.1 to 5) and µ (from 0.01 to 1.5), i.e., our method is not sensitive to such twohyper-parameters.
In our experiments, the default values of γ and µ are 1.0 and 1.0 simply.
Conclusion
In this paper, we explore a deformation transformation operation to address the major challenge inweakly-supervised semantic segmentation with image-level class labels.
A novel training pipelinefor the WSSS task, Expansion and Shrinkage, is proposed to first recover the entire target objectregions as much as possible while the network is driven by an inverse image-level supervision.
Then,we apply a feature Clipping operation to provide even high-recall regions for Shrinkage to samplehigh-precision regions, while the network is optimized by two loss functions, classification loss andarea loss.
Our method significantly improves the quality of the initial localization maps, exhibiting acompetitive performance on the PASCAL VOC 2012 and MS COCO 2014 datasets.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[No]
(c) Did you discuss any potential negative societal impacts of your work?
[No](d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[N/A]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] Please refer to Section 4.1.
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] Please refer to Section 4.1.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]
A Appendix