Unsupervised Object Detection Pretraining with Joint Object Priors Generation and Detector Learning
Abstract
Unsupervised pretraining methods for object detection aim to learn object discrimination and localization ability from large amounts of images.
Typically, recent works design pretext tasks that supervise the detector to predict the defined object priors.
They normally leverage heuristic methods to produce object priors, e.g., selective search, which separates the prior generation and detector learning and leads to sub-optimal solutions.
In this work, we propose a novel object detection pretraining framework that could generate object priors and learn detectors jointly by generating accurate object priors from the model itself.
Specifically, region priors are extracted by attention maps from the encoder, which highlights foregrounds.
Instance priors are the selected high-quality output bounding boxes of the detection decoder.
By assuming objects as instances in the foreground, we can generate object priors with both region and instance priors.
Moreover, our object priors are jointly refined along with the detector optimization.
With better object priors as supervision, the model could achieve better detection capability, which in turn promotes the object priors generation.
Our method improves the competitive approaches by +1.3 AP , +1.7 AP in 1% and 10% COCO low-data regimes object detection.
Introduction
Object detection is a fundamental task in computer vision, whose goal is to predict a set of bounding boxes and category labels for all objects of interest in the images.
The success of modern detectors [4, 35, 22, 14, 27, 29, 31, 32, 41, 42] relies on the large-scale datasets with precious annotations which are very costly and even infeasible for enormous images available in the Internet.
Therefore, unsupervised pretraining methods [12, 3] for object detection are proposed to ease the burden of these human annotations, which enable the detectors to be fast deployed in the real world by finetuning the pretrained model with only a few annotated images.
While various unsupervised pretraining methods, e.g., SwAW [5], BYOL [18], DetCo [51], have been shown to learn good backbones for object detection [28, 52, 46, 37], only a few methods are able to pretrain the detection head, which is also a key component of a complete detection architecture [24, 20, 25].
To enable the localization of the pretrained detection head, a few recent
SenseTime
[Figure]
methods use hand-craft methods [58, 45, 11, 1] to generate object priors on unlabeled images to construct the pretext tasks.
Specifically, UP-DETR [12] and DETReg [3] generate object priors by randomly cropping patches and selective search [45], respectively.
Then, both of them require the detection model to learn to detect those object priors for pretraining.
Although these methods bring possibilities of pretraining detection heads, we argue that they suffer from two main drawbacks: (1) Both random cropping and selective search can only provide limited object-related supervision.
For random cropping, it does not explicitly focus on objects.
Selective search, as a hand-crafted and heuristic box generation method, relies on low-level features of input images to generate object priors, which is an extra time-consuming non-deep-learning process.
The supervision can not be progressively refined during training.
For example, the object priors of selective search are fixed through the pretraining process, which limits the upper bound of the methods.
In this paper, we propose a novel unsupervised object detection pretraining method that can generate object priors and learn the detector jointly.
In our method, we mine the supervision (object priors) based on the model itself, and show that the generated object priors can be jointly refined with detection optimization.
Inspired by DINO [6], the self-attention maps can generate region prior bounding boxes, as they highlight the foreground regions (see Fig. 1(a)).
Meanwhile, the output bounding boxes of the detector can generate instance prior bounding boxes, although may be located in the backgrounds (see Fig. 1(a)).
By combining knowledge in region priors and instance priors together, we can generate reliable object priors only from the detector model.
Furthermore, the object prior generation and detector optimization can be mutually evolved (see Fig. 1(b)).
Supervised by reliable object priors, the model can gradually learn better object localization information, producing better region and instance priors for better object priors.
Given the object priors generated along with detector optimization, the supervision signal for the detector learning shifts quickly at the beginning, which leads to undesirable learning instabilities.
In order to stabilize the pretraining process, both preceding and current object priors should be considered, and the update should be implemented in a slowly progressing way.
To this end, we accomplish this regularization by designing a momentum box update strategy.
Specifically, we match the current object priors to their neighboring and proceeding priors, and implement the update as a momentum-based average of their object prior locations.
Our contributions are three-fold.
(1) We propose to mine supervision from the knowledge of the model to be learned itself for unsupervised object detection pretraining, which can generate supervision signals.
(2) We design an unsupervised object detection pretraining method with Joi nt regio n priors generation and Det ector learning (JoinDet) which jointly generates reliable object priors and learns object detection in a progressive way.
(3) We propose a Box Smooth Module to stabilize the pretraining process on JoinDet.
Our method outperforms state-of-the-arts on low data-regime object detection on MS COCO, and full-data finetuning on PASCAL VOC.
Detailed related works are provided in supplementary materials.
[Figure]
Methodology
Our goal is to mine effective supervision from the knowledge learned by the model itself for pretraining without annotations.
To this effect, we propose an unsupervised object detection pretraining method with joint object priors generation and detector learning (JoinDet) which dynamically generates reliable object priors and learns object detection.
Specifically, we first extract the eigen attention map from the transformer to highlight foregrounds.
We then generate region priors from the eigen attention map using the Region Generation Module and choose reliable predictions as instance priors using the Instance Generation Module.
By combining region and instance priors, reliable object priors are generated and set as the pseudo-labels for the object detection pretraining process.
To stabilize the optimization process, we further propose a Box Smooth Module to smooth the refinement of object priors during pretraining.
During pretraining, the supervision is alternated between two steps: (1) generating reliable object priors by combining region priors and instance priors, including Step 1-3 below.
(2) optimizing the detection model with reliable object priors (Step 4).
Specifically,
Step1 : Generate region priors (Sec. 2.1).
At n -th epoch, given an image x , the output patch features
F of the transformer encoder E can be computed by F = E ( G ( x )) , where G is the backbone network.
To better highlight foregrounds, we generate the eigen attention map M with M = K ( F ) , which is detailed later.
The region priors P r can then be generated by the Region Generation Module .
Step2 : Generate instance priors (Sec. 2.2).
With the DETR-based detection frameworks, object queries q and patch features F are fed into a transformer decoder D together to generate foreground prediction boxes, i.e., P n = D ( F , q ) .
Guided by the eigen attention map M , instance priors can be generated by the Instance Generation Module denoted as IG , i.e., P in = IG ( P n , M ) .
Step3 : Smooth object priors refinement by the Box Smooth Module (Sec. 2.3).
Reliable object priors
R n can be generated by combining region priors P rn and instance priors P in with R n = P rn ⊕ P in .
Given R n and the last object priors R n − 1 , the momentum averaged object priors are computed by
R n ← BS ( R n , R n − 1 ) , where BS represents the Box Smooth Module .
The R 0 is initialized with the region priors P r 0 before pretraining the detector.
Step4 : Train the model with updated supervision using set prediction loss (Sec. 2.4).
Given model predictions and updated supervision R n , we train the transformer parameters using set prediction loss.
2.1 Region priors generation
There are two steps of region priors generation.
First, compute the eigen attention map on patch features from the transformer encoder.
Second, extract the foreground bounding boxes as region priors.
Generate the eigen attention map.
Our main motivation follows [47, 39] is that the eigen attention maps in the vision transformer can well highlight salient foreground objects.
Specifically, given an input image x and DETR with a backbone G , a transformer encoder E , the eigen attention map M is extracted from the transformer encoder by
[Equation]
where K represents the computation method of the eigen attention map mentioned in [47], and
F ∈ R h × w represents output patch features from transformer encoder.
The details of the eigen attention map computation method K are elaborated in supplementary materials.
Extract region priors.
As foregrounds are highlighted in the eigen attention map, the bounding boxes of foreground regions can be extracted as region priors using Region Generation Module
RG .
Concretely, M is firstly partitioned into background parts (denoted by 0) and foreground parts (denoted by 1) by a bi-partition mask m :
[Equation]
where i , j represent the position index, α = 1 hw (cid:80) hi =1 (cid:80) wj
=1 M ij is the average score of M .
Region priors P r are then generated by computing the minimum bounding rectangles of all connected foreground regions.
In addition, at the beginning of pretraining, to generate the initial supervision R 0 , we compute P rusing the unsupervised pretrained backbone G by
[Equation]
Although region priors represent highlighted foregrounds in the eigen attention map, they do not divide foregrounds into foreground instances, and therefore instance priors should be mined.
2.2 Instance priors generation
The foreground predictions from the box prediction branch of DETR provide instance information.
With object queries q and output patch features F passing to transformer decoder D , DETR generates a set of box predictions P by
[Equation]
where N denotes the number of output predictions and ˆ b j , ˆ p j denotes the predicted bounding box, and logits in foreground-background predictions, respectively.
The prediction score ˆ s j can be computed by ˆ s j = Softmax (ˆ p j ) .
As the model only predicts background (class label 0) and foreground (class label 1), we only reserve foreground predictions P f .
[Equation]
Although foreground predictions can define instances, some of them might correspond to the background, which should be removed by the Instance Generation Module to improve reliability.
Instance Generation Module IG , in which foreground predictions
P f and the eigen attention map
M interact together to eliminate background predictions and generate instance priors P i .
Specifically, boxes with low prediction scores in P f are removed with
[Equation]
where P f j is the element in P f , N is the number of elements in the P f , N f is the number of foreground pixels in the eigen attention map M bounded by P fj , N t is the number of total pixels in M bounded by P fj .
Here, t is a threshold set to 0.5.
Guided by highlighted foregrounds in M , background predictions in P f can be removed.
The remaining predictions are treated as reliable instance priors P i .
2.3 Box Smooth Module
Given generated region priors P r and instance priors P i , reliable object priors R can therefore be generated by
[Equation]
where ⊕ denotes the concatenation operation.
We use reliable object priors R to supervise the pretraining process.
As region priors P r and instance priors P i are dynamically updated along with network optimization, the fast evolution of reliable object priors R may lead to the instability of the training process.
Motivated by Weight Box Fusion [40], we propose a Box Smooth Module to overcome these problems, which preserves accurate object priors from previous supervision and reduces the perturbation using a momentum update strategy.
Specifically, given current object priors R n , together with previous supervision R n − 1 , we cluster all boxes in R n and R n − 1 using IoU metric to generate K box clusters.
We recompute the box coordinates and scores using all boxes in the k -th cluster to generate a new box b k .
The supervision R n = { ( b k , s k ) } Kk =1 is then updated in a momentum strategy by
[Equation]
where k -th cluster contains T boxes in R n − 1 and U boxes in R n , and m s ∈ [0 , 1] represents the momentum coefficient to adjust the influence of previous supervision R n − 1 and current object priors
R n .
m s is empirically set as 0.45.
When m s is set to 1, the supervision will not be updated.
When m s is set to 0, the supervision will be directly changed to current object priors R n .
When initializing the pretraining ( n = 0 ), we set R
= P r 0 as the initial supervision.
With such a design, we smooth the refinement of object priors to stabilize the training process.
Discussion.
As no labels are used to supervise the model, generated region priors and instance priors have relatively large variations between training epochs.
The Box Smooth Module is a critical design in the framework, which suppresses inaccurate previous supervision and slows down the shifting speed of supervision to guarantee stable optimization of the model.
Note that Weighted Box Smooth is just a simple implementation to refine the supervision in a momentum update strategy, which is different from its original use to ensemble boxes from different object detection models.
2.4 Objective function
Here, we describe the objective function of JoinDet during pretraining.
Following DETReg [3],
JoinDet has three prediction heads: f cls which predicts the class (foreground or background), f box which predicts bounding boxes, and f emb which reconstructs features of the predicted regions.
Assume K boxes { ( b k ) }
Kk =1 left after the Box Smooth Module, use a pretrained SWAV [5] model to extract features Z = { z k } Kk =1 of box regions, assign class label c
k
= 1 , and let targets y =
{ y k
=
( c k , b k , z k ) }
Kk =1 .
Let { v j } Nj =1 denote N query embedding outputted by transformer decoder, the JoinDet outputs can be denoted as: ˆ p j = f cls ( v j ) , ˆ b j = f box ( v j ) , ˆ z j = f emb ( v j ) .
The model predictions can be defined as ˆ y =
{ ˆ y j
=
(ˆ p j , ˆ b j , ˆ z j ) } Nj =1 .
Following DETR [4], we use Hungarian bipartite matching algorithm [26] to match y j and ˆ y j by finding the permutation σ that minimizes the optimal matching cost between y j and ˆ y j :
[Equation]
where L match is the pairwise matching cost matrix.
After finding the optimal σ , the set prediction loss is defined as:
[Equation]
[Equation]
where L cls is the classification loss which can be implemented via the cross-entropy loss or Focal Loss,
L box is based on L 1 loss and GIoU Loss [36].
Following DETReg, we set L emb as L emb ( z j , ˆ z ˆ σ ( j ) ) =
∥
z j − ˆ z ˆ σ ( j ) ∥ 1 .
[Table]
Experiment 3.1 Implementation details
Architecture.
We choose Deformable-DETR [57] as the baseline model for its high accuracy and fast convergence.
Following DETReg [3], in prediction heads, f box and f emb are multi-layer perceptrons (MLPs) with 2 hidden layers of size 256 followed by a ReLU [34] layer.
The output dimensions of f box and f emb are 4 and 256, respectively.
We use a single fully-connect layer with output dimension 2 as f cls .
Datasets.
For a fair comparison, we pretrain our JoinDet on ImageNet-1K [13], ImageNetand MS COCO [30] without labels, and evaluate our method on MS COCO and PASCAL VOC.
ImageNet-1K contains 1.28M object-centric images with 1000 class categories.
Similar to other works [38, 21, 56], we use a subset of ImageNet-1K that contains 125K images and 100 classes as ImageNet-100.
MS COCO is a popular object detection dataset that contains 121K scene images with 80 object categories labeled with bounding boxes.
PASCAL VOC contains 20K images withobject categories.
Training.
Following DETReg, we initialize the ResNet50 [23] backbone of JoinDet with SwAV [5], which is fixed throughout the training stage.
We pretrain JoinDet for 5 and 50 epochs on ImageNet-1K and MS COCO, respectively, where the pretraining schedules are set the same as DETReg.
We update object priors every 1 and 10 epochs for ImagNet-1K and MS COCO, respectively.
As our method relies on the self-attention map in the transformer encoder, most generated object priors focus on big objects, so we use large-scale jittering mentioned in [17] to alleviate the imbalance problem.
More implementation details and experimental results are provided in supplementary materials.
All experiments are implemented on 16 NVIDIA V100 GPUs.
3.2 Comparison with State-of-the-art methods
Low-Data regimes object detection.
To evaluate the effectiveness of JoinDet, we conduct experiments in a low-data regimes object detection setting on COCO, where only 1% or 10% amounts of COCO labeled data are used during finetuning.
The experimental results are presented in Tab. 1.
Firstly, our method outperforms existing backbone pretraining methods, with a significant performance gain of 4.68 AP, and 4.53 AP over the supervised pretraining baseline on 1% and 10% COCO data, respectively.
Second, when compared with DETReg, which also fully pretrains the model, our method remains 1.31 AP, and 1.75 AP performance gain on 1% and 10% COCO data, respectively, showing the effectiveness of progressively refined object priors.
Full-data finetuning.
To further evaluate JoinDet, we finetune it on two datasets: MS COCO and
PASCAL VOC.
On MS COCO, following previous works [57, 3], we finetune JoinDet using a similar training schedule on MS COCO train 2017 and evaluate on val 2017 .
On PASCAL VOC, we finetune on trainval 07 + 12 and evaluate on test 07 .
Comparison with state-of-the art methods is presented in Tab. 2 and Tab. 3.
Tab. 2 shows that JoinDet achieves higher performance on VOC.
Concretely,
JoinDet outperforms DETReg with +0.2 AP and +1.0 AP when pretraining on ImageNet-1K and
COCO, respectively.
When using COCO as the pretraining dataset, JoinDet shows considerable performance gain.
We suggest that, as JoinDet is based on mining object priors on the pretraining dataset, more diverse objects contained in COCO provide more diverse supervision on JoinDet, which further boosts its performance when transferred to downstream datasets. Second, JoinDet achieves [Table]
[Table] [Figure] [Figure] +0.4 AP performance gain on SwAV and comparable performance with DETReg on full-data COCO finetuning.
Second, as shown in Fig.
3 and Fig. 4, JoinDet is finetuning-efficient, which shows considerable improvement when finetuning with small epochs.
Class agnostic object proposal evaluation.
To demonstrate that JoinDet learns better to detect objects without labels, following DETReg, we examine the class agnostic performance of JoinDet on Tab. 4.
First, JoinDet outperforms other pretraining approaches, indicating that using progressive region priors improves the localization ability.
Second, pretraining on COCO shows a larger performance gain of +1.7 AP, which shows our method benefits more on scene images with various foreground objects.
Ablation
Default Settings .
We implement ablations on single-scale deformable DETR.
JoinDet is unsupervised pretrained on COCO for 50 epochs with a learning rate drop at epoch 40.
The supervision mined from the model outputs is updated every 10 epochs.
The results in this section are evaluated by full-data finetuning on PASCAL VOC with 25 epochs.
We train 3 different models with different random seeds and report the mean result of AP.
Select effective object priors.
To assess the effectiveness of joint usage of region priors and instance priors, we compare the performance of JoinDet variants using different object priors: (a) box predictions P , (b) initial region priors P r 0 , (c) region priors P r , (d) instance priors
P i , (e) region priors
P r + instance priors P i .
As shown in Tab. 5, using both region and instance priors (Tab. 5(e)) outperforms DETReg by +3.0 AP, +1.5 AP with 5 epochs and 25 epochs finetuning, respectively, which verifies the effectiveness of joint usage of two priors generated by the model.
Second, when region priors or instance priors are removed, we observe a -0.6 , -1.9 AP performance drop withepochs finetuning, respectively, which suggests that region and instance priors can mutually benefit each other.
Third, when no region priors are used to initialize the pretraining, directly using the box predictions of the model can only generate unreliable object priors for supervision, leading to a significant performance drop of -13.4 AP with 25 epochs finetuning. [Table] [Table]
Smooth the refinement of object priors.
To demonstrate the effectiveness of the proposed Box
Smooth Module, we conduct two other updating methods, including directly changing and updating by NMS (see Fig. 5).
Directly changing, which updates the supervision without considering previous generated object priors, leads to a fast change on the supervision signal.
When updating by NMS, object priors are only updated when they have a large IoU with current object priors, making a relatively slow updating speed.
As summarized in Tab. 6, noticeable performance drops are observed without the Box Smooth Module, especially when the updating method is altered by directly changing, which verifies the effectiveness of the Box Smooth Module.
The large performance gaps are due to the facts the: (1) Directly changing neglects useful object priors in previous supervision, which makes object-related object priors appear and disappear during pretraining, leading to an unstable training process; (2)
Although updating by NMS avoids the disappearance of reliable object priors in previous supervision, it fails to refine the box coordinates in a slow and smooth way.
The quick shifting of the supervision signal leads to undesirable learning instabilities.
Augmentations during pretraining.
As our method mines object priors on the self-attention map in the transformer encoder, most generated priors focus on big objects.
To assess the importance of small object priors during pretraining, we ablate on different augmentations during pretraining.
We use the initial supervision R 0 , which bounds the highlighted regions in the eigen attention map generated by the backbone as fixed supervision.
All experiments are pretrained over 50 epochs on
COCO with Deformable DETR and evaluated on PASCAL VOC with 100 epochs finetuning.
It can be observed in Tab. 7 that both scale jittering and copy & paste improve the pretraining with a large margin, indicating the importance of adding small object priors when mining supervision from the self-attention map.
Visualization
To qualitatively evaluate the generated object priors in our method, we visualize the evolution of the eigen attention map and generated object priors during pretraining, and compare them with that in a supervised trained Deformable DETR in Fig.
First, generated object priors are progressively refined to objects in images during pretraining.
Second, supervised by updating object priors, foregrounds in the eigen attention map can be further highlighted (see Fig. 6(a)).
Third, with the help of progressive instance priors, the generated object priors can still become object-related and similar to ground truth [Figure] [Table] [Figure]
[Table] [Table] even when it is hard for the eigen attention map to be further refined [Table]
e
[Table]
g. when foreground objects fill images (Fig. 6(b)) or objects appear in a simple background (Fig. 6(c)).
In the former condition, our method highlights all objects while the supervised method focuses on labeled people.
In the latter condition, foregrounds and backgrounds have been well divided at the beginning epochs.
Related Works
Unsupervised pretraining for backbone.
Recent unsupervised pretraining methods, which rely on pretext tasks to learn visual representations [9, 21, 18, 7, 10, 8, 44, 49, 33, 54, 55, 43], have shown considerable performance on transfer learning tasks, outperforming their supervised counterparts.
However, compared with considerable performance gains on classification-related tasks, the improvement on dense-prediction tasks [30, 16] is limited.
To this end, a growing number of works explore pretext tasks for object detection and instance segmentation.
DenseCL [46] and PixPro [53] contrast pixel feature on the same physical location under different views to learn pixel-level representations.
DetCo [51] exploits supervision on features from different stages of the backbone and from global and local patches to learn consistent representations on image-level and patch-level.
[2] proposes point-level region contrast, which enables the model to learn at the point-level to help localization, and at the region-level to help holistic object recognition.
Despite the good performance, all these works only focus on pretraining the backbone of object detectors, neglecting the detector heads.
When these methods are transferred to object detection, the detector heads are initialized from scratch and do not benefit from pretraining, which limits their performance on object detection.
In contrast, our JoinDet, which utilizes object priors generated by the model itself as supervision, pretrains the entire model to promote detector learning.
Unsupervised pretraining for object detector.
Pretraining the backbone with a pretext task for dense-prediction tasks leaves untrained detection heads which are also a core component when transferring to object detection [24].
Few works attempt to remedy this problem by pretraining the entire detector with various unsupervised pretext tasks.
SoCo [48] utilizes selective search to generate object priors and perform contrastive learning on object-level features from the detector head.
UP-DETR [12] and DETReg [3] pretrain the detection heads of DETR [4] by forcing them to predict object priors generated by randomly cropping and selective search, respectively.
However, randomly cropping hardly provides any effective object prior, and selective search is a heuristic method that is time-consuming, independent from the pretraining process. In contrast to these methods, our G T O u r m e t hod
P r e t r a i n i ng epo c
h s
[Figure] Progressive refined ℛ and ℳ Progressive refined ℛ Progressive refined ℛ
[Figure]
proposed JoinDet jointly generates object priors and learns detection, which can gradually update the object priors with learned and improved ones for better supervision during pretraining.
Attention in unsupervised pretraining as supervision.
NNCLR [15] and DINO [6] show that the attention maps of the visual transformer can generate semantic segmentation masks even though the model is pretrained without labels.
This suggests that self-learned attention can provide effective supervision for dense-prediction tasks.
STEGO [19] utilizes off-the-shelf pretrained DINO to extract cross-image feature correspondence (cross-image attention) as supervision to distill segmentation features and train an unsupervised segmentation model.
Different from STEGO, our JoinDet exploits the self-attention maps in the transformer encoder to generate multiple object priors as supervision during training.
We also show that the self-attention maps can be jointly refined during training to generate progressive object priors for better supervision.
Conclusion
In this paper, we mine the information from the knowledge learned by the object detection model itself to pretrain the model without labels.
We propose an unsupervised object detection pretraining method which can jointly generate reliable object priors and learn to detect objects.
The core is at generating region priors from the transformer encoder and instance priors from the transformer decoder to achieve reliable object priors for supervision.
With careful design, the model can gradually learn better object localization capability, producing better object priors to optimize itself.
Our method shows considerable improvements in low-data regimes object detection and fast convergence on full-data finetuning.
However, there still exists a large gap from supervised training on class agnostic object proposal evaluation, which calls for further studies.
Acknowledgment
This work is supported by the Key R&D Project of Zhejiang Province (No.2022C01056) and
the
National Natural Science Foundation of China (No.62127803).
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information on how to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] or [N/A] .
You are strongly encouraged to include a justification to your answer , either by referencing the appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section ?? .
• Did you include the license to the code and datasets?
[No] The code and the data are proprietary.
• Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.
For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] (c) Did you discuss any potential negative societal impacts of your work?
[N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[N/A] (b) Did you include complete proofs of all theoretical results?
[N/A] 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[No] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] See Section 3.1.
See supplementary materials for more details.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[No] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] (b) Did you mention the license of the assets?
[No] (c) Did you include any new assets either in the supplemental material or as a URL?
[No]
(d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
[No] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]
A Appendix
Optionally include extra information (complete proofs, additional experiments and plots) in the appendix.
This section will often be part of the supplemental material.