Proposal-Contrastive Pretraining for Object Detection from Fewer Data
Authors: Quentin Bouniot
Romaric Audigier
Angelique Loesch
Amaury Habrard
Venue: ICLR 2023 notable top 25% (2023)
Keywords: Object Detection
Unsupervised
Pretraining
Contrastive Learning

Abstract: The use of pretrained deep neural networks represents an attractive way to achieve strong results with few data available. When specialized in dense problems such as object detection, learning local rather than global information in images has proven to be more efficient. However, for unsupervised pretraining, the popular contrastive learning requires a large batch size and, therefore, a lot of resources. To address this problem, we are interested in transformer-based object detectors that have recently gained traction in the community with good performance and with the particularity of generating many diverse object proposals. 
    In this work, we present Proposal Selection Contrast (ProSeCo), a novel unsupervised overall pretraining approach that leverages this property. ProSeCo uses the large number of object proposals generated by the detector for contrastive learning, which allows the use of a smaller batch size, combined with object-level features to learn local information in the images. To improve the effectiveness of the contrastive loss, we introduce the object location information in the selection of positive examples to take into account multiple overlapping object proposals. When reusing pretrained backbone, we advocate for consistency in learning local information between the backbone and the detection head. 
    We show that our method outperforms state of the art in unsupervised pretraining for object detection on standard and novel benchmarks in learning with fewer data. 

Published as a conference paper at ICLR 2023
PROPOSAL-CONTRASTIVE PRETRAINING FOR OBJECT DETECTION FROM FEWER DATA
Quentin Bouniot Romaric Audigier', AAnglliue Loesch', Amaury Habrard
1 Université Paris-Saclay, CEA, LIST, F-91 Palaiseau, France
2 Université Jean Monnet Etinne,, CNRS, Institut d Optique Graduate School, Laboratoire Hubert Curien UMR 5516, F-42023, Saint-Etienne, France
3 Institut Universitaire de France (IUF) fix iirstname. last nanetn (cea. fr, univ-st-etienne. fr}
ABSTRACT
The use of pretrained deep neural networks represents an attractive way to achieve strong results with few data available. When specialized dense problems such as object detection, learning local rather than global information in images has proven to be more efficient. However, for unsupervised pretraining, the popular contrastive learning requires large batch size and, therefore, a lot of resources. To address this problem, we are interested in hansformer-basee object detectors that have recently gained traction in the community with good performance and with the particularity of generating many diverse object proposals. this work, we present Proposal Selection Contrast (ProSeCo), novel unsupervised overall pretraining approach that leverages this property. ProSeCo uses the large number of object proposals generated by the detector for contrastive learning, which al- lows the use of a smaller batch size, combined with object-level features to learn local information in the images. To improve the effectiveness of the contrastive loss, we introduce the object location information in the selection of positive ex- amples to take into account multiple overlapping object proposals. When reusing pretrained backbone, we advocate for consistency in learning local information between the backbone and the detection head. We show that our method outper- forms state of the art in unsupervised pretraining for object detection standard and novel benchmarks in learning with fewer data.
INTRODUCTION
In recent years, we have seen a surge in research on unsupervised pretraining. For some popular tasks such as Image Classification Object detection, initializing with pretrained backbone helps train big architectures more efficiently (Chen et al., 2020b; Caron et al., 2020; He et al., 2020). While gathering data is not difficult in most cases, its labeling always time-consuming and costly. Pretraining leverages huge amounts of unlabeled data and helps achieve better performance with fewer data and fewer iterations, when finetuning the pretrained models afterwards.
The design of pretraining tasks for dense problems such as Object Detection has to take into account the graingraiedd information in the image. Furthermore, complex object detectors contain different specific parts that can be either pretrained independently (Xiao et al., 2021; Xie et al., 2021; Wang et al., 2021a; Hénaff al., 2021: Dai et al., 2021b; Bar al., 2022) or jointly (Wei et al., 2021). The different pretraining possibilities for Object Detection in the literature are illustrated in Figure 1. A pretraining of the backbone tailored to dense tasks has been the subject of many recent efforts (Xiao et al,, 2021; Xie et al., 2021; Wang et al., 202 1021a; Hénaff et al., 2021) (Backbone Pretraining). but few have been interested in incorporating the detection-specifii components of the architectures during pretraining (Dai et al., b02lb; Bar et al., 2022; Wei et al., 2021) (Overall Pretraining). Among them, SoCo (Wei et al., 2021) focuses on convolutional architectures and pretrains the whole detector, f.e. the backbone along with the detection heads (approach in Figure 1), whereas UP-DETR (Dai et al., 2021b) and DETReg (Bar et al., 2022) pretrain only the transformers (Vaswani et al., 2017) in ransformer-based object detectors (Carion et al., 2020; Zhu al., 2021) and keep the backbone

---

Published as a conference paper at ICLR 2023
Backbone Pretraining
Overall Pretraining
Global
Discrepancy in features Less costly
Local
C.
Back bone
Detection Heads
Back bone
Detection Heads
a.
(Ours) Consistency in features Less costly
Back bone
Detection Heads
b.
Back
Detection Heads
Consistency in features Costly to train
Back bone
Detection Heads
e.
Figure I: Illustration of the different pretraining possibilities for Object Detection. The pretraining can be either limited to the backbone (left), or overall including the detection heads (right). The few previous overall approaches either suffer from a discrepancy in the features between the backbone, that is trained at the image-level (global), and the detection heads, trained to encode object-level (local) information (c.), or from the cost training lots of parameters with large batch size (e.).
fixed (approach c. in Figure 1). Due to the numerous parameters that must be learned and the huge number of iterations needed because of random initialization, pretraining the entire detection model is expensive (Figure 1, e.). On the other hand, pretraining only the detection-specine parts with fixed backbone leads fewer parameters and allows leveraging strong pretrained backbones al- ready available. However, fully relying on aligning embeddings given by the fixed backbone during pretraining and those given by the detection head, as done in DETReg or UP-DETR, introduces discrepancy in the information contained in the features (Figure 1, c.). Indeed, while the pretrained backbone has been trained learn image-level features, the object detector must understand object- level information in the image. Aligning inconsistent features hinders the pretraining quality.
In this work, we propose Proposal Selection Contrast (ProSeCo), an unsupervised pretraining method using transformer- detectors with fixed pretrained backbone. ProSeCo makes use of two models. The first one aims to alleviate the discrepancy in the features by maintaining a copy of the whole detection model. This model is referred to teacher in charge of the object proposals embeddings, and is updated through an Exponential Moving Average (EMA) of another student net- work making the object predictions and using similar architecture. This latter network is trained by a contrastive learning approach leveraging the high number of object proposals that can be obtained from the detectors. This methodology. addition to the absence of batch normalization in the archi- tectures. reduces the need for a large batch size. We further adapt the contrastive loss commonly used pretraining to take into account the locations of the object proposals in the image, which is crucial in object detection. In addition, the localization task is independently learned through region pro- posals generated by Selective Search (Uijlings et al., 2013). Our contributions are summarized as:
We propose Proposal Selection Contrast (ProSeCo), contrastive learning method tailored for pretraining transformer based object detectors.
We introduce the information of the localization of object proposals for the selection of positive examples in the contrastive loss to improve its efficiency for pretraining.
We show that our proposed ProSeCo outperforms previous pretraining methods for transformer-based object detectors on standard benchmarks as well as novel benchmarks.
2 RELATED WORK
Supervised Object Detection with transformer-basee architectures Object Detection is an im- portant and extensively researched problem in computer vision (Girshick al., 2014; Girshick,
2

---

Published as a conference paper at ICLR 2023
2015; Ren et al., 2015; Redmon et al., 2016; Liu et 2016; Lin et al., 2017; Tian et al., 2019). es- sentially combines object localization and classification tasks. Recently, novel detector based on an encoder-decoder architecture using transformers (Vaswani et al., 2017) has been proposed in Carion et al. (2020). The training complexity of this architecture was subsequently improved in Deformable DETR (Def. DETR) (Zhu et al., 2021), by changing the attention operations into deformable atten- tion, resulting in a faster convergence speed. Several other follow-up works (Dai et al., 2021a; Meng et 2021; Wang et al., 2021b; Liu et al., 2022a; Yang et al., 2022; Li et al., 2022) have also fo- cused on increasing the training efficiency of DETR. Transformer-basee architectures now represent a strong alternative traditional convolutional object detectors, reaching better performance for a similar training cost. Furthermore, recent work have shown strong results of transformer- berseeee----dddd--- de- tectors in a data-scarce setting (Bar et al., 2022; Bouniot et al., 2023), compared to convolutional architectures (Liu et al., 2020; 2022b), which we also observe and discuss in Appendix H.
Self-supervised and unsupervised pretraining backbone architectures Recent advances in self-supervised pretraining (Grill et al., 2020; Caron et al., 2020; Chen et al., 2020b; 2021; Zheng al., 2021; Denize et al., 2023) have achieved strong results for obtaining general representations that transfer well to image classification (He et al., 2020). Early works proposed pretext tasks (Alexey et al., 2015; Noroozi & Favaro, 2016; Komodakis & Gidaris, 2018), which are now out- performed by Contrastive Learning (Oord et al., 2018; Wu et al., 2018; He et al., 2020; Chen et al., 2020a; Misra & Maaten, 2020; Grill et al., 2020; Caron al., 2020; Chen al., 2020b; 2021; Denize et al., 2023). This paradigm relies on instance discrimination using pair of positive views from the same imput contrasted with all other instances in the batch, called negatives. However, the InfoNCE objective function (Oord et al., 2018) widely used for contrastive learning and its recent improved version, SCE (Denize et al., 2023), both require a large amount of negative instances to be effec- tive (Wang & Isola, 2020). The improvements observed using a general self-supervised pretraining are less significant for more complex, dense downstream tasks (He et al., 2019; Reed et al., 2022). To address this issue, recent approaches have started investigating pretraining approaches tailored for these tasks by imposing local consistency, either at the pixel or region-level they respectively propose to match in the representation space the features corresponding to the same location in the input space (O Pinheiro et al., 2020; Xie et al., 2021; Wang et al., 2021a), or apply local consistency between features from regions in the image (Roh et al., 2021; Yang et al., 2021; Xiao et al., 2021).
Unsupervised Pretraining for the overall detection model Few approaches in the literature have tackled the problem of pretraining the detection-specifie parts of the architecture. along with the backbone (Wei al., 2021), or independently (Dai et al.,, 2021b; Bar et al., 2022). SoCo (Wei al., 2021) proposes pretraining strategy for convolutional detectors inspired by BYOL (Grill et al., 2020). Object locations are generated using Selective Search (Uijlings et al., 2013), and then object- level features are extracted and contrasted with each other using teacher-student strategy. The small amount of object proposals and object features generated requires using a large batch size for the contrastive loss to be effective. Pretraining the backbone along with the detection modules this way makes the method difficult and costly to train due to the high amount of parameters to learn. For transformer-basee architectures, UP-DETR (Dai et al., 2021b) and DETReg (Bar et al., 2022) use a fixed pretrained backbone to extract features respectively from random patch, or from object locations given by Selective Search (Uijlings et al., 2013), then pretrain the detector by localizing and reconstructing the features of the patch extracted from the input images. However, since the features to reconstruct are obtained by a backbone which was trained encode image-level information, there is discrepancy in the information between the features to match.
Our proposed ProSeCo designed specifically for transformer- based detectors, and use a fixed backbone pretrained for local information. In this work, we leverage the high amount of object proposals generated by wansformer-based detectors as instances for contrastive learning. Target object-level features and localizations are provided by a teacher detection model updated through EMA, inspired by recent advances in self-supervised and semi-supervised learning (Liu et al., 2020; Grill et al., 2020; Denize et al., 2023; Wei et al., 2021). The student detection model is pretrained by computing the contrastive loss between object proposals given by the student and teacher detectors. The large number of proposals generated by transformer-bar based detectors alleviates the need for large batch size for the contrastive loss. The contrastive loss function used is further improved by introducing the location of objects for selecting positive proposals.
3

---

Published as a conference paper at ICLR 2023
Strong view x = (x))) t ~ T
Object Predictions
b
Loard
Selective Search
Student
Box
Matching
+
MLP
L lou
b
Back bone
Transformer Detector
Boxes
b
Foatures
2 b
EMA
MLP
Proj.
Proposal Matching
LOCSCE
Features xi
Back bone
Transformer Detector
Unlabeled image
Boxes
Stop gradients
Box
Weak view X (x)
Teacher
MLP
Object
e p1-11
Proposals
Figure 2: Overview of our proposed ProSeCo for unsupervised pretraining. The method follows student-teacher architecture, with the teacher updated through Exponential Moving Average (EMA) of the student. For each input image, K random boxes are computed using the Selective Search algorithm, and two different views are generated through an asymmetric of weak aug- mentations T and strong augmentations T Then, object predictions are obtained from the stu- dent model for the strongly augmented view, and object proposals from the teacher model with the weakly augmented view. Finally, the boxes predicted by the student are matched with the boxes sampled from Selective Search compute the localization losses Looord and Lgiou. and the full pre- dictions are matched with the object proposals to compute novel contrastive loss LLoSCEE
OVERVIEW OF THE APPROACH
We present in this section our proposed unsupervised pretraining approach, illustrated in Figure 2, beginning with the data processing pipeline. Then, we detail the contrastive loss used with the localization-wware positive object proposal selection. The waasformer-basee detectors built a general architecture that consists of a backbone encoder (e.g. ResNet-50), followed by several transformers encoder and decoder layers, and finally prediction heads for the bounding boxes coordinates and class logits (Carion et al., 2020; Zhu et al., 2021).
3.1 DATA PROCESSING PIPELINE
Throughout the section, assume to have sampled batch of unlabeled images {xx, with x, the Hth image and N₂ the batch size.
For each input image X,, we compute two different views with two asymmetric distributions of augmentations T and 2 weakly augmented view x,' = = (x:), with e T and strongly augmented view obtained from the weakly augmented one x," = (x)(x))) with 12 T
To guide the model into discovering localization of objects in unlabeled images and prevent collapse, we use bounding boxes obtained from the Selective Search algorithm (Uijlings ct al., 2013). similarly to previous work (Wei et al., 2021; Bar al., 2022). Since Selective Search is deterministic and the generated boxes are not ordered, we compute the boxes for all images offline and, at training time, randomly sample K boxes 5. (bss R R k for each image in the batch. Then, the two views and the corresponding boxes sampled are used as input for our method.
3.2 PRETRAINING METHOD
As shown in Figure 2, our approach is composed of Denize al., 2023; Wei et al., 2021). With ProSeCo, extend the student-teade teacher pretraining for transformer- based detectors to tackle the discrepancy in information-level when aligning features, and introduce dual unsupervised bipartite matching presented below.
architecture (Grill et al., 2020;
4

---

Published as a conference paper at ICLR 2023
First, the backbone and the detection heads are respectively initialized from pretrained weights and randomly for both the student and teacher models. The teacher model is updated through an Expo- nential Moving Average (EMA) of the student's weights at every training iteration. For both mod- els, the classification heads the detectors are replaced by an MLP, called projector, for obtaining latent representations of the objects.
From the weakly augmented view x,', the teacher model provides object proposals Y; = (M(1)))f=1 ((z() ((..)))=11 with ((,)) the latent embedding and ((a,)) the coordinates of the 5th object found. The student model infers predictions y: = (f = {(2(2s)) b( )y from the corresponding strongly augmented view x.".
Then, we apply an unsupervised Hungarian algorithm for proposal matching to find from all the permutations of N elements GN the optimal bipartite matching dr between the predictions y; of the student and the object proposals y, the teacher:
prop arg min dEEN
proppmmtth ((,()))))))))))
Therefore, for each image x;, the jth proposal Y(i.)) found by the teacher associated to the (1)tt prediction of the student ()), Our matching cost Sprpp.mach for the Hungarian al- gorithm takes into account both features and bounding box predictions through a linear combination (tt,),2 of features similarity Ssim (Z(i.)) ((.o(i))) [| laa alll) the & loss of the box coordinates Lovndd [[b())) - b(r.a.6)) Il,, and the generalized IoU loss Lgina from Rezatofighi et al. (2019):
[Asim Caim (Z(..) (((o()))))
ppopp manch
(2)
+ Leaved (bra.)): b(aou)) Agion
Similarly, we also use an unsupervised Hungarian algorithm for box matching, find the optimal bipartite matching h E &. between the predicted boxes b, the student and the sampled boxes b. from Selective Search, using the matching cost machh
bbox arg min Σ
Laoch ( ((eo))));
(3)
GEGN
j=1
mooch (bss S8 (i.)) ((o,()))) Cooord
( b((.()))) + AgionCgion (b b(so()))
(4)
Finally, the global unsupervised loss fu used for training is a combination of a loss function between the object latent embeddings the teacher and student models, and between the object localization of the student predictions and Selective Search boxes. More formally, it is computed as:
(x)
Acontrast LOSSCE. (v. (.PPP))
1
N₂
Loourdd ((sstrra Σ Agiou (bss) ((oo)))) (5)
N₂K
In the above equations, we define Aaim Acouet Agious Agantrast E IR as the coefficients for the different losses. For the consistency in the latent embeddings of the objects, we introduce the object locations information in contrastive loss LLoSSCT. This loss is used to contrast the predictions of the student with object proposals y = [ found by the teacher, matched according to the proposal matching Grop = {6ptop) No in the next section.
{}:) No.
11=1 over the batch. We detail the computations behind this loss
5

---

Published as a conference paper at ICLR 2023
Strong view
Iou 2.8
IoU 2
Weak view
Figure 3: Illustration of the effect of the localized contrastive loss used. Predictions of the student and teacher models are contrasted with each other to leverage the large number of object proposals obtained from transformer-based detectors. To introduce the object locations information, overlap- ping proposals (in green) in each weak view, according to an loU threshold 8, are also considered positive along with the matched proposal. Proposals that neither match nor overlap the matched proposal, are considered as negative (in red) in the contrastive loss.
3.3 LOCALIZATION-AWARI CONTRASTIVE LOSS
Inspired by advances in self-superviser learning, we propose contrastive objective function (Oord at, 2018; Chen al., 2020a; He at,,, 2020) between object proposals that also maintains relations (Zheng et al., 2021; ;; Denize et al., 2023) among these proposals. This objective function extends the latest SCE (Denize et al., 2023) for instance discrimination between object proposals and is illustrated in Figure 3. We compute the contrastive loss between all the object latent embeddings from a batch image. The positive pair of objects in an image x, is given by the proposal matching of prop First, we define the distributions of similarity between objects embeddings:
Ligen 1l jm exp(Z(a.)) 2(a,n)/TT))
P(in.jm)
NN Ex 1 i exp (Z(1.) 2(x.)//T)))
(6)
exp(2((.)) /T)
P(nnjjm) E EN exp( 2(x))/T))
The distribution P(in.jm) represents the relations between weakly augmented object embeddings scaled by the temperature Tr, and P'(..jmm the similarity between the strongly augmented embed- dings and the weakly augmented ones, scaled by Then, the target similarity distribution for the ob- jective function is a weighted combination of one-hot label and the teacher's embeddings relations:
Win.jmm) AScr 1,=n1,mm + >SCE. P(in.jm)"
To introduce the localization information in our objective, we compute the pairwise Intersection over Union (IoU) between object proposals of the same image and consider overlapping objects as other positives when computing the target similarity distribution:
Loc
AscE
1 (i.))) + (1 ASCE) P((n.j)))
(in.jm)
where ToU,(j, m) corresponds to the loU between teacher proposals Y(t.)) and Y(s.) found in the same image and 8 is the loU threshold to consider the proposal as a positive example. Finally, we use this tailored target similarity distribution in our Localized SCE (LocSCE) loss:
N
CLoesCE(y. y, (PTP)
1
ΣΣΣΣ
w
Loc (mn log(p" (an (m))).
(10)
NON

---

Published as a conference paper at ICLR 2023
Note that we do not match the proposals according to @PmP in the target similarity, proposals obtained by the teacher model. We also require the full proposal as input of the loss to compute the pairwise IoU using the box coordinates. Furthermore, tion of SCE when 5 = 1. This formulation leads to an effective batch size of Ng. N.
compare
recover the original formula-
This localization- aware contrastive loss function aims pull together the objects embeddings that overlap subsequently with each others, as they should correspond to the same object in the image.
4 EXPERIMENTS
In this section, we present comparative study of the results of our proposed method on standard and novel benchmarks for learning with fewer data, as well as an ablative study on the most relevant parts. First, we introduce the datasets, evaluation and training settings.
4.1 IMPLEMENTATION DETAILS
Datasets and evaluation We use ImageNet ILSVRC 2012 (IN) (Russakovsky al., 2015) for pre- training. MS-COCO (COCO) (Lin et al., 2014) and Pascal VOC 2007 and 2012 (Everingham et al., 2010) for finetuning. To evaluate the performance in learning with fewer data, following previous work (Wei et al., 2021; Bar et al., 2022), we consider the Mini-COCO benchmarks, where we ran- domly sample 1 %, 5% or 10% of the training data. Similarly, we also introduce the novel Mini-VOC benchmark, in which we randomly sample 5% or 10% of the training data. We also use the Few- Shot Object Detection (FSOD) dataset (Fan et al., 2020) in the novel FSOD-test and FSOD-train benchmarks. We separate the FSOD test set with 80% of the data randomly sampled for training and the remaining 20% data for testing, by taking care of having at least image for each class in both subsets, and do the same for the FSOD train set. In all benchmarks, the image ids selected for training and testing will be made available for reproducibility. More details in Appendix A.
Pretraining We initialize the backbone with the publicly available pretrained SCRL (Roh et al., 202 1) checkpoint and pretrain ProSeCo for 10 epochs on IN. The hyperparameters are set as follows: the EMA keep rate parameter to 0.999, the IoU threshold 8 0.5, a batch size of N, = 64 images over 8 A100 GPUs, and the coefficients in the different losses Asim = Acontrast = 2 which is the same value used for the coefficient governing the class cross-entropy in the supervised loss. The projector is defined as a 2-layer MLP with hidden layer of 4096 and a last layer of 256, without batch normalization. Following SCE (Denize et al., 2023), we set the temperatures T = 0.1, Te = 0.07 and the coefficient ASCE 0.5. We sample K = 30 random boxes from the outputs of Selective Search for each image at every iteration. Other training and architecture hyperparameters defined as in Def. DETR (Zhu et al., 2021) with, specifically, the coefficients Acoord 5 and Agiou = 2, the number of object proposals (queries) N = 300, and the learning rate is set to i = 2- 10 4, For weak augmentations T we use a random combination of flip, resize and crop, and for strong augmentations T we use random combination of color jittering, grayscale and Gaussian blur. In T we resize images with the same range of scales as the supervised training protocol on COCO (Large-scale). The exact parameters for the augmentations are detailed in Appendix B, and a discussion about the pretraining cost can be found in Appendix C.
Finetuning protocols For finetuning the pretrained models, follow the standard supervised learning hyperparameters of Def. DETR (Zhu et al., (.21). In all experiments, we train the models with batch size of 32 images over 8 A100 GPUs until the validation performance stops increasing, i.e. for Mini-COCO, up to 2000 epochs for 1%, 500 epochs for 5%, 400 epochs for 10%, for Mini- VOC, up to 2000 epochs for both 5% and 10%, up to 100 epochs for both FSOD-test and PASCAL VOC, and up to 50 epochs for FSOD-train. We always decay the learning rates by factor of 0.1 after about 80% of training. Experiments with more annotated data are discussed in Appendix G. To compare our method to DETReg (Bar et al., 2022) our novel benchmarks, we use their publicly available checkpoints from github.
4.2 FINETUNING AND TRANSFER LEARNING
We evaluate the transfer learning ability of our pretrained model on several datasets. Tables la and Ib present the results obtained compared to previous methods in the literature when learning from fewer
7

---

Published as a conference paper at ICLR 2023
labeled data. We see that our method outperforms state-of-the-an art results in unsupervised pre- training on all benchmarks and datasets, and obtain even more strong results when training data is scarce. The improvement is even more significant the overall performance with few training data low. When using 5% of the COCO training data (i.e. Mini-COCO 5% Table la), corresponding about 5.9k images, ProSeCo achieves 28.8 mAP, which represents improvement of +5.2 per- centage point (p.p.) over the supervised pretraining baseline and +2 p.p. over both state-of-the-an overall pretraining methods. Results with all the evaluation metrics are presented in Appendix E.
Table 1: Performance (mAP in %) of our proposed pretraining approach after finetuning using different percentage of training data (with the corresponding number of images reported). We show that our ProSeCo outperforms previous pretraining methods all benchmarks.
Pretrain.
Mini-COCO
Method
Detector
Dataset
1% (1.2k)
5% (5.9k)
10% (11.8k)
Supervised
Def. DETR
IN
13.0
23.6
28.6
SwAV (Caron et al., 2020)
Def. DETR
IN
13.3
24.5
29.5
SCRL (Roh al., 2021)
Def. DETR
IN
16.4
26.2
30.6
DETReg (Bar et al., 2022)
Def. DETR
COCO
15.8
26.7
30.7
DETReg (Bar et al., 2022)
Def. DETR
IN
15.9
26.1
30.9
Supervised (Wei al., 2021)
Mask R-CNN
IN
19.4
24.7
SoCo* (Wei et al., 2021)
Mask R-CNN
IN
26.8
31.1
ProSeCo (Ours)
Def. DETR
IN
18.0
28.8
32.8
(a)
FSOD-test
FSOD-train
PASCAL VOC
Mini VOC
Method
100% (11k)
100% (42k)
100% (16k)
5% (0.8k)
10% (1.6k)
Supervised
39.3
42.6
59.5
33.9
40.8
DETReg (Bar et al., 2022)
43.2
43.3
63.5
43.1
48.2
ProSeCo (Ours)
46.6
47.2
65.1
46.1
51.3
(b)
4.3 ABLATION STUDIES
In the following, provide several ablation studies for proposed approach. All experiments and results are compared on the Mini-COCO 5% benchmark with the pretrained SwAV backbone unless explicitly stated. Additional ablation on the number of queries be found in Appendix F,
Pretraining dataset and backbone In Table 2a, we show the effect of changing the pretraining dataset the fixed backbone used. We
that using a backbone more adapted dense tasks that learned local information (e.g. SCRL) helps the model by having consistent features (+) p.p.), compared backbone pretrained for global features (e.g. SwAV). Furthermore, even with a less adapted backbone, our ProSeCo initialized with the SwAV backbone outperforms DETReg (+1.7 p.p.). Pretraining with DETReg improves when using a more adapted backbone, but ProSeCo still reaches better performance. To compare with IN, we also pretrain ProSeCo on COCO for 120 epochs. obtain better results when pretraining the model on IN than using COCO thanks to the large number of different images in IN (about 10 times the number of images of COCO, leading to +0.4 p.p.), which is consistent with previous findings (Wei et al., 2021).
Localization information in contrastive loss In Table 2b, we show the effect of the localization information in the contrastive loss SCE. We can see that when introducing multiple positive exam- ples for each image based the IoU threshold 5 (i.e. Vō < 1), achieve better results than with the original SCE loss (i.e. = 1). Notably, the best results are achieved with = 0.5 (+1.7 p.p.). More experiments with the InfoNCE loss (Oord et al., 2018) can be found in Appendix D.
Hyperparameters Table 3 presents an ablation study on different important hyperparameters of our approach. We experimented first with the same batch size applied in Bar al. (2022) (Abl. Batch),
8

---

Published as a conference paper at ICLR 2023
but found that using smaller batch size (Base) leads to improved results (+0.2 p.p.). We evalu- ated different image scales as parameter of the weak data augmentations distribution in Abl. Scale. Mid-scale corresponds to resizing of the images such that the shortest edge is between 320 and 480 pixels, as used in previous work (Dai et al., b021b; Bar et al., 2022), and Large-scale to resize between 480 and 800 pixels, used for supervised learning on COCO. Exact values for these parame- ters can be found in Appendix B. We found that increasing the size of the images during pretraining important to have more meaningful information in the boxes, and more precise localization of the boxes (+0.7 P.p.). Following the best results from Denize et al. (2023), we evaluated the perfor- mance for T E {0.05, 0.07}. We found that 7, = 0.07 leads to the best performance (+0.3 p.p.). We considered several EMA keep rate parameter values following previous work (He et al., 2020; Wei al., 2021; ; Denize al., 2023), and found that 0.999 achieves the best results (+0.1 p.p.).
Table 2: (a) Comparison after finetuning when using different pretrained backbone and/or pretrain- ing datasets. (b) Comparison of the effect of the localization information using different IoU thresh- old All performance (mAP in %) are measured on Mini-COCO 5%.
Pretraining
Dataset
mAP
Loss
8
mAP
ProSeCo w/ SWAV
COCO
27.4
SCE
1.0
26.1
ProSeCo w/ SwAV
IN
27.8
LocSCE (Ours)
0.2
27.0
DETReg w/ SCRL
IN
28.0
LocSCE (Ours)
0.7
27.1
ProSeCo w/ SCRL
IN
28.8
LocSCE (Ours)
0.5
27.8
(a)
(b)
Table 3: Ablation studies on different hyperparameters for the proposed method. The performance 5%. Green and bold columns names indicate positive effect on the performance, and red columns a negative effect.
(mAP in %) measured on
Batch size
Images scale
Temperature T
EMA
Ablative Variant
mAP
192
64
Mid
Large
0.05
0.07
0.99
0.996
0.999
Base
V
26.7
Abl. Batch
26.5
Abl. Scale
v
/
27.4
Abl. Temp.
27
V
26.3
Abl. EMA
/
V
26.8
Best
27.8
CONCLUSION
In this work, we aim large unlabeled datasets for unsupervised pretraining of the overall detection model to improve performance when having access to fewer labeled data. In this end, we propose Proposal Selection Contrast (ProSeCo), novel pretraining approach for Object Detection. Our method leverages the large number of object proposals generated by transformer- based detectors for contrastive learning, reducing the necessity of a large batch size, and introducing the localization information of the objects for the selection of positive examples to contrast. We show from various experiments on standard and novel benchmarks in learning with few training data that ProSeCo outperforms previous pretraining methods. Throughout this work, we advocate for consistency in the level of information encoded in the features when pretraining. Indeed, learning object-level features during pretraining more important than image-level when applied to dense downstream task such as Object Detection. Future work could update the backbone during pretraining to further improve the consistency between the backbone and the detection heads.

---

Published as a conference paper at ICLR 2023
REPRODUCIBILITY STATEMENT
Throughout this paper, we made sure that the experiments and results are fully reproducible. We explicitly state the exact values of hyperparameters used in Section 4.1, and describe in details the datasets and evaluation protocols in Appendix A. All image ids randomly selected when evaluating with few training data (in Mini-COCO, Mini-VOC, FSOD-test, FSOD-train) will be made available.
ACKNOWLEDGEMENTS
This work was made possible by the use of the Factory-Al supercomputer, financially supported by the Franne Regional Council, and also performed using HPC resources from GENCI-IDRIS (Grant 2022-AD011013478)
REFERENCES
Dosovitskiy Alexey, Philipp Fischer, Jost Tobias, Martin Riedmiller Springenberg, and Thomas Brox. Discriminative unsupervised feature learning with exemplar convolutional neural networks. IEEE Trans. Pattern Analysis and Machine Intelligence, 2015.
Amir Bar, Xin Wang, Vadim Kantorov, Colorado Reed, Roei Herzig, Gal Chechik, Anna Rohrbach, Trevor Darrell, and Amir Globerson. DETReg: Unsupervised pretraining with region priors for object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
Quentin Bouniot, Angélique Loesch, Romaric Audigier, and Amaury Habrard. Towards few- annotation learning for object detection: Are transformer-based models more efficient? In Pro- ceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 75 14, January 2023.
Tiffany Tianhui Cai, Jonathan Frankle, David Schwab, and Ari S Morcos. Are all negatives created equal in contrastive instance discrimination? arXiv preprint arXiv:2010.0668,, 2020.
Nicolas Carion, Francisco Massa, Gabriel Synnaeve. Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European conference on computer vision. Springer, 2020.
Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. Advances in Neural Information Processing Systems, 2020.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning. PMLR, 2020a.
Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey E Hinton. Big self-supervised models are strong semi-supervised learners. Advances in neural information pro- cessing systems, 2020b.
Xinlei Chen, Saining Xie, and Kaiming He. An empirical study of training self-supervised vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021.
Xiyang Dai, Yinpeng Chen, Jianwei Yang. Pengchuan Zhang, Lu Yuan, and Lei Zhang. Dynamic DETR: End-to-end object detection with dynamic attention. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021a.
Zhigang Dai. Bolun Cai, Yugeng Lin, and Junying Chen. Up-DETR: Unsupervised pre-training for object detection with transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021b.
10

---

Published as a conference paper at ICLR 2023
Julien Denize, Jaonary Rabarisoa, Astrid Orcesi, Romain Hérault, and Stéphane Canu. Similar- ity contrastive estimation for self-supervised soft contrastive learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision pp. 2706 2716, 2023.
Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International journal of computer vision, 2010.
Qi Fan, Wei Zhuo, Chi-Keung Tang. and Yu- fu-Wing Tai. Few-shot object detection with attention-rpn and multi-relation detector. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4013-4022, 2020.
Ross Girshick. Fast r-cnn. In Proceedings of the IEEE international conference on computer vision, 2015.
Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accu- rate object detection and semantic segmentation. In Proceedings of the IEEE conference com- puter vision and pattern recognition, 2014.
Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to self-superrised learning. Advances in neural information processing systems, 2020.
Kaiming He, Ross Girshick, and Piotr Dollar. Rethinking imagenet pre-training In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020.
Olivier Hénaff, Skanda Koppula, Jean-Baptiste Alayrac, Aaron Van den Oord, Oriol Vinyals, and João Carreira. Efficient visual pretraining with contrastive detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021.
Nikos Komodakis and Spyros Gidaris. Unsupervised representation learning by predicting image rotations. In International Conference on Learning Representations (ICI.R, 2018.
Feng Li, Hao Zhang, Shilong Liu, Jian Guo, Lionel Ni, and Lei Zhang. Dn-DETR: Accelerate DETR training by introducing query denoising. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2022.
Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision. Springer, 2014.
Tsung- Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection. In Proceedings of the IEEE conference on com- puler vision and pattern recognition, 2017.
Shilong Liu, Feng Li, Hao Zhang, Xiao Yang, Xianbiao Qi, Hang Su, Jun Zhu, and Lei Zhang DAB-DETR: Dynamic anchor boxes are better queries for DETR. In International Conference on Learning Representations, 2022a.
Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng- Yang Fu, and Alexander C Berg. Ssd: Single shot multibox detector. In European conference on computer vision. Springer, 2016.
Yen-Cheng Liu, Chih-Yao Ma, Zijian He, Chia-War Wen Kuo, Kan Chen, Peizhao Zhang, Bichen Wu, Zsolt Kira, and Peter Vajda. Unbiased teacher for semi-supervised object detection. Interna- tional Conference on Learning Representations, 2020.
Yen-Cheng Liu, Chih- Yao Ma, and Zsolt Kira. Unbiased teacher v2: Semi-supervised object detec- tion for anchor-fir free and anchor-basee detectors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9819-9828, June 2022b.
11

---

Published as a conference paper at ICLR 2023
Depu Meng, Xiaokang Chen, Zejia Fan, Gang Zeng, Houqiang Li, Yuhui Yuan, Lei Sun, and Jing- dong Wang. Conditional DETR for fast training convergence. In Proceedings of the IEEE/CVF International Conference Computer Vision, 2021.
Ishan Misra and Laurens van der Maaten. Self-supervised learning of pretext-invariant representa- tions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.
Mehdi Noroozi and Paolo Favaro. Unsupervised learning of visual representations by solving jigsaw puzzles. In European conference on computer vision. Springer, 2016.
Pedro O Pinheiro, Amjad Almahairi, Ryan Benmalek, Florian Golemo, and Aaron C Courville. Unsupervised learning of dense visual representations. Advances Neural Information Process- ing Systems, 2020.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding. arXiv preprint 18Xiv:1807.03744. 2018.
Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and patlern recognition, 2016
Colorado J Reed, Xiangyu Yue, Ani Nrusimha, Sayna Ebrahimi, Vivek Vijaykumar, Richard Mao, Bo Li, Shanghang Zhang, Devin Guillory, Sean Metzger, et al. Self-supervised pretraining im- proves self-supprvised pretraining. In Proceedings of the IEEE/CVF Winter Conference on Ap- plications of Computer Vision, 2022.
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems, 2015.
Hamid Rezatofighi, Nathan Tsoi, Jun Young Gwak, Amir Sadeghian, Ian Reid, and Silvio Savarese. Generalized intersection over union: A metric and loss for bounding box regression. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019.
Byungseok Roh, Wuhyun Shin, Iodoo Kim, and Sungwoong Kim. Spatially consistent represen- tation learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2021.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 2015.
Zhi Tian, Chunhua Shen, Hao Chen, and Tong He. Feos: Fully convolutional one-stage object detection. In Proceedings of the IEEF/CVF International Conference on Computer Vision (ICCV), October 2019.
Jasper RR Uijlings, Koen EA Van De Sande, Theo Gevers, and Arnold WM Smeulders. Selective search for object recognition. International journal of computer vision, 2013.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 2017.
Tongzhou Wang and Phillip Isola. Understanding contrastive representation learning through align- ment and uniformity on the hypersphere. In International Conference Machine Learning. PMLR, 2020.
Xinlong Wang, Rufeng Zhang, Chunhua Shen, Tao Kong, and Lei Li. Dense contrastive learning for self-supervised visual pre-training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021a.
Yingming Wang, Xiangyu Zhang, Tong Yang, and Jian Sun. Anchor DETR: Query design for transformer-hased object detection. arXiv preprint 10X:022109.07107, 2021b.
12

---

Published as a conference paper at ICLR 2023
Fangyun Wei, Yue Gao, Zhirong Wu, Han Hu, and Stephen Lin. Aligning pretraining for detection via object-level contrastive learning. Advances in Neural Information Processing Systems, 2021.
Zhirong Wu, Yuanjun Xiong, Stella X parametric instance discrimination. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2018.
Yu, and Dahua Lin. Unsupervised feature learning via non-
Tete Xiao, Colorado Reed, Xiaolong Wang, Kurt Keutzer, and Trevor Darrell. Region similarity representation learning. Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021.
Zhenda Xie, Yutong Lin, Zheng Zhang, Yue Cao, Stephen Lin, and Han Hu. Propagate yourself: Exploring pixel-level consistency for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2021.
Ceyuan Yang, Zhirong Wu, Bolei Zhou, and Stephen Lin. Instance localization for self-supervised detection pretraining. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021.
Chenhongyi Yang. Zehao Huang, and Naiyan Wang. Querydet: Cascaded sparse query for accel- erating high-resolution small object detection. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2022.
Mingkai Zheng, Shan You, Fei Wang, Chen Qian, Changshui Zhang, Xiaogang Wang, and Chang Xu. Ressl: Relational self-supervised learning with weak augmentation. Advances in Neural Information Processing Systems, 2021.
Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable DETR: De- formable transformers for end-to-end object detection. In International Conference on Learning Representations, 2021.
13


Reviews:
[{"review_id": "UcmHk1XU3nP", "reviewer": "Reviewer_9EDD", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a novel contrastive pre-training strategy for object detection. The method uses object proposals generated by Selective Search for contrastive learning, which avoids large batch requirement in the existing work. And the localization of the proposals are considered during sampling stage, to further improve the training efficiency. The proposed strategy shows clear improvement over baselines.", "summary_of_the_review": "Overall speaking I think it's a good paper. ", "strength_and_weaknesses": "The paper is easy to understand and the experiment are very comprehensive. The idea of using localization information to sample proposals for pre-training is interesting to me. Overall speaking I think it's a good paper. \n\nI have two questions of the paper: (i) what's the training time of the proposed method compared with the existing work. In addition to large batch training, the contrastive pre-training is also time-consuming; (ii) the proposed pre-training model is used for fsod, andI feel the proposed strategy is general and can also be applied in generic object detection. What's the performance for generic object detection? ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "SqApJBTY-o", "reviewer": "Reviewer_cxY8", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper presents an overall pretraining approach for object detection with a fixed pretrained backbone. A transformer-based detector is used.\nThe main contribution of the paper is the approach to selecting positive examples in a contrastive learning method. ", "summary_of_the_review": "My recommendation rating for the paper is to reject. The main reason is that some claims are not well-supported as identified in the weaknesses.", "strength_and_weaknesses": "Strength:\n- The overall paper is well-written and defined. It is easy enough to follow and reproduce\n\nWeaknesses:\n- The reasoning for the design choice of a transformer-based detector is not well-supported. The paper's claim of the high amount of object proposals is suitable for contrastive learning. How about other types of detectors or considering low-score predictions as object proposals?\n- Considering overlapping objects as other positives may not work well in other cases. How about crowd scenes?\n- It is not proven the paper's claim of less costly than previous approaches.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "4Nlu53y7a9", "reviewer": "Reviewer_Sfkk", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes to solve the challenging large batch size for contrast learning in unsupervised object detection pretraining. They conducted experiments with the Transformer-based detector and found that the diversity of proposals generated by the detector can effectively mitigate this problem. Thus, they propose ProSeCo, which improves the state-of-the-art on the fewer data pretraining task. They also propose a novel localization-aware proposal matching loss and apply it in the feature-consistent student-teacher architecture, which will be very useful for object detection. I think the precision of the proposed method is obviously improved, and the pre-training for unsupervised object detection deserves more in-depth study.", "summary_of_the_review": "See Strength And Weaknesses", "strength_and_weaknesses": "Strengths\n+ The contributions are clearly illustrated and validated. The ablation experiments clearly reflect these contributions. I think ProSeCo is on the right way of exploration.\n+ The illustrations and explanations are clear.\n+ The proposed method outperforms others by a large margin.\n\n\nWeaknesses\n- In the Introduction, after analyzing the local feature consistency, there is no description of its application and advantages, and the relationship with the whole approach is not clearly elaborated.\n- The red part in Figure 3 is not clearly explained, which raises confusion.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: notable-top-25%