Original: P ROPOSAL -C ONTRASTIVE P RETRAINING FOR O BJECT D ETECTION FROM F EWER D ATA
Revised: P ROPOSAL -C ONTRASTIVE P RETRAINING FOR O BJECT D ETECTION FROM F EWER D ATA

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: The use of pretrained deep neural networks represents an attractive way to achieve strong results with few data available.
Revised: The use of pretrained deep neural networks represents an attractive way to achieve strong results with few data available.

Original: When specialized in dense problems such as object detection, learning local rather than global information in images has proven to be more efficient.
Revised: When specialized in dense problems such as object detection, learning local rather than global information in images has proven to be more efficient.

Original: However, for unsupervised pretraining, the popular contrastive learning requires a large batch size and, therefore, a lot of resources.
Revised: However, for unsupervised pretraining, the popular contrastive learning requires a large batch size and, therefore, a lot of resources.

Original: To address this problem, we are interested in transformer-based object detectors that have recently gained traction in the community with good performance and with the particularity of generating many diverse object proposals.
Revised: To address this problem, we are interested in transformer-based object detectors that have recently gained traction in the community with good performance and with the particularity of generating many diverse object proposals.

Original: In this work, we present Proposal Selection Contrast (ProSeCo), a novel unsupervised overall pretraining approach that leverages this property.
Revised: In this work, we present Proposal Selection Contrast (ProSeCo), a novel unsupervised overall pretraining approach that leverages this property.

Original: ProSeCo uses the large number of object proposals generated by the detector for contrastive learning, which allows the use of a smaller batch size, combined with object-level features to learn local information in the images.
Revised: ProSeCo uses the large number of object proposals generated by the detector for contrastive learning, which allows the use of a smaller batch size, combined with object-level features to learn local information in the images.

Original: To improve the effectiveness of the contrastive loss, we introduce the object location information in the selection of positive examples to take into account multiple overlapping object proposals.
Revised: To improve the effectiveness of the contrastive loss, we introduce the object location information in the selection of positive examples to take into account multiple overlapping object proposals.

Original: When reusing pretrained backbone, we advocate for consistency in learning local information between the backbone and the detection head.
Revised: When reusing pretrained backbone, we advocate for consistency in learning local information between the backbone and the detection head.

Original: We show that our method outperforms state of the art in unsupervised pretraining for object detection on standard and novel benchmarks in learning with fewer data.
Revised: We show that our method outperforms state of the art in unsupervised pretraining for object detection on standard and novel benchmarks in learning with fewer data.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: In recent years, we have seen a surge in research on unsupervised pretraining.
Revised: In recent years, we have seen a surge in research on unsupervised pretraining.

Original: For some popular tasks such as Image Classification or Object detection, initializing with a pretrained backbone helps train big architectures more efficiently (Chen et al., 2020b; Caron et al., 2020;
Revised: For some popular tasks such as Image Classification or Object detection, initializing with a pretrained backbone helps train big architectures more efficiently (Chen et al., 2020b; Caron et al., 2020;

Original: He et al., 2020).
Revised: He et al., 2020).

Original: While gathering data is not difficult in most cases, its labeling is always time-consuming and costly.
Revised: While gathering data is not difficult in most cases, its labeling is always time-consuming and costly.

Original: Pretraining leverages huge amounts of unlabeled data and helps achieve better performance with fewer data and fewer iterations, when finetuning the pretrained models afterwards.
Revised: Pretraining leverages huge amounts of unlabeled data and helps achieve better performance with fewer data and fewer iterations, when finetuning the pretrained models afterwards.

Original: The design of pretraining tasks for dense problems such as Object Detection has to take into account the fine-grained information in the image.
Revised: The design of pretraining tasks for dense problems such as Object Detection has to take into account the fine-grained information in the image.

Original: Furthermore, complex object detectors contain different specific parts that can be either pretrained independently (Xiao et al., 2021; Xie et al., 2021; Wang et al., 2021a; H´enaff et al., 2021; Dai et al., 2021b; Bar et al., 2022) or jointly (Wei et al., 2021).
Revised: Furthermore, complex object detectors contain different specific parts that can be either pretrained independently (Xiao et al., 2021; Xie et al., 2021; Wang et al., 2021a; H´enaff et al., 2021; Dai et al., 2021b; Bar et al., 2022) or jointly (Wei et al., 2021).

Original: The different pretraining possibilities for Object Detection in the literature are illustrated in Figure 1.
Revised: The different pretraining possibilities for Object Detection in the literature are illustrated in Figure 1.

Original: A pretraining of the backbone tailored to dense tasks has been the subject of many recent efforts (Xiao et al., 2021; Xie et al., 2021;
Revised: A pretraining of the backbone tailored to dense tasks has been the subject of many recent efforts (Xiao et al., 2021; Xie et al., 2021;

Original: Wang et al., 2021a; H´enaff et al., 2021) ( Backbone Pretraining ), but few have been interested in incorporating the detection-specific components of the architectures during pretraining (Dai et al., 2021b; Bar et al., 2022; Wei et al., 2021)
Revised: Wang et al., 2021a; H´enaff et al., 2021) ( Backbone Pretraining ), but few have been interested in incorporating the detection-specific components of the architectures during pretraining (Dai et al., 2021b; Bar et al., 2022; Wei et al., 2021)

Original: ( Overall Pretraining ).
Revised: ( Overall Pretraining ).

Original: Among them,
Revised: Among them,

Original: SoCo (Wei et al., 2021) focuses on convolutional architectures and pretrains the whole detector, i.e. the backbone along with the detection heads (approach e. in Figure 1), whereas UP-DETR (Dai et al., 2021b) and
Revised: SoCo (Wei et al., 2021) focuses on convolutional architectures and pretrains the whole detector, i.e. the backbone along with the detection heads (approach e. in Figure 1), whereas UP-DETR (Dai et al., 2021b) and

Original: DETReg (Bar et al., 2022) pretrain only the transformers (Vaswani et al., 2017) in transformer-based object detectors (Carion et al., 2020; Zhu et al., 2021) and keep the backbone fixed (approach c. in Figure 1).
Revised: DETReg (Bar et al., 2022) pretrain only the transformers (Vaswani et al., 2017) in transformer-based object detectors (Carion et al., 2020; Zhu et al., 2021) and keep the backbone fixed (approach c. in Figure 1).

Original: Due to the numerous parameters that must be learned and the huge number of iterations needed because of random initialization, pretraining the entire detection model is expensive (Figure 1, e. ).
Revised: Due to the numerous parameters that must be learned and the huge number of iterations needed because of random initialization, pretraining the entire detection model is expensive (Figure 1, e. ).

Original: On the other hand, pretraining only the detection-specific parts with a fixed backbone leads to fewer 
Revised: On the other hand, pretraining only the detection-specific parts with a fixed backbone leads to fewer parameters and allows leveraging strong pretrained backbones already available. However, fully relying on aligning embeddings given by the fixed backbone during

Original: [Figure]
Revised: [Figure]

Original: Backbone Pretraining
Revised: Backbone Pretraining

Revised: Overall Pretraining

Original: [Figure]
Revised: [Figure]

Original: parameters and allows leveraging strong pretrained backbones already available.

Original: However, fully relying on aligning embeddings given by the fixed backbone during pretraining and those given by the detection head, as done in DETReg or UP-DETR, introduces a discrepancy in the information contained in the features (Figure 1, c. ).
Revised: pretraining and those given by the detection head, as done in DETReg or UP-DETR, introduces a discrepancy in the information contained in the features (Figure 1, c. ).

Original: Indeed, while the pretrained backbone has been trained to learn image-level features, the object detector must understand object-level information in the image.
Revised: Indeed, while the pretrained backbone has been trained to learn image-level features, the object detector must understand objectlevel information in the image.

Revised: Aligning inconsistent features hinders the pretraining quality.

Original: In this work, we propose Proposal Selection Contrast (ProSeCo), an unsupervised pretraining method using transformer-based detectors with a fixed pretrained backbone.
Revised: In this work, we propose Proposal Selection Contrast (ProSeCo), an unsupervised pretraining method using transformer-based detectors with a fixed pretrained backbone.

Original: ProSeCo makes use of two models.
Revised: ProSeCo makes use of two models.

Original: The first one aims to alleviate the discrepancy in the features by using the whole detection model.
Revised: The first one aims to alleviate the discrepancy in the features by maintaining a copy of the whole detection model.

Original: This model is referred to as a teacher in charge of the object proposals embeddings, and is updated through an Exponential Moving Average (EMA) of another student network making the object predictions and using a similar architecture.
Revised: This model is referred to as a teacher in charge of the object proposals embeddings, and is updated through an Exponential Moving Average (EMA) of another student network making the object predictions and using a similar architecture.

Original: This latter network is trained by a contrastive learning approach that leverages the high number of object proposals that can be obtained from the detectors.
Revised: This latter network is trained by a contrastive learning approach that leverages the high number of object proposals that can be obtained from the detectors.

Original: This methodology, in addition to the absence of batch normalization in the architectures, reduces the need for a large batch size.
Revised: This methodology, in addition to the absence of batch normalization in the architectures, reduces the need for a large batch size.

Original: We further adapt the contrastive loss commonly used in pretraining to take into account the locations of the object proposals in the image, which is crucial in object detection.
Revised: We further adapt the contrastive loss commonly used in pretraining to take into account the locations of the object proposals in the image, which is crucial in object detection.

Original: In addition, the localization task is independently learned through region proposals generated by Selective Search (Uijlings et al., 2013).
Revised: In addition, the localization task is independently learned through region proposals generated by Selective Search (Uijlings et al., 2013).

Original: Our contributions are summarized as:
Revised: Our contributions are summarized as:

Original: • We propose Proposal Selection Contrast (ProSeCo) , a contrastive learning method tailored for pretraining transformer-based object detectors.
Revised: • We propose Proposal Selection Contrast (ProSeCo) , a contrastive learning method tailored for pretraining transformer-based object detectors.

Original: • We introduce the information of the localization of object proposals for the selection of positive examples in the contrastive loss to improve its efficiency for pretraining.
Revised: • We introduce the information of the localization of object proposals for the selection of positive examples in the contrastive loss to improve its efficiency for pretraining.

Original: • We show that our proposed ProSeCo outperforms previous pretraining methods for transformer-based object detectors on standard benchmarks as well as novel benchmarks.
Revised: • We show that our proposed ProSeCo outperforms previous pretraining methods for transformer-based object detectors on standard benchmarks as well as novel benchmarks.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Supervised Object Detection with transformer-based architectures Object Detection is an important and extensively researched problem in computer vision (Girshick et al., 2014; Girshick, 2015;
Revised: Supervised Object Detection with transformer-based architectures Object Detection is an important and extensively researched problem in computer vision (Girshick et al., 2014; Girshick, 2015;

Original: Ren et al., 2015;
Revised: Ren et al., 2015;

Original: Redmon et al., 2016; Liu et al., 2016; Lin et al., 2017; Tian et al., 2019).
Revised: Redmon et al., 2016; Liu et al., 2016; Lin et al., 2017; Tian et al., 2019).

Original: It essentially combines object localization and classification tasks.
Revised: It essentially combines object localization and classification tasks.

Original: Recently, a novel detector based on an encoder-decoder architecture using transformers (Vaswani et al., 2017) has been proposed in Carion et al.
Revised: Recently, a novel detector based on an encoder-decoder architecture using transformers (Vaswani et al., 2017) has been proposed in Carion et al.

Original: The training complexity of this architecture was subsequently improved in
Revised: The training complexity of this architecture was subsequently improved in

Original: Deformable DETR (Def. DETR)
Revised: Deformable DETR (Def. DETR)

Original: (Zhu et al., 2021), by changing the attention operations into de- formable attention, resulting in a faster convergence speed.
Revised: (Zhu et al., 2021), by changing the attention operations into de- formable attention, resulting in a faster convergence speed.

Original: Several other follow-up works (Dai et al., 2021a;
Revised: Several other follow-up works (Dai et al., 2021a;

Original: Meng et al., 2021; Wang et al., 2021b;
Revised: Meng et al., 2021; Wang et al., 2021b;

Original: Liu et al., 2022; Yang et al., 2022; Li et al., 2022) have also focused on increasing the training efficiency of DETR.
Revised: Liu et al., 2022; Yang et al., 2022; Li et al., 2022) have also focused on increasing the training efficiency of DETR.

Original: Transformer-based architectures now represent a strong alternative to traditional convolutional object detectors, reaching better performance for a similar training cost.
Revised: Transformer-based architectures now represent a strong alternative to traditional convolutional object detectors, reaching better performance for a similar training cost.

Original: Furthermore, recent work have shown strong results of transformer-based detectors in a data-scarce setting (Bar et al., 2022), compared to convolutional architectures (Liu et al., 2020).
Revised: Furthermore, recent work have shown strong results of transformer-based detectors in a data-scarce setting (Bar et al., 2022), compared to convolutional architectures (Liu et al., 2020), which we also observe and discuss in Appendix H.

Original: Self-supervised and unsupervised pretraining backbone architectures Recent advances in self-supervised pretraining (Grill et al., 2020; Caron et al., 2020;
Revised: Self-supervised and unsupervised pretraining backbone architectures Recent advances in self-supervised pretraining (Grill et al., 2020; Caron et al., 2020;

Original: Chen et al., 2020b; 2021; Zheng et al., 2021; Denize et al., 2021) have achieved strong results for obtaining general representations that transfer well to image classification (He et al., 2020).
Revised: Chen et al., 2020b; 2021; Zheng et al., 2021; Denize et al., 2021) have achieved strong results for obtaining general representations that transfer well to image classification (He et al., 2020).

Original: Early works proposed pretext tasks (Alexey et al., 2015; Noroozi & Favaro, 2016; Komodakis & Gidaris, 2018), which are now outperformed by Contrastive Learning (Oord et al., 2018; Wu et al., 2018;
Revised: Early works proposed pretext tasks (Alexey et al., 2015; Noroozi & Favaro, 2016; Komodakis & Gidaris, 2018), which are now outperformed by Contrastive Learning (Oord et al., 2018; Wu et al., 2018;

Original: He et al., 2020; Chen et al., 2020a; Misra & Maaten, 2020; Grill et al., 2020; Caron et al., 2020; Chen et al., 2020b; 2021; Denize et al., 2021).
Revised: He et al., 2020; Chen et al., 2020a; Misra & Maaten, 2020; Grill et al., 2020; Caron et al., 2020; Chen et al., 2020b; 2021; Denize et al., 2021).

Original: This paradigm relies on instance discrimination using a pair of positive views from the same input contrasted with all other instances in the batch, called negatives.
Revised: This paradigm relies on instance discrimination using a pair of positive views from the same input contrasted with all other instances in the batch, called negatives.

Original: However, the InfoNCE objective function (Oord et al., 2018) widely used for contrastive learning and its recent improved version, SCE (Denize et al., 2021), both require a large amount of negative instances to be effective (Wang & Isola, 2020).
Revised: However, the InfoNCE objective function (Oord et al., 2018) widely used for contrastive learning and its recent improved version, SCE (Denize et al., 2021), both require a large amount of negative instances to be effective (Wang & Isola, 2020).

Original: The improvements observed using a general self-supervised pretraining are less significant for more complex, dense downstream tasks (He et al., 2019;
Revised: The improvements observed using a general self-supervised pretraining are less significant for more complex, dense downstream tasks (He et al., 2019;

Original: Reed et al., 2022).
Revised: Reed et al., 2022).

Original: To address this issue, recent approaches have started investigating pretraining approaches tailored for these tasks by imposing local consistency, either at the pixel or region-level : they respectively propose to match in the representation space the features corresponding to the same location in the input space (O Pinheiro et al., 2020; Xie et al., 2021; Wang et al., 2021a), or apply local consistency between features from regions in the image (Roh et al., 2021; Yang et al., 2021; Xiao et al., 2021).
Revised: To address this issue, recent approaches have started investigating pretraining approaches tailored for these tasks by imposing local consistency, either at the pixel or region-level : they respectively propose to match in the representation space the features corresponding to the same location in the input space (O Pinheiro et al., 2020; Xie et al., 2021; Wang et al., 2021a), or apply local consistency between features from regions in the image (Roh et al., 2021; Yang et al., 2021; Xiao et al., 2021).

Original: Unsupervised Pretraining for the overall detection model Few approaches in the literature have tackled the problem of pretraining the detection-specific parts of the architecture, along with the backbone (Wei et al., 2021), or independently (Dai et al., 2021b;
Revised: Unsupervised Pretraining for the overall detection model Few approaches in the literature have tackled the problem of pretraining the detection-specific parts of the architecture, along with the backbone (Wei et al., 2021), or independently (Dai et al., 2021b;

Original: Bar et al., 2022).
Revised: Bar et al., 2022).

Original: SoCo (Wei et al., 2021) proposes a pretraining strategy for convolutional detectors inspired by BYOL (Grill et al., 2020).
Revised: SoCo (Wei et al., 2021) proposes a pretraining strategy for convolutional detectors inspired by BYOL (Grill et al., 2020).

Original: Object locations are generated using Selective Search (Uijlings et al., 2013), and then objectlevel features are extracted and contrasted with each other using a teacher-student strategy.
Revised: Object locations are generated using Selective Search (Uijlings et al., 2013), and then objectlevel features are extracted and contrasted with each other using a teacher-student strategy.

Original: The small amount of object proposals and object features generated requires using a large batch size for the contrastive loss to be effective.
Revised: The small amount of object proposals and object features generated requires using a large batch size for the contrastive loss to be effective.

Original: Pretraining the backbone along with the detection modules this way makes the method difficult and costly to train due to the high amount of parameters to learn.
Revised: Pretraining the backbone along with the detection modules this way makes the method difficult and costly to train due to the high amount of parameters to learn.

Original: For transformer-based architectures, UP-DETR
Revised: For transformer-based architectures, UP-DETR

Original: (Dai et al., 2021b) and DETReg (Bar et al., 2022) use a fixed pretrained backbone to extract features respectively from random patch, or from object locations given by Selective Search (Uijlings et al., 2013), then pretrain the detector by localizing and reconstructing the features of the patch extracted from the input images.
Revised: (Dai et al., 2021b) and DETReg (Bar et al., 2022) use a fixed pretrained backbone to extract features respectively from random patch, or from object locations given by Selective Search (Uijlings et al., 2013), then pretrain the detector by localizing and reconstructing the features of the patch extracted from the input images.

Original: However, since the features to reconstruct are obtained by a backbone which was trained to encode image-level information, there is a discrepancy in the information between the features to match.
Revised: However, since the features to reconstruct are obtained by a backbone which was trained to encode image-level information, there is a discrepancy in the information between the features to match.

Original: Our proposed ProSeCo is designed specifically for transformer-based detectors, and use a fixed backbone pretrained for local information .
Revised: Our proposed ProSeCo is designed specifically for transformer-based detectors, and use a fixed backbone pretrained for local information .

Original: In this work, we leverage the high amount of object proposals generated by transformer-based detectors as instances for contrastive learning.
Revised: In this work, we leverage the high amount of object proposals generated by transformer-based detectors as instances for contrastive learning.

Original: Target object-level features and localizations are provided by a teacher detection model updated through
Revised: Target object-level features and localizations are provided by a teacher detection model updated through

Original: EMA, inspired by recent advances in self-supervised and semi-supervised learning (Liu et al., 2020;
Revised: EMA, inspired by recent advances in self-supervised and semi-supervised learning (Liu et al., 2020;

Original: Grill et al., 2020; Denize et al., 2021; Wei et al., 2021).
Revised: Grill et al., 2020; Denize et al., 2021; Wei et al., 2021).

Original: The student detection model is pretrained by computing the contrastive loss between object proposals given by the student and teacher detectors.
Revised: The student detection model is pretrained by computing the contrastive loss between object proposals given by the student and teacher detectors.

Original: The large number of proposals generated by transformer-based detectors alleviates the need for a large batch size for the contrastive loss.
Revised: The large number of proposals generated by transformer-based detectors alleviates the need for a large batch size for the contrastive loss.

Original: The contrastive loss function used is further improved by introducing the location of objects for selecting positive proposals.
Revised: The contrastive loss function used is further improved by introducing the location of objects for selecting positive proposals.

Original:  O VERVIEW OF THE APPROACH
Revised:  O VERVIEW OF THE APPROACH

Original: We present in this section our proposed unsupervised pretraining approach, illustrated in Figure 2, beginning with the data processing pipeline.
Revised: We present in this section our proposed unsupervised pretraining approach, illustrated in Figure 2, beginning with the data processing pipeline.

Original: Then, we detail the contrastive loss used with the
Revised: Then, we detail the contrastive loss used with the

Original: [Figure]
Revised: [Figure]

Original: localization-aware positive object proposal selection.
Revised: localization-aware positive object proposal selection.

Original: The transformer-based detectors are built on a general architecture that consists of a backbone encoder ( e.g. a ResNet-50), followed by several transformers encoder and decoder layers, and finally two prediction heads for the bounding boxes coordinates and class logits (Carion et al., 2020; Zhu et al., 2021).
Revised: The transformer-based detectors are built on a general architecture that consists of a backbone encoder ( e.g. a ResNet-50), followed by several transformers encoder and decoder layers, and finally two prediction heads for the bounding boxes coordinates and class logits (Carion et al., 2020; Zhu et al., 2021).

Original: 3. D ATA PROCESSING PIPELINE
Revised: 3. D ATA PROCESSING PIPELINE

Original: Throughout the section, we assume to have sampled a batch of unlabeled images x = { x i } N b i =1 , with x i the i th image and N b the batch size.
Revised: Throughout the section, we assume to have sampled a batch of unlabeled images x = { x i } N b i =1 , with x i the i th image and N b the batch size.

Original: For each input image x i , we compute two different views with two asymmetric distributions of augmentations T 1 and T 2 : a weakly augmented view x i ′ =
Revised: For each input image x i , we compute two different views with two asymmetric distributions of augmentations T 1 and T 2 : a weakly augmented view x i ′ =

Original: t 1 ( x i ) , with t 1 ∼ T 1 , and a strongly augmented view obtained from the weakly augmented one x i ′′ =
Revised: t 1 ( x i ) , with t 1 ∼ T 1 , and a strongly augmented view obtained from the weakly augmented one x i ′′ =

Original: t 2 ( t 1 ( x i )) , with t 2 ∼ T 2 .
Revised: t 2 ( t 1 ( x i )) , with t 2 ∼ T 2 .

Original: To guide the model into discovering localization of objects in unlabeled images, we use bounding boxes obtained from the Selective Search algorithm (Uijlings et al., 2013), similarly to previous work (Wei et al., 2021; Bar et al., 2022).
Revised: To guide the model into discovering localization of objects in unlabeled images, we use bounding boxes obtained from the Selective Search algorithm (Uijlings et al., 2013), similarly to previous work (Wei et al., 2021; Bar et al., 2022).

Original: Since Selective Search is deterministic and the generated boxes are not ordered, we compute the boxes for all images offline and, at training time, randomly sample K boxes b SSi = { b SS ( i,j ) ∈ R 4 } Kj =1 for each image in the batch.
Revised: Since Selective Search is deterministic and the generated boxes are not ordered, we compute the boxes for all images offline and, at training time, randomly sample K boxes b SSi = { b SS ( i,j ) ∈ R 4 } Kj =1 for each image in the batch.

Original: Then, the two views and the corresponding boxes sampled are used as input for our method.
Revised: Then, the two views and the corresponding boxes sampled are used as input for our method.

Original: 3. P RETRAINING METHOD
Revised: 3. P RETRAINING METHOD

Original: As shown in Figure 2, our approach is composed of a student-teacher architecture (Grill et al., 2020;
Revised: As shown in Figure 2, our approach is composed of a student-teacher architecture (Grill et al., 2020;

Original: Denize et al., 2021; Wei et al., 2021).
Revised: Denize et al., 2021; Wei et al., 2021).

Original: With ProSeCo, we extend the student-teacher pretraining for transformer-based detectors to tackle the discrepancy in information-level when aligning features, and introduce a dual unsupervised bipartite matching presented below.
Revised: With ProSeCo, we extend the student-teacher pretraining for transformer-based detectors to tackle the discrepancy in information-level when aligning features, and introduce a dual unsupervised bipartite matching presented below.

Original: First, the backbone and the detection heads are respectively initialized from pretrained weights and randomly for both the student and teacher models.
Revised: First, the backbone and the detection heads are respectively initialized from pretrained weights and randomly for both the student and teacher models.

Original: The teacher model is updated through an Exponential Moving Average (EMA) of the student’s weights at every training iteration.
Revised: The teacher model is updated through an Exponential Moving Average (EMA) of the student’s weights at every training iteration.

Original: For both mod- els, the classification heads in the detectors are replaced by an MLP, called projector , for obtaining latent representations of the objects.
Revised: For both models, the classification heads in the detectors are replaced by an MLP, called projector , for obtaining latent representations of the objects.

Original: From the weakly augmented view x i ′ , the teacher model provides object proposals
Revised: From the weakly augmented view x i ′ , the teacher model provides object proposals

Original: y i
Revised: y i

Original: =
Revised: =

Original: y ( i,j ) } Nj =

Revised: y ( i,j ) } Nj =

Original: = { ( z ( i,j ) , b ( i,j ) } Nj =1 , with z ( i,j ) the latent embedding and b ( i,j ) the coordinates of the j th object found.
Revised: = { ( z ( i,j ) , b ( i,j ) } Nj =1 , with z ( i,j ) the latent embedding and b ( i,j ) the coordinates of the j th object found.

Original: The student model infers predictions ˆ y i
Revised: The student model infers predictions ˆ y i

Original: =
Revised: =

Original: { ˆ y ( i,j ) } Nj =

Revised: { ˆ y ( i,j ) } Nj =

Original: = { (ˆ z ( i,j ) , ˆ b ( i,j ) } Nj =1 from the corresponding strongly augmented view x i ′′ .
Revised: = { (ˆ z ( i,j ) , ˆ b ( i,j ) } Nj =1 from the corresponding strongly augmented view x i ′′ .

Original: Then, we apply an unsupervised Hungarian algorithm for proposal matching to find from all the permutations of N elements S N , the optimal bipartite matching ˆ σ prop i between the predictions
Revised: Then, we apply an unsupervised Hungarian algorithm for proposal matching to find from all the permutations of N elements S N , the optimal bipartite matching ˆ σ prop i between the predictions

Original: ˆ y i of the student and the object proposals
Revised: ˆ y i of the student and the object proposals

Original: y i of the teacher:
Revised: y i of the teacher:

Original: [Equation]
Revised: [Equation]

Original: Therefore, for each image x i , the j th proposal y ( i,j ) found by the teacher is associated to the ˆ σ prop i ( j ) th prediction of the student ˆ y ( i, ˆ σ prop
Revised: Therefore, for each image x i , the j th proposal y ( i,j ) found by the teacher is associated to the ˆ σ prop i ( j ) th prediction of the student ˆ y ( i, ˆ σ prop

Original: i
Revised: i

Original: ( j )) .
Revised: ( j )) .

Original: Our matching cost L prop match for the Hungarian algorithm takes into account both features and bounding box predictions through a linear combination of features similarity L sim ( z ( i,j ) ,
Revised: Our matching cost L prop match for the Hungarian algorithm takes into account both features and bounding box predictions through a linear combination of features similarity L sim ( z ( i,j ) ,

Original: ˆ z ( i,σ ( j )) )
Revised: ˆ z ( i,σ ( j )) )

Original: =
Revised: =

Original: ⟨ z ( i,j ) , ˆ z ( i,σ ( j ))
Revised: ⟨ z ( i,j ) , ˆ z ( i,σ ( j ))

Original: ⟩ ∥ z ( i,j ) ∥ 2 ·∥ ˆ z ( i,σ ( j ))
Revised: ⟩ ∥ z ( i,j ) ∥ 2 ·∥ ˆ z ( i,σ ( j ))

Original: ∥ 2 , the ℓ 1 loss of the box coordinates
Revised: ∥ 2 , the ℓ 1 loss of the box coordinates

Original: L coord =
Revised: L coord =

Original: ∥ b ( i,j ) − ˆ b ( i, ˆ σ i ( j ))
Revised: ∥ b ( i,j ) − ˆ b ( i, ˆ σ i ( j ))

Original: ∥ 1 , and the generalized IoU loss L giou from Rezatofighi et al.
Revised: ∥ 1 , and the generalized IoU loss L giou from Rezatofighi et al.

Original: L prop match
Revised: L [Equation]

Original: [Equation]
Revised: [Equation]

Original: Similarly
Revised: Similarly

Original: [Equation]
Revised: [Equation]

Original: we also use an unsupervised Hungarian algorithm for box matching , to find the optimal bipartite matching ˆ σ box i ∈ S N between the predicted boxes ˆ b i of the student and the sampled boxes b SSi from Selective Search, using the matching cost L box match :
Revised: we also use an unsupervised Hungarian algorithm for box matching , to find the optimal bipartite matching ˆ σ box i ∈ S N between the predicted boxes ˆ b i of the student and the sampled boxes b SSi from Selective Search, using the matching cost L box match :

Original: [Equation]
Revised: [Equation]

Original: Finally
Revised: Finally

Original: [Equation]
Revised: [Equation]

Original: the global unsupervised loss L u used for training is a combination of a loss function between the object latent embeddings of the teacher and student models
Revised: the global unsupervised loss L u used for training is a combination of a loss function between the object latent embeddings of the teacher and student models

Original: [Equation]
Revised: [Equation]

Original: and between the object localization of the student predictions and Selective Search boxes.
Revised: and between the object localization of the student predictions and Selective Search boxes.

Original: More formally
Revised: More formally

Original: [Equation]
Revised: [Equation]

Original: it is computed as:
Revised: it is computed as:

Original: [Equation]
Revised: [Equation]

Original: In the above equations, we define λ sim , λ coord , λ giou , λ contrast ∈ R as the coefficients for the different losses.
Revised: In the above equations, we define λ sim , λ coord , λ giou , λ contrast ∈ R as the coefficients for the different losses.

Original: For the consistency in the latent embeddings of the objects, we introduce the object locations information in our contrastive loss L LocSCE .
Revised: For the consistency in the latent embeddings of the objects, we introduce the object locations information in our contrastive loss L LocSCE .

Original: This loss is used to contrast the predictions ˆ y
Revised: This loss is used to contrast the predictions ˆ y

Original: =
Revised: =

Original: { ˆ y i } N b
Revised: { ˆ y i } N b

Original: i
Revised: i

Original: = of the student with object proposals y
Revised: = of the student with object proposals y

Original: = { y i } N b
Revised: = { y i } N b

Original: i =1 found by the teacher, matched according to the proposal matching ˆ σ prop = { ˆ σ prop i } N b
Revised: i =1 found by the teacher, matched according to the proposal matching ˆ σ prop = { ˆ σ prop i } N b

Original: i =1 over the batch.
Revised: i =1 over the batch.

Original: We detail the computations behind this loss in the next section.
Revised: We detail the computations behind this loss in the next section.

Original: 3. L OCALIZATION - AWARE CONTRASTIVE LOSS
Revised: 3. L OCALIZATION - AWARE CONTRASTIVE LOSS

Original: Inspired by advances in self-supervised learning, we propose a contrastive objective function (Oord et al., 2018;
Revised: Inspired by advances in self-supervised learning, we propose a contrastive objective function (Oord et al., 2018;

Original: Chen et al., 2020a; He et al., 2020) between object proposals that also maintains relations 
Revised: Chen et al., 2020a; He et al., 2020) between object proposals that also maintains relations (Zheng et al., 2021; Denize et al., 2021) among these proposals. This objective function extends

Original: [Figure]
Revised: [Figure]

Original: (Zheng et al., 2021; Denize et al., 2021) among these proposals. This objective function extends the latest SCE (Denize et al., 2021) for instance discrimination between object proposals and is illustrated in Figure 3.
Revised: the latest SCE (Denize et al., 2021) for instance discrimination between object proposals and is illustrated in Figure 3.

Original: We compute the contrastive loss between all the object latent embeddings from a batch of image.
Revised: We compute the contrastive loss between all the object latent embeddings from a batch of image.

Original: The positive pair of objects in an image x i is given by the proposal matching σ prop i .
Revised: The positive pair of objects in an image x i is given by the proposal matching σ prop i .

Original: First, we define the distributions of similarity between objects embeddings :
Revised: First, we define the distributions of similarity between objects embeddings :

Original: [Equation]
Revised: [Equation]

Original: The distribution p ′ ( in,jm ) represents the relations between weakly augmented object embeddings scaled by the temperature τ t , and p ′′ ( in,jm ) the similarity between the strongly augmented embeddings and the weakly augmented ones, scaled by τ .
Revised: The distribution p ′ ( in,jm ) represents the relations between weakly augmented object embeddings scaled by the temperature τ t , and p ′′ ( in,jm ) the similarity between the strongly augmented embeddings and the weakly augmented ones, scaled by τ .

Original: Then, the target similarity distribution for the objective function is a weighted combination of one-hot label and the teacher’s embeddings relations:
Revised: Then, the target similarity distribution for the objective function is a weighted combination of one-hot label and the teacher’s embeddings relations:

Original: [Equation]
Revised: [Equation]

Original: To introduce the localization information in our objective, we compute the pairwise Intersection over Union (IoU) between object proposals of the same image and consider overlapping objects as other positives when computing the target similarity distribution:
Revised: To introduce the localization information in our objective, we compute the pairwise Intersection over Union (IoU) between object proposals of the same image and consider overlapping objects as other positives when computing the target similarity distribution:

Original: [Equation]
Revised: [Equation]

Original: where IoU i ( j, m ) corresponds to the IoU between teacher proposals
Revised: where IoU i ( j, m ) corresponds to the IoU between teacher proposals

Original: y
Revised: y

Original: ( i,j ) and y ( i,m ) found in the same image x
Revised: ( i,j ) and y ( i,m ) found in the same image x

Original: i , and δ is the IoU threshold to consider the proposal as a positive example.
Revised: i , and δ is the IoU threshold to consider the proposal as a positive example.

Original: Finally, we use this tailored target similarity distribution in our Localized SCE (LocSCE) loss:
Revised: Finally, we use this tailored target similarity distribution in our Localized SCE (LocSCE) loss:

Original: [Equation]
Revised: [Equation]

Original: Note that we do not match the proposals according to ˆ σ prop in the target similarity, as we compare proposals obtained by the teacher model.
Revised: Note that we do not match the proposals according to ˆ σ prop in the target similarity, as we compare proposals obtained by the teacher model.

Original: We also require the full proposal as input of the loss to compute the pairwise IoU using the box coordinates.
Revised: We also require the full proposal as input of the loss to compute the pairwise IoU using the box coordinates.

Original: Furthermore, we recover the original formulation of SCE when δ = 1 .
Revised: Furthermore, we recover the original formulation of SCE when δ = 1 .

Original: This formulation leads to an effective batch size of N b · N .
Revised: This formulation leads to an effective batch size of N b · N .

Original: This localization-aware contrastive loss function aim to pull together the objects embeddings that overlap subsequently with each others, as they should correspond to the same object in the image.
Revised: This localization-aware contrastive loss function aims to pull together the objects embeddings that overlap subsequently with each others, as they should correspond to the same object in the image.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In this section, we present a comparative study of the results of our proposed method on standard and novel benchmarks for learning with fewer data, as well as an ablative study on the most relevant parts.
Revised: In this section, we present a comparative study of the results of our proposed method on standard and novel benchmarks for learning with fewer data, as well as an ablative study on the most relevant parts.

Original: First, we introduce the datasets, evaluation and training settings.
Revised: First, we introduce the datasets, evaluation and training settings.

Original: 4. I MPLEMENTATION DETAILS
Revised: 4. I MPLEMENTATION DETAILS

Original: Datasets and evaluation We use ImageNet ILSVRC 2012 (IN) (Russakovsky et al., 2015) for pretraining, MS-COCO (COCO)
Revised: Datasets and evaluation We use ImageNet ILSVRC 2012 (IN) (Russakovsky et al., 2015) for pretraining, MS-COCO (COCO)

Original: (Lin et al., 2014) and Pascal VOC 2007 and 2012 (Everingham et al., 2010) for finetuning.
Revised: (Lin et al., 2014) and Pascal VOC 2007 and 2012 (Everingham et al., 2010) for finetuning.

Original: To evaluate the performance in learning with fewer data, following previous work (Wei et al., 2021; Bar et al., 2022), we consider the Mini-COCO benchmarks, where we randomly sample 1%, 5% or 10% of the training data.
Revised: To evaluate the performance in learning with fewer data, following previous work (Wei et al., 2021; Bar et al., 2022), we consider the Mini-COCO benchmarks, where we randomly sample 1%, 5% or 10% of the training data.

Original: Similarly, we also introduce the novel
Revised: Similarly, we also introduce the novel

Original: Mini-VOC benchmark, in which we randomly sample 5% or 10% of the training data.
Revised: Mini-VOC benchmark, in which we randomly sample 5% or 10% of the training data.

Original: We also use the Few-Shot Object Detection (FSOD) dataset (Fan et al., 2020) in a novel FSOD-test benchmark.
Revised: We also use the Few-Shot Object Detection (FSOD) dataset (Fan et al., 2020) in a novel FSOD-test benchmark.

Original: We separate the FSOD test set with 80% of the data randomly sampled for training and the remaining 20% data for testing, by taking care of having at least 1 image for each class in both subsets.
Revised: We separate the FSOD test set with 80% of the data randomly sampled for training and the remaining 20% data for testing, by taking care of having at least 1 image for each class in both subsets.

Original: In all benchmarks, the image ids selected for training and testing will be made available for reproducibility.
Revised: In all benchmarks, the image ids selected for training and testing will be made available for reproducibility.

Original: More details on the datasets and protocols can be found in Appendix A.
Revised: More details on the datasets and protocols can be found in Appendix A.

Original: Pretraining We initialize the backbone with the publicly available pretrained SCRL (Roh et al., 2021) checkpoint and pretrain ProSeCo for 10 epochs on IN.
Revised: Pretraining We initialize the backbone with the publicly available pretrained SCRL (Roh et al., 2021) checkpoint and pretrain ProSeCo for 10 epochs on IN.

Original: The hyperparameters are set as follows:
Revised: The hyperparameters are set as follows:

Original: the EMA keep rate
Revised: the EMA keep rate

Original: parameter to 0.999, the IoU threshold δ = 0 .
Revised: parameter to 0.999, the IoU threshold δ = 0 .

Original: 5 , a batch size of N b = 64 images over 8 A100 GPUs, and the coefficients in the different losses λ sim
Revised: 5 , a batch size of N b = 64 images over 8 A100 GPUs, and the coefficients in the different losses λ sim

Original: = λ contrast = 2 which is the same value used for the coefficient governing the class cross-entropy in the supervised loss.
Revised: = λ contrast = 2 which is the same value used for the coefficient governing the class cross-entropy in the supervised loss.

Original: The projector is defined as a 2-layer MLP with a hidden layer of 4096 and a last layer of 256, without batch normalization.
Revised: The projector is defined as a 2-layer MLP with a hidden layer of 4096 and a last layer of 256, without batch normalization.

Original: Following SCE (Denize et al., 2021), we set the temperatures τ = 0 .
Revised: Following SCE (Denize et al., 2021), we set the temperatures τ = 0 .

Original: 1 , τ t = 0 .
Revised: 1 , τ t = 0 .

Original: 07 and the coefficient λ SCE = 0 .
Revised: 07 and the coefficient λ SCE = 0 .

Original: Similarly to DETReg (Bar et al., 2022), we sample K = 30 boxes from the outputs of Selective Search for each image at every iteration.
Revised: Similarly to DETReg (Bar et al., 2022), we sample K = 30 boxes from the outputs of Selective Search for each image at every iteration.

Original: Other training and architecture hyperparameters are defined as in Def.
Revised: Other training and architecture hyperparameters are defined as in Def.

Original: DETR (Zhu et al., 2021) with, specifically, the coefficients λ coord = 5 and λ giou = 2 , the number of object proposals (queries) N = 300 , and the learning rate is set to lr = 2 · 10 − 4 .
Revised: DETR (Zhu et al., 2021) with, specifically, the coefficients λ coord = 5 and λ giou = 2 , the number of object proposals (queries) N = 300 , and the learning rate is set to lr = 2 · 10 − 4 .

Original: For weak augmentations T 1 , we use a random combination of flip, resize and crop, and for strong augmentations T 2 , we use a random combination of color jittering, grayscale and Gaussian blur.
Revised: For weak augmentations T 1 , we use a random combination of flip, resize and crop, and for strong augmentations T 2 , we use a random combination of color jittering, grayscale and Gaussian blur.

Original: In T 1 , we resize images with the same range of scales as the supervised training protocol on COCO ( Large-scale ).
Revised: In T 1 , we resize images with the same range of scales as the supervised training protocol on COCO ( Large-scale ).

Original: The exact parameters for the augmentations are detailed in Appendix B.
Revised: The exact parameters for the augmentations are detailed in Appendix B, and a discussion about the pretraining cost can be found in Appendix C.

Original: Finetuning protocols For finetuning the pretrained models, we follow the standard supervised learning hyperparameters of Def.
Revised: Finetuning protocols For finetuning the pretrained models, we follow the standard supervised learning hyperparameters of Def.

Original: DETR (Zhu et al., 2021).
Revised: DETR (Zhu et al., 2021).

Original: In all experiments, we train the models with a batch size of 32 images over 8 A100 GPUs until the validation performance stops increasing, i.e. for Mini-COCO, up to 2000 epochs for 1%, 500 epochs for 5%, 400 epochs for 10%, for MiniVOC, up to 2000 epochs for both 5% and 10%, and up to 100 epochs for FSOD-test and PASCAL
Revised: In all experiments, we train the models with a batch size of 32 images over 8 A100 GPUs until the validation performance stops increasing, i.e. for Mini-COCO, up to 2000 epochs for 1%, 500 epochs for 5%, 400 epochs for 10%, for MiniVOC, up to 2000 epochs for both 5% and 10%, and up to 100 epochs for FSOD-test and PASCAL

Original: VOC.
Revised: VOC.

Original: We always decay the learning rates by a factor of 0.1 after about 80% of the number of training epochs.
Revised: We always decay the learning rates by a factor of 0.1 after about 80% of training.

Original:  To compare our method to DETReg (Bar et al., 2022) on our novel benchmarks, we use their publicly available checkpoints from github.
Revised: Experiments with large-scale annotated data are presented in Appendix G. To compare our method to DETReg (Bar et al., 2022) on our novel benchmarks, we use their publicly available checkpoints from github.

Original: 4. F INETUNING AND TRANSFER LEARNING
Revised: 4. F INETUNING AND TRANSFER LEARNING

Original: We evaluate the transfer learning ability of our pretrained model on several datasets.
Revised: We evaluate the transfer learning ability of our pretrained model on several datasets.

Original: Tables 1a and 1b present the results obtained compared to previous methods in the literature when learning from fewer labeled data.
Revised: Tables 1a and 1b present the results obtained compared to previous methods in the literature when learning from fewer labeled data.

Original: We can see that our method outperforms state-of-the-art results in unsupervised pretraining on all benchmarks and datasets, and obtain even more strong results when training data is scarce.
Revised: We can see that our method outperforms state-of-the-art results in unsupervised pretraining on all benchmarks and datasets, and obtain even more strong results when training data is scarce.

Original: The improvement is even more significant as the overall performance with few training data is low.
Revised: The improvement is even more significant as the overall performance with few training data is low.

Original: When using 5% of the COCO training data ( i.e. Mini-COCO 5% in Table 1a), corresponding to about 5.9k images, ProSeCo achieves 28.8 mAP, which represents an improvement of +5.2 percentage point (p.p.) over the supervised pretraining baseline and +2 p.p. over both state-of-
Revised: When using 5% of the COCO training data ( i.e. Mini-COCO 5% in Table 1a), corresponding to about 5.9k images, ProSeCo achieves 28.8 mAP, which represents an improvement of +5.2 percentage point (p.p.) over the supervised pretraining baseline and +2 p.p. over both state-of-the-art overall pretraining methods. Results with all the evaluation metrics are presented in Appendix E.

Original: [Table]
Revised: [Table]

Original: 4. A BLATION S TUDIES
Revised: 4. A BLATION S TUDIES

Original: In the following, we provide several ablation studies for our proposed approach.
Revised: In the following, we provide several ablation studies for our proposed approach.

Original: All experiments and results are compared on the Mini-COCO 5% benchmark.
Revised: All experiments and results are compared on the Mini-COCO 5% benchmark with the pretrained SwAV backbone unless explicitly stated.

Revised: Additional ablation on the number of queries can be found in Appendix F.

Original: Pretraining dataset and backbone In Table 2a, we show the effect of changing the pretraining dataset or the fixed backbone used in our pretraining method.
Revised: Pretraining dataset and backbone In Table 2a, we show the effect of changing the pretraining dataset or the fixed backbone used in our pretraining method.

Original: We can see that using a backbone more adapted to dense tasks that learned local information ( e.g. SCRL) helps the model by having consistent features (+1 p.p.), compared to a backbone pretrained for global features ( e.g. SwAV).
Revised: We can see that using a backbone more adapted to dense tasks that learned local information ( e.g. SCRL) helps the model by having consistent features (+1 p.p.), compared to a backbone pretrained for global features ( e.g. SwAV).

Original: Furthermore, even with a less adapted backbone, our ProSeCo initialized with the SwAV backbone outperforms DETReg (+1.7 p.p.).
Revised: Furthermore, even with a less adapted backbone, our ProSeCo initialized with the SwAV backbone outperforms DETReg (+1.7 p.p.).

Original: To compare with IN, we also pretrain ProSeCo on COCO forepochs.
Revised: To compare with IN, we also pretrain ProSeCo on COCO forepochs.

Original: We obtain better results when pretraining the model on IN than using COCO thanks to the large number of different images in IN (about 10 times the number of images of COCO, leading to +0.4 p.p.), which is consistent with previous findings (Wei et al., 2021).
Revised: We obtain better results when pretraining the model on IN than using COCO thanks to the large number of different images in IN (about 10 times the number of images of COCO, leading to +0.4 p.p.), which is consistent with previous findings (Wei et al., 2021).

Original: The difference in the number of images might be slightly compensated in COCO by the closeness to the downstream task and the diversity of objects.
Revised: The difference in the number of images might be slightly compensated in COCO by the closeness to the downstream task and the diversity of objects.

Original: Localization information in contrastive loss In Table 2b, we show the effect of the localization information in the contrastive loss SCE.
Revised: Localization information in contrastive loss In Table 2b, we show the effect of the localization information in the contrastive loss SCE.

Original: We can see that when introducing multiple positive examples for each image based on the IoU threshold δ ( i.e. ∀ δ < 1 ), we achieve better results than with the original SCE loss ( i.e. δ = 1 ).
Revised: We can see that when introducing multiple positive examples for each image based on the IoU threshold δ ( i.e. ∀ δ < 1 ), we achieve better results than with the original SCE loss ( i.e. δ = 1 ).

Original: Notably, the best results are achieved with δ = 0 .
Revised: Notably, the best results are achieved with δ = 0 .

Original: 5 (+1.7 p.p.).
Revised: 5 (+1.7 p.p.).

Original: More experiments with the InfoNCE loss (Oord et al., 2018) can be found in Appendix C.
Revised: More experiments with the InfoNCE loss (Oord et al., 2018) can be found in Appendix D.

Original: Hyperparameters Table 3 presents an ablation study on different important hyperparameters of our approach.
Revised: Hyperparameters Table 3 presents an ablation study on different important hyperparameters of our approach.

Original: We experimented first with the same batch size applied in Bar et al.
Revised: We experimented first with the same batch size applied in Bar et al.

Original: ( Abl. Batch ), but found that using a smaller batch size ( Base ) leads to improved results (+0.2 p.p.).
Revised: ( Abl. Batch ), but found that using a smaller batch size ( Base ) leads to improved results (+0.2 p.p.).

Original: We evaluated different image scales as a parameter of the weak data augmentations distribution in Abl.
Revised: We evaluated different image scales as a parameter of the weak data augmentations distribution in Abl.

Original: Scales .
Revised: Scales .

Original: Mid-scale corresponds to a resizing of the images such that the shortest edge is betweenand 480 pixels, as used in previous work (Dai et al., 2021b;
Revised: Mid-scale corresponds to a resizing of the images such that the shortest edge is between 320 and 480 pixels, as used in previous work (Dai et al., 2021b;

Original: Bar et al., 2022), and Large-scale to a resize between 480 and 800 pixels, used for supervised learning on COCO.
Revised: Bar et al., 2022), and Large-scale to a resize between 480 and 800 pixels, used for supervised learning on COCO.

Original: Exact values for these parameters can be found in Appendix B.
Revised: Exact values for these parameters can be found in Appendix B.

Original: We found that increasing the size of the images during pretraining is important to have more meaningful information in the boxes, and a more precise localization of the boxes (+0.7 p.p.).
Revised: We found that increasing the size of the images during pretraining is important to have more meaningful information in the boxes, and a more precise localization of the boxes (+0.7 p.p.).

Original: Following the best results from Denize et al.
Revised: Following the best results from Denize et al.

Original: (2021), we evaluated the performance for τ t ∈ { 0 . 05 , 0 . 07 } .
Revised: (2021), we evaluated the performance for τ t ∈ { 0 . 05 , 0 . 07 } .

Original: We found that τ t = 0 .
Revised: We found that τ t = 0 .

Original: 07 leads to the best performance (+0. p.p.).
Revised: 07 leads to the best performance (+0.3 p.p.).

Original: We considered several EMA keep rate parameter values following previous work (He et al., 2020; Wei et al., 2021; Denize et al., 2021), and found that 0.999 achieves the best results (+0.1 p.p.). 
Revised: We considered several EMA keep rate parameter values following previous work (He et al., 2020; Wei et al., 2021; Denize et al., 2021), and found that 0.999 achieves the best results (+0.1 p.p.). [Table]

Original: [Table]
Revised: [Table]

Revised: Best ✓

Original: [Table]
Revised: [Table]

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this work
Revised: In this work

Original: [Table]
Revised: [Table]

Original: we aim to use large unlabeled datasets for an unsupervised pretraining of the overall detection model to improve performance when having access to fewer labeled data.
Revised: we aim to use large unlabeled datasets for an unsupervised pretraining of the overall detection model to improve performance when having access to fewer labeled data.

Original: In this end
Revised: In this end

Original: [Table]
Revised: [Table]

Original: we propose Proposal Selection Contrast (ProSeCo) , a novel pretraining approach for Object Detection.
Revised: we propose Proposal Selection Contrast (ProSeCo) , a novel pretraining approach for Object Detection.

Original: Our method leverages the large number of object proposals generated by transformer-based detectors for contrastive learning, reducing the necessity of a large batch size
Revised: Our method leverages the large number of object proposals generated by transformer-based detectors for contrastive learning, reducing the necessity of a large batch size

Original: [Table]
Revised: [Table]

Original: and introducing the localization information of the objects for the selection of positive examples to contrast.
Revised: and introducing the localization information of the objects for the selection of positive examples to contrast.

Original: We show from various experiments on standard and novel benchmarks in learning with few training data that ProSeCo outperforms previous pretraining methods.
Revised: We show from various experiments on standard and novel benchmarks in learning with few training data that ProSeCo outperforms previous pretraining methods.

Original: Throughout this work, we advocate for consistency in the level of information encoded in the features when pretraining.
Revised: Throughout this work, we advocate for consistency in the level of information encoded in the features when pretraining.

Original: Indeed, learning object-level features during pretraining is more important than image-level when applied to a dense downstream task such as Object Detection.
Revised: Indeed, learning object-level features during pretraining is more important than image-level when applied to a dense downstream task such as Object Detection.

Original: Future work could update the backbone during pretraining to further improve the consistency between the backbone and the detection heads.
Revised: Future work could update the backbone during pretraining to further improve the consistency between the backbone and the detection heads.

Original: R EPRODUCIBILITY S TATEMENT
Revised: R EPRODUCIBILITY S TATEMENT

Original: Throughout this paper, we made sure that the experiments and results are fully reproducible.
Revised: Throughout this paper, we made sure that the experiments and results are fully reproducible.

Original: We explicitly state the exact values of hyperparameters used in Section 4.1, and describe in details the datasets and evaluation protocols in Appendix A.
Revised: We explicitly state the exact values of hyperparameters used in Section 4.1, and describe in details the datasets and evaluation protocols in Appendix A.

Original: All image ids randomly selected when evaluating with few training data (in Mini-COCO, Mini-VOC, FSOD-test) will be made available
Revised: All image ids randomly selected when evaluating with few training data (in Mini-COCO, Mini-VOC, FSOD-test) will be made available

Original: A D ATASETS AND EVALUATION DETAILS

Original: We use different datasets throughout this work that we present below.

Original: • For pretraining purposes, we use the standard ImageNet ILSVRC 2012 (IN) (Russakovsky et al., 2015) dataset, which contains 1.2M training images separated in 1000 class categories.

Original: • We also use the MS-COCO (COCO)

Original: (Lin et al., 2014) dataset for pretraining, but mainly for finetuning and evaluation purposes.

Original: This dataset contains 80 classes of objects and about 118k training images (train2017 subset) .

Original: Performance is evaluated on the val2017 subset.

Original: • For finetuning, we also use the Pascal VOC 2007 and 2012 (VOC 07-12) (Everingham et al., 2010) datasets.

Original: This dataset contains 20 classes of objects, and we use the combination of the trainval subsets from both VOC2007 and VOC2012 for training, corresponding to about 16k training images in total.

Original: Performance is evaluated on the test subset from VOC2007.

Original: • For a more complicated dataset, we use the Few-Shot Object Detection dataset (Fan et al., 2020).

Original: Since the dataset is designed as open-set ,

Original: i

Original: e. with different classes between training and testing, we only use the test set for benchmarking.

Original: We separate the test set into a training and testing subset, by randomly taking 80% of images for training and the 20% remaining for testing.

Original: We make sure that all classes appears at least once in both training and testing subsets.

Original: The images selected for training and testing will be made available for reproducibility.

Original: This separation leads to about 11k training images and 3k testing images.

Original: B A UGMENTATIONS USED

Original: We detail in Table 4 the distributions of augmentations used to create the weak view and the strong view.

Original: The Weak Augmentations follow standard supervised training for transformer-based detectors (Carion et al., 2020; Zhu et al., 2021).

Original: The Strong Augmentations follow typical contrastive learning augmentations (Chen et al., 2020a;

Original: Bar et al., 2022).

Original: [Table]

Original: Color

Original: [Table]

Original: GrayScale

Original: [Table]

Original: Gaussian Blur

Original: [Table]

Original: x

Original: [Table]

Original: sigma y)

Original: =

Original: [Table]

Original: C U SING ANOTHER CONTRASTIVE LOSS

Original: The popular InfoNCE loss (Oord et al., 2018) used for contrastive learning is a similarity based function scaled by the temperature τ that maximizes agreement between the positive pair of in- stances and push negatives away.

Original: However, it suffers from the class collision problem (Cai et al., 2020; Denize et al., 2021), where semantically close instances can be used as negatives in the loss computation, which damages the quality of the representation learned.

Original: Recent work (Zheng et al., 2021; Denize et al., 2021) have tackled this problem by introducing the relational aspect between instances.

Original: In our experimental study, we also considered the InfoNCE loss for ProSeCo.

Original: We formulate the loss as follows:

Original: [Equation]

Original: This formulation leads to an effective batch size of N b · N for the contrastive loss.

Original: Similarly to our

Original: LocSCE , the localization of the objects can be introduced in the InfoNCE loss function to obtain the

Original: LocNCE objective as follows:

Original: [Equation]

Original: In Table 5, we compare the two different contrastive objectives for pretraining.

Original: We can see that using the InfoNCE loss leads to slightly better results (+0.3 p.p.).

Original: However, when using the localization information, SCE benefits much more than InfoNCE (+1.7 p.p. compared to +0.6 p.p.).

Original: This might be that the selection of positives from the localization information helps to introduce easy positive examples, and thanks to this, the relational aspect of SCE can focus on the more difficult positives.

Original: In the end, LocSCE achieves stronger results than LocNCE (+0.8 p.p.).

Original: [Table]

Original: D F ULL EVALUATION METRICS

Original: Tables 6 and 7 provide the results with full evaluation metrics (mAP, AP 50 and AP 75 in %) on

Original: PASCAL VOC, Mini-VOC and FSOD-test benchmarks

Original: [Table]

Original: [Table]

Original: E I NCREASING THE NUMBER OF QUERIES

Original: In Table

Original: [Table]

Original: we provide an ablation on the number of object proposals (queries) N in Def.

Original: DETR, when pretraining with ProSeCo and finetuning afterwards.

Original: A higher N leads to more parameters in the model and longer computing time, but also better performance when initializing with a supervised backbone.

Original: However, it does not benefit to our unsupervised pretraining, even though increasing the number of queries means a higher effective batch size

Original: [Table]

Original: [Table]