Title: PrObeD: Proactive Object Detection Wrapper
Venue: NeurIPS 2023
Accepted: yes
Keywords: Object detection

proactive

Camouflage

2D
Abstract: Previous research in $2D$ object detection focuses on various tasks, including detecting objects in generic and camouflaged images. These works are regarded as passive works for object detection as they take the input image as is. However, convergence to global minima is not guaranteed to be optimal in neural networks; therefore, we argue that the trained weights in the object detector are not optimal. To rectify this problem, we propose a wrapper based on proactive schemes, PrObeD, which enhances the performance of these object detectors by learning a signal. PrObeD consists of an encoder-decoder architecture, where the encoder network generates an image-dependent signal termed templates to encrypt the input images, and the decoder recovers this template from the encrypted images. We propose that learning the optimum template results in an object detector with an improved detection performance. The template acts as a mask to the input images to highlight semantics useful for the object detector. Finetuning the object detector with these encrypted images enhances the detection performance for both generic and camouflaged. Our experiments on MS-COCO, CAMO, COD$10$K, and NC$4$K datasets show improvement over different detectors after applying PrObeD. Our models/codes are available at https://github.com/vishal3477/Proactive-Object-Detection.

PrObeD: Proactive Object Detection Wrapper

Abstract

Previous research in 2 D object detection focuses on various tasks, including detecting objects in generic and camouflaged images. These works are regarded as passive works for object detection as they take the input image as is. However, convergence to global minima is not guaranteed to be optimal in neural networks; therefore, we argue that the trained weights in the object detector are not optimal. To rectify this problem, we propose a wrapper based on proactive schemes, PrObeD, which enhances the performance of these object detectors by learning a signal. PrObeD consists of an encoder-decoder architecture, where the encoder network generates an image-dependent signal termed templates to encrypt the input images, and the decoder recovers this template from the encrypted images. We propose that learning the optimum template results in an object detector with an improved detection performance. The template acts as a mask to the input images to highlight semantics useful for the object detector. Finetuning the object detector with these encrypted images enhances the detection performance for both generic and camouflaged. Our experiments on MS-COCO, CAMO, COD 10 K, and NC 4 K datasets show improvement over different detectors after applying PrObeD. Our models/codes are available at https: //github.com/vishal3477/Proactive-Object-Detection .

1 Introduction

Generic 2 D object detection (GOD) has improved from earlier traditional detectors [15,20,64,65] to the deep-learning-based object detectors [8, 10, 26, 32, 52, 58]. Advancements in deep-learning-based methods underwent many architectural change over recent years, including one-stage [5,43,46,52-54], two-stage [23,24,58], CNN-based [5,14,16,21-23,52,54], transformer-based [8,74], and diffusionbased [10] methods. All these methods aim to predict the 2 D bounding box of the objects in the images and their category class.

Another emerging area related to generic object detection is camouflaged object detection [17,18,2729,34,40] (COD). COD aims to detect and segment objects blended with the background [17,18] via object-level mask supervision. Applications of COD include medical [19,45], surveillance [11] and autonomous driving [69]. Early COD detectors exploit hand-crafted features [50,61] and optical flow [33], while current methods are deep-learning-based. These methods utilize attention [9,63], joint learning [40], image gradient [34], and transformers [48, 70].

All these methods take input images as is for the detection task and hence are called passive methods. However, there is a line of research on proactive methods for a wide range of vision tasks such as disruption [59, 60], tagging [68], manipulation detection [1], and localization [2]. Proactive methods

Abhinav Kumar

Passive object detection

OGOD

Idealweights

OcoD

OPassiveweights

OProactiveweights

Encryption

Template

OGOD

OcoD

W1

Proactive object detection

Applying proactive wrapper

(a)

(b)

(c)

use signals, called templates, to encrypt the input images and pass the encrypted images as the input to the network. These are trained in an end-to-end manner by using either a fixed [68] or learnable template [1, 2, 59, 60] to improve the performance. A major advantage of proactive schemes is that such methods generalize better on unseen data/models [1,2]. Motivated by this, we propose a plug-and-play Proactive Object Detection wrapper, PrObeD, to improve GOD and COD detectors.

Designing PrObeD as a proactive scheme involves several challenges and key factors. First, the proactive wrapper needs to be a plug-and-play module that can be applied to both GOD and COD detectors. Secondly, the encryption process should be intuitive to benefit the object detection task. e.g. , an ideal template for detection should highlight the foreground objects in the input image. Lastly, the choice of supervision to estimate the template for encryption is hard to formulate.

Previous proactive methods [1,2] use learnable but image-independent templates for manipulation and localization tasks. However, the object detection task is scene-specific; therefore, the ideal template should be image-dependent. Based on this key insight, we propose a novel plug-and-play proactive wrapper in which we apply object detectors to enhance detection performance. The PrObeD wrapper utilizes an encoder network to learn an image-dependent template. The learned template encrypts the input images by applying a transformation, defined as an element-wise multiplication between the template and the input image. The decoder network recovers the templates from the encrypted images. We utilize regression losses for supervision and leverage the ground-truth object map to guide the learning process, thereby imparting valuable object semantics to be integrated into the template. We then fine-tune the proactive wrapper with the GOD and COD detectors to improve their detection performance. Extensive experiments on MS-COCO, CAMO, COD 10 K, and NC 4 K datasets show that PrObeD improves the detection performance for both GOD and COD detectors.

In summary, the contributions of this work include:

We propose a novel proactive approach PrObeD for the object detection task. To the best of our knowledge, this is the first work to develop a proactive approach to 2 D object detection.

We mathematically prove that the proactive method results in a better-converged model than the passive detector under assumptions and, consequently, a better object detector.

PrObeD wraps around both GOD and COD detectors and improves detection performance on MS-COCO, CAMO, COD10K, and NC 4 K datasets

2 Related works

Proactive Schemes. Earlier works adopt to add signals like perturbation [60], adversarial noise [59], and one-hot encoding [68] messages while focusing on tasks like disruption [59,60] and deepfake tagging [68]. Asnani et al . [1] propose to learn an optimized template for binary detection by unseen generative models. Recently, MaLP [2] adds the learnable template to perform generalized

manipulation localization for unknown generative models. Unlike these works, PrObeD uses imagedependent templates and is a plug-and-play wrapper for a different task of object detection.

Generic Object Detection Detection of generic objects, instead of specific object categories such as pedestrians [7], apples [13], and others [4, 37, 38], has been a long-standing objective of computer vision. RCNN [24,25] employs the extraction of object proposals. He et al . [31] propose a spatial pooling layer to extract a fixed-length representation of all the objects. Modifications of RCNN [23,41,58,72] increase the inference speed. Feature pyramid network [42] detects objects with a wide variety of scales. The above methods are mostly two-stage, so inference is an issue. Single-stage detectors like YOLO [5,52-54,66], SSD [46], HRNet [67] and RetinaNet [43] increase the speed and simplicity of the framework compared to the two-stage detector. Recently, transformerbased methods [8, 74] use a global-scale receptive field. Chen et al . [10] use diffusion models to denoise noisy boxes at every forward step. PrObeD functions as a wrapper around the pre-existing object detector, facilitating its transformation into an enhanced object detector. The comparison of PrObeD with prior works is summarized in Tab. 1.

Camouflaged Object Detection Early COD works rely on hand-crafted features like co-occurrence matrices [61], 3 D convexity [50], optical flow [33], covariance matrix [35], and multivariate calibration components [57]. Later on, [9, 63] incorporate an attention-based cross-level fusion of multi-scale features to recover contextual information. Mei et al . [49] take motivation by predators to identify camouflaged objects using a position and focus ideology. SINet [18] uses a search and identification module to perform localization. SINET-v2 [17] uses group-reversal attention to extract the camouflaged maps. [36] explores uncertainty maps and [75] utilizes cube-like architecture to integrate multi-layer features. ANet [39], LSR [47], and JCSOD [40] employ joint learning with different tasks to improve COD. Lately, [12,48,70] apply a transformer-based architecture for difficult-aware learning, uncertainty modeling, and temporal consistency. Zhai et al . [73] use a graph learning model to disentangle input into different features for localization. DGNet [34] uses image gradients to exploit intensity changes in the camouflaged object from the background. Unlike these methods, PrObeD uses proactive methods to improve camouflaged object detection.

3 Proposed Approach

Our method originates from understanding what makes proactive schemes effective. We first overview the two detection problems: GOD and COD in Sec. 3.1. We next derive Lemma 1, where we show that the proactive schemes with the multiplicative transformation of images are better than passive schemes by comparing the deviation of trained network weights from the optimal. Based on this result, we derive that Average Precision (AP) from the proactive model is better than AP from the passive model in Theorem 1. At last, we present our proactive scheme-based wrapper, PrObeD, in Sec. 3.3, which builds upon the Theorem 1 to improve generic 2D objects and camouflaged detection.

3.1 Background

3.1.1 Passive Object Detection

Although generic 2 D object detection and camouflage detection are similar problems, they have different objective functions. Therefore, we treat them as two different problems and define their objectives separately.

Generic 2D Object Detection. Let I j be the set of input images given to the generic 2D object detector O with trainable parameters θ . Most of these detectors output two sets of predictions per image: (1) bounding box coordinates, O ( I j ) 1 = ˆ T ∈ R 4 , (2) class logits, O ( I j ) 2 = ˆ C ∈ R C , where N is the number of foreground object categories. If the ground-truth bounding box coordinates are T j , and the ground-truth category label is C , the objective function of such detector is:

Camouflaged Object Detection. Let I j be the input image set given to the camouflaged object detector O with trainable parameters θ , and G j be the ground-truth segmentation map. Prior passive works predict a segmentation map with the following objective:

3.1.2 Proactive Object Detection

Proactive schemes [1, 2] encrypt the input images with the template to aid manipulation detection/localization. Such schemes take an input image I j ∈ R H × W × 3 and learns a template S j ∈ R H × W . PrObeD uses image-dependent templates to improve object detection. Given an input image I j ∈ R H × W × 3 , PrObeD learns to output a template S j ∈ R H × W , which can be used by a transformation T resulting in encrypted images T ( I j ) . PrObeD uses element-wise multiplication as the transformation T , which is defined as:

3.2 Mathematical Analysis of Passive and Proactive Detectors

PrObeD optimizes the template to improve the performance of the object detector. We argue that this template helps arrive at a better global minima representing the optimal parameters θ . We now define the following lemma to support our argument:

Lemma 1. Converged weights of proactive and passive detectors. Consider a linear regression model that regresses an input image I j under an additive noise setup to obtain the 2D coordinates. Assume the noise under consideration e is a normal random variable N (0 , σ 2 ) . Let w and w ∗ denote the trained weights of the pretrained linear regression model and the optimal weights of the linear regression model. Also, assume SGD optimizes the model parameters with decreasing step size s such that the steps are square summable i.e., S = lim t →∞ t ∑ k =1 s 2 k exist, and the noise is independent of

the image. Then, there exists a template S j ∈ [0 , 1] for the image I j such that the multiplicative transformation of images as the input results in a trained weight w ′ closer to the optimal weight than the originally trained weight w . In other words,

The proof of Lemma 1 is in supplementary. We use the variance of the gradient of the encrypted images to arrive at this lemma. We next use Lemma 1 to derive the following theorem:

Theorem 1. AP comparison of proactive and passive detectors. Consider a linear regression model that regresses an input image I j under an additive noise setup to obtain the 2D coordinates. Assume the noise under consideration e is a normal random variable N (0 , σ 2 ) . Let w and w ∗ denote the

Proactivewrapper

Networks

Stage 2

Stage 3

Lossfunctions

S'

Mathematical

operation

P

Training

Inference

S

E

*

Stage 1

T(I; S)

TG

trained weights of the pretrained linear regression model and the optimal weights of the linear regression model. Also, assume SGD optimizes the model parameters with decreasing step size s

t

such that the steps are square summable i.e., S = lim t →∞ ∑ k =1 s 2 k exist, and the noise is independent of the image. Then, the AP of the proactive detector is better than the AP of the passive detector.

The proof of Theorem 1 is in the supplementary. We use the Lemma 1 and the non-decreasing nature of AP w.r.t. IoU to arrive at this theorem. Next, we adapt the objectives of Eqs. (1) and (2) to incorporate the proactive methods as follows:

3.3 PrObeD

Our proposed approach comprises of three stages: template generation, template recovery, and detector fine-tuning. First, we use an encoder network to generate an image-dependent template for image encryption. This encrypted image is further used to recover the template through a decoder network. Finally, the object detector is fine-tuned using the encrypted images. All three stages are trained in an end-to-end fashion. While all the stages are used for training PrObeD, we specifically use only stages 1 and 3 for inference. We will now describe each stage in detail.

3.3.1 Proactive Wrapper

Our proposed approach consists of three stages, as shown in Fig. 2. However, only the first two stages are part of our proposed proactive wrapper, which can be applied to object detector to improve its performance.

Stage 1: Template Generation. Prior works learn a set of templates [1,2] in their proactive schemes. This set of templates is enough to perform the respective downstream tasks as the generative model manipulates the template, which is easy to capture with a set of learnable templates. However, for object detection tasks, every image has unique object characteristics such as size, appearance, and color that can vary significantly. This variability present in the images may exceed the descriptive capacity of a finite set of templates, thereby necessitating the use of image-specific templates to

accurately represent the range of object features present in each image. In other words, a fixed set of templates may not be sufficiently flexible to capture the diversity of visual features across the given set of input images, thus demanding more adaptable, image-dependent templates.

Motivated by the above argument, we propose to generate the template S j for every image using an encoder network. We hypothesize that highlighting the area of the key foreground objects would be beneficial for object detection. Therefore, for GOD, we use the ground-truth bounding boxes T G to generate the pseudo ground-truth segmentation map. Specifically, for any image I j , if the bounding box coordinates are T G j = { x 1 , x 2 , y 1 , y 2 } , we define the pseudo ground-truth segmentation map as:

However, for COD, the dataset already has the ground-truth segmentation map G j , which we use as the supervision for the encoder to output the templates with semantic information of the image to be restricted only in the region of interest for the detector. For both GOD and COD, we minimize the cosine similarity (Cos) between S j and G j as the supervision for the encoder network. The encoder loss J E is as follows:

This generated template acts as a mask for the input image to highlight the object region of interest for the detector. We use this template with the transformation T to encrypt the input image as T ( I j ; S j ) = I j ⊙ S j . As we start from the pretrained model of object detector O , we initialize the bias of the last layer of the encoder as 0 so that for the first few iterations, S j ≈ 1 . This is to ensure that the distribution of I j and T ( I j ; S j ) remains similar for the first few iterations, and O doesn't encounter a sudden change in its input distribution.

Stage 2: Template Recovery. So far, we have discussed the generation of template S j using E , which will be used as a mask to encrypt the input image. The encrypted images are used for two purposes: (1) recovery of templates and (2) fine-tuning of the object detector. The main intuition of recovering the templates is from the prior works on image steganalysis [55, 56] and proactive schemes [1, 2]. Motivated by these works, we draw the following insight: 'To properly learn the optimal template and embed it onto the input images, it is beneficial to recover the template from encrypted images."

To perform recovery, we exploit an encoder-decoder approach. Using this approach leverages the strengths of the encoder network E for feature extraction, capturing the most useful salient details, and the decoder network D for information recovery, allowing for efficient and effective encryption and decryption of the template. We also empirically show that not using the decoder to recover the templates harms the object detection performance.

To supervise D in recovering S j from T ( I j ; S j ) , we propose to maximize the cosine similarity between the recovered template, S ′ j and S j . The decoder loss is as follows:

Stage 3: Detector Fine-tuning. Due to our encryption, the distribution of the images input to the pretrained O changes. Thus, we fine-tune O on the encrypted images T ( I j ; S ) . As proposed in Theorem 1 , given the encrypted images T ( I j ; S ) , we use the pretrained detector O with parameters θ to arrive at a better local minima. Therefore, the general objective of GOD and COD in Eq. (5) and Eq. (6) change to as follows:

We use the detector-specific loss function J OBJ of O along with the encoder and decoder loss in Eq. (7) and Eq. (8) to train all the three stages. The overall loss function J to train PrObeD is as follows:

4 Experiments

We apply PrObeD for two categories of object detectors: GOD and COD.

GOD Baselines. For GOD, we apply PrObeD on four detectors with varied architectures: two-stage, one-stage, and transformer-based detectors, namely, Faster R-CNN [58], YOLO [52], Sparse R-CNN, and DeTR [8]. We use these works as baselines for three reasons: (1) varied architecture types, (2) their increased prevalence in the community, and (3) varied timelines (from earlier to recent detectors). We use the PyTorch [51] code of the respective detectors for our GOD experiments and use the corresponding GODs as our baseline. For YOLOv5 and DeTR, we use the official repositories released by the authors; for Faster R-CNN, we use the public repository "Faster R-CNN.pytorch". For other GOD detectors, we use Detectron2 library as the pre-trained detector. We use the ResNet101 backbone for Faster R-CNN, Sparse R-CNN and DeTR, and CSPDarknet53 for YOLOv5.

COD Baselines. For COD, we apply PrObeD on the current SoTA camouflage detector DGNet [34] and use DGNet as our baseline. For all object detectors, we use the pretrained model released by the authors and fine-tune them with PrObeD. Please see the supplementary for more details.

Datasets. Our experiments use the MS-COCO 2017 [44] dataset for GOD, while we use CAMO [39], COD 10 K [17], and NC 4 K [47] datasets for COD. We use the following splits of these datasets:

MS-COCO 2017 Val Split [44]: It includes 118 , 287 images for training and 5 K for testing.

COD 10 K Val Split [17]: It includes 4 , 046 camouflaged images for training and 2 , 026 for testing.

CAMO Val Split [39]: It includes 1 K camouflaged images for training and 250 for testing.

NC 4 K Val [47]: It includes 4 , 121 NC 4 K images. We use it for generalization testing as in [34].

Evaluation Metrics. We use mean average precision average at multiple thresholds in [0 . 5 , 0 . 95] (AP) for GOD as in [44]. We also report results at threshold of 0 . 5 (AP 50 ), threshold of 0 . 75 (AP 75 ) and at different object sizes: small (AP S ), medium (AP M ), and large (AP L ). For COD, we use E-measure E m , S-measure S m , weighted F1 score wF β and mean absolute error MAE as [34].

4.1 GODResults

Quantitative Results. Tab. 2 shows the results of applying PrObeD on GOD networks. PrObeD improves the average precision of all three detectors. The performance gain is significant for Faster R-CNN. As Faster R-CNN is an older detector, it was at a worse minima to start with. PrObeD improves the convergence weight of Faster R-CNN by a significant margin, thereby improving the performance. We further experiment with two variations of Faster R-CNN, namely, Faster R-CNN +

(a)

(b)

(c)

(d)

(e)

1.2

1.0

1.0

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0.0

0.0

1.2

1.0

1.0

0.8

80

0.6

0.6

0.4

0.4

0.2

0.2

0.0

0.0

(a)

(b)

(c)

(d)

(e)

(f)

2.5

2.0

1.5

1.0

0.5

FPN and Sparse-RCNN. We observe an increase in the performance of both detectors. PrObeD also improves newer detectors like YOLOv5 and DeTR, although the gains are smaller compared to Faster R-CNN. We believe this happens because the newer detectors leave little room for improvement due to which PrObeD improves the performance slightly. We next compare PrObeD with a work that leverage segmentation map as a mask for object detection. We compare our performance with Mask R-CNN [30], which uses an image segmentation branch to help with object detection. Tab. 2 shows that the gains using Mask R-CNN are lower than using our proactive wrapper.

Qualitative Results. Fig. 3 shows qualitative results for the MS-COCO 2017 dataset. PrObeD clearly improves the performance of pretrained Faster R-CNN for three types of errors: Missed predictions, false negatives, and localization errors. PrObeD has a lower number of missed predictions, fewer false positives, and better bounding box localization. We also visualize the generated and recovered templates. We see that the template has object semantics of the input images. When the template is multiplied with the input image, it highlights the foreground objects, thereby making the task of object detector easier.

Error Analysis. We show the error analysis [6] for GOD section 4 of the supplementary. We observe that all GOD detectors make mistakes mainly due to five types of errors: classification, localization, duplicate detection, background detection, and missed detection. The main reason for the degraded performance is the errors in which the foreground-background boundary is missed. These errors include localization, background detection, and missed detection. Our proactive wrapper significantly corrects these errors, as the template has object semantics, which, when multiplied with the input image, highlights the foreground objects, consequently simplifying the task of object detection.

4.2 COD Results

Quantitative Results. Tab. 3 shows the result of applying PrObeD to DGNet [34] on three different datasets. PrObeD, when applied on top of DGNet, outperforms DGNet on all four metrics for all datasets. The biggest gain appears in COD 10 K and NC 4 K datasets. This is impressive as these datasets have more diverse testing images than CAMO. As NC 4 K is only a testing set, the higher performance of PrObeD demonstrates its superior generalizability as compared to DGNet [34]. This result agrees with the observation in [1, 2], where proactive-based approaches exhibit improved generalization on manipulation detection and localization tasks.

Qualitative Results. Fig. 4 visualizes the predicted camouflaged map for DGNet before and after applying PrObeD on testing samples of all three datasets. PrObeD improves the predicted camouflaged map, with less blurriness along the boundaries and better localization of the camouflaged object. As observed before for GOD, the generated and recovered template has the semantics of the camouflaged objects, which after multiplication intensifies the foreground object, resulting in better segmentation by DGNet.

4.3 Ablation Study

Comparison with Proactive Works. The prior proactive works perform a different task of image manipulation detection and localization. Therefore, these works are not directly comparable to our proposed proactive wrapper, which performs a different task of object detection as described in Tab. 1. However, manipulation localization and COD both involve a prediction of a localization map, segmentation, and fakeness map, respectively. This inspires us to experiment with MaLP [2] for the task of COD. We train the localization module of MaLP supervised with the COD datasets. The results are shown in Tab. 4. We see that MaLP is not able to perform well for all three datasets. MaLP is designed for estimating universal templates rather than templates tailored to specific images. It shows the significance of image-specific templates in object detection. While MaLP's design with image-independent templates is effective for localizing image manipulation, applying it to object detection has a negative impact on performance.

Framework Design. PrObeD consists of blocks to improve the object detector. Tab. 5 ablates different versions of PrObeD to highlight the importance of each block in our design. PrObeD utilizes an encoder network E to learn image-dependent templates aiding the detector. We remove the encoder E from our network, replacing it with a fixed template. We observe that the performance deteriorates by a large margin. Next, we make this template learnable as proposed in PrObeD, but only a single template would be used for all the input images. This choice also results in worse performance, highlighting that image-dependent templates are necessary for object detection. Finally, we remove the decoder network D , which is used to recover the template from the encrypted images. Although this results in a better performance than the pretrained Faster R-CNN, we observe a drop as compared to PrObeD. Therefore, as discussed in Sec. 3.3, the recovery of templates is indeed a necessary and beneficial step for boosting the performance of the proactive schemes.

Encryption Process. PrObeD includes an encryption process as described in Eq. (3), which involves multiplying the template with the input image. This process makes the template act as a mask, highlighting the foreground for better detection. However, prior proactive works [1, 2] consider adding templates to achieve better results. Thus, we ablate by changing the encryption process to template addition. Tab. 5 shows that template addition degrades performance by a significant margin w.r.t. our multiplication scheme. This shows that encryption is a key step in formulating proactive schemes, and the same encryption process may not work for all tasks.

More Training Time. We perform an ablation to show that the performance gain of the detector is due to our proactive wrapper instead of training for more iterations of the pretrained object detector. Results in Tab. 6 show that although more training iterations for the detector has a performance gain, it's not enough to get the significant margin in performance as achieved by PrObeD. This shows that extra training can help, but only up to a certain extent.

Inference Time. We evaluate the overhead computational cost after applying PrObeD on different object detectors are shown in Tab. 6, averaged across 1 , 000 images, on a NVIDIA V 100 GPU. Our encoder network has 17 layers, which adds extra cost for inference. For detectors with bulky architectures like Faster R-CNN (ResNet101) and DeTR (transformer), the overhead computational cost is quite small, 8 . 7% and 7 . 2% , respectively. This additional cost is minor compared to the performance gain of detectors, especially Faster R-CNN. For a lighter detector like YOLOv5, our overhead computational cost increases to 29 . 1% . So, there is a trade-off of applying PrObeD to different detectors with varied architectures. PrObeD is more beneficial to bulky detectors like two-staged/transformer-based as compared to one-stage detectors.

5 Conclusion

We mathematically prove that the proactive method results in a better-converged model than the passive detector under assumptions and, consequently, a better 2D object detector. Based on this finding, we propose a proactive scheme wrapper, PrObeD, which enhances the performance of camouflaged and generic object detectors. The wrapper outputs an image-dependent template using an encoder network, which encrypts the input images. These encrypted images are then used to fine-tune the object detector. Extensive experiments on MS-COCO, CAMO, COD 10 K, and NC 4 K datasets show that PrObeD improves the overall object detection performance for both GOD and COD detectors.

Limitations. Our proposed scheme has the following limitations. First, PrObeD does not provide a significant gain for recent object detectors such as YOLO and DeTR. Second, the proactive wrapper should be thoroughly tested on other object detectors to show the generalizability of PrObeD. Finally, we only experiment with simple multiplication and addition as the encryption scheme. A more sophisticated encryption process might further improve the object detectors' performance. We leave these for our future avenues.

References

Vishal Asnani, Xi Yin, Tal Hassner, Sijia Liu, and Xiaoming Liu. Proactive image manipulation detection. In CVPR , 2022. 1, 2, 3, 4, 5, 6, 9, 10

Vishal Asnani, Xi Yin, Tal Hassner, and Xiaoming Liu. MaLP: Manipulation localization using a proactive scheme. In CVPR , 2023. 1, 2, 3, 4, 5, 6, 9, 10

Vishal Asnani, Xi Yin, Tal Hassner, and Xiaoming Liu. Reverse engineering of generative models: Inferring model hyperparameters from generated images. TPAMI , 2023.

Yousef Atoum, Joseph Roth, Michael Bliss, Wende Zhang, and Xiaoming Liu. Monocular video-based trailer coupler detection using multiplexer convolutional neural network. In ICCV , 2017. 3

Alexey Bochkovskiy, Chien-Yao Wang, and Hong-Yuan Mark Liao. YOLOv4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934 , 2020. 1, 3

Daniel Bolya, Sean Foley, James Hays, and Judy Hoffman. TIDE: A general toolbox for identifying object detection errors. In ECCV , 2020. 8

Garrick Brazil and Xiaoming Liu. Pedestrian detection with autoregressive network phases. In CVPR , 2019. 3

Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In ECCV , 2020. 1, 3, 7, 10

Geng Chen, Si-Jie Liu, Yu-Jia Sun, Ge-Peng Ji, Ya-Feng Wu, and Tao Zhou. Camouflaged object detection via context-aware cross-level fusion. IEEE Transactions on Circuits and Systems for Video Technology , 32(10), 2022. 1, 3

Shoufa Chen, Peize Sun, Yibing Song, and Ping Luo. DiffusionDet: Diffusion model for object detection. In CVPR , 2023. 1, 3

Wei-Chen Chen, Xin-Yi Yu, and Lin-Lin Ou. Pedestrian attribute recognition in video surveillance scenarios based on view-attribute attention localization. Machine Intelligence Research , 2022. 1

Xuelian Cheng, Huan Xiong, Deng-Ping Fan, Yiran Zhong, Mehrtash Harandi, Tom Drummond, and Zongyuan Ge. Implicit motion handling for video camouflaged object detection. In CVPR , 2022. 3

Pengyu Chu, Zhaojian Li, Kyle Lammers, Renfu Lu, and Xiaoming Liu. Deepapple: Deep learning-based apple detection using a suppression mask R-CNN. PRL , 2021. 3

Jifeng Dai, Yi Li, Kaiming He, and Jian Sun. R-FCN: Object detection via region-based fully convolutional networks. NeurIPS , 2016. 1

Navneet Dalal and Bill Triggs. Histograms of oriented gradients for human detection. In CVPR , 2005. 1

Mohammad Derakhshani, Saeed Masoudnia, Amir Shaker, Omid Mersa, Mohammad Sadeghi, Mohammad Rastegari, and Babak Araabi. Assisted excitation of activations: A learning technique to improve object detectors. In CVPR , 2019. 1

Deng-Ping Fan, Ge-Peng Ji, Ming-Ming Cheng, and Ling Shao. Concealed object detection. TPAMI , 2021. 1, 3

Deng-Ping Fan, Ge-Peng Ji, Guolei Sun, Ming-Ming Cheng, Jianbing Shen, and Ling Shao. Camouflaged object detection. In CVPR , 2020. 1, 3

Deng-Ping Fan, Ge-Peng Ji, Tao Zhou, Geng Chen, Huazhu Fu, Jianbing Shen, and Ling Shao. Pranet: Parallel reverse attention network for polyp segmentation. In MICCAI , 2020. 1

Pedro Felzenszwalb, David McAllester, and Deva Ramanan. A discriminatively trained, multiscale, deformable part model. In CVPR , 2008. 1

Sanja Fidler, Roozbeh Mottaghi, Alan Yuille, and Raquel Urtasun. Bottom-up segmentation for top-down detection. In CVPR , 2013. 1

Spyros Gidaris and Nikos Komodakis. Object detection via a multi-region and semantic segmentation-aware cnn model. In ICCV , 2015. 1

Ross Girshick. Fast R-CNN. In ICCV , 2015. 1, 3

Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR , 2014. 1, 3

Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Region-based convolutional networks for accurate object detection and segmentation. TPAMI , 2015. 3

Agrim Gupta, Piotr Dollár, and Ross Girshick. LVIS: A dataset for large vocabulary instance segmentation. In CVPR , 2019. 1

Chunming He, Kai Li, Yachao Zhang, Longxiang Tang, Yulun Zhang, Zhenhua Guo, and Xiu Li. Camouflaged object detection with feature decomposition and edge reconstruction. In CVPR , 2023. 1

Chunming He, Kai Li, Yachao Zhang, Guoxia Xu, Longxiang Tang, Yulun Zhang, Zhenhua Guo, and Xiu Li. Weakly-supervised concealed object segmentation with SAM-based pseudo labeling and multi-scale feature grouping. arXiv preprint arXiv:2305.11003 , 2023. 1

Chunming He, Kai Li, Yachao Zhang, Yulun Zhang, Zhenhua Guo, Xiu Li, Martin Danelljan, and Fisher Yu. Strategic preys make acute predators: Enhancing camouflaged object detectors by generating camouflaged objects. arXiv preprint arXiv:2308.03166 , 2023. 1

Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask R-CNN. In ICCV , 2017. 7, 8

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Spatial pyramid pooling in deep convolutional networks for visual recognition. TPAMI , 2015. 3

Yihui He, Chenchen Zhu, Jianren Wang, Marios Savvides, and Xiangyu Zhang. Bounding box regression with uncertainty for accurate object detection. In CVPR , 2019. 1

Jianqin Yin Yanbin Han Wendi Hou and Jinping Li. Detection of the mobile object with camouflage color under dynamic background based on optical flow. Procedia Engineering , 2011. 1, 3

Ge-Peng Ji, Deng-Ping Fan, Yu-Cheng Chou, Dengxin Dai, Alexander Liniger, and Luc Van Gool. Deep gradient learning for efficient camouflaged object detection. Machine Intelligence Research , 2023. 1, 3, 7, 8, 9

Ge-Peng Ji, Lei Zhu, Mingchen Zhuge, and Keren Fu. Fast camouflaged object detection via edge-based reversible re-calibration network. Pattern Recognition , 123, 2022. 3

Nobukatsu Kajiura, Hong Liu, and Shin'ichi Satoh. Improving camouflaged object detection with the uncertainty of pseudo-edge labels. In ACM Multimedia Asia , 2021. 3

Abhinav Kumar, Garrick Brazil, Enrique Corona, Armin Parchami, and Xiaoming Liu. DEVIANT: Depth EquiVarIAnt NeTwork for Monocular 3 D Object Detection. In ECCV , 2022. 3

Abhinav Kumar, Garrick Brazil, and Xiaoming Liu. GrooMeD-NMS: Grouped mathematically differentiable nms for monocular 3 D object detection. In CVPR , 2021. 3

Trung-Nghia Le, Tam Nguyen, Zhongliang Nie, Minh-Triet Tran, and Akihiro Sugimoto. Anabranch network for camouflaged object segmentation. CVIU , 2019. 3, 7

Aixuan Li, Jing Zhang, Yunqiu Lv, Bowen Liu, Tong Zhang, and Yuchao Dai. Uncertainty-aware joint salient object and camouflaged object detection. In CVPR , 2021. 1, 3

Zeming Li, Chao Peng, Gang Yu, Xiangyu Zhang, Yangdong Deng, and Jian Sun. Light-head R-CNN: In defense of two-stage object detector. arXiv preprint arXiv:1711.07264 , 2017. 3

Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection. In CVPR , 2017. 3, 7

Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In ICCV , 2017. 1, 3

Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and Lawrence Zitnick. Microsoft COCO: Common objects in context. In ECCV , 2014. 7

Jiannan Liu, Bo Dong, Shuai Wang, Hui Cui, Deng-Ping Fan, Jiquan Ma, and Geng Chen. Covid-19 lung infection segmentation with a novel two-stage cross-domain transfer learning framework. Medical image analysis , 2021. 1

Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander Berg. SSD: Single shot multibox detector. In ECCV , 2016. 1, 3

Yunqiu Lv, Jing Zhang, Yuchao Dai, Aixuan Li, Bowen Liu, Nick Barnes, and Deng-Ping Fan. Simultaneously localize, segment and rank the camouflaged objects. In CVPR , 2021. 3, 7

Yuxin Mao, Jing Zhang, Zhexiong Wan, Yuchao Dai, Aixuan Li, Yunqiu Lv, Xinyu Tian, DengPing Fan, and Nick Barnes. Transformer transforms salient object detection and camouflaged object detection. arXiv preprint arXiv:2104.10127 , 2021. 1, 3

Haiyang Mei, Ge-Peng Ji, Ziqi Wei, Xin Yang, Xiaopeng Wei, and Deng-Ping Fan. Camouflaged object segmentation with distraction mining. In CVPR , 2021. 3

Yuxin Pan, Yiwang Chen, Qiang Fu, Ping Zhang, and Xin Xu. Study on the camouflaged target detection method based on 3D convexity. Modern Applied Science , 2011. 1, 3

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An imperative style, high-performance deep learning library. In NeurIPS , 2019. 7

Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object detection. In CVPR , 2016. 1, 3, 7, 10

Joseph Redmon and Ali Farhadi. YOLO9000: better, faster, stronger. In CVPR , 2017. 1, 3

Joseph Redmon and Ali Farhadi. YOLOv3: An incremental improvement. arXiv preprint arXiv:1804.02767 , 2018. 1, 3

Atique Rehman, Rafia Rahim, Shahroz Nadeem, and Sibt Hussain. End-to-end trained CNN encoder-decoder networks for image steganography. In ECCVW , 2018. 6

Atique-ur Rehman, Rafia Rahim, Shahroz Nadeem, and Sibt-ul Hussain. End-to-end trained CNN encoder-decoder networks for image steganography. In ECCVW , 2019. 6

Jingjing Ren, Xiaowei Hu, Lei Zhu, Xuemiao Xu, Yangyang Xu, Weiming Wang, Zijun Deng, and Pheng-Ann Heng. Deep texture-aware features for camouflaged object detection. IEEE Transactions on Circuits and Systems for Video Technology , 2023. 3

Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster R-CNN: Towards real-time object detection with region proposal networks. NeurIPS , 2015. 1, 3, 7, 8, 10

Nataniel Ruiz, Sarah Adel Bargal, and Stan Sclaroff. Disrupting deepfakes: Adversarial attacks against conditional image translation networks and facial manipulation systems. In ECCV , 2020. 1, 2, 3

Eran Segalis and Eran Galili. OGAN: Disrupting deepfakes with an adversarial attack that survives training. arXiv preprint arXiv:2006.12247 , 2020. 1, 2, 3

P Sengottuvelan, Amitabh Wahi, and A Shanmugam. Performance of decamouflaging through exploratory image analysis. In ICETET , 2008. 1, 3

Peize Sun, Rufeng Zhang, Yi Jiang, Tao Kong, Chenfeng Xu, Wei Zhan, Masayoshi Tomizuka, Lei Li, Zehuan Yuan, Changhu Wang, and Ping Luo. Sparse R-CNN: End-to-end object detection with learnable proposals. In CVPR , 2021. 7

Yujia Sun, Geng Chen, Tao Zhou, Yi Zhang, and Nian Liu. Context-aware cross-level fusion network for camouflaged object detection. arXiv preprint arXiv:2105.12555 , 2021. 1, 3

Paul Viola and Michael Jones. Rapid object detection using a boosted cascade of simple features. In CVPR , 2001. 1

Paul Viola and Michael Jones. Robust real-time face detection. IJCV , 2004. 1

Chien-Yao Wang, Alexey Bochkovskiy, and Hong-Yuan Liao. YOLOv7: Trainable bag-offreebies sets new state-of-the-art for real-time object detectors. In CVPR , 2023. 3

Jingdong Wang, Ke Sun, Tianheng Cheng, Borui Jiang, Chaorui Deng, Yang Zhao, Dong Liu, Yadong Mu, Mingkui Tan, Xinggang Wang, Wenyu Liu, and Bin Xiao. Deep high-resolution representation learning for visual recognition. TPAMI , 2020. 3

Run Wang, Felix Juefei-Xu, Meng Luo, Yang Liu, and Lina Wang. FakeTagger: Robust safeguards against deepfake dissemination via provenance tracking. In ACM Multimedia , 2021. 1, 2, 3

Jian-Ru Xue, Jian-Wu Fang, and Pu Zhang. A survey of scene understanding by event reasoning in autonomous driving. International Journal of Automation and Computing , 2018. 1

Fan Yang, Qiang Zhai, Xin Li, Rui Huang, Ao Luo, Hong Cheng, and Deng-Ping Fan. Uncertainty-guided transformer reasoning for camouflaged object detection. In ICCV , 2021. 1, 3

Chin-Yuan Yeh, Hsi-Wen Chen, Shang-Lun Tsai, and Sheng-De Wang. Disrupting imagetranslation-based deepfake algorithms with adversarial attacks. In WACVW , 2020. 3

Matthew Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In ECCV , 2014. 3

Qiang Zhai, Xin Li, Fan Yang, Chenglizhao Chen, Hong Cheng, and Deng-Ping Fan. Mutual graph learning for camouflaged object detection. In CVPR , 2021. 3

Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable DETR: Deformable transformers for end-to-end object detection. In ICLR , 2020. 1, 3

Mingchen Zhuge, Xiankai Lu, Yiyou Guo, Zhihua Cai, and Shuhan Chen. CubeNet: X-shape connection for camouflaged object detection. Pattern Recognition , 2022. 3

Peer reviews:
comments: SUMMARY:
The authors propose a proactive object wrapper for the object detection method. The warper is similar to an input transformation that makes the object detection model focus more on the salient regions. The experiments show that a consistent improvement after the proposed method is applied on top of existing objection methods.

STRENGTHS:
The proposed method is somewhat novel in the sense that it is the first to use the proactive method on object detection tasks. Both GOD and COD situations are evaluated and the proposed method consistently improves the existing methods.

WEAKNESSES:
There are several weaknesses of the method, and some of them are also mentioned by the authors

- Performance gain: The performance gain becomes marginal when it comes to advanced object detection methods like YOLOv5 and DeTR. Considering two-stage methods, if Sparse R-CNN is evaluated instead of Faster R-CNN, would the improvement is also incremental?

- Simple transformation: The proactive scheme is a simple multiplication or addition between the input and the template. Is this process equivalent to having an image-dependent transformation that makes object detection focus on salient regions and improve performance? If yes, then I think there may be more different sophisticated techniques to be explored. 

For the training time overhead, in table 5, it is mentioned the training time with and without PrObeD is reported. However, only the inference time is mentioned.

QUESTIONS:
See the Weakness.

LIMITATIONS:
See the Weakness.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-06-28

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a wrapper based on proactive object detection which can be utilized for both general object detection and camouflaged object detection problems. The proposed algorithm is flexible to different detection algorithms like FasterRCNN, Yolo and DeTR. Reasonable performance have been reported in benchmarks in general object detections like COCO and camouflaged object detection like CAMO, COD10K, and NC4K.\n\nSTRENGTHS:\n1. The proposed proactive object detection wrapper is flexible and can be applied to different object detectors. \n2. The proposed algorithm has been verified on general object detection as well as camouflaged object detection benchmarks.\n\nWEAKNESSES:\n1. The motivation (as well as the advantages) of using the \"template\" is not clear.  As the generation of template relies on the ground-truth seg map, how about compairing the proposed algorithm with the algorithm which first perform segmentation to obtain the mask and then doing the object detection?\n\n2. The performance gain for the general object detectors is not obvious, for example YOLOV5 and DeTR, both of which are not state-of-the-art object detectors. Especially, the proposed wrapper will introduce additional computational cost, which is not negligible. \n\n3. The benchmark results on camouflaged object detection benchmarks (CAMO, COD10K, and NC4K) are slightly different from the results reported in other papers. For example, the baseline result reported in Table 3 is different from the results reported in DGNet[20]. Also, more comparison should be provided to validate the effectiveness of the proposed algorithm.\n\nQUESTIONS:\nPlease address the questions raised in the weakness section, especially on the experimental results.\n\nLIMITATIONS:\nThe authors have already provided the limitation discussion of the proposed algorithm.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-29"}

comments: SUMMARY:
Authors propose a new object detection wrapper, PrObeD, based on proactive scheme, which tries to enhance the performance of existing object detectors. PrObeD generate a custom template for each input images which makes the weights of object detector closer to the ideal weights. Besides, a decoder who recovers the template from the encrypted images is presented to further boost the performance.

STRENGTHS:
- PrObeD is the first work to develop a proactive approach upon object detection tasks, providing a new aspect to develop object detection models.
- Authors provide mathematical analysis of passive and proactive detectors, and prove that the proactive method results in a better-converged model than the passive detector under specific assumptions.
- Experiments show that PrObeD achieves SOTA performance on several GOD and COD datasets.

WEAKNESSES:
- The templates are applied on raw images with element-wise multiplication, which can be regarded as a soft mask. COD tasks may get benefit from it because it can mask useless background, but GOD tasks strongly dependent on image context, it's unclear why it's beneficial to GOD tasks. 
- In Sec 3.2,  when the authors try to prove the rationality of template, they assume the object detector is a linear regression model, which is contrary to the factual circumstances.
- In line 191, authors said "it is beneficial to recover the template from encrypted images", but didn't provide explanations on it. Please provide the necessity or some intuitions behind this.

QUESTIONS:
see Weaknesses.

LIMITATIONS:
N/A
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors study the problem of proactive object detection. To be specific, the authors introduce a wrapper around an existing object detector that can produce a template image and improve the performance of the detector if the image is multiplied with this template. Prior works have proposed learning this template in an image independent manner. One of the important contributions of the paper is to make the template image dependent.\n\nAfter the rebuttal:\n\nI've read the other reviews and the responses provided by the authors. The authors have addressed my concerns and I have increased my recommendation. However, since the reported improvements are only minor, I feel unable to make a stronger recommendation.\n\nSTRENGTHS:\n1. Novel extension over prior work.\n2. Significant improvements over baselines.\n\nWEAKNESSES:\nI don't work on proactive object detection. So, my evaluation maybe limited.\n\n1. An important issue with the paper is that it fails to provide intuition on several design choices.  For example:\n\n1.1. Lemma 1: The lemma is clear. But, what is the intuition for the template to improve the weights? Since the model is linear and the element-wise multiplication is linear, I don't see that adding S improves model capacity. The authors should provide some intuition here.\n\n1.2. Why is template recovery helpful? Why is the recovered template too different from the generated one (in Fig 3)?\n\n2. Theorem 1: \"Then, the AP of the proactive detector is better than the AP of the passive detector\" => They can be equal as well, if you check your proof carefully.\n\n3. Experimental evaluation has major issues:\n\n3.1. Certain crucial details are not provided:\n  - It is not clear what framework (mmdetection, detectron, ..) is used.\n  - Table 1: It is not clear what the backbones are.\n\n3.2. Faster R-CNN's baseline performance (19.3 AP) is way too low compared to the known scores. See e.g. https://github.com/open-mmlab/mmdetection/tree/main/configs/faster_rcnn\n\n3.3. No comparison is provided with the closest work [1,2].\n\n\nMinor comments:\n\n- \"convergence to global minima is not optimal in neural networks\" => Did you mean to say \"not guaranteed\" instead of \"not optimal\"? \n\n- \"weights in the object detector are not optimal\" => \"weights in object detectors are not optimal\"?\n\n- \"schemes,PrObeD,\" => \"schemes, PrObeD,\".\n\n- The abstract is overloaded with undefined unconventional terms (for the object detection literature) such as proactive scheme, template, ideal object detector... These make the abstract difficult to read.\n\n- \"has slowly improved from earlier traditional detectors\" => I am not sure \"slowly\" is a suitable term here.\n\n- Fig 1: The images are too small to comprehend what's being performed with the proposed method.\n\n- Line 111: The symbol should be T for box coordinates.\n\n- Eq 1: This is an overly simplistic formulation of the loss functions used in GOD. I would suggest using generic terms (such as L_box and L_cls) to keep the formulation general. Moreover, you should use \\cdot instead of \".\" for multiplication (same for Eq 5).\n\n- Eq 3: It is better to use \\odot for element-wise multiplication.\n\n- Proof of Lemma 1: It would be better to stick to the notation in the paper. E.g., for the template, you should use S instead of \\alpha, and I for image instead of h.\n\n- Fig 2: There are two arrows for training in the legend.\n\n- Line 183: What is T here? The transformation or the box coordinates?\n\n- \"Faster-RCNN\" => \"Faster R-CNN\".\n\nQUESTIONS:\nPlease see above.\n\nLIMITATIONS:\nNone.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a proactive approach to enhance the performance of object detection through adding a learnable image-dependent template into the image. The optimal template can effectively bridge the gap between the learned and the ideal weights for object detectors. As demonstrated by the results, the template acts as a mask of the input images to provide relevant information of the objects to guide the object detector for improved detection. The proposed approach can effectively improve the results of both generic and camouflaged object detection as shown in the extensive experimental results of diverse object detection benchmarks.\n\nSTRENGTHS:\n1. The proposed encoder-based learnable template can effectively highlight the semantic useful information to guide the object detector to perform improved detection.\n2. The proposed learnable template can help significantly improve the performance of Faster-RCNN and other object detectors while not introducing much additional training and computational costs.\n3. The authors have conducted extensive experimental evaluation to demonstrate their effectiveness.\n\nWEAKNESSES:\n1. For the generic object detection, the performance improvement for more modern object detectors, YOLOv5 and DeTR, is marginal and is not as significant as the Faster-RCNN. There is no discussions about this situation.\n2. The performance of Faster-RCNN seems to be the old one. With proper parameter tuning and backbone selection as other modern object detectors, the performance should be much higher than 19.3, or even close to 40 on the MSCOCO dataset. The results of Faster-RCNN are not representative. However, most of the ablation studies of the proposed method on GOD are based on Faster-RCNN.\n3. Some relevant works to leverage ground truth to generate a mask to help improve the object detection are not included for discussion in the related work.\n\nDerakhshani, Mohammad Mahdi, Saeed Masoudnia, Amir Hossein Shaker, Omid Mersa, Mohammad Amin Sadeghi, Mohammad Rastegari, and Babak N. Araabi. \"Assisted excitation of activations: A learning technique to improve object detectors.\" In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9201-9210. 2019.\n\nQUESTIONS:\nAfter reading the proposed method, I think its functionality is close to a certain of region proposal generator to guide the object detector to better detect the objects. However, as shown in the Table 2, for one-stage detectors, YOLOv5 and DeTR both get marginal improvement. In contrast, the two-stage detector with the regional proposal network, Faster-RCNN gets significant performance improvement. It means that the proposed method can provide better object proposal information than the regional proposal network of Faster-RCNN. However, the performance of the original Faster-RCNN is very low as shown in the original NeurIPS paper. The updated Faster-RCNN with proper tuning and architecture selection can achieve much better performance. To consider these factors, I worry that the proposed method does not work as claimed for generic object detection. In addition, it is not clear for me why the authors choose to minimze the cosine similarity between the learnable template and the ground truth mask. I think the cross entropy loss with dice loss which are widely used in the segmentation task could be more appropriate to achieve the desired effect. Perhaps, the authors could further elaboration the loss design.\n\nFor camouflaged object detection, since it is formulated as a segmentation problem, it is reasonable that if the proposed method could provide object information, it will be very helpful for the following segmentation. However, I am curious about the comparisons of the region proposal network (if trained appropriately) and the proposed method in terms of the effectiveness and efficiency.\n\nHow do the authors actually conduct the element-wise multiplication between the template S_j and the input image I_j? The dimensions of both tensors seems mismatched where S_j is HxW but I_j is HxWx3.\n\nLIMITATIONS:\nYes, the authors have addressed the limitation of their proposed method with sufficient context. For the negative societal impact, the authors could discuss more about avoiding the object detection from malicious monitoring use, etc.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-09"}