PrObeD: Proactive Object Detection Wrapper
Authors: Vishal Asnani
Abhinav Kumar
Suya You
Xiaoming Liu
Venue: NeurIPS 2023 poster (2023)
Keywords: Object detection
proactive
Camouflage
2D
TL;DR: We propose a proactive wrapper for 2D object detection, which exploits an optimum template to enhance the performance of different generic and camouflaged object detectors.

Abstract: Previous research in $2D$ object detection focuses on various tasks, including detecting objects in generic and camouflaged images. These works are regarded as passive works for object detection as they take the input image as is. However, convergence to global minima is not guaranteed to be optimal in neural networks; therefore, we argue that the trained weights in the object detector are not optimal. To rectify this problem, we propose a wrapper based on proactive schemes, PrObeD, which enhances the performance of these object detectors by learning a signal. PrObeD consists of an encoder-decoder architecture, where the encoder network generates an image-dependent signal termed templates to encrypt the input images, and the decoder recovers this template from the encrypted images. We propose that learning the optimum template results in an object detector with an improved detection performance. The template acts as a mask to the input images to highlight semantics useful for the object detector. Finetuning the object detector with these encrypted images enhances the detection performance for both generic and camouflaged. Our experiments on MS-COCO, CAMO, COD$10$K, and NC$4$K datasets show improvement over different detectors after applying PrObeD. Our models/codes are available at https://github.com/vishal3477/Proactive-Object-Detection.

PrObeD: Proactive Object Detection Wrapper
Vishal Asnani Michigan State University asmanivi@msu.ce. edu
Abhinav Kumar
Michigan State University kumarab6@msu.cn edda
Suya You
Xiaoming Liu
DEVCOM Army Research Laboratory suya - you. wrw...iviarmyy... mil
Michigan State University 1 i luxmmcss ese. msu. edu
Abstract
Previous research in 2D object detection focuses various tasks, including de- tecting objects in generic and camouflaged images. These works are regarded passive works for object detection they take the input image is. How- ever, convergence to global minima not guaranteed to be optimal in neural argue that the trained weights in the object detector
networks; therefore, not optimal. To rectify this problem, we propose wrapper based on proactive schemes, PrObeD, which enhances the performance these object detectors by learning a signal. PrObeD consists of an encoder-decoder architecture, where the encoder network generates an image-dependent signal termed templates encrypt the input images, and the decoder recovers this template from the en- crypted images. We propose that learning the optimum template results in an object detector with an improved detection performance. The template acts mask to the input images to highlight semantics useful for the object detec- tor. Finetuning the object detector with these encrypted images enhances the detection performance for both generic and camouflaged. Our experiments on MS-COCO, CAMO, COD10K, and NC4K datasets show improvement over differ- ent detectors after applying PrObeD. Our models/codes are available at ht tps::: / //githuss wwt.w//w.....a.cc wwttt//rratiivviiaaa -eet-oeett--eeeeeeeeeeeeeeee---- defect-eetectin..
Introduction
Generic 2D object detection (GOD) has improved from earlier traditional detectors [15,20, 64, 65] to the deep-learming-batt based object detectors [8, 10, 26, 32, 58). Advancements in beep-learning-basee methods underwent many architectural change over recent years, including one-stage (5,,43,6,552-- two-stage [23, 24,58), CNN-based [5, 14, 16, 21,231,2,, 54], transformer-based [8, 74], and diffusion- based [10] methods. All these methods aim to predict the 2D bounding box the objects in the images and their category class.
Another emerging area related generic object detection is camouflaged object detection [17, 18, 27- 29.34, 22,4,,40 (COD). COD aims to detect and segment objects blended with the background [17, 7, via object-level mask supervision. Applications of COD include medical [19,45], surveillance [11] and autonomous driving [69]. Early COD detectors exploit hand-crafted features [50, 61] and optical flow [33], while current methods are deep-learning
These methods utilize attention [9,63] 63),
joint learning [40], image gradient [34], and transformers [48, 70].
All these methods take input images is for the detection task and hence called passive methods. However, there is a line of research on proactive methods for a wide range of vision tasks such as disruption [59, 60], tagging [68], manipulation detection [1], and localization [2]. Proactive methods
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Passive object detection
Ideal weights
O
GOD
O COD
weights
Proactive weights
Encryption
Template
OOOD
OCOD
W1
w
Proactive object detection
Applying proactive wrapper
(a)
(b)
(c)
Figure 1: (a) Passive vs. Proactive object detection. A learnable template encrypts the input images, which are further used to train the object detector. (b) PrObeD serves as wrapper both generic and camouflaged object detectors, enhancing the detection performance. (c) For the linear regression model under additive noise and other assumptions, the converged weights the proactive detector are closer to the optimal weights as compared to the converged weights of the passive detector. See Sec. 3.2 for details and proof.
use signals, called templates, encrypt the input images and pass the encrypted images as the input to the network. These are trained in an end-to-end manner by using either a fixed [68] or learnable template [1, 59, 60] improve the performance. that such methods generalize better on unseen data/models [1,2]. Motivated by this, we propose Blug-and-play Proactive Object Detection wrapper, PrObeD, improve GOD and COD detectors.
major advantage of proactive schemes is
Designing PrObeD as a proactive scheme involves several challenges and key factors. First, the proactive wrapper needs be plug-and-play module that be applied both GOD and COD detectors. Secondly, the encryption process should be intuitive to benefit the object detection task. e.g., an ideal template for detection should highlight the foreground objects in the input image. Lastly, the choice of supervision to estimate the template for encryption is hard to formulate.
Previous proactive methods [1,2] use learnable but amage-indeeendeen templates for manipulation and localization tasks. However, the object detection task is scene-specific; therefore, the ideal template should be image-dependent. Based this key insight, propose novel plug-and-play proactive wrapper in which we apply object detectors to enhance detection performance. The PrObeD wrapper utilizes encoder network to learn image-dependent template. The learned template encrypts the input images by applying a transformation, defined as an element-wise multiplication between the template and the input image. The decoder network recovers the templates from the encrypted images. We utilize regression losses for supervision and leverage the ground-truth object map to guide the learning process, thereby imparting valuable object semantics to integrated into the template. We then fine-tune the proactive wrapper with the GOD and COD detectors to improve their detection performance. Extensive experiments on MS-COCO, CAMO, COD10K, and NC4K datasets show that PrObeD improves the detection performance for both GOD and COD detectors.
In summary, the contributions of this work include:
propose novel proactive approach PrObeD for the object detection task. To the best of our knowledge, this is the first work to develop proactive approach to 2D object detection.
mathermaticlll prove that the proactive method results in a better-converged model than passive detector under assumptions and, consequently, a better object detector. PrObeD wraps around both GOD and COD detectors and improves detection performance on MS-COCO, CAMO, CODIOK, and NC4K datasets
2 Related works
Proactive Schemes. Earlier works adopt to signals like perturbation [60], adversarial noise [59], and one-hot encoding [68] messages while focusing on tasks like disruption [59, 60] and deepfake tagging [68]. Asnani [1] propose to learn an optimized template for binary detection by
unseen generative models. Recently, MaLP [2] adds the learnable template to perform generalized

---

Table Comparison of PrObeD with prior works.
Template
Method
Proactive
Task
COD GOD Plug- Plug-Play
Number
Type
Faster R-CNN [58]
Object Detection
X
X
YOLO [52]
Object Detection
X
DeTR [8]
X
Object Detection
X
X
DGNet [34]
Object Detection
X
SINet-v2 [17]
Object Detection
JCSOD [40]
Object Detection
OGAN [60]
Disrupt
Learnabee
Ruiz al. [59]
Disrupt
Learnable
Ych et at. [?]]
Disrupt
Learnablc
X
Fake'Tagger [68]
Tagging
Fixed. Id-dependent
X
Asnani al. [1]
Manipulation Detection
Learnable set, Image hange-independent
MaLP
Manipulation Localization
Learnable sct, image-independent
PrObeD (Ours)
Object Detection
Learnable, Image-dependent
manipulation localization for unknown generative models. Unlike these works, PrObeD uses image- dependent templates and is plug-and-pla and-play wrapper for different task of object detection.
Generic Object Detection Detection of generic objects, instead of specific object categories such as pedestrians [7]. apples [13], and others [4, 37,338,, has been a long-standing objective of computer vision. RCNN [24,25] employs the extraction of object proposals. He er al. [31] propose spatial pooling layer to extract fixed-length representation of all the objects. Modifications of RCNN [23,41,58, 58, 72] increase the inference speed. Feature pyramid network [42] detects objects with a wide variety of scales. The above methods are mostly two-stage, so inference is an issue. Single-stage detectors like YOLO 15,52554, 66), SSD [46], HRNet [67] and RetinaNet [43] increase the speed and simplicity of the framework compared to the two-stage detector. Recently, transformer- based methods [8, 74] use a global-scale receptive field. Chen et al. [10] use diffusion models to denoise noisy boxes every forward step. PrObeD functions
wrapper around the pre-existing object detector, facilitating its transformation into an enhanced object detector. The comparison of PrObeD with prior works is summarized in Tab. 1.
Camouflaged Object Detection Early COD works rely on hand-crafted features like co-occurrence matrices [61], 3D convexity [50], optical flow [33], covariance matrix [35], and multivariate cal- ibration components [57]. Later on, [9, 63] incorporate an attention-
cross-level fusion of multi-scale features to recover contextual information. Mei et al. [49] take motivation by predators to identify camouflaged objects using position and focus ideology. SINet [18] uses a search and identification module to perform localization. SINET-v2 [17] uses group-reversal attention to extract the camouflaged maps. [36] explores uncertainty maps and [75] utilizes cube-like architecture to integrate multi-layer features. ANet [39], LSR [47], and JCSOD [40] employ joint learning with different tasks to improve COD. Lately, [12,48, 70] apply a transformer-baat difficult-aware learning, uncertainty modeling, and temporal consistency. Zhai et al. [73] use graph learning model to disentangle input into different features for localization. DGNet [34] uses image gradients exploit intensity changes in the camouflaged object from the background. Unlike these methods, PrObeD uses proactive methods to improve camouflaged object detection.
architecture for
3 Proposed Approach
Our method originates from understanding what makes proactive schemes effective. We first overview the two detection problems: GOD and COD in Sec. 3.1. We next derive Lemma where we show that the proactive schemes with the multiplicative transformation of images are better than passive schemes by comparing the deviation of trained network weights from optimal. Based on this result, we derive that Average Precision (AP) from the proactive model is better than AP from the passive model in Theorem 1. At last, we present our proactive scheme-based wrapper, PrObeD, in Sec. 3.3, which builds upon the Theorem to improve generie 2D objects and camouflaged detection.

---

3.1 Background
3.1.1 Passive Object Detection
Although generic 2D object detection and camouflage detection are similar problems, they have different objective functions. Therefore, we treat them two different problems and define their objectives separately.
Generic 2D Object Detection. Let 1 be the set of input images given to the generic 2D object detector O with trainable parameters O. Most of these detectors output two sets of predictions per
image: (1) bounding box coordinates, O1))) f E R.,, (2) class logits, O(12)2 = Č R where N is the number of foreground object categories. If the ground-truth bounding box coordinates are T3, and the ground-truth category label is C, the objective function of such detector is:
min Σ - -Ty11e)
ΣΣ J 1=1
(cj log 1og(0((1;0)))))
(I)
Camouflaged Object Detection. Let I be the input image set given to the camouflaged object detector O with trainable parameters 6, and Gj be the ground-truth segmentation map. Prior passive works predict segmentation map with the following objective:
min &
3.1.2 Proactive Object Detection
Proactive schemes [1, 2] encrypt the input images with the template to aid manipulation detec- tion/localization. Such schemes take input image I, "xxw3 and learns template S, E #xW PrObeD uses image-dependent templates improve object detection. Given an input image I #R x3 PrObeD learns to output template Sj- R#xW, which can be used by a transformation resulting in encrypted images T(;).. PrObeD uses element-wise multiplication as the transformation T, which is defined as:
T(1) S,); O
3.2 Mathematical Analysis Passive and Proactive Detectors
PrObeD optimizes the template improve the performance of the object detector. We argue that this template helps arrive at better global minima representing the optimal parameters 0. We now define the following lemma to support our argument:
Lemma 1. Converged weights of proactive and passive detectors. Consider linear regression model that regresses an input image I under an additive noise setup to obtain the 2D coordinates. Assume the noise under consideration is normal random variable (0, o²). Let and w" denote the trained weights of the pretrained linear regression model and the optimal weights of the linear regression model. Also, assume SGD optimizes the model parameters with decreasing step size
such that the steps square summable i.e., S= -im Σ st exist, and the noise is independent of k=1
the image. Then, there exists template S, E [0, 11. the image I such that the multiplicative transformation of images as the input results in a trained weight w' closer to the optimal weight than the originally trained weight w. In other words,
E(Iw w (²1l2) < [8[[w w²112).
(4)
The proof of Lemma in supplementary. We use the variance of the gradient of the encrypted images to arrive at this lemma. We next use Lemma 1 to derive the following theorem:
Theorem AP comparison of proactive and passive detectors. Consider a linear regression model that regresses input image I under additive noise setup obtain the 2D coordinates. Assume the noise under consideration is normal random variable (0, o2). Let and w denote the

---

Proactive wrapper
Networks
Stage 2
Stage 3
Loss functions
S'
D
Mathematical
operation
P
Training
Inference
JE
JOBI
*
Stage 1
J(I
G
T
Figure 2: Overview of PrObeD. PrObeD consists of three stages: (I) template generation, (2) template recovery, and (3) detector fine-tuning. The templates generated by encoder network & to encrypt the input images. The decoder network D is used to recover the template from the encrypted images. Finally, the encrypted images are used to tun-tune the object detector perform detection. We train all the stages an end ao-to-end manner. However, for inference, we only use stages and 3. Best viewed in color.
trained weights of the pretrained linear regression model and the optimal weights of the linear regression model. Also, assume SGD optimizes the model parameters with decreasing step size
such that the steps are square summable i.e., S= lim Σ sy exist, and the noise is independent of 1-000
the image. Then, the AP of the proactive detector is better than the AP of the passive detector.
The proof of Theorem is in the supplementary. We use the Lemma and the non-decreasing nature of AP w.r.t. IoU to arrive at this theorem. Next, adapt the objectives of Eqs. (1) and (2) to incorporate the proactive methods as follows:
N ΣΣ i=
min 0,5,
Σ
T3112)
c (0) -
(5)
min S,S,
(6)
3.3 PrObeD
Our proposed approach comprises of three stages: template generation, template recovery, and detector fine-tuning. First, we use an encoder network to generate an image-dependent template for image encryption. This encrypted image further used to recover the template through decoder network. Finally, the object detector is fine-tuned using the encrypted images. All three stages are trained in end-to-end fashion. While all the stages used for training PrObeD, we specifically use only stages and for inference. We will now describe each stage in detail.
3.3.1 Proactive Wrapper
Our proposed approach consists of three stages, shown in Fig. 2. However, only the first two stages are part our proposed proactive wrapper, which can be applied to object detector to improve its performance.
Template Generation. Prior works learn set of templates [1,2] their proactive schemes. This set of templates is enough to perform the respective downstream tasks as the generative model manipulates the template, which is easy to capture with set of learnable templates. However, for object detection tasks, every image has unique object characteristics such size, appearance, and color that can vary significantly. This variability present in the images may exceed the descriptive capacity of finite set of templates, thereby necessitating the use of image-specific templates to
Stage 1:

---

accurately represent the range of object features present in each image. In other words, a fixed set of templates may not be sufficiently flexible to capture the diversity of visual features across the given set of input images, thus demanding more adaptable, image-dependent templates.
Motivated by the ahove argument, we propose to generate the template Sj for every image using an encoder network. We hypothesize that highlighting the area of the key foreground objects would be beneficial for object detection. Therefore, for GOD, we use the ground-truth bounding boxes 7 to generate the pseudo ground-truth segmentation map. Specifically, for any image 1,, if the bounding box coordinates are T = {21, 22, , y2), we define the pseudo ground-truth segmentation map as:
Vm E [0, H E [0, W1. we have
Gm if 2f S 22 and y1 I S 12, otherwise 0
However, for COD, the dataset already has the ground-truth segmentation map Gj. which we use as the supervision for the encoder to output the templates with semantic information of the image to be restricted only in the region of interest for the detector. For both GOD and COD, minimize the cosine similarity (Cos) between Sj and G2 as the supervision for the encoder network. The encoder loss JE is as follows:
JE = 1-
= 1- -o__)))( Gj).
(7)
This generated template acts as a mask for the input image to highlight the object region of interest for the detector. We use this template with the transformation TT to encrypt the input image as
T[ S;) = 1 @ Sj. As we start from the pretrained model of object detector O, we initialize the bias of the last layer of the encoder 0 that for the first few iterations, Sj 1. This is to ensure that the distribution of 1 and T g S2) remains similar for the first few iterations, and O doesn't t encounter a sudden change in its input distribution.
Stage 2: Template Recovery. So far, we have discussed the generation of template S, using E,
which will be used mask to encrypt the input image. The encrypted images are used for two purposes: (1) recovery of templates and (2) fine-tuning of the object detector. The main intuition of recovering the templates is from the prior works on image steganalysis [55,56] and proactive schemes [1.2]. Motivated by these works, we draw the following insight: "To properly learn the optimal template and embed it onto the input images, it is beneficial recover the template from encrypted images."
To perform recovery, we exploit an encoder-decoder approach. Using this approach leverages the strengths of the encoder network & for feature extraction, capturing the most useful salient details, and the decoder network DD for information recovery, allowing for efficient and effective encryption and decryption of the template. We also empirically show that not using the decoder to recover the templates harms the object detection performance.
To supervise 'D in recovering S, from T(I;: we propose to maximize the cosine similarity between the recovered template, S/j and Sj. The decoder loss is as follows:
Jn = - Cos(S S,) = (,)
(8)
Stage 3: Detector Fine-tuning. Due to our encryption, the distribution of the images input to the pretrained O changes. Thus, we fine-tune O on the encrypted images T(I;; S). As proposed in Theorem 1, given the encrypted images T T(I; we use the pretrained detector O with parameters 0 to arrive at a better local minima. Therefore, the general objective of GOD and COD in Eq. (5) and Eq. (6) change to as follows:
{
min 0.0g 9p
Tyl Σ (CC.,
(9)
min {I( -., (,
(10)
6,0g
We use the detector-specific loss function JOBJ of O along with the encoder and decoder loss in Eq. (7) and Eq. (8) train all the three stages. The overall loss function to train PrObeD as follows:
J AonsJons XEJE AnJD.
(11)

---

Table 2: GOD results MS-COCO val split. PrObeD improves the performance of all GOD all thresholds
across all categories.
Method
AP
APs0 T
AP75
APs 1
APM
APz.
Faster R-CNN [58]
19.3
42.5
16.9
1.8
17.9
39.3
Faster R-CNN [58] PrObeD
31.7
52.6
33.3
11.0
35.5
51.1
Faster R-CNN FPN [42]
37.3
40.6
21.4
41.0
48.4
Faster R-CNN FPN [42
Seg. Mask [30]
38.2
60.3
41.7
22.1
43.2
51.2
Faster R-CNN FPN [42]
PrObeD
38.5
60.4
41.9
22.5
43.4
49.8
Sparse R-CNN [2]]
37.6
55.6
40.2
20.5
39.6
52.9
Sparse R-CNN [62]+ + PrObeD
39.2
57.5
41.5
21.7
40.1
53.6
YOLOvv [52]
48.9
67.6
53.
31.8
54.4
62.3
YOLOv5 [52] + PrObeD
49.4
67.9
53.5
32.0
55.1
62.6
DeTR
41.9
62.3
44.1
20.3
45.8
61.0
DeTR [8]
PrOheD
42.1
62.6
44.4
20.4
46.0
61.3
Table 3: COD results on CAMO, COD10K and NC4K datasets, PrObeD outperforms DGNet on all datasets and metrics.
CAMO
CODIOK
NC4K
Method
Em 1
Sm WF MAE,
Em f Sm f what MAEL Em 1 S. I WF MAET
DGNet [34]
0.859 0.791 0.68T 0.079
0.833
0.776 0.603 0.046 0.876
0.815 0.710 0.059
PrObeD 0.871
0.797 0.702 0.071
0.869 0.803
6.661 0.037 0.900 0.838
0.755 0.049
4
Experiments
We apply PrObeD for two categories of object detectors: GOD and COD.
GOD Baselines. For GOD, we apply PrObeD on four detectors with varied architectures: two-stage, one-stage, and transformer-based detectors, namely, Faster R-CNN [58], YOLO [52], Sparse R-CNN, and DeTR [8]. We use these works as baselines for three reasons: (1) varied architecture types, (2) their increased prevalence in the community, and (3) varied timelines (from earlier to recent detectors). We use the PyToorch [51] code of the respective detectors for our GOD experiments and use the corresponding GODs as our baseline. For YOLOOv5 and DeTR, we use the official repositories released by the authors; for Faster R- R-CNN, we use the public repository "Faster R L-CNN.pyttrrdl For other GOD detectors, we use Detectron2 library as the pre-trained detector. We use the ResNet101 backbone for Faster R-CNN, Sparse R- R-CNN and DeTR, and CSPDarknet53 for YOLOv5.
COD Baselines. For COD, we apply PrObeD the current SoTA camouflage detector DGNet [34] and use DGNet as our baseline. For all object detectors, we use the pretrained model released by the authors and fine-tune them with PrObeD, Please see the supplementary for more details.
Datasets. Our experiments use the MS-COCO 2017 [44] dataset for GOD, while we use CAMO [39], COD10K [17], and NC4K [47] datasets for COD. We use the following splits of these datasets:
MS-COCO 2017 Val Split [44]: includes 118,287 images for training and 5K for testing.
COD10K Val Split [17]: It includes 4,046 camouflaged images for training and 2,026 for testing.
CAMO Val Split [39]: It includes 1K camouflaged images for training and 250 for testing.
NC4K Val [47]: It includes 4,121 NC4K images. We use it for generalization testing as in [34].
Evaluation Metrics. We use mean average precision average at multiple thresholds in 00.5, 0.95] (AP) for GOD as in [44]. We also report results at threshold of 0.5 (APso). threshold of 0.75 (AP7s) and at different object sizes: small (APs), medium (APM). and large (APz). For COD, use E-measure E,,, S-measure 5m, weighted F1 score F.3 and mean absolute error M AE as [34].
4.1 GOD Results
Quantitative Results. Tab. shows the results of applying PrObeD on GOD networks. PrObeD improves the average precision of three detectors. The performance gain is significant for Faster R-CNN. As Faster R-CNN is an older detector, it was at worse minima to start with. PrObeD improves the convergence weight of Faster R-CNN by a significant margin, thereby improving the performance. We further experiment with two variations of Faster R- R-CNN, namely, Faster R-CNN +
7

---

(a)
(b)
(c)
(d)
(e)
Figure 3: Qualitative GOD Results on MS-COCO 2017 dataset. (a) ground-truth annotations, (b) Faster R-CNN [58] predictions, (c) Faster R-CNN [58]+ PrOhel) predictions, (d) generated template, and (c) recovered template. We highlight the objects responsible for improvement in (c) compared to (b). The yellow box represents better localization, the blue box represents false positives, and the red box represents missed predictions. PrObeD improves on these errors made by (b).
(a)
(b)
(c)
(d)
(e)
(f)
Figure 4: Qualitative COD Results on CAMO, COD1OK, and NC4K datasets from top to battom, after applying
PrObeD. (a) input images, (b) ground-truth camouflaged map. (c) DGNet [34] predictions, (d) DGNet [34]
PrObeD predictions, (e) generated PrObeD template, and (f) recovered PrObeD template. PrObeD template has the semantics of the camouflaged object, which aids DGNet detection.
FPN and SCNrse-RCNN. We observe an increase in the performance of both detectors. PrObeD also improves newer detectors like YOLOv5 and DeTR, although the gains are smaller compared to Faster R-CNN. We believe this happens because the newer detectors leave little room for improvement due to which PrObeD improves the performance slightly. We next compare PrObeD with a work that leverage segmentation map mask for object detection. We compare our performance with Mask R-CNN [30], which uses an image segmentation branch to help with object detection. Tab. 2 shows that the gains using Mask R-CNN are lower than using our proactive wrapper.
Qualitative Results. Fig. shows qualitative results for the MS-COCO 2017 dataset. PrObeD clearly improves the performance of pretrained Faster R-CNN for three types of errors: Missed predictions, false negatives, and localization errors. PrObeD has lower number missed predictions, fewer false positives, and better bounding box localization. We also visualize the generated and recovered templates. We see that the template has object semantics the input images. When the template is multiplied with the input image, it highlights the foreground objects, thereby making the task of object detector easier.
Error Analysis. We show the error analysis [6] for GOD section 4 of the supplementary. We observe that all GOD detectors make mistakes mainly due five types of errors: classification, localization, duplicate detection, background detection, and missed detection. The main reason for the degraded performance is the errors in which the breegrundd-bccgrooung include localization, background detection, and missed detection. Our proactive wrapper significantly corrects these errors, as the template has object semantics, which, when multiplied with the input image, highlights the foreground objects, consequently simplifying the task of object detection.
boundary missed. These errors

---

Table 4: Performance comparison with proactive works. MaLP [2] has significantly deteriorated performance than PrObeD.
CAMO
COD10K
NC4K
Method
Em t
Sm.
what MAEL
Bm
Sm
wFat MAE]
Em
S..
WFa 1 MAE
MaLP [
0.474
0.514
0.218 0.254
0.491
0.520
0.150 0.202
0.503
0.548
0.228 0.222
PrOheD 0.871 0.797
O. 702 0.071
0.869
0.803 0.661 0.037 0.900
0.838 0.755 0.049
Table 5: Ablation studies of PrObeD using Faster R-CNN GOD on MS-COCO 2017 dataset. Removing the
encoder/decoder network adding the template results in degraded performance.
Changed
From
AP
APso f
AP76 f
APs f
APar A
AP,.
Image Dependent + Fixed
17.6
37.9
15.1
1.3
15.4
39.5
Template
Image Dependent Universal
19.4
42.6
17.1
1.9
18.0
39.4
Decoder
Yes + No
25.2
46.1
26.2
5.3
26.6
Transformation
Multiply- + Add
19.2
42.3
20.1
1.7
17.9
39.1
PrObeD
31.7
52.6
33.3
11.0
35.5
51.1
4.2 COD Results
Quantitative Results. Tab. 3 shows the result of applying PrObeD to DGNet [34] on three different datasets. PrObeD, when applied on top of DGNet, outperforms DGNet on all four metrics for all datasets. The biggest gain appears in COD10K and NC4K datasets. This is impressive datasets have more diverse testing images than CAMO. As NC4K is only a testing set, the higher performance of PrObeD demonstrates its superior generalizability as compared DGNet [34]. This result agrees with the observation in [1, 2], where proactive-based approaches exhibit improved generalization on manipulation detection and localization tasks.
these
Qualitative Results. Fig. 4 visualizes the predicted camouflaged map for DGNet before and after applying PrObeD testing samples of all three datasets. PrObeD improves the predicted camouflaged map, with less blurriness along the boundaries and better localization of the camouflaged object. As observed before for GOD, the generated and recovered template has the semantics of the camouflaged objects, which after multiplication intensifies the foreground object, resulting in better segmentation by DGNet.
4.3 Ablation Study
Comparison with Proactive Works. The prior proactive works perform different task of image manipulation detection and localization. Therefore, these works are not directly comparable to our proposed proactive wrapper, which performs a different task of object detection as described in Tab. 1. However, manipulation localization and COD both involve a prediction of localization map, segmentation, and fakeness map, respectively. This inspires us experiment with MaLP [2] for the task of COD. We train the localization module of MaLP supervised with the COD datasets. The results are shown in Tab. We see that MaL.P not able to perform well for all three datasets. MaLP is designed for estimating universal templates rather than templates tailored to specific images. It shows the significance of image-specific templates object detection. While Mal.P's design with meage-independent templates is effective for localizing image manipulation, applying it to object detection has a negative impact on performance.
Framework Design. PrObeD consists of blocks to improve the object detector. Tab. ablates different versions PrObeD to highlight the importance of each block in our design. PrObeD utilizes an encoder network & to learn image-dependent templates aiding the detector. We remove the encoder from our network, replacing with fixed template. We observe that the performance deteriorates by large margin. Next, we make this template learnable proposed in PrObeD, but only a single template would be used for all the input images. This choice also results in worse performance, highlighting that mnngeddeeenddet templates are necessary for object detection. Finally, we remove decoder network D, which used to recover template from the encrypted images. Although this results in better performance than the pretrained Faster R-CNN, we observe drop as compared PrObeD. Therefore, as discussed in Sec. 3.3, the recovery templates is indeed a necessary and beneficial step for boosting the performance of the proactive schemes.

---

Table 6: Ablation of training iterations Faster R-CNN. YOLOvS, and DeTR for more iterations similar
to after applying PrObeD. also report the inference time the delectors before and after applying PrObeD. Training object detectors proactively with PrObeD results in more performance gain compared to
training passively for more iterations. PrObeD adds an overhead cost on top of the inference cost of detectors.
Method
Irerations
AP
APso
APrs
APs
APM
A
T Time (ms)
Faster R- RNNN [58]
1x
19.3
42.5
16.9
188
17.9
39.3
161.1
Faster R-CNN
[58]
2x
20.1
46.6
21.5
3.3
20.3
41.2
Faster R- R-CNN
58
PrObeD
2x
31.7
52.6
33.3
11.0
35.5
51.1
175.3 (1 8,7%)
YOLOvw [2]
1x
48.9
67.6
53.1
31.8
54.4
62.3
48.5
YOLOv5 [52]
2x
48.8
67.7
53.0
31.8
54.7
62.4
YOLOv5 [2]] PrObeD
2x
49.4
67.9
53.5
32.0
55.1
62.6
62.7 (1 29.1%)
DeTR [8]
1x
41.9
62.3
44.1
20.3
45.8
61.0
194.2
DeTR [8]
41.9
62.4
44.0
20.1
45.9
61.1
DeTR [8] PrObeD
2x
42.1
62.6
44.4
20.4
46.0
61.3
208.4 (1 7.2%)
Encryption Process. PrObeD includes an eneryption process as described in Eq. (3), which involves multiplying the template with the input image. This process makes the template act as a mask, highlighting the foreground for better detection. However, prior proactive works [1, 2] consider adding templates to achieve better results. Thus, we ablate by changing the encryption process to template addition. Tab. 5 shows that template addition degrades performance by a sigmificant margin w.r.t. our multiplication scheme. This shows that encryption is key step in formulating proactive schemes, and the same encryption process may not work for all tasks.
More Training Time. We perform an ablation to show that the performance gain of the detector is due to proactive wrapper instead of training for more iterations of the pretrained object detector. Results in Tab, 6 show that although more training iterations for the detector has a performance gain, it's not enough to get the significant margin in performance achieved by PrObeD. This shows that extra training can help, but only up to a certain extent.
Inference Time. We evaluate the overhead computational cost after applying PrObeD on different object detectors are shown in Tab. averaged across 000 images, on NVIDIA V V00 GPU, Our encoder network has 17 layers, which adds extra cost for inference. For detectors with bulky architectures like Faster R-CNN (ResNet101) and DeTR (transformer), the overhead computational cost is quite small, 8.7% and 7.2%. respectively. This additional cost is minor compared to the performance gain of detectors, especially Faster R-CNN. For a lighter detector like YOLOv5, our overhead computational cost increases to 29.1% %. So, there is a trade-off of applying PrObeD to different detectors with varied architectures. PrObeD more beneficial bulky detectors like (wo-staged//tansffommes based as compared to one-stage detectors.
5
Conclusion
We mathematically prove that the proactive method results in a better-converged model than the passive detector under assumptions and, consequently, a better 2D object detector. Based on this finding, we propose a proactive scheme wrapper, PrObeD, which enhances the performance of camouflaged and generic object detectors. The wrapper outputs an image-dependent template using an encoder network, which encrypts the input images. These encrypted images are then used to fine-tune the object detector. Extensive experiments on MS-COCO, CAMO, COD10K, and NC4K datasets show that PrObeD improves the overall object detection performance for both GOD and COD detectors.
Limitations. Our proposed scheme has the following limitations. First, PrObeD does not provide a significant gain for recent object detectors such as YOLO and DeTR. Second, the proactive wrapper should be thoroughly tested on other object detectors to show the generalizability of PrObeD. Finally, only experiment with simple multiplication and addition as the encryption scheme. A more sophisticated encryption process might further improve the object detectors' performance. We leave these for future avenues.
10

---

References
[1] Vishal Asnani, Xi Yin, Tal Hassner, Sijia Liu, and Xiaoming Liu. Proactive image manipulation
detection. In CVPR, 2022. 1, 2, 3, 4, 5, 6, 9, 10
[2] Vishal Asnani. Xi Yin, Tal Hassner, and Xiaoming Liu. MaLP: Manipulation localization using
a proactive scheme. In CVPR. 2023. 2, 3.4.3. 5, 6, 9, 10
[3] Vishal Asnani, Xi Yin. Tal Hassner, and Xiaoming Liu. Reverse engincering of generative models: Inferring model hyperparameters from generated images. 7PAMI, 2023.
[4] Yousef Atoum, Joseph Roth, Michael Bliss, Wende Zhang, and Xiaoming Liu. Monocular video-basee trailer coupler detection using multiplexer convolutional neural network. In ICCV,
2017. 3
[5] Alexey Bochkovskiy, Chien-Yao Wang, and Hong- Yuan Mark Liao. YOLOv4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004. 10934, 2020. 1,3
[6] Daniel Bolya, Sean Foley, James Hays, and Judy Hoffman. TIDE: A general toolbox for identifying object detection errors. ECCV, 2020.
[7] Garrick Brazil and Xiaoming Liu. Pedestrian detection with autoregressive network phases. In
CVPR, 2019. 3
[8] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In ECCV, 2020. 3, 7, 10 [9] Geng Chen, Si-Jie Liu, Yu-Jia Sun, Ge-Peng Ji, Ya-Feng Wu, and Tao Zhou. Camouflaged object detection via context-aware cross-level fusion. IEEE Transactions on Circuits and Systems for Video Technology, 32(10), 2022. 1,3
[10] Shoufa Chen, Peize Sun, Yibing Song, and Ping Luo. DiffusionDet: Diffusion model for object detection. In CVPR, 2023. 1,3
[11] Wei-Chen Chen, Xin-Yi Yi Yu, and Lin Ou. Pedestrian attribute recognition in video surveil- lance scenarios based on view-attribute attention localization. Machine Intelligence Research, 2022.
[12] Xuelian Cheng, Huan Xiong, Deng-Ping- Fan, Yiran Zhong, Mehrtash Harandi, Tom Drummond, and Zongyuan Ge. Implicit motion handling for video camouflaged object detection. In CVPR,
2022. 3
[13] Pengyu Chu, Zhaojian Li, Kyle Lammers, Renfu Lu, and Xiaoming Liu. Deepapple: Deep learning-based apple detection using a suppression mask R-CNN. PRI, 2021. 3
[14] Jifeng Dai, Yi Li, Kaiming He, and Jian Sun. R-FCN: Object detection via region-based fully convolutional networks. NeurIPS, 2016.
[15] Navneet Dalal and Bill Triggs. Histograms of oriented gradients for human detection. In CVPR, 2005.
[16] Mohammad Derakhshani, Saeed Masoudnia, Amir Shaker, Omid Mersa, Mohammad Sadeghi,
Mohammad Rastegari, and Babak Araabi. Assisted excitation of activations: A learning technique to improve object detectors. In CVPR, 2019. 1
[17] Deng-Ping Fan, Ge-Peng Ji. Ming-Ming Cheng, and Ling Shao. Concealed object detection. TPAMI, 2021. 1,
[18] Deng-Ping Fan, Ge-Peng Ji, Guolei Sun, Ming-Ming Cheng, Jianbing Shen, and Ling Shao. Camouflaged object detection. In CVPR, 2020. 1,3
[19] Deng-Ping Fan, Ge-Peng Ji, Tao Zhou, Geng Chen, Huazhu Fu, Jianbing Shen, and Ling Shao. Pranet: Parallel reverse attention network for polyp segmentation. MICCAI, 2020.
[20] Pedro Felzenszwalb, David McAllester, and Deva Ramanan. A discriminatively trained, multi- scale, deformable part model. In CVPR, 2008. 1
[21] Sanja Fidler, Roozbeh Mottaghi, Alan Yuille, and Raquel Urtasun. Bottom-up segmentation for top-down detection. In CVPR, 2013.
[22] Spyros Gidaris and Nikos Komodakis. Object detection via a multi-region and semantic
segmentation-ang awwatt aware enn model. In ICCV, 2015. 1
[23] Ross Girshick. Fast R-CNN. In ICCV, 2015. 1,3
[24] Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR, 2014. 3,
[25] Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Region-based convolutional networks for accurate object detection and segmentation. TPAMI, 2015. 3
[26] Agrim Gupta, Piotr Dollár, and Ross Girshick. LVIS: A dataset for large vocabulary instance segmentation. In CVPR, 2019. 1
11

---

[27] Chumming He, Kai Li, Yachao Zhang, Longxiang Tang, Yulun Zhang, Zhenhua Guo, and Xiu Li. Camouflaged object detection with feature decomposition and edge reconstruction. In CVPR,
2023.
[28]
Chunming He, Kai Li, Yachao Zhang, Guoxia Xu, Longxiang Tang, Yulun Zhang, Zhenhua Guo, and Xiu Li. Weakly-supervised concealed object segmentation with SAMM-asedd pseudo labeling and multi-scale feature grouping. arXiv preprint ar arXiv:2305. 10022005..003,, 2023. 1
[29]
Chunming He, Kai Li. Yachao Zhang, Yulun Zhang, Zhenhua Guo, Xiu Li, Martin Danelljan, and Fisher Yu. Strategic preys make acute predators: Enhancing camouflaged object detectors by generating camouflaged objects. arXiv preprint vrXiv:2308.03100 2023. 1
[30] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask R-CNN. In ICCV, 2017.
7, 8
[31] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Spatial pyramid pooling in deep convolutional networks for visual recognition. TPAMI, 2015. 3
[32] Yihui He, Chenchen Zhu, Jianren Wang, Marios Savvides, and Xiangyu Zhang. Bounding box regression with uncertainty for accurate object detection. In CVPR, 2019. 1
[33] Jianqin Yin Yanbin Han Wendi Hou and Jinping Li. Detection of the mobile object with camouflage color under dynamic background based on optical flow. Procedia Engineering,
2011. 1.3
[34] Ge-Peng Ji, Deng- Fan, Yu-Cheng Chou, Dengxin Dai, Alexander Liniger, and Luc Van Gool. Deep gradient learning for efficient camouflaged object detection. Machine Intelli- gence Research, 2023. 1, 7, 9.
[35] Ge-Peng Ji, Lei Zhu, Mingchen Zhuge, and Keren Fu. Fast camouflaged object detection via edge-based reversible re-calibration network. Puttern Recognition, 123, 2022.
[36] Nobukatsu Kajiura, Hong Liu, and iiini Satoh. Improving camouflaged object detection
with the uncertainty of pseudo-edge labels. In ACM Multimedia Asia, 20 3
[37] Abhinav Kumar. Garrick Brazil, Enrique Corona, Armin Parchami, and Xiaoming Liu. DE- VIANT: Depth EquiVarlAni VarlAnt NeTwork for Monocular 3D Object Detection. In ECCV, 2022.
3
[38] Abhinav Kumar, Garrick Brazil, and Xiaoming Liu. GrooMeD-NMS: Grouped mathematically
differentiable nms for monocular 3D object detection. In CVPR, 2021. 3
[39] Trung-Nghia Le, Tam Nguyen, Zhongliang Nie, Minh-Triet Tran, and Akihiro Sugimoto.
Anabranch network for camouflaged object segmentation. CVIU, 2019. 3, 7
[40] Aixuan Li, Jing Zhang, Yunqiu Lv, Bowen Liu, Tong Zhang, and Yuchao Dai. Uncertainty-aware
joint salient object and camouflaged object detection. In CVPR, 2021. 1,3
[41] Zeming Li, Chao Peng, Gang Yu, Xiangyu Zhang, Yangdong Deng, and Jian Sun. Light-head
2017. 3 [42] Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie.
R- R-CNN: In defense of two-stage object detector. arXiv preprint arXiv:1711.0721
Feature pyramid networks for object detection. In CVPR, 2017. 3.7
[43] Tsung- Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense
object detection. In ICCV, 2017. 3,3
[44] Tsung-Ys Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and Lawrence Zitnick. Microsoft COCO: Common objects in context. In ECCV, 2014.7
[45] Jiannan Liu, Bo Dong, Shuai Wang, Hui Cui, Deng-Ping Fan, Jiquan Ma, and Geng Chen. 19 lung infection segmentation with a novel two-stage cross-domain transfer learning framework. Medical image analysis, 2021.
[46] Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng- Yang Fu, and Alexander Berg. SSD: Single shot multibox detector. In ECCV, 2016. 1,
[47] Yunqiu Lv, Jing Zhang, Yuchao Dai, Aixuan Li, Bowen Liu, Nick Barnes, and Deng-Ping Fan. Simultaneously localize, segment and rank the camouflaged objects. In CVPR, 2021. 3, 7
[48] Yuxin Mao, Jing Zhang, Zhexiong Wan, Yuchao Dai, Aixuan Li, Yunqiu Lv, Xinyu Tian, Deng- Ping Fan, and Nick Barnes. Transformer transforms salient object detection and camouflaged object detection. arXiv preprint arXiv:2104. 10127, 2021. 1,3
[49] Haiyang Mei, Ge-Peng Ji, Ziqi Wei, Xin Yang. Xiaopeng Wei, and Deng-Ping Fan. Camouflaged object segmentation with distraction mining. In CVPR, 2021. 3
[50] Yuxin Pan, Yiwang Chen, Qiang Fu, Ping Zhang, and Xin Xu. Study on the camouflaged target detection method based on 3D convexity. Modern Applied Science, 2011. 1,3
12

---

Unified, [51] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary Vito,, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang. Junjie Bai, and Soumith Chintala. PyTorch: An imperative style,
geghhperformace deep learning library. In NeurlPS, 2019. 7
[52] Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once:
real-time object detection. In CVPR, 2016. 1, 3, 7, 10
[53] Joseph Redmon and Ali Farhadi. YOLO9000: better, faster, stronger. In CVPR, 2017. 1,3
[54] Joseph Redmon and Ali Farhadi. YOLOvv: An incremental improvement. arXiv preprint
arXiv: 02067, 2018. 1.3
[55] Atique Rehman, Rafa Rahim, Shahroz Nadeem, and Sibt Hussain. End-to-end trained CNN
encoder-decoder networks for image steganography. In ECCVW, 2018. 6
[56] Atique-ur Rehman, Rafia Rahim, Shahroz Nadeem, and Sibt-ul Hussain. End-to-end trained
CNN encoder-decoder networks for image steganography. In ECCVW, 2019. 6
[57] Jingjing Ren, Xiaowei Hu, Lei Zhu, Xuemiao Xu, Yangyang Xu, Weiming Wang, Zijun Deng, and Pheng-Ann Heng. Deep texture-aware features for camouflaged object detection. IEEE
Transactions on Circuits and Systems for Video Technology, 2023. 3
[58] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster R-CNN: Towards real-time object detection with region proposal networks. NeurIPS, 2015. 1, 7, 8, 10
[59] Nataniel Ruiz, Sarah Adel Bargal, and Stan Sclaroff. Disrupting deepfakes: Adversarial attacks against conditional image translation networks and facial manipulation systems. In ECCV, 2020.
1, 2, 3
[60] Eran Segalis and Eran Galili. OGAN:
Disrupting deepfakes with an adversarial attack that survives training. arXiv preprint arXiv:2006. /2247, 2020. 2,
[61] Sengottuvelan, Amitabh Wahi, and A Shanmugam. Performance of decamouflaging through exploratory image analysis. In ICETET, 2008 1,3
[62] Peize Sun, Rufeng Zhang, Yi Jiang. Tao Kong. Chenfeng Xu, Wei Zhan, Masayoshi Tomizuka, Lei Li, Zehuan Yuan, Changhu Wang. and Ping Luo. Sparse R-CNN: End-to-end object
detection with learnable proposals. In CVPR, 2021. 7
[63] Yujia Sun, Geng Chen, Tao Zhou, Yi Zhang, and Nian Liu. Context-aware cross-level fusion network for camouflaged object detection. arXiv preprint arXiv:2 /2555, 2021. 1,3
[64] Paul Viola and Michael Jones. Rapid object detection using boosted cascade of simple features.
In CVPR, 2001. 1
[65]
Paul Viola and Michael Jones. Robust real-time face detection. IJCV, 2004.
[66] Chien-Yao Wang, Alexey Bochkovskiy, and Hong- Yuan Liao. YOLOv7: Trainable bag-of-
freebies sets new state-of-the-ar for real-time object detectors. In CVPR, 2023. 3
[67] Jingdong Wang, Ke Sun, Tianheng Cheng, Borui Jiang, Chaorui Deng, Yang Zhao, Dong Liu, Yadong Mu, Mingkui Tan, Xinggang Wang, Wenyu Liu, and Bin Xiao. Deep high-resolution
representation learning for visual recognition. TPAMI, 2020. 3
[68] Run Wang, Felix Juefei-Xu, Meng Luo, Yang Liu, and Lina Wang. FakeTagger: Robust safeguards against deepfake dissemination via provenance tracking. In ACM Multimedia, 2021.
1, 2.
[69] Jian Ru Xue, Jian- Wu Fang, and Pu Zhang. A survey of scene understanding by event reasoning
in autonomous driving. International Journal of Automation and Computing. 2018. 1
[70] Fan Yang, Qiang Zhai, Xin Li, Rui Huang, Ao Luo, Hong Cheng, and Deng-Ping Fan. Uncertainty-guided transformer reasoning for camouflaged object detection. In ICCV, 2021. 1,
3
[71] Chin-Yuan Yuan Yeh, Hsi-Wen Chen, Shang-Lun Tsai, and Sheng-De Wang. Disrupting image- translation-basel deepfake algorithms with adversarial attacks. In WACVW, 2020. 3
[72] Matthew Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In ECCV, 2014.
[73] Qiang Zhai, Xin Li, Fan Yang, Chenglizhao Chen, Hong Cheng, and Deng-Ping Fan. Mutual graph learning for camouflaged object detection. In CVPR, 2021. 3
[74] Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable DETR:
Deformable transformers for end-to-ena object detection. In ICLR, 2020. 1,3
[75] Mingehen Zhuge, Xiankai Lu, Yiyou Guo, Zhihua Cai, and Shuhan Chen. CubeNet: X-shape connection for camouflaged object detection. Pattern Recognition, 2022. 3
13


Reviews:
[{"review_id": "9zq75uCfjI", "reviewer": "Reviewer_V1FT", "summary": "The paper proposes a proactive approach to enhance the performance of object detection through adding a learnable image-dependent template into the image. The optimal template can effectively bridge the gap between the learned and the ideal weights for object detectors. As demonstrated by the results, the template acts as a mask of the input images to provide relevant information of the objects to guide the object detector for improved detection. The proposed approach can effectively improve the results of both generic and camouflaged object detection as shown in the extensive experimental results of diverse object detection benchmarks.", "questions": "After reading the proposed method, I think its functionality is close to a certain of region proposal generator to guide the object detector to better detect the objects. However, as shown in the Table 2, for one-stage detectors, YOLOv5 and DeTR both get marginal improvement. In contrast, the two-stage detector with the regional proposal network, Faster-RCNN gets significant performance improvement. It means that the proposed method can provide better object proposal information than the regional proposal network of Faster-RCNN. However, the performance of the original Faster-RCNN is very low as shown in the original NeurIPS paper. The updated Faster-RCNN with proper tuning and architecture selection can achieve much better performance. To consider these factors, I worry that the proposed method does not work as claimed for generic object detection. In addition, it is not clear for me why the authors choose to minimze the cosine similarity between the learnable template and the ground truth mask. I think the cross entropy loss with dice loss which are widely used in the segmentation task could be more appropriate to achieve the desired effect. Perhaps, the authors could further elaboration the loss design.\n\nFor camouflaged object detection, since it is formulated as a segmentation problem, it is reasonable that if the proposed method could provide object information, it will be very helpful for the following segmentation. However, I am curious about the comparisons of the region proposal network (if trained appropriately) and the proposed method in terms of the effectiveness and efficiency.\n\nHow do the authors actually conduct the element-wise multiplication between the template S_j and the input image I_j? The dimensions of both tensors seems mismatched where S_j is HxW but I_j is HxWx3.", "limitations": "Yes, the authors have addressed the limitation of their proposed method with sufficient context. For the negative societal impact, the authors could discuss more about avoiding the object detection from malicious monitoring use, etc.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed encoder-based learnable template can effectively highlight the semantic useful information to guide the object detector to perform improved detection.\n2. The proposed learnable template can help significantly improve the performance of Faster-RCNN and other object detectors while not introducing much additional training and computational costs.\n3. The authors have conducted extensive experimental evaluation to demonstrate their effectiveness.", "weaknesses": "1. For the generic object detection, the performance improvement for more modern object detectors, YOLOv5 and DeTR, is marginal and is not as significant as the Faster-RCNN. There is no discussions about this situation.\n2. The performance of Faster-RCNN seems to be the old one. With proper parameter tuning and backbone selection as other modern object detectors, the performance should be much higher than 19.3, or even close to 40 on the MSCOCO dataset. The results of Faster-RCNN are not representative. However, most of the ablation studies of the proposed method on GOD are based on Faster-RCNN.\n3. Some relevant works to leverage ground truth to generate a mask to help improve the object detection are not included for discussion in the related work.\n\nDerakhshani, Mohammad Mahdi, Saeed Masoudnia, Amir Hossein Shaker, Omid Mersa, Mohammad Amin Sadeghi, Mohammad Rastegari, and Babak N. Araabi. \"Assisted excitation of activations: A learning technique to improve object detectors.\" In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9201-9210. 2019.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback. We provide a global response containing positives comments and major concerns of all reviewers. We also provide a global pdf file containing tables and figures with reference to added evaluation by some reviewers. We address all of your concerns below:\n\n$\\textbf{Performance of YOLOv5 and DeTR.}$\nThank you for your comment. We agree that the performance gain is not significant enough for YOLOv5 and DeTR as compared to Faster-RCNN. We argue that the new $2$D object detectors have very little room for improvement, therefore making it harder for PrObeD to improve the performance by a significant margin. We show that the idea of using a proactive wrapper on top of various object detectors (GOD/COD) will result in the increase in the performance. As this is the first novel work to propose a proactive wrapper to improve object detection, we believe that PrObeD will motivate the future works to propose more sophisticated wrappers to enhance the margin of improvement in the performance of object detection. Our error analysis in Figure $1$ of supplementary also proves the effectiveness of applying the idea of proactive schemes to improve the performance of the detectors. \n\nWe will add this discussion in the manuscript. \n\n\n$\\textbf{Performance of Faster-RCNN.}$ \nThat’s an interesting point. We chose the original Faster R-CNN for our experiments. We report the results from the original Faster R-CNN paper [Ren $\\textit{et al.}$ in NIPS $2015$] in our Table $2$ of paper. We agree that fine-tuning the model with optimal parameters and architecture selection boosts the performance. Having said that, we experiment with two variations of Faster R-CNN, namely, Faster R-CNN + FPN and Sparse-RCNN. The results are shown in Table $1$ of global pdf file. We observe an increase in the performance for both the detectors. This shows that PrObeD generalizes well to all the types of object detectors for increasing the performance. Also, the point to be noted here is that before and after applying our proposed PrObeD, the object detector is same in both cases. We show in the paper Table $5$ that even fine-tuning the detectors without our wrapper does increase the performance across various object detectors including GOD and COD, but the gain is less compared to after applying PrObeD. Stating this, if any variation in the object detector hyperparameters or in architecture is helping the object detector, it will help even after applying PrObeD too.\n\n$\\textbf{Related works to leverge ground truth.}$ \nWe will added the following related prior works which leverage ground truth to generate a mask for improving the object detection into our manuscript. \n\n\n$1.$ Mahdi $\\textit{et al.}$ \"Assisted excitation of activations: A learning technique to improve object detectors.\" In CVPR $2019$.\n\n$2.$ Fidler $\\textit{et al.}$. Bottom-up segmentation for top-down detection. In CVPR $2019$.\n\n$3.$ Gidaris $\\textit{et al.}$. Object detection via a multi-region and semantic segmentation-aware cnn model. In ICCV $2015$ .\n\n$4.$ He, Kaiming $\\textit{et al.}$. \"Mask r-cnn.\" In ICCV. $2017$.\n\nWe will add more works related to the ones above in the manuscript.\n\n$\\textbf{Ablation of using cross entropy loss with dice loss.}$\nOur loss design is inspired by the prior proactive works [Asnani $\\textit{et al.}$ CVPR $2022$, Asnani $\\textit{et al.}$ CVPR $2023$]  which estimate the learnable template by applying a cosine similarity loss. The authors experiment with various loss types, showing the effectiveness of cosine similarity loss design. However, we agree that a cross-entropy loss with dice loss inspired by segmentation works might be beneficial for object detection. Due to the time constraint, we perform an ablation of applying cross-entropy loss with dice loss for the task of object detection only for COD. The results are shown in Table $2$ of global pdf file. We see that our proactive wrapper is not benefiting by removing the cosine similarity loss, proving the study of the prior proactive works. \n\n\n$\\textbf{Predicting object information.}$\nWe agree that predicting the object information for COD can be useful. However, the dataset used by the COD works don’t have this label information to help train the detector for predicting this information. We propose PrObeD as a wrapper on top of the existing object detectors to improve the performance. We don’t change the design of the object detector in any form. \n\nAnother point to be stated here is that PrObeD is able to reduce the classification and localization errors of some of the GOD object detectors as shown in the figure 1 of the supplementary. Improvement of the performance across both COD and GOD detectors show the generalizability of PrObeD, for both in terms of detection and classification of objects.\n\n$\\textbf{Clarification regarding  element-wise multiplication of template and image}$\nDue to the template having only one channel as compared to the input image with three channels, we repeat the single channel information of the template thrice to match the dimensions of the input images. \n"}, {"review_id": "JZlt00OCQW", "reviewer": "Reviewer_rzBs", "summary": "In this paper, the authors study the problem of proactive object detection. To be specific, the authors introduce a wrapper around an existing object detector that can produce a template image and improve the performance of the detector if the image is multiplied with this template. Prior works have proposed learning this template in an image independent manner. One of the important contributions of the paper is to make the template image dependent.\n\nAfter the rebuttal:\n\nI've read the other reviews and the responses provided by the authors. The authors have addressed my concerns and I have increased my recommendation. However, since the reported improvements are only minor, I feel unable to make a stronger recommendation.", "questions": "Please see above.", "limitations": "None.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 1, "contribution": 3, "strengths": "1. Novel extension over prior work.\n2. Significant improvements over baselines.", "weaknesses": "I don't work on proactive object detection. So, my evaluation maybe limited.\n\n1. An important issue with the paper is that it fails to provide intuition on several design choices.  For example:\n\n1.1. Lemma 1: The lemma is clear. But, what is the intuition for the template to improve the weights? Since the model is linear and the element-wise multiplication is linear, I don't see that adding S improves model capacity. The authors should provide some intuition here.\n\n1.2. Why is template recovery helpful? Why is the recovered template too different from the generated one (in Fig 3)?\n\n2. Theorem 1: \"Then, the AP of the proactive detector is better than the AP of the passive detector\" => They can be equal as well, if you check your proof carefully.\n\n3. Experimental evaluation has major issues:\n\n3.1. Certain crucial details are not provided:\n  - It is not clear what framework (mmdetection, detectron, ..) is used.\n  - Table 1: It is not clear what the backbones are.\n\n3.2. Faster R-CNN's baseline performance (19.3 AP) is way too low compared to the known scores. See e.g. https://github.com/open-mmlab/mmdetection/tree/main/configs/faster_rcnn\n\n3.3. No comparison is provided with the closest work [1,2].\n\n\nMinor comments:\n\n- \"convergence to global minima is not optimal in neural networks\" => Did you mean to say \"not guaranteed\" instead of \"not optimal\"? \n\n- \"weights in the object detector are not optimal\" => \"weights in object detectors are not optimal\"?\n\n- \"schemes,PrObeD,\" => \"schemes, PrObeD,\".\n\n- The abstract is overloaded with undefined unconventional terms (for the object detection literature) such as proactive scheme, template, ideal object detector... These make the abstract difficult to read.\n\n- \"has slowly improved from earlier traditional detectors\" => I am not sure \"slowly\" is a suitable term here.\n\n- Fig 1: The images are too small to comprehend what's being performed with the proposed method.\n\n- Line 111: The symbol should be T for box coordinates.\n\n- Eq 1: This is an overly simplistic formulation of the loss functions used in GOD. I would suggest using generic terms (such as L_box and L_cls) to keep the formulation general. Moreover, you should use \\cdot instead of \".\" for multiplication (same for Eq 5).\n\n- Eq 3: It is better to use \\odot for element-wise multiplication.\n\n- Proof of Lemma 1: It would be better to stick to the notation in the paper. E.g., for the template, you should use S instead of \\alpha, and I for image instead of h.\n\n- Fig 2: There are two arrows for training in the legend.\n\n- Line 183: What is T here? The transformation or the box coordinates?\n\n- \"Faster-RCNN\" => \"Faster R-CNN\".", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback. We provide a global response containing positives comments and major concerns of all reviewers. We also provide a global pdf file containing tables and figures with reference to added evaluation by reviewers. We address all of your concerns below:\n\n$\\textbf{Clarification regarding Lemma 1.}$\nWe want to clarify that our claim is not about model capacity. We only propose that using a template proactively leads to the optimum weights of object detector.  \nRegarding the proof of lemma $1$, we will make the proof and text consistent in terms of variables for easy understanding.\n\n$\\textbf{Clarification regarding Theorem 1.}$\nWe agree that it could be equal since we state that AP is a non-decreasing function of IoU. However, our experience says that IoU improvements, especially close to $1$, lead to significant AP improvements. \nCurrent SoTA detectors already achieve decent IoU; hence, even a slight improvement in IoU improves the AP score.\n\n$\\textbf{Motivation for template recovery.}$\nPrior proactive works [Asnani \\etal in CVPR 2022, Asnani \\etal in CVPR 2023] on image manipulation detection and localization show that the template recovery is useful for learning the optimum template. Our design of proactive wrapper is inspired by these works. We perform an ablation on removing the template recovery decoder from our wrapper. The ablation results in main paper Table $4$ show that the performance decreases if the decoder is removed. \n\nFor Figure $3$, we apologize for overlooking this mistake. We came to know that the recovered template visualization in the manuscript is wrong. We remade the figure and show it in the global pdf file (Figure $1$). \n\n$\\textbf{Experimental details.}$\nFor YOLOv5 and DeTR, we use the official repositories released by authors. For Faster R-CNN, we use public repository \"faster-rcnn.pytorch\". We use the ResNet101 backbone for Faster R-CNN, CSPDarknet53 for YOLOV5, and ResNet101 for DeTR. \n\n$\\textbf{Performance of Faster R-CNN.}$\nThat’s an interesting point. We chose the original Faster R-CNN for our experiments. We report the results from the original Faster R-CNN paper [Ren $\\textit{et al.}$ in NIPS $2015$] in our Table $2$ of paper. We agree that fine-tuning the model with optimal parameters and architecture selection boosts the performance. Having said that, we experiment with two variations of Faster R-CNN, namely, Faster R-CNN + FPN and Sparse-RCNN. The results are shown in Table $1$ of global pdf file. We observe an increase in the performance for both the detectors. This shows that PrObeD generalizes well to all the types of object detectors for increasing the performance. Also, the point to be noted here is that before and after applying our proposed PrObeD, the object detector is same in both cases. We show in the paper Table $5$ that even fine-tuning the detectors without our wrapper does increase the performance, but the gain is less compared to after applying PrObeD. Stating this, if any variation in the object detector hyperparameters or in architecture is helping the object detector, it will help even after applying PrObeD too.\n\n$\\textbf{Comparison with proactive works.}$\nThe prior proactive works perform a different task of image manipulation detection and localization. Therefore, these works are not directly comparable to our proposed proactive wrapper, which performs a completely different task of object detection as described in Table $1$ of the paper. However, manipulation localization and COD both involve a prediction of a localization map, segmentation and fakeness map respectively. This inspires us to experiment with MaLP (Asnani~\\etal CVPR $23$) for the task of COD. We train the localization module of MaLP supervised with the COD datasets. The results are shown in Table $3$ in the global pdf file. We see that MaLP is not able to perform well for all three datasets. This was expected as MaLP is designed for image manipulation localization and extending it to object detection is not useful.\n\n$\\textbf{Convergence to global minima.}$\nGreat suggestion! Yes, we meant \"not guaranteed\" and we have fixed in paper.\n\n$\\textbf{Abstract.}$\nWe have updated the abstract to reduce and redefine some terms.\n\nPrior works in $2$D object detection focus on two different problems to detect objects in generic and camouflaged images. These works are regarded as passive works for object detection as they take the input image as is. However, convergence to global minima is not optimal in neural networks; therefore, we argue that the trained weights in the object detector are not optimal. To rectify this problem, we propose a wrapper based on proactive schemes, PrObeD, which enhances the performance of these object detectors by learning a signal. PrObeD consists of an encoder-decoder architecture, where the encoder network generates an image-dependent signals termed as templates to encrypt the input images, and the decoder recovers this template back from the encrypted images. We propose that learning the optimum template results in an object detector with an improved detection performance. The template acts as a mask to the input images to highlight semantics useful for the object detector. When the encrypted images are fine-tuned with the object detector, the detection performance for both generic and camouflaged object detectors enhances. Our experiments on MS-COCO, CAMO, COD$10$K, and NC$4$K datasets show improvement over different detectors on various datasets after applying PrObeD. Models and codes will be released upon publication. \n\n$\\textbf{Figure 1.}$\nThank you for pointing it out. We will incorporate your comment. \n\n$\\textbf{Eq. $1$ and Eq. $3$.}$ Thank you for this suggestion! We will update these equations.\n\n$\\textbf{Figure $2$.}$\nWe want to specify that both orange and black arrow connected components are used for training. However, only black arrow components are needed for inference. \n\n$\\textbf{Typos.}$\nWe will update the text. "}, {"review_id": "zPg5xgZP4b", "reviewer": "Reviewer_93HY", "summary": "Authors propose a new object detection wrapper, PrObeD, based on proactive scheme, which tries to enhance the performance of existing object detectors. PrObeD generate a custom template for each input images which makes the weights of object detector closer to the ideal weights. Besides, a decoder who recovers the template from the encrypted images is presented to further boost the performance.", "questions": "see Weaknesses.", "limitations": "N/A", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- PrObeD is the first work to develop a proactive approach upon object detection tasks, providing a new aspect to develop object detection models.\n- Authors provide mathematical analysis of passive and proactive detectors, and prove that the proactive method results in a better-converged model than the passive detector under specific assumptions.\n- Experiments show that PrObeD achieves SOTA performance on several GOD and COD datasets.", "weaknesses": "- The templates are applied on raw images with element-wise multiplication, which can be regarded as a soft mask. COD tasks may get benefit from it because it can mask useless background, but GOD tasks strongly dependent on image context, it's unclear why it's beneficial to GOD tasks. \n- In Sec 3.2,  when the authors try to prove the rationality of template, they assume the object detector is a linear regression model, which is contrary to the factual circumstances.\n- In line 191, authors said \"it is beneficial to recover the template from encrypted images\", but didn't provide explanations on it. Please provide the necessity or some intuitions behind this.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback. We provide a global response containing positives comments and major concerns of all reviewers. We also provide a global pdf file containing tables and figures with reference to added evaluation by some reviewers. We address all of your concerns below:\n\n\n$\\textbf{Benefit of adding Template for GOD task.}$\nThat’s a good question! All the GOD detectors make mistakes mainly due to five types of errors as reported in the supplementary which are, classification, localization, duplicate detection, background detection, and missed detection. The main reason for performance to be reduced are the errors in which foreground-background boundary is missed. These errors include localization, background detection and missed detection. These errors are significantly corrected by our proactive wrapper, as the template has object semantics, which when multiplied with the input image highlights the foreground objects, thereby making the task of object detector easier. We hope this clears up the motivation of adding templates for GOD task. \n\n$\\textbf{Proof assumptions}$\nA point to note is that mathematical proof and real world models do not share a one-to-one relationship. We must make assumptions to arrive at closed-form expressions to shed light on the problem we are trying to solve. While our analysis depends on these assumptions, it is significant that our results are evident even in real scenarios, even when the premises do not hold well. We leave the analysis after relaxing some or all of these assumptions for future avenues. Moreover, we believe our technical design and experimental results well justify our theoretical findings' effectiveness.\n\n\n$\\textbf{Template recovery.}$\nPrior proactive works [Asnani $\\textit{et al.}$ in CVPR 2022, Asnani $\\textit{et al.}$ in CVPR 2023] on image manipulation detection and localization have showed that the template recovery is useful for learning the optimum template. Our design of proactive wrapper is inspired by these works. We back this claim by performing an ablation on removing the decoder responsible for template recovery from our wrapper. The ablation results are shown in main paper Table $4$. The performance decreases if the decoder is removed, proving the effectiveness of template recovery."}, {"review_id": "k1i0UzhWhl", "reviewer": "Reviewer_rENj", "summary": "The paper presents a wrapper based on proactive object detection which can be utilized for both general object detection and camouflaged object detection problems. The proposed algorithm is flexible to different detection algorithms like FasterRCNN, Yolo and DeTR. Reasonable performance have been reported in benchmarks in general object detections like COCO and camouflaged object detection like CAMO, COD10K, and NC4K. ", "questions": "Please address the questions raised in the weakness section, especially on the experimental results.", "limitations": "The authors have already provided the limitation discussion of the proposed algorithm. ", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The proposed proactive object detection wrapper is flexible and can be applied to different object detectors. \n2. The proposed algorithm has been verified on general object detection as well as camouflaged object detection benchmarks. ", "weaknesses": "1. The motivation (as well as the advantages) of using the \"template\" is not clear.  As the generation of template relies on the ground-truth seg map, how about compairing the proposed algorithm with the algorithm which first perform segmentation to obtain the mask and then doing the object detection?\n\n2. The performance gain for the general object detectors is not obvious, for example YOLOV5 and DeTR, both of which are not state-of-the-art object detectors. Especially, the proposed wrapper will introduce additional computational cost, which is not negligible. \n\n3. The benchmark results on camouflaged object detection benchmarks (CAMO, COD10K, and NC4K) are slightly different from the results reported in other papers. For example, the baseline result reported in Table 3 is different from the results reported in DGNet[20]. Also, more comparison should be provided to validate the effectiveness of the proposed algorithm. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback. We provide a global response containing positives comments and major concerns of all reviewers. We also provide a global pdf file containing tables and figures with reference to added evaluation by some reviewers. We address all of your concerns below:\n\n\n$\\textbf{Motivation of using the template.}$\nIn our work, we propose to use a proactive scheme for improving the performance of object detection. As stated in the text, the major advantage of using the proactive schemes is for improving generalization. Prior works have proved this for different tasks such as image manipulation detection and localization. We aim to design a proactive scheme which is generalizable to different object detector, with an objective of increasing the performance. The learned template, which is the outcome of proactive learning, is multiplied with the input image. This can be seen as transforming the input data distribution to a perturbed data distribution which results in a better convergence and performance of the detector. \n\n$\\textbf{Comparison with segmentation works.}$\nThat’s a good point! It is beneficial to compare our method with the works that leverage segmentation map as a mask for object detection. We compare our performance with Mask R-CNN [He $\\textit{et al.}$ in ICCV $2017$] which uses an image segmentation branch to help the object detection. Table 1 in global pdf file shows that the gains using Mask R-CNN are lower than using our proactive wrapper. Having said that, we also want to point out our baseline for COD, i.e. DGNET. The authors of DGNET leverage a gradient map as a mask to perform COD, and our proactive wrapper improves the performance of DGNET. This gradient map is analogous to using the segmentation maps as an extra information which is leveraged to perform the task of COD. Our wrapper is able to boost the performance of DGNet, highlighting the value of templates as extra information, and proactive schemes in general.\n\n$\\textbf{Performance of YOLOv5 and DeTR.}$\nThank you for your comment. We agree that the performance gain is not significant enough for YOLOv5 and DeTR as compared to Faster-RCNN. We argue that the new $2$D object detectors have very little room for improvement, therefore making it harder for PrObeD to improve the performance by a significant margin. We show that the idea of using a proactive wrapper on top of various object detectors (GOD/COD) will result in the increase in the performance. As this is the first novel work to propose a proactive wrapper to improve object detection, we believe that PrObeD will motivate the future works to propose more sophisticated wrappers to enhance the margin of improvement in the performance of object detection. Our error analysis in Figure $1$ of supplementary also proves the effectiveness of applying the idea of proactive schemes to improve the performance of the detectors. \n\nWe will add this discussion in the manuscript. \n\n\n$\\textbf{COD baseline performance.}$\nThat’s a valid argument. We report the performance of the pretrained model released by the authors and use the same evaluation before and after applying PrObeD. Although the performance reported in the DGNET paper is higher than what is reported in our manuscript, we argue that it is very common to not have the exact replicated performance in object detection. Moreover, this issue has been raised by a lot of people in the publicly available repository, and the authors don’t comment on this issue. Further, we compare with the pretrained model released on the official repository and so, we believe our comparison is fair.\n"}, {"review_id": "5gF3RKex2f", "reviewer": "Reviewer_NMUy", "summary": "The authors propose a proactive object wrapper for the object detection method. The warper is similar to an input transformation that makes the object detection model focus more on the salient regions. The experiments show that a consistent improvement after the proposed method is applied on top of existing objection methods. ", "questions": "See the Weakness.", "limitations": "See the Weakness.", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The proposed method is somewhat novel in the sense that it is the first to use the proactive method on object detection tasks. Both GOD and COD situations are evaluated and the proposed method consistently improves the existing methods. ", "weaknesses": "There are several weaknesses of the method, and some of them are also mentioned by the authors\n\n- Performance gain: The performance gain becomes marginal when it comes to advanced object detection methods like YOLOv5 and DeTR. Considering two-stage methods, if Sparse R-CNN is evaluated instead of Faster R-CNN, would the improvement is also incremental?\n\n- Simple transformation: The proactive scheme is a simple multiplication or addition between the input and the template. Is this process equivalent to having an image-dependent transformation that makes object detection focus on salient regions and improve performance? If yes, then I think there may be more different sophisticated techniques to be explored. \n\nFor the training time overhead, in table 5, it is mentioned the training time with and without PrObeD is reported. However, only the inference time is mentioned. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback. We provide a global response containing positives comments and major concerns of all reviewers. We also provide a global pdf file containing tables and figures with reference to added evaluation by some reviewers. We address all of your concerns below:\n\n$\\textbf{Comparison with Sparse-RCNN}$\nThat’s an excellent suggestion! We experiment with two variations of Faster-RCNN, namely, Faster-RCNN $+$ FPN and Sparse-RCNN (Faster-RCNN $+$ FPN $+$ Segmentation mask). The results are shown in Table $1$ of the global pdf file. We observe an increase in the performance for both the detectors, further proving the generalizability of our proactive wrapper. \n\n$\\textbf{Image transformation.}$\nThat is correct. Our proactive wrapper’s encryption scheme is an image-dependent transformation that makes object detectors focus on salient regions. We agree that there might be more sophisticated techniques to formulate the encryption scheme. We show that the idea of using a proactive wrapper on top of various object detectors (GOD/COD) increases the performance. As this is the first novel work to propose a proactive wrapper to improve object detection, we believe that PrObeD will motivate the future works to propose more sophisticated methods, which includes sophisticated encryption schemes to enhance the margin of improvement in the performance of object detection. Our error analysis in Figure 1 of supplementary proves the effectiveness of applying the idea of proactive schemes to improve the detection performance. \n\n$\\textbf{Training/inference time overhead clarification. }$\nThank you for pointing it out. We accept this mistake on our end. We meant to report the inference time overhead due to our proactive wrapper. \n"}]

Author rebuttal:
$\textbf{Positives}$

We thank all the reviewers for their feedback regarding our work. All the reviewers have greatly appreciated our proposed work. The positives comments by the reviewers are mentioned below:

     

$1.$ Novelty of PrObeD (Reviewers rzBs, 93HY, NMUy).

$2.$ A new aspect to develop object detection models. (Reviewer 93HY)

$3.$ A strong mathematical analysis (Reviewer 93HY)

$4.$ Good performance on several GOD and COD detectors. (Reviewer V1FT, 93HY, rENj, NMUy)

$5.$ PrObeD is an effective approach to guide the object detector to perform improved detection. (Reviewers V1FT, rzBs)

$6.$ Extensive experimental evaluation to demonstrate the effectiveness of PrObeD. (Reviewers V1FT, 93HY, rENj)

$7.$ Important contribution of making the template image independent. (Reviewer rzBs)

$\textbf{Reviewer concerns}$

We carefully address each of the concerns of the reviewers concerns in their individual responses. However, we mention here some of main concerns of all the reviewers collectively providing a good summary of the major concerns of the reviewers. 

$\textbf{Performance of Faster-RCNN. (Reviewer V1FT, rzBs, NMUy)}$ 
That’s an interesting point. We chose the original Faster R-CNN for our experiments. We report the results from the original Faster R-CNN paper [Ren $\textit{et al.}$ in NIPS $2015$] in our Table $2$ of paper. We agree that fine-tuning the model with optimal parameters and architecture selection boosts the performance. Having said that, we experiment with two variations of Faster R-CNN, namely, Faster R-CNN + FPN and Sparse-RCNN. The results are shown in Table $1$ of global pdf file. We observe an increase in the performance for both the detectors. This shows that PrObeD generalizes well to all the types of object detectors for increasing the performance. Also, the point to be noted here is that before and after applying our proposed PrObeD, the object detector is same in both cases. We show in the paper Table $5$ that even fine-tuning the detectors without our wrapper does increase the performance across various object detectors including GOD and COD, but the gain is less compared to after applying PrObeD. Stating this, if any variation in the object detector hyperparameters or in architecture is helping the object detector, it will help even after applying PrObeD too.


$\textbf{Performance of YOLOv5 and DeTR. (Reviewer V1FT, rENj)}$ 
Thank you for your comment. We agree that the performance gain is not significant enough for YOLOv5 and DeTR as compared to Faster-RCNN. We argue that the new $2$D object detectors have very little room for improvement, therefore making it harder for PrObeD to improve the performance by a significant margin. We show that the idea of using a proactive wrapper on top of various object detectors (GOD/COD) will result in the increase in the performance. As this is the first novel work to propose a proactive wrapper to improve object detection, we believe that PrObeD will motivate the future works to propose more sophisticated wrappers to enhance the margin of improvement in the performance of object detection. Our error analysis in Figure $1$ of supplementary also proves the effectiveness of applying the idea of proactive schemes to improve the performance of the detectors. 

We will add this discussion in the manuscript. 



$\textbf{Motivation for template recovery. (Reviewer rzBS, 93HY)}$
Prior proactive works [Asnani $\textit{et al.}$ in CVPR 2022, Asnani $\textit{et al.}$ in CVPR 2023] on image manipulation detection and localization have showed that the template recovery is useful for learning the optimum template. Our design of proactive wrapper is inspired by these works. We back this claim by performing an ablation on removing the decoder responsible for template recovery from our wrapper. The ablation results are shown in main paper Table $4$. The performance decreases if the decoder is removed, proving the effectiveness of template recovery.


$\textbf{Benefit of adding Template for GOD task. (Reviewer 93HY)}$
That’s a good question! All the GOD detectors make mistakes mainly due to five types of errors as reported in the supplementary which are, classification, localization, duplicate detection, background detection, and missed detection. The main reason for performance to be reduced are the errors in which foreground-background boundary is missed. These errors include localization, background detection and missed detection. These errors are significantly corrected by our proactive wrapper, as the template has object semantics, which when multiplied with the input image highlights the foreground objects, thereby making the task of object detector easier. We hope this clears up the motivation of adding templates for GOD task. 



$\textbf{Comparison with segmentation works. (Reviewer rENj)}$
That’s a good point! It is beneficial to compare our method with the works that leverage segmentation map as a mask for object detection. We compare our performance with Mask R-CNN [He $\textit{et al.}$ in ICCV $2017$] which uses an image segmentation branch to help the object detection. Table 1 in global pdf file shows that the gains using Mask R-CNN are lower than using our proactive wrapper. Having said that, we also want to point out our baseline for COD, i.e. DGNET. The authors of DGNET leverage a gradient map as a mask to perform COD, and our proactive wrapper improves the performance of DGNET. This gradient map is analogous to using the segmentation maps as an extra information which is leveraged to perform the task of COD. Our wrapper is able to boost the performance of DGNet, highlighting the value of templates as extra information, and proactive schemes in general.

Meta-review: The paper introduces a "proactive" paradigm for object detection where the authors propose introducing a learnable template that guides the object detector towards semantically relevant parts of the image. They provide both theoretical and experimental analysis of their work. While the reviews were mixed, many of the comments were supportive of the work. Reviewers liked the novelty of the idea and the thorough analysis of the method. The negative reviewers were generally satisfied with the answers to their queries, but still questioned the performance improvement and generalizability of the approach. The AC looked at the paper and reviews and believes that the idea has enough novelty for the paper to be presented at NeurIPS. The evaluation is quite extensive and the AC did not feel that the authors should be penalized on that ground.
Decision: Accept (poster)