An Empirical Study of Adder Neural Networks for Object Detection
Authors: Xinghao Chen
Chang Xu
Minjing Dong
Chunjing Xu
Yunhe Wang
Venue: NeurIPS 2021 Poster (2021)
Keywords: Object Detection
Adder Neural Networks

Abstract: Adder neural networks (AdderNets) have shown impressive performance on image classification with only addition operations, which are more energy efficient than traditional convolutional neural networks built with multiplications. Compared with classification, there is a strong demand on reducing the energy consumption of modern object detectors via AdderNets for real-world applications such as autonomous driving and face detection. In this paper, we present an empirical study of AdderNets for object detection. We first reveal that the batch normalization statistics in the pre-trained adder backbone should not be frozen, since the relatively large feature variance of AdderNets. Moreover, we insert more shortcut connections in the neck part and design a new feature fusion architecture for avoiding the sparse features of adder layers. We present extensive ablation studies to explore several design choices of adder detectors. Comparisons with state-of-the-arts are conducted on COCO and PASCAL VOC benchmarks. Specifically, the proposed Adder FCOS achieves a 37.8% AP on the COCO val set, demonstrating comparable performance to that of the convolutional counterpart with an about $1.4\times$ energy reduction.

An Empirical Study of Adder Neural Networks for Object Detection
Xinghao Chen', Chang Xu², Minjing Donga.1, Chunjing Xu', Yunhe Wang Huawei Noah's Ark Lab
School of Computer Science, University of Sydney {xinghao. chen, yunhe wang) com, c.xu@sydney edu.
Abstract
Adder neural networks (AdderNets) have shown impressive performance image classification with only addition operations, which are more energy efficient than traditional convolutional neural networks built with multiplications. Compared with classification, there is a strong demand on reducing the energy consumption of modern object detectors via AdderNets for real-world applications such as autonomous driving and face detection. In this paper, we present an empirical study AdderNets for object detection. We first reveal that the batch normalization statistics in the pre-trained adder backbone should not be frozen, since the relatively large feature variance of AdderNets. Moreover, we insert more shortcut connections in the neck part and design a new feature fusion architecture for avoiding the sparse features adder layers. We present extensive ablation studies to explore several design choices of adder detectors. Comparisons with state-of-the-arts are conducted on COCO and PASCAL VOC benchmarks. Specifically, the proposed Adder FCOS achieves 37.8% AP on the COCO val set, demonstrating comparable performance to that the convolutional counterpart with an about 1.4x energy reduction.
Introduction
Object detection is foundational problem in computer vision and has attracted tremendous interests from both academic and industrial communities for decades [2]]. has wide range of applications for various areas, e.g., video surveillance, autonomous driving and robotic vision. Deep neural networks have indeed dominated the research of object detection recent years since the pioneering work of R-CNN 122 The performance of object detectors has been considerably improved by various designs of architecture, loss function etc. However, most modern accurate object detectors require massive computation, making them quite challenging in resource-constraint applications, e.g., mobile phones and embedded devices.
Various approaches have been proposed to compress and accelerate convolutional neural networks
(CNNs) for classification tasks, including channel pruning 24 26 38 low-bit quantization (50)
(25) and lightweight network design [33 (7). These methods reduce the number of parameters or inference latency while maintaining the accuracy the maximum extent. Such model compression
methods have also been explored in variety of down-stream tasks, such as semantic segmentation 7
and image super resolution 44 etc.
There have been a few methods aiming at fast and efficient object detectors. One family of solutions
using new architecture design 23
29 41.. For example, YOLO series [30 31 have achieved
good trade-off between running speed and accuracy via a novel one-stage detection framework. Another family of solutions is use common model compression methods for accelerating object
"ooresponding author.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

detectors, e.g., knowledge distillation 4 42 and pruning M. Moreover, some recent works utilize neural architecture search (NAS) approach for searching better architectures for different components of object detectors 3 13 11. Although these methods mentioned above show strong performance while improving the efficiency, they are mainly built with traditional convolutional neural networks, which contain massive inefficient multiplications.
Recently, Chen et al. [5] proposed the adder neural networks (AdderNets) to re- place traditional convolutional filters with adder filters. Since addition is more en- ergy efficient than multiplication [36] 477, AdderNets shed light on the design of effi- cient neural networks and have the poten- tial of much fewer chip areas and less en- ergy consumption. AdderNets have shown impressive performance in large scale im- age classification [45) via a kernel-basee progressive distillation method, and also successfully been applied for other applica- tions like image super resolution. Song et al. [34] proposed to utilize self shortcuts and learnable power activations to build super resolution networks via adder filters.
40
A50
RepPoints
Frookncher
Indiralte
MAP (96
35-
R15
FCOSRT
Addee FCOS
Addler TOO5-RT
-08
hoveatoox
<08-
RetinaNesMM5
Adder TovesBox
-
Adder Rettraket-M5
200
400
600
200
1000
1200
1400
Energy Im])
Figure 1: Comparisons of mAP on COCO va/2017 and
energy costs for different object detectors.
9 or super- resolution tasks [34] is not clear yet how will the AdderNets perform for object detection, which often has sophisticated framework design and various objectives. This much more challenging com- puter vision task therefore brings in new challenges and opportunities for the research on AdderNets. So how to build accurate and efficient object detectors via AdderNets? The straightforward idea is to directly replace the original convolutional filters by adder filters. However, this naive adder detector cannot be casily trained classification networks. At first. most modern detectors tend to pre-traue pre-train the backbone on the ImageNet and then fine-tune the whole model on the target dataset, but this straightforward fine-tuning might worsen adder detectors, because of the sensitivity of adder filter. neural architectures were all developed for convolution based
Existing variants of AdderNets mainly deal with either image classification S 45
Moreover, the performance-prove detectors, and whether they still applicable for the adder detectors is unclear.
In this paper, we propose series of strategies to reform efficient object detectors with adder filters. In contrast with the frozen batch normalization widely exploited during fine-tuning of the detector, we empirically observe an opposite conclusion that adder detectors are better to unlock the statistics of batch normalization in pre- are-trained adder backbone for a performance improvement. Extensive ablation studies are conducted to explore the properties of batch normalization layers and the impact of batch size. In addition, new feature fusion network with more residual connections and better fusion module is explored to compensate for sparse adder features. Experimental results are reported on PASCAL VOC and COCO benchmarks, and the results carefully analyzed and discussed. In particular, the proposed Adder FCOS achieves a 37.8% mAP on COCO val set, which is comparable with sfate-of-the-art object detectors, while saving much energy consumption In summary, we present an extensive empirical study for how to build object detectors via adder neural networks. We believe that the discussions and analysis in this paper will be beneficial for the research of efficient object detection and adder neural networks.
shown in Figure
2 Related Work
Object Detection. Object detection has attracted tremendous interests for decades. Although there have been huge improvements for accurate object detection (21), the struggle for high-speed and energy-efficient detectors is largely unsolved. Several recent approaches exploit neural architecture search (NAS) to discover efficient and accurate object detectors [3 13
On the meanwhile,
there are some attempts to utilize model compression methods for improving the efficiency of detectors, e.g..channel pruning I1. quantization [43] and knowledge distillation convolutional neural networks (CNNs) have in fact dominated the design of object detectors, which consist of massive multiplications and are quite energy inefficient.
42) Nevertheless,

---

(a) Adder BN running mean. (b) Adder BN running var.
(c) Adder weight.
(d) Cony weight.
Figure 2: The statistics of batch normalization and filter weights in backbone during training.
Adder Neural Networks. Chen es al. [5] provided new perspective for designing energy efficient neural networks. They proposed to utilize é distance to measure the similarity between the input features and filter weights. The devised AdderNets consist only additions instead of massive multiplication, which potentially energy efficient. Full precision gradients along with adaptive local learning rate are exploited to optimize this novel neural networks. AdderNets show impressive performance large-scale image classification benchmark. i.e., ImageNet. To further improve the performance of AdderNets, Xu el al. [45] proposed a kernel based knowledge distillation method to learn from homogeneous CNNs. Using this method, AdderNets achieve ResNet-50 on ImageNet. You et al. (47) further proposed to use bit-shift operations and additions improve the energy efficiency. Song et al. (34) analyzed several key problems of AdderNets and successfully applied AdderNets for image super-resolution. Nevertheless, designing accurate object detectors with AdderNets is still less explored and is the main focus of this paper.
better accuracy than
3 AdderNets for Object Detection
Traditional deep convolutional neural networks are mainly constructed by convolutional filters, which have massive manipulations and are energy inefficient. To this end, Chen al. (5) proposed new kind of neural architecture called AdderNet, which adopts é -nomm to compute the similarity between the input features X and the filter weights F, as shown in Eq. I
d
(m, n, 1) =
X(m + + i, +,kk Fi,,,k,
(1)
1=0
1=0 k=0
We aim to extend the success of AdderNet to the object detection task. We build our adder object detectors upon modern convolution- based detection frameworks, e.g., FCOS (39]. is straightforward to replace the convolutional filters in the detector with adder filters. However, it is non-trivia to train an adder detector of decent performance. We first analyze several key strategies for applying adder filters for detectors. We then propose novel multi-scale feature fusion architecture which is more suitable for adder detectors.
3.1 Making it Work: Towards Strong Baseline
Intuitively if we replace the convolutional filters with adder filters in modern object detectors like FCOS [39], we could obtain a vanilla adder detector, which may be denoted as Adder FCOS. However, it is not easy to train the vanilla Adder FCOS in the same way as convolutional FCOS. Here we elaborate on some special designs for adder detectors.
3.1.1 Revisiting Batch Normalization in Adder Detector
Most modern object detection methods follow the paradigm of pre-training the backbone network on large scale image classification dataset (e.g.. ImageNet) and then tuneitunig the whole detector on the target detection dataset. In object detection, the batch size is usually much smaller than that of image classification tasks, due to relatively higher resolution of the input images. For example, generally there are only 2 or 4 images on one GPU for training an modern object detector. Therefore, the statistics batch normalization (BN) are often frozen (denoted as FrozenBN) the fine-tuning stage, which brings in a considerable improvement than the unfrozen counterpart (simply denoted as BN here) [6]. However, we empirically observe that FrozenBN backbone leads to an unstable training of adder detectors. As shown in Fig. 3 with FrozenBN in backbone, the training loss converges

---

much slower than the unfrozen BN, resulting in a quite unsatisfied performance with mAP of zero. Therefore, updating the statistics of BN layers backbone network is critical for training a detector built with adder filters.
We attribute this phenomenon to the variance of fea- tures in AdderNets. As analyzed in [5], the addition operations in AdderNets tend to have much larger variances for the features before batch normaliza- tion. Dong et al. [T0] also analyze that the mean and variance of output feature are dominated by the variance of adder weights. Therefore, slightly tuning the adder filters would bring in drastic changing of the feature distribution, which makes the previous statistic of batch normalization incompatible with the input features. We visualize the statisties of batch normalization for random layer in backbone dur- ing training in Fig. We can see that the weights for adder network (Fig. 22) and convolutional net- work (Fig. 2d exhibit quite different properties. The weights for convolutional network only have slight changes from the first epoch to the last epoch. How- ever, the weights for adder detector become much smaller the training goes on. The changing of adder weights brings drastic variance for the output features, thus making the running means of BN for the last epoch quite different from that of the first epoch. If the statistics of BN are frozen during training, it would be quite challenging for training.
loss
3.00
FroceniN
2.75
UnfrozenBN
2.50
2.25
2.00
1.75
1.30
1.25
10000
20000
30000
40000
Iter
Figure 3: Training loss of fine-tuning adder detectors with frozen BN (final mAP: 0%) and unfrozen BN (final mAP: 32.4%).
To address this problem, it is necessary to unfreeze the statistic of batch normalization when fine- tuning the adder detectors from a pre-trained backbone. However, recomputing statistics for BN layers would be critical for the final performance, especially when the batch size is small, i.e., 2 images per GPU. Therefore, is necessary to use larger batch size for training adder detectors.
3.1.2 Better Pre-trained Backhone and Gradients
The performance of vanilla AdderNet [5] ImageNet is still poorer than CNNs. For better perfor- mance, we use the pre-trained model using the knowledge distillation proposed in [45]. The Top-1 accuracy of AdderNet-50 on ImageNet is 76.8%, which is comparable with the performance of its convolutional counterpart ResNet-50. More discussions for the pre- pre-trainte trained models will be included in the ablation studies.
The original AdderNets (5] require a special design of gradient descent rule for optimization. More specifically, instead of using the actual sign gradients, AdderNets exploit the full precision gradients of f2-norm to update the filters. To avoid the magnitude accumulation brought by the gradient chain rule, AdderNets clip the gradients of input features X to [-1, 1] using HardTanh function. Though this practice works well in image classification tasks, we argue that this gradient approximation may introduce challenges for optimization for object detection which often exhibits more complicated architectures. Since the gradients of X only play the role of accumulate gradients in chain rule, is better to the sign gradients for X. Therefore, exploit the gradients of & e-roorm for adder weights and sign gradients for the input features.
3.2 Better Feature Fusion for Adder Detector
Using all strategies discussed in the above sections, obtain baseline Adder FCOS with the mAP of 34.8% which is quite close to the convolutional counterpart. We then move one step forward to explore better architecture for adder detectors.
We first visualize features from the pretrained backbone, as shown in Fig. The feature maps from the last block of AdderNet-50 is much sparser than ResNet-50. More specifically, over 92% of the features for the last output from AdderNet-50 are zeros, while the percentage for ResNet-50 is only 63%. These sparse features may be enough for image classification task. However, it poses great challenges for object detection which needs dense predictions for class classification and bounding box regression.

---

(4) Features of the last block for ResNet-50
(b) Features of the last block for AdderNet-50
Figure 4: Features of the last block for ResNet-50 and AdderNet-50.. The features from AdderNet are much sparser than CNNs and pose great challenges for object detection which needs dense predictions for bounding box regression and category classification.
We attribute the problem of sparse features the calculation of adder filters. As shown in Eq. ( the output features of adder operation are always negatíve. Although the normalization procedure in BN makes these features have the mean of zero and the variance of one, the scaling and shift parameters in BN are learned to restore the representation power of the original features and tends to move features towards negative. The following ReLU activation would eliminate the negative features, making features in deeper layers to be more sparse.
Multi-scale feature fusion module is widely adopted for accurate object detection 19
As shown in Fig. 5 (a), feature Pyramid Network (FPN) [19] utilizes a top-down architecture to aggregate features from different levels and enhance the high-level semantic features for all scales. shown in Fig. (b). The feature fusion module somehow alleviates the feature sparsity problem of adder detectors. However, since the pre-trained model of AdderNet has sparse features in top layers, the top-down path may not bring meaningful information for feature fusion.
PAFPN [2] added extra bottom-up path for better feature aggregation
Therefore, we hereby propose a novel feature fusion module to alleviate this problem. Unlike most previous multi-scale feature fusion methods [19 22 37 that first adopt top-down path for feature aggregation, we propose to exploit a reverse pattern. Specifically, the proposed R-PAFPN first exploits bottom-up path propagate features from bottom layers to top layers, and then utilizes top-down path for fusing semantically strong features. This simple yet effective design is more suitable for adder detectors. Moreover, as discussed in [34, identity mapping is challenging for AdderNets. Therefore, we also add extra skip connections in our proposed feature fusion module. Specifically, we add residual connections for each 3x3 adder filters for better feature propagation, as shown Fig.
Experiments
In this section we first conduct extensive ablation experiments to analyze the effectiveness of different components of the proposed method. After that, we compare our proposed method with state-of-the- art object detectors.
4.1 Experimental Settings
We conduct experiments the bounding box detection track of MS COCO 2017 and PASCAL VOC benchmarks, which have 80 and 20 object classes, respectively. On all experiments, the AdderNet backbone keeps the first layer as convolution and has all the rest layers built with adder filters, following the practice in 5 45
COCO. Following the common practices, for COCO benchmark we use the COCO train2017 split that contains 118 images for training, va/2017 split for validation (5k) and test-dev split (20k) for testing. We report the average precision (AP) w.E.t. different IoU thresholds and different object scales, i.e. APso, AP7s. APs, AP and APz. All models are trained with stochastic gradient descent (SGD) over 8 GPUs. Unless otherwise specified, all models are trained for 12 epochs (also known as 1x schedule) with cosine learning rate decay strategy. Weight decay and momentum are set to 0.0001 and 0.9, respectively.
PASCAL VOC. For PASCAL VOC benchmarks, we train our models on the VOC 2007 and 2012 trainval sets, which contain about 16, 551 images, and evaluate the VOC 2007 rest set (4952
5

---

maps
Adder 1x1
Adder 3x3
D
O
P6
0
P4
O
(a)
(b) PAFPN
(c) R-PAFPN
Bottom-up
Top-down
Addel
3-MM 502
(d) Detailed architecture of the proposed R-PAFPN
Figure 5: Sketches for different multi-scale feature fusion architectures. (a) Feature Pyramid Networks (FPN). (b) Path Aggregation Network (PAFPN). (c) The proposed R-PAFPN architecture for adder detectors. (d) More detailed architecture of R-PAFPN.
Table 1:
Ablation studies for the baseline of adder detector. "NAN" indicates that the training is non-convergent. "LR" means using cosine learning rate and sypper-paramter tuning for learning rate.
UnfrozenBN 4 Gradients
KD pre-trained
LR
mAP
APso
APrs
APs
APM
AP1
NAN
32.4
50.3
34.2
18.2
35,8
41.5
32.6
50.5
34.5
18.3
35.8
42.1
33.2
51.2
35.1
18.3
36.2
42.7
34.8
52.6
37.1
19.7
38.4
images). We use the VOC style mAP (i..,mAP at IoU=0.5) as the evaluation metric. The input images resized to have shorter side being 600 while the longer side not exceed 1000. Other pprammrrreeters are similar to COCO benchmark. Our implementation based on the popular object detection framework MMDetection 6
4.2 Ablation Studies
Steps towards a strong baseline. We first analyze the strategies introduced Section 3.1 on FCOS with adder backbone and neck. As shown in Table [ if we train an adder FCOS detector with FrozenBN, the loss could not converge properly and the detector gets zero mAP. Updating the statistics of running mean and variance for BN (i.e., Unfrozen BN) is critical for training the adder detector and achieve reasonable mAP of 32.4%. We further exploit gradients for input features instead of clipped & gradients in [5). which harvest about 0.2 mAP improvement.

---

Detector
Conv
bs=2
bs=4
bs=8
VFNet 48
44.5
34.6
40.5
41.5
ATSS 49
39.6
29.3
34.5
36.9
RepPoints
46
37.1
27.2
33.0
34.7
RetinaNct
20
36,5
30.3
33.8
34.1
GFL 18
40.2
31.4
35.8
37.9
-D- Corty Frocendh
Como-IN Adder BN
FoveaBox 15
36.5
31.9
33.8
34.7
134
Sparse R-CNN 35
37.9
31.1
34.4
37.0
batch dize
Table 3: Performance on COCO val2017 for different
Figure 6: Comparisons of conv and adder detectors with various batch sizes.
adder detectors with different batch sizes (bs). The sec- ond column shows the results of convolutional baselines.
To further improve the performance of the adder detector, we instead use better pre-trained backbone, which is trained on ImageNet via kernel based knowledge distillation [5]. This strategy boosts the mAP to 33.2%. We also introduce cosine learning rate and hyper-parameter tuning for learning rate, which obtain 0.6% mAP improvement. Now we have got a strong baseline for adder detector, which achieves 34.8% mAP on COCO val2017 set.
The AdderNet-50 trained with knowledge distillation 45 has 76.8% top-1 accuracy on ImageNet, which is slightly higher than that of ResNet-50. To explore what extent the pre-trained backbones affect the final detection performance, we evaluate the backbones with and without knowledge distillation, shown Table Using a vanilla AdderNet-S0 with 1.9% top-1 accuracy drop on ImageNet suffers only 0.4% mAP drop compared with the KD counterpart. The AdderNet- 18 trained with KD and the vanilla AdderNet-100 101 perform worse than their convolutional counterparts on ImageNet but still obtain considerably good performance for object detection.
Impacts of batch size. As discussed in Sectionn.11.1 is necessary to recom- pute the statistics of batch normalization layers when training the adder detec- tors. We first explore the performance of convolutional detectors with differ- ent batch sizes. As shown in Figure 6 when the batch size is relatively large enough, i.e,, batch size of 8, the perfor- mance of using FrozenBN and normal BN are similar. Reducing the batch size to 4 or 2, the accuracies of using BN (i.e., recomputing statistics) decrease drastically while the counterpart with FrozenBN suffers slight performance degradation. Similar results are observed for adder detectors with normal BN, which demonstrates the intuitive fact that training detectors when recomputing statistics for BN with small batch size challenging. Moreover, adder detectors suffer severer accuracy drop when reducing the batch size, e.g., 5.4 mAP drop when decreasing batch size from 8 to 2 for adder detector while only 3.4 mAP drop for convolutional one. We perform more analysis on several modern detectors, as shown in Table 3 We replace the backbone of detectors with adder networks and explore the performance with different batch sizes. For most detectors, larger batch size consistently improves the performance, which demonstrates that large batch size is critical for training adder detectors.
Table 2: Ablation studies for different pre-trained back- bones. The values before the slash are Top-1 accuracy (%) on ImageNet and values after the slash are mAP on COCO for FCOS with corresponding backbones.
Top-1 Acc. (%) Detection mAP (%)
Backbone
R-18
R-50
R-101
Conv
69.8 35.2
76.2 / 39.0
77.37 41.0
Adder
67.0 33.3
74.9 / 37.4
76.08 38.9
Adder KD
68.8 33.9
76.8 37.8
Impacts of different neck structures. In Section 3.2 we propose a novel multi-scale feature fusion architecture (R-PAFPN) for improving the performance of adder detectors. Here we elaborate the effectiveness of the proposed design choices. Original FCOS [39] adopts feature pyramid network (FPN) for feature aggregation. Its adder counterpart achieves mAP of 34.8%, as shown in Table 4 We simply replace the neck architecture to PAFPN (22), which adds an extra bottom-up feature fusion path to FPN. This modification only brings minor improvement, i.e....1% mAP, which demonstrates that simply exploiting more fusion paths is not enough for adder detectors and special design is urgent.
We further add extra shortcut connections to PAFPN (denoted as PAFPN shortcut), motivated by the discussions in AdderSR (34]. Experiments show that adding skip connections is also beneficial
7

---

Table 4: Ablation studies for neck structures. The proposed R-PAFPN neck structure achieves the mAP of 36.5% % on COCO va/20/7 set.
Neck
Type
mAP
AP75
APs
AP M
AP
L
FPN [9
Adder
34.8
52.6
37.1
19.7
38.4
44.5
PAFPN 22
Adder
34.9 (+0.1)
52.4
37.2
20.1
38.3
44.5
PAFPN w/ shortcut
Adder
36. (+1.3)
53.8
38.9
20.4
39.8
46.5
R-PAFPN
Adder
37.0 (+2.2)
55.5
39.7
21.7
40.5
47.4
for detection task, which bring 1.3% mAP improvement. As discussed in Section 3.2 the pre-trained backbone for adder detector exhibits sparser feature maps for deeper layers. Therefore, we propose to utilized bottom-up feature aggregation path to fuse features of different scales from the backbone network. As shown in Table 4 the proposed R-PAFPN neck structure harvests additional 0.9% mAP gains, achieving the mAP of 37.0% on COCO val2017 set. It justifies our motivation that bottom-up path should be applied first since the low-level features could compensate for the sparse high-level features.
4.3 Experiments on COCO
We evaluate our proposed method on COCO benchmark for several state-of-the-art object detectors, including FCOS (3). FoveaBox (15), RetinaNet [20) and Sparse R-CNN [35]. We replace various components for these detection frameworks, e.g., Backbone (B), Neck (N) and Head (H), as shown in 5 For example, FCOS with adder backbone and neck is denoted as Adder FCOS (B+H). Similar to prior method (34,, we calculate the energy costs of different detectors. As discussed in prior literature 88 47), one operation of floating-poin addition and multiplication have energy costs of 0.9 p. and 3.7 pJ,, respectively.
For standard x schedule (i.e., 12 epochs), our adder detectors with AdderNet-50 backbone and convolutional neck, achieve comparable performance COCO val2017 with their convolutional baselines. For example, Adder FCOS (B) achieves mAP of 37.2%, which is only 1.2% mAP lower than FCOS [39]. Since the backbone is built with adder filters which get rid of massive multiplications, Adder FCOS (B) has considerably fewer number of multiplications than FCOS (129.9 vs. 214.7). Sparse R-CNN (35 with adder backbone only suffers 0.9% mAP drop while reducing the potential energy costs from 719.5 to 482. We also build detector with adder backbone and neck, i.e., Adder FCOS (B+N), which achieves 37.0 .0% mAP and have further fewer multiplications. Compared with RetinaNet (20), our adder FCOS achieves the same detection accuracy but have much higher potential for energy efficiency.
We also conduct experiments on longer schedule (i.e., 2x). Adder FCOS (B+N) achieves 37.8% mAP and outperforms RetinaNet (20) by 0.4% mAP. obtains quite competitive performance with FoveaBox (15), Faster R-CNN 32) and RepPoints [46) but have much fewer multiplications. For ResNet-101 backbone, the adder counterpart suffers from 2.4% mAP drop while having 1.6x energy reduction.
Table 6: Comparisons of mAP on PASCAL VOC.
We further evaluated method on two detectors with smaller input sizes so that they can be trained with larger batch size. Specifically, FCOS-RT [40] is a real-time version of FCOS, with the in- put image of 736 x 512, multi-scale training and longer training iterations is
Model
Backbone
Neck
mAP
Faster R-CNN 33
Conv R50
Conv
79.5
FCOS [39
Conv R50
Conv
79.1
RetinaNet 20
Conv R50
Conv
77.3
FoveaBox 15
Conv R50
Conv
76.6
Adder FCOS (Ours)
Adder R50
Adder
76.5
(i.e., 48 epochs). RetinaNet-MS trained with similar settings as (.Л. with 640 x 640 input images and also multi-scale training strat-
egy. Adder FCOS-RI RT (B+N) achieves 1.3% mAP less that FCOS-RT while reducing the energy cost from 344.4 244.4. We try to replace all layers except the first and last layers in RetinaNet-MS. -60 and obtain 35.2% mAP, which is 2.7% lower that the convolutional baseline but having 2 energy cost reduction. Similarly, Adder RetinaNet-MS--35 baseline with 2x fewer energy cost. More comparisons with state-of-the-an detectors are shown in Figure The proposed adder detectors present better accuracy and energy cost trade-off for various detection frameworks.
achieves 3.0% mAP lower that the convolutional
8

---

Table 5: Comparisons of object detection results COCO val2017. We estimate the energy costs according to prior literature 47), ine operation of floating-pon addition and multiplication have energy costs of 0.9 p. and 3.7 pJ, respectively. B: Backbone, N: Neck, H: Head.
Adder?
#Mul (G)
#Add
Energy (mJ)
AP val (%)
Detectors
Epoch
H
(G)
GHM
17
R550
12
250.3
250.3
1152
37.0
PAFPN 22
R50
12
241.7
241.7
1112
37.5
RetinaNet 20
R50
12
239.3
239.3
1100
36.5
Libra R-CNN 28
R50
12
216.9
216.9
997.9
38.3
Faster R-CNN 13
R50
12
215.8
215.8
992.8
37.4
PISA 2
R50
12
215.8
215.8
992.8
38.4
FSAF [51]
R50
12
215.8
215.8
992.8
37.4
RepPoints |46
R50
199.0 12
199.0
915.4
38.1
FoveaBox 15
R50
12
215.8
215.8
992.7
36.5
Adder FoveaBox
R50
12
131
300.6
755.2 (0.76x)
34.7 (-1.8)
Adder FoveaBox
R550
/
/
12
112.9
318.7
704.4 (0.71x)
33.3 (.3.2)
Sparse R-CNN 35
R50
12
156.4
156.4
719.5
37.9
Adder Sparse R-CNN
R50
12
71.59
241.2
482.0 (0.67 x)
37.0 (-0.9)
FCOS 339
R50
12
214.7
214.7
987.7
38.4
Adder FCOS
R50
12 129.9
299.5
750.2 (0.76%)
37.2 (-1.2)
Adder FCOS
R50
/
/
12
112.9
316.5
702.7 (0.71x)
37.0 (-1.4)
RetinaNet 20
R50
24
239.3
239.3
1100
37.4
Faster R-CNN
32
R50
24
215.8
215.8
992.8
38.4
RepPoints 46
R50
24
199.0
199.0
91.5.4
38.6
FoveaBox 15
R50
24
215.8
21.5.8
992.7
37.9
Adder FoveaBox
R50
24
131
300.6
755.2 (0.76x)
35.8 (-2.1)
FCOS
R50
214.7
214.7
987.7
39.0
Adder FCOS
R50
24
129.9
299.5
750,2 (0.76x)
38.2 (+0.8)
Adder FCOS
R50
/
24
112.9
316.5
702.7 (0.71x)
37.8 (-1.2)
FCOS
R18
24
162.7
162.7
748.5
35.2
Adder FCOS
R18
24
141.9
183.5
690.3 (0.92x)
33.9 (-1.3)
Adder FCOS
R18
/
/
24
112.4
213.0
607.6 (0.81x)
33.6 (-1.6)
FCOS
R101
24
294.6
294.6
1355
41
Adder FCOS
R101
24
145.7
443.5
938.1 (0.69
38.9 (-2.1)
Adder FCOS
R101
24
113.3
475.9
847.4 (0.63x)
(.2.4)
FCOS-RT 40
R50
48
74.86
74.86
344.4
40
Adder FCOS-RT
R50
48
39.16
110.6
244.4 (0.71x)
38.7 (-1.3)
RetinaNet-MS-500
R50
50
61.24
61.24
281.7
35.3
Adder RetinaNet-MS-500
R50
/
/
50
36.08
86.40
211.3 (0.75x)
33.2 (-2.1)
Adder RetinaNet-MSS5000
R50
/
10.34
112.1
139.2 (0.49x)
32.3 (-3.0)
RetinaNct-MS-640
R50
50
95.68
95.68
440.1 1
37.9
Adder RetinaNct-MSS-66
R50
/
50
55.13
136.2
326.6 (0.74x)
36.6 (-1.3)
Adder RetinaNet-MSS-400
R50
/
50
16.15
175.2
217.4 (0.49
35.2 (-2.7)
Figure 7 shows some qualitative results of our proposed adder detectors and state-of-the-arrs detectors, including RetinaNet (20) and FCOS [39]. Adder FCOS works well for variety of challenging scenarios and have similar predictions with other detectors.
4.4 Experiments PASCAL VOC
We also evaluate our proposed method on PASCAL VOC dataset. Our Adder FCOS with adder backbone and neck structure achieves mAP of 76.5, which is comparable with FoveaBox [5] and RetinaNet 20). The performance is a bit poorer than Faster R-CNN and FCOS. However, considering the energy cost reduction, the proposed adder detector is good trade-off for object detection accuracy and energy efficiency.

---

perport
(a) RetinaNet
(b) FCOS
(c) Adder FCOS (Ours)
Figure 7: Qualitative results of RetinaNet 20
FCOS 39
and the Adder FCOS.
5 Conclusion and Discussion
In this paper, we present an empirical study for accurate object detectors via adder neural networks. We first reveal that unfreezing statistics of batch normalization in backbone is crucial for adder detectors. We empirically analyze the properties of batch normalization and the impact of batch size. We also move extra step forward to exploit better architecture for adder detector. More specifically, we propose a novel reverse multi-scale feature fusion module called R-PAFPN, which compensates for the sparse high-level features with feature aggregation. Extensive experiments conducted on COCO and PASCAL VOC benchmarks. In details, Adder FCOS achieved 37.8% AP on COCO val set, demonstrating comparable performance with convolutional counterpart but having much more potential energy reduction.
There still some unsolved limitations for adder detector. For example, it still suffers from a bit accuracy degradation when compared with its convolutional counterpart, iee,, FCOS (39. What's s more, the prediction heads some adder detectors still stacked with convolutions, and we empirically find that it would bring considerably large performance reduction if we replace convolutional filters cost saving brought by AdderNets, it is still promising solution for efficient object detectors. In further work, it would be interesting to design new prediction head architecture which is suitable for accurate adder detectors. We hope this study will be helpful for the research of adder neural networks and energy-efficiert object detection.
heads with adder filters for FCOS. Nevertheless, considering the energy
Acknowledgments and Disclosure of Funding
The authors sincerely thank anonymous reviewers and ACS for their helpful comments. Chang Xu was supported in part by the Australian Research Council under Projects DE18010143 and DP210101859.
10

---

References
[1] Yuxuan Cai, Hongjia Li, Geng Yuan, Wei Niu, Yanyu Li, Xulong Tang, Bin Ren, and Yanzhi Wang. Yolobile: Real-time object detection on mobile devices via compression-compilatio co-design. arXiv preprint STt::2009.05699 2020.
[2] Yuhang Cao, Kai Chen, Chen Change Loy, and Dahua Lin. Prime sample attention in object detection. In CVPR, 2020.
[3] Bo Chen, Golnaz Ghiasi, Hanxiao Liu, Tsung- Yi Lin, Dmitry Kalenichenko, Hartwig Adam, and Quoe Le. MnasFPN: Learning aadency-wware pyramid architecture for object detection on mobile devices. In CVPR, pages 13607-13616, 2020.
[4] Guobin Chen, Wongun Choi, Xiang Yu, Tony Han, and Manmohan Chandraker. Learning efficient object detection models with knowledge distillation. In NeurIPS, pages 742-751, 2017.
[5] Hanting Chen, Yunhe Wang, Chunjing Xu, Boxin Shi, Chao Xu, Qi Tian, and Chang Xu. AdderNet: Do we really need multiplications in deep learning? In CVPR, 2020.
[6] Kai Chen, Jiaqi Wang, Jiangmiao Pang. Yuhang Cao, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jiarui Xu, Zheng Zhang, Dazhi Cheng, Chenchen Zhu, Tianheng Cheng, Qijie Zhao, Buyu Li, Xin Lu, Rui Zhu, Yue Wu, Jifeng Dai, Jingdong Wang, Jianping Shi, Wanli Ouyang, Chen Change Loy, and Dahua Lin. MMDetection: Open mmlab detection toolbox and benchmark. arXiv preprint (906vv/990607700
2019.
[7] Xinghao Chen, Yunhe Wang, Yiman Zhang, Peng Du, Chunjing Xu, and Chang Xu. Multi-task pruning for semantic segmentation networks. arXiv preprint arXiv:2007.0838 2020. [8] William Dally. High-performance hardware for machine learning. NIPS Tutorial, 2, 2015.
[9] Minjing Dong, Yunhe Wang, Xinghao Chen, and Chang Xu. Handling long- tailed feature distribution in addernets. In NeurlPS, 2021.
[10] Minjing Dong. Yunhe Wang. Xinghao Chen, and Chang Xu. Towards stable and robust addernets. In NewrlPS, 2021.
[1]] Golnaz Ghiasi, Tsung-Yi Yi Lin, and Quoc V Le. NAS-FPN: Learning scalable feature pyramid architecture for object detection. In CVPR, pages 7036-7045, 2019.
[12] Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR, pages 580-587, 2014. [13] Jianyuan Guo, Kai Han, Yunhe Wang, Chao Zhang. Zhaohui Yang, Han Wu, Xinghao Chen, and Chang Xu. Hit-Detector: Hierarchical trinity architecture search for object detection. In CVPR, pages 1 14144 2020.
[14) Yihui He, Xiangyu Zhang, and Jian Sun. Channel pruning for accelerating very deep neural networks. ICCV, pages 1389-1397, 2017.
[15] Tao Kong, Fuchun Sun, Huaping Liu, Yuning Jiang, Lei Li, and Jianbo Shi. FoveaBox: Beyound anchor-based object detection. TIP, 29:73899 77398,999888,,,, 2020.
[16] Tao Kong, Fuchun Sun, Anbang Yao, Huaping Liu, Ming Lu, and Yurong Chen. Ron: Reverse connection with objectness prior networks for object detection. In CVPR, pages 5936-5944, 2017.
[17] Buyu Li, Yu Liu, and Xiaogang Wang. Gradient harmonized single-stage detector. In AAAI, 2019.
(18) Xiang Li, Wenhai Wang, Lijun Wu, Shuo Chen, Xiaolin Hu, Jun Li, Jinhui Tang, and Jian Yang. Generalized focal loss: Learning qualified and distributed bounding boxes for dense object detection. In NeurIPS, 2020.
[19] Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection. In CVPR, pages 21 17-2125, 2017.
[20] Tsung-Yi Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In ICCV, pages 2980-2988 2017.
[21] Li Liu, Wanli Ouyang, Xiaogang Wang, Paul Fieguth, Jie Chen, Xinwang Liu, and Matti Pietikäinen. Deep learning for generic object detection: A survey. IJCV, 128(2):261-31,, 2020.
[22] Shu Liu, Lu Qi. Haifang Qin, Jianping Shi, and Jiaya Jia. Path aggregation network for instance segmentation. In CVPR, pages 8759-8768, 2018.
[23] Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng- Yang Fu, and Alexander C Berg. SSD: Single shot multibox detector. In ECCV, pages 21-37, 2016.
[24] Zhuang Liu, Jianguo Li, Zhiqiang Shen, Gao Huang, Shoumeng Yan, and Changshui Zhang. Learning efficient convolutional networks through network slimming. In ICCV, 2017.
11

---

[25]
Zechun Liu, Baoyuan Wu, Wenhan Luo, Xin Yang, Wei Liu, and Kwang- Ting Cheng. Bi-Real
Net: Enhancing the performance of 1-bit cnns with improved representational capability and advanced training algorithm. In ECCV, pages 722-737, 2018.
[26] Jian-Hao Luo, Jianxin Wu, and Weiyao Lin. ThiNet: A filter level pruning method for deep
neural network compression. In ICCV, pages (058-5066, 2017.
[27] Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun. ShuffleNet V2: Practical guidelines for efficient enn architecture design. In ECCV, 2018.
[28] Jiangmiao Pang, Kai Chen, Jianping Shi, Huajun Feng, Wanli Ouyang. and Dahua Lin. Libra
R- R-CNN: Towards balanced learning for object detection. In CVPR, 2019.
[29] Zheng Qin, Zeming Li, Zhaoning Zhang, Yiping Bao, Gang Yu. Yuxing Peng, and Jian Sun. ThunderNet: Towards real-time generic object detection on mobile devices. In ICCV, 2019. [30] Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object detection. In CVPR, pages 779-788, 2016.
[31] Joseph Redmon and Ali Farhadi. Yolov3: An incremental improvement. arXiv preprini
arXiv:1804.02761 2018.
[32] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster R-CNN: Towards real-time object detection with region proposal networks. In NIPS, pages 91-99, 2015.
[33] Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chies Chen. MobileNetV2: Inverted residuals and linear bottlenecks. In CVPR, pages 4510-4521 2018. Dehua Song, Yunhe Wang. Hanting Chen, Chang Xu, Chunjing Xu, and DaCheng Tao. AdderSR: Towards energy efficient image super-resolution. In CVPR, 2021.
[34]
[35] Peize Sun, Rufeng Zhang, Yi Jiang. Tao Kong, Chenfeng Xu, Wei Zhan, Masayoshi Tomizuka,
Lei Li, Zehuan Yuan, Changhu Wang, and Ping Luo. SparseR- RansseR-NN:::: End-to-end object detection with learnable proposals. arXiv preprint Xix::220 1. 12450 2020.
[36] Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang, and Joel S Emer. Efficient processing of deep neural networks: A tutorial and survey. Proceedings of the IEEE, 105(12):2295-23) 22229, 2017. [37] Mingxing Tan, Ruoming Pang. and Quoc Le. EfficientDet: Scalable and efficient object detection. In CVPR. pages 10781-10790, 2020.
[38] Yehui Tang. Yunhe Wang. Yixing Xu, Yiping Deng, Chao Xu, Dacheng Tao, and Chang Xu. Manifold regularized dynamic network pruning. In CVPR, pages 5018-5028, 2021.
[39] Zhi Tian, Chunhua Shen, Hao Chen, and Tong He. FCOS: Fully convolutional one-stage object detection. ICCV, pages 9627-9636, 2019.
[40] Zhi Tian, Chunhua Shen, Hao Chen, and Tong He. FCOS: A simple and strong anchor-free object detector. TPAMI, 2020.
[41] Robert Wang, Xiang Li, and Charles X Ling. Pelee: A real-time object detection system on mobile devices. In NewrlPS, pages 1963-1972, 2018.
[42] Tao Wang, Li Yuan, Xiaopeng Zhang, and Jiashi Feng. Distilling object detectors with fine-
grained feature imitation. In CVPR, pages 4933 4942, 2019.
[43] Yi Wei, Xinyu Pan, Hongwei Qin, Wanli Ouyang, and Junjie Yan. Quantization mimic: Towards very tiny enn for object detection. In ECCV, pages 267-283, 2018
[44] Jingwei Xin, Nannan Wang, Xinrui Jiang, Jie Li, Heng Huang, and Xinbo Gao. Binarized neural network for single image super resolution. In ECCV, pages 91-107. Springer, 2020. [45] Yixing Xu, Chang Xu, Xinghao Chen, Wei Zhang, Chunjing Xu, and Yunhe Wang. Kernel based progressive distillation for adder neural networks. In NewrIPS, 2020.
[46] Ze Yang, Shaohui Liu, Han Hu, Liwei Wang, and Stephen Lin. RepPoints: Point set representa- tion for object detection. In ICCV, Oct 2019.
[47] Haoran You, Xiaohan Chen, Yongan Zhang, Chaojian Li, Sicheng Li, Zihao Liu, Zhangyang Wang, and Yingyan Lin. ShiftAddNet: A hardware-inspired deep network. In NeurlPS, 2020. [48] Haoyang Zhang. Ying Wang, Feras Dayoub, and Niko Sünderhauf. Varifocalnet: An iou-aware dense object detector. In CVPR, 2021.
[49] Shifeng Zhang, Cheng Chi, Yongqiang Yao, Zhen Lei, and Stan Z Li. Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection. In CVPR, pages 9759-9768, 2020.
[50] Shuchang Zhou, Yuxin Wu, Zekun Ni, Xinyu Zhou, He Wen, and Yuheng Zou. Dorefa- Training low bitwidth convolutional neural networks with low bitwidth gradients. arXiv preprint arXiv: 1606666666666 2016
[51] Chenchen Zhu, Yihui He, and Marios Savvides. Feature selective anchor- free module for single-shot object detection. In CVPR, pages 840-849, 2019.
12


Reviews:
[{"review_id": "mftd_30HHcY", "reviewer": "Reviewer_h7uh", "summary": "The authors look at using AdderNets for object detection. AdderNets are neural networks which replace dot product between a weight filter and an activation window with the L1 distance between that weight and activation window instead. In hardware, AdderNets would replace multiplications with additions which are simpler and use less energy.\n\nThe authors use empirical experiments to discover a number of strategies that improve AdderNet AP, including:\n1. Unfreezing the batchnorm statistics during downstream fine-tuning\n2. Using direct gradients for the L1 distance (which are +`1/-1) instead of a continuous approximate gradient\n3. Finetuning from a knowledge-distilled AdderNet backbone (as opposed to a regularly trained AdderNet)\n4. Architectural changes specific to object detection, adding more skip connections\n\nWith these ideas, AdderNet versions of object detectors can approach 1-3% of their convolutional counterparts in AP while reducing multiplies by about 50% and energy consumption by `about 25%.", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n - Clearly written paper with practical ideas that seem easy to apply to real object detection systems\n - Large amount of empirical data validating claims, Table 5 especially shows that their technique achieves similar results across a large variety of baseline CNN object detectors. I think the paper is very trustworthy.\n\nWeaknesses:\n - Impact of each individual tweak seems small. Other than unfreezing the BN, the other ideas only seem to boost AP by a small amount (Table 1). This amount seems comparable to what a practitioner would get by doing hyperparameter tuning, cleaning data, or some other generic tuning.\n - Energy savings seem underwhelming. Table 5 seems to show that we can save 25% energy at a cost of 1-3% loss of AP. That doesn't seem much better than using a 25% shallower convnet. I'm a bit surprised - I would have thought converting all multiplies in the convs to adds would save a lot more.\n\nOne way to improve the paper: right now the energy calculation assumes floating-point adds/muls. What about comparing fixed-point (e.g. 16-bit) operations instead? Are there other benefits to AdderNets (throughput, latency, faster training) that could be leveraged to make the results more impressive?\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Paper topic is efficient ML, will help reduce energy footprint of DNN inference for object detection.", "review_text": "", "extra_scores": {"ethical_concerns": "None", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1"}, "extra_text": {}}, {"review_id": "mYmhE0sbDET", "reviewer": "Reviewer_NNwT", "summary": "This paper empirically explores the impact of adder-based network architecture for developing object detectors. It studies the impact of batch normalization and batch size on the performance of adder networks and proposes unfreezing batch normalization for adder networks. It also explored the effect of different neck architectures by investigating the limitations of adder networks and how to improve them. ", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper proposes a series of strategies to reform object detectors with adder filters. Overall, it sheds more light into how to make adder networks work better for object detection. There are some points that should be incorporated into the paper.\n\n- Section 2.1.2 needs to be elaborated more. \n\n- The way Figure 6 and Table 3 are placed makes it confusing to read.\n\n- In figure 3 the accuracy (mAP) should also be included. The lower loss may not always lead to hiegher mAP values.\n\n- The energy savings ignore memory read and write operations which depend on feature map size and parameters and constitute a large part of the energy consumption hence, the fraction of improvement might not be that significant if all these factors are considered. Can you please elaborate more on this and add a relevant explanation in the paper.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "A major limitation not properly addressed is the fact that in some cases the adder networks cannot be trained. Additional hyperparameter tuning would be necessary in this case beyond the standard procedures.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "lb7e72cuvC_", "reviewer": "Reviewer_uAeA", "summary": "The purpose of the paper is to utilize addernet to replace the convolutional layer in the object detection network, so as to optimize the power consumption of the model. To address the problem of directly replacing the convolutional layer with the addernet, the author has made two contributions: (1) When training the addernet, the BN layers in the pre-trained weights are not frozen (2) Considering the sparse nature of the adder network, the paper designed a new FPN to alleviates the problem.", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "- The paper is well-written and the authors introduced the contribution and related work in detail, and verified the generalization on many detectors.\n- The innovation of this article is relatively lacking. Among the two contributions of this article, bn-no-freeze during training is more like a training trick. The second contribution of this article is to redesign the FPN for the sparsity of addernet. Although the paper compares the feature sparsity of normal convolution and addernet, it does not show the effect of R-PAFPN on normal convolution experimentally. And the largest gain part is on the shortcut, which obviously increases the amount of calculation of the network, which makes the design of R-PAFPN more like an incremental improvement to FPN. In general, although the contribution of this article presents a certain technical contribution, the novelty of the paper to the NIPS community may be relatively small.\n-  There are some unfair setting between the Conv and Adder detectors. For example, distillation is used in Pre-training stage of adder network, but the pre-trained weight of the Conv network does not have distillation.\n- There is no comparison with the performance of Conv network when using R-PAFPN.\n- For the power-efficient design, there is no comparison over model pruning to show whether addernet has a better power-accuracy trade-off. Besides, there are certain doubts whether addernet and prune can coexist well as there is no discussion and experiments.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "No obvious limitations and potential negative societal impact.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "0nU1webyFLp", "reviewer": "Reviewer_ZgMW", "summary": "This paper presents an empirical study for exploiting the AdderNet for low-cost object detection including the considerations about batch normalization, feature pyramid networks. Experiments have demonstrated that the proposed Adder FCOS can achieve competitive performance with less energy cost.", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": " Strength:\n1. This paper is well-organized and easy to understand.\n2. The proposed Adder FCOS and other variants are effective for low-cost object detection and experimental results are good.\n\n Weakness:\n1. The technical novelty of this paper is limited. The authors exploit the AdderNet to object detection approaches with some modifications (e.g., unfreezing Batch Normalization, better pretrained backbone, etc.) to stabilize the training and improve the performance.\n2. In Sec. 2.2, the authors claim the sparse features caused by adder filters and propose a reversed feature pyramid network (R-PAFPN) to alleviate the problem while there no evidence to prove it. In Table 4, it seems that the shortcut connections matter more than the bottom-up features. Besides, providing the results of R-PAFPN w/o shortcut will make it more clear for us to determine whether using bottom-up features can alleviate the sparse problem. In addition, visualizations of features after the R-PAFPN are required for comparisons.\n3. I’m concerned about whether Synchronized Batch Normalization is adopted in the implementation since small batch in one GPU (<8) will degrade the performance. \n4. With longer training schedule, the performance gaps between Adder FCOS and FCOS become larger (as shown in Table 5). Is this a general limitation of the proposed approach?\n5. How about the inference speed? Current computing bottlenecks of low-cost processors lie in memory access [1]. The Adder FCOS reduces the energy cost but still involves higher memory access cost, which might limit the performance in practical use.\n\n[1] Ma et.al. ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design.  ECCV 2018.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes. The authors have discussed partial of the limitations of the proposed method.\n\nTypos: \n* L259: Adder FCOS (B+H) -> Adder FCOS (B+N)\n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8"}, "extra_text": {}}]

Author rebuttal:

Meta-review: On the whole the reviewers agreed that the paper was well presented and thoroughly evaluated. There were some points raised on how much of an improvement this provides and some clarity questions, but these concerns were addressed by the authors in the rebuttal, and I think the paper results will be of interest to the NeurIPS community.
Decision: Accept (Poster)