Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: This paper is an empirical study of using adder networks for the task of object detection. These networks (mostly) use additions over multiplications, and are a good choice for improving computational performance for this task, specially for edge and embedded devices. I belive that the authors in this paper only consider adder networks only for the backbone and neck architectures of an object detector, leaving out the prediction heads which still use full convolution.

The contributions are: some tricks and strategies to train AdderNet's for object detection, a comprehensive evaluation on COCO and PASCAL VOC, a residual version of FPN which has slightly better performance, knowledge distillation for an improved backbone network for the object detector, and an analysis of freezing batch normalization and the effect of the batch size on the adder object detection networks.

main_review: Positives

- The paper is well motivated, as object detection computational performance is an important target for improvement, and is very important for edge devices. AdderNets are a good candidate for performance improvement as they are a meta-architecture and reducing the number of multiplications makes a lot of sense for good computational performance.
- The paper is very well written, with good use of space and words, including a lot of results and ablation studies that support the paper's conclusions and overall scientific decision making.
- The evaluation has a good selection of methods from the state of the art and detectors that are transformed into its AdderNet version. I have no complains about the overall evaluation, it looks correct. All decisions are being correctly motivated and justified with a scientific experiment.
- While there is no improvement to the state of the art in terms of mAP, there is improvement in terms of number of additions and multiplications, and overall amount of energy required for a forward pass, and this means a small sacrifice in mAP, both in COCO and PASCAL VOC (around 3 mAP points).

Negatives

- There seems not to be any comparison to other techniques that also target a reduction in computational cost, some of these techniques are mentioned in the introduction, like pruning, weight quantization, and lightweight network architectures. This comparison would make the paper much more strong, in order to connect with the original motivation of computational performance in object detection.
- The whole evaluation metric for computational performance is the amount of energy required for a forward pass, I believe that the computation time on a standard platform would also be useful. The number of FLOPs can also be inferred from number of additions and multiplications, and this metric can also be plotted directly with raw mAP. I have more suggestions below in the minor comments on how to provide additional views of your results.
- Overall performance of the best model (mAP 37.8% in COCO) is quite far from the greater state of the art, where in the COCO minival the best performance  I could find according to [PapersWithCode](https://paperswithcode.com/sota/object-detection-on-coco-minival?tag_filter=12%2C13) is around 57.0% mAP. This is only a minor issue since obtaining this kind of performance requires a lot of hacks that are not being applied here.
- In the conclusion it is mentioned that full adder networks (including prediction heads) would also provide additional computational performance improvement, but why were these results left out of the paper?

Minor Comments

- In Figure 1, I think it would be better to use different marks (diamond, triangle, sqiuare) for the original object detection method and its Adder variant, while using different colors for different object detection methods. This would make Figure 1 easier to read and understand.
- Table 4, please mention in the caption on which dataset these results were obtained.
- It would be very interesting to plot the results of Table 5, I am aware that some of these results are included in Figure 1, but for example, the trade-off between performance and computation could be plotted as mAP vs Energy, or number of additions vs number of multiplications, with different colors for a scatterplot that represents mAP performance, etc. This could show the trade-off between # of additions, # of multiplications, and performance in terms of mAP, for a large selection of detectors. This could be added to the supplementary material.
- One thing I notice in Figure 7 is that overall (not always), seems that FCOS and Adder FCOS are less confident (by looking at the bounding box probabilities) than RetinaNet, and Adder FCOS seems to be a bit less confident than FCOS. This is something to consider in the future as detection confidences are also quite important.
- The paper could benefit from supplementary material, for example, expanding the results in Figure 7 to all detectors and a larger selection of images both for PASCAL VOC and COCO.

Overall I believe this is a worthy contribution to the state of the art and should be accepted.
limitations_and_societal_impact: The paper has very little limitations, I have mentioned before, comparison with other techniques meant to improve object detector computational performance, and some visualizations of the results. Overall I think the limitations are not a big concern.

The paper has no broad impact statement, and I believe that the results from this paper should not have a large impact on society, at least in the short term.
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: This work aims to design an object detection framework based on AdderNets, studies the problems (i.e., the frozen batch normalization and sparse features caused by adder layers) when applying AdderNets on the object detection tasks, and provides solutions for these problems. The proposed AdderNet-based detector can archive comparable performance to CNN-based counterparts.
main_review: Strengths:
1. AdderNets for object detection is an interesting and worth studying direction. This work studies this problem for the first time.
2. The technical part is well written.

Weaknesses:
1. This paper claims that CNN contains massive inefficient multiplications, which leads to more energy consumption than AdderNets. But I cannot see inference speed comparison between AdderNet-based and CNN-based detectors. Inference speed is an important metric in real-world applications.
2. BN statistics change drastically when the weights pre-trained on ImageNet are used in COCO, as shown in Figure 2. Therefore, the AdderNet-based detector seems to be less robust to the domain shift.
3. The AdderNet-based detector uses some tricks (see Table 1) to improve the performance. These tricks can also be used to improve CNN-based models. But the reported CNN-based method does not include these improvements.
4.  The author provides some ways to alleviate these problems, but I think there will be more essential solutions, such as avoiding these problems when designing the adder filter.
limitations_and_societal_impact: Yes.
review_title: null

final_score: rating: 3: Clear rejection
review_content: summary: The paper investigates the use of AdderNets (networks that do not use multiply operations) for object detection. Several modifications to the original AdderNets architecture are proposed including "unfreezing" batch norm statistics during inference as well as residual connections. Knowledge distillation as well as "other tricks" (L206) are used to further improve performance. Results are reported for some modified (replace traditional convs with adders) detection architectures are presented. The results show a moderate drop in performance, while supposedly using less power (estimated in mJ). 
main_review: The proposed work applies the recently published AdderNets to the detection problem. The modifications proposed are highly ad-hoc, and the motivations behind them are questionable. Specifically, I do not find the results shown in Figure 2 convincing for unfreezing the batchNorm params during inference. I would appreciate at least an attempt at some analysis as to why training becomes unstable without unfreezing BatchNorm. Why are AdderNets more sensitive than traditional convnets? Perhaps start by noting that: exp(log(a) + log(b)) = a*b. Also proposing to unfreeze these params is highly limiting, for example consider a mobile device setting: can inference even be done on one image at a time (i.e. single sample batch stats are unreliable)? Finally, the baseline models used by the authors are far from modern state of the art (https://paperswithcode.com/sota/object-detection-on-coco) which severely limits the impact of these results. 
limitations_and_societal_impact: Although the authors claim that AddrNets use less power they have not demonstrated this in realistic settings.
review_title: null

Meta-review: The crux of the paper is to enable object detection using Adder Networks -- architectures optimized for energy efficiency. The paper proposes a couple of empirically validated heuristics that certainly achieve the objective. Consequently, reviewer 2TRF argues that the paper should be accepted. However, reviewers 3Btm and cnqS argue that the paper should be rejected.

- I acknowledge Author's concerns about cnqS's opinion that this paper is not a good fit for NeurIPS. I don't think cnqS's concerns in this context are valid. 

- The main concern of 3Btm is that the proposed changes are trivial. I don't consider the proposal to be trivial, since it does lead to an increase in performance. After all, even the AlexNet paper talked about specific ways of tuning / architectural choices to enable CNN training. 

My main concern really is that there are many ways of increasing energy efficiency -- reduced precision, distillation, pruning, etc. 2TRF raised this point in the review and authors compared the performance with one such paper. However, to really make an impact and ensure we are doing good science, a thorough comparison should be made with different ways of achieving energy efficiency -- with adderNets being one of them. I agree with the Authors that pruning etc. can be applied to AdderNets also, but it's unclear how much pruning is possible with these architectures. 

While this paper is certainly an advance in the field of "AdderNets", I think it's critical to put it in the context of advances in energy-efficient computing, which is not done. Such a comparison would greatly strengthen the paper. I, therefore, encourage the authors to resubmit with this change. 
Decision: Accept (Poster)