Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper deals with the problem of uncertainty estimation. It proposed spatial smoothing, a method that ensembles neighboring feature map points of CNNs to alleviate the computational issues with Bayesian NNs. 
main_review: Strengths:
 
+ The paper proposes spatial smoothing to alleviate the computational issues with Bayesian NNs.

+ The paper empirically shows that spatial smoothing improves accuracy, uncertainty estimation, and robustness of BNNs across a whole range of ensemble sizes.

Weaknesses:

- The experimental evaluations are relatively weak or too some extent insufficient

- The paper is not well self-contained, which highly relies on the analysis, experiments in the appendix.
summary_of_the_review: While the theoretical analysis of the proposed spatial smoothing is interesting and valuable, the experimental evaluations are relatively weak or too some extent insufficient. Furthermore, the paper is not well self-contained, which highly relies on the analysis, experiments in the appendices.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposed a spatial `smooth` layer including a feature range bounding layer `prob` and `blur` the intermediate feature map in a CNN. 'Smooth' improves the accuracy and uncertainty of both deterministic CNN and a Bayesian NN approximated by MC-dropout. Authors tried to justify how `smooth` improves the optimization of neural networks by 1. interpolating the `blur` operations as an ensemble of the neighboring features 2. showing `smooth` filter out the high-frequency noises introduced by MC-dropout and smoothen the loss landscapes perturbed by MC-dropout. Authors empirically evaluated `smooth` on image classification and semantic segmentation tasks and showed that it improves both accuracy and uncertainty. Authors also tried to connect common pieces in CNNs like global average pooling, ReLU + BN as special cases of `smooth`.
main_review: Strength:
- The spatial smooth layer is an interesting design that intuitively makes sense to be helpful for removing high-frequency feature noise in CNN especially with MC-dropout.
- Authors also studied/explained their design from multiple angles, e.g. Fourier analysis of feature maps, changes to the loss landscape, understood as a spatial ensemble of BNN inference. 
- Authors did a great job of providing an extensive study on various tasks, datasets, and architectures.
- This paper tries to build connections between their designs with some common practices in CNNs which are interesting and valuable for understanding why those designs are important for getting the performance.

Weakness:
- Smoothing the feature map for improving CNN's performance and robustness has been previously studied e.g. Sinha et a. NeurIPS2020 [1]. [1] even did the smoothing in a curriculum learning style. The authors should discuss related papers and compare them. The existing work also discounts the novelty of this paper.
- This paper represents the spatial smooth design in the context of Bayesian neural network and claims that the smoothing e.g. averaging can be seen as an ensemble of neighboring features. I did not see strong support for this assumption. For computer vision task, different location of feature maps is usually be treated as independent variables. Using neighboring feature points for the ensemble is a coarse approximation even considering the neighboring pixels are often similar. Especially for downsampled feature maps, such an assumption can be dangerous. It is questionable if `smooth`'s improvement should be interpolated in this way. 
- I am concerned about why this technique is presented in a way strongly correlated with BNNs. After reading the introduction I got the impression that this paper's design is particularly for BNN although still helpful for deterministic models. However, only MC-dropout approximated BNN is studied in the paper. For other types of BNN, this technique seems not going to help. Considering there is no strong support for smoothing = ensembling, it may be more appropriate to understand the technique as removing the high-frequency noise inside the network and this issue is particularly important when dropout is present.
- On the empirical value of the proposed technique. `Smooth` is more helpful are smaller datasets and architectures with irregular loss landscapes e.g. VGG but get marginal improvement or even hurts the ECE metric on ResNet (Table E.1). Smoothing feature maps can be dangerous for dense prediction tasks e.g. segmentation and detections. Not surprisingly, the limited results on semantic segmentation shown in the paper are weak. The limitations of the proposed method should be further discussed.

[1] Sinha, Samarth, Animesh Garg, and Hugo Larochelle. "Curriculum by smoothing." NeurIPS 2020
summary_of_the_review: This is a paper with simple designs but a lot of analysis and studies theoretically and empirically. The authors presented the proposed technique motivated in the context of Bayesian neural networks, which is not very strongly supported. The technique is clearly helpful in some settings and not very much in other settings. It needs further discussion on the limitation of the proposed technique and related works. I am slightly lean to reject this paper but may change my rating later.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The motivation of this work is on the computational cost of using BNNs in practice, where applications might require a large number of BNNs in an ensemble formation for achieving good performance.
The work in this manuscript aims to reduce the computational cost ensemble.

The manuscript's insight for solving the computational cost is to exploit similarities in spatial neighbouring from images. This is a similar approach used in convolutional neural networks. Experimental results show improved efficiency in image tasks, while also considerably reducing the computational cost when compared to competitor methods.
main_review: The work explores the idea of spatial consistency to improve the computational cost of posteriors. The main idea is well presented and edge cases are explored. Although this is not my area of expertise, the theoretical analysis seems relevant and sound. Experimental results are convincing. Some more discussion on applications beyond the imaging domain could provide stronger significance to this method. The unifying perspective, when compared to global average pooling, pre-activation, and ReLU6, is a welcoming result.

* Minor comments
    - Figure 1 appears on page 1 without being referenced in the text
summary_of_the_review: The work presents a novel method for reducing the computational cost when using BNNs in real-world image tasks.
Although I am not an expert in the area, I judge the work as being worth publishing since the theoretical analysis seems relevant and experimental results convincing.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper provides an improvement to Bayesian NNs computation needs and accuracy by incorporating spatial smoothing (blur). 
main_review: The authors have conducted many experiments to test their ideas, which is commendable. The paper is, however, too difficult to follow since much of these results are put in an appendix which significantly increases the page numbers, and for which, without the appendix, the paper does not have enough evidence. It is also difficult to follow the paper with the appendix referred to every 3rd sentence. The structure of the paper also lends to difficult reading, with results included in the methodology, figures out of place, the literature review at the end, and a too-brief conclusion.  
summary_of_the_review: I believe the content and ideas to be of interest but the paper requires a major revision and proper thought put into reducing the content without the appendix. 
The language of the paper should also be improved. 
Acronyms are not defined throughout, assuming the reader knows what they stand for. 
The references are mostly from arXiv - I do not find this appropriate. 
I am also not convinced the ideas are novel. The authors do not provide enough evidence to prove this.
review_title: null

Meta-review: This paper proposed a spatial smoothing layer for CNNs which is composed out of a feature range bounding layer (referred to as prob) and  a bluring layer (referred to as blur). An empirical analyses shows that the proposed layer improves the accuracy and uncertainty of both deterministic CNNs and Bayesian NN (BNNs) approximated by MC-dropout. The paper further provides theoretical arguments for the hypothesis that bluring corresponds to an ensemble and represents the proposed method as a strategy to reduce the sample amount during inference in BNNs.

Reviewers valued the extensive (theoretical as well as practical) analyses. However, the theoretical analysis should still be improved. First of all, the  the proposed technique is motivated in the context of BNNs, which is not very strongly supported. Second, the argument that „the smoothing layer is an ensemble“ is based on the observation that it has some properties ensembles have as well: (1) they reduce feature map variances, (2) filter out high-frequency signals, and (3) flatten the loss landscape. But two things sharing the same properties do not need to be the same thing. Moreover, the proofs of the prepositions stating the properties are difficult to follow and may contain some flaws. Furthermore, the paper is not well self-contained and highly depends on the appendix.
Given these, the paper can not be accepted in its current state. 

A future version could improve over the current manuscript by making the theoretical statements and proofs more clear. Another option would be to analyze the contribution without connecting it to a Bayesian setting and ensembles, and instead focus on showing that the proposed smoothing layer has those good properties, doing detailed empirical studies, and showing that CNN components like global average pooling and ReLU + BN are special cases of the propose method.
Decision: Reject