Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper tries to identify the most salient parameters in a neural network and shows that when these parameters are modified the predictions change. They also show that there is a relation between these parameters and regions in the input that impact the classification result.
main_review: The paper devises an approach so look at saliency of filters by aggregating the gradients w.r.t. the parameters belonging to that filter. In that sense it is an extension of sensitivity analysis. The paper then shows that only a limited number of filters need updating to modify the prediction. Given that some of these filters might have very many parameters the resulting change can be quite big. 

I believe there are 2 clear limitations in the paper.
- it is not clear how this differs from an adversarial attack in parameter space/simply exploiting the sensitivity to certain parameters. I.e. changing parts with large gradients will change the behavior. The fact that this is done for a limited number of parameters is similar to using L1 normed adversarial attacks. 
- The evaluation in section 3.4 is quite limited in scope and looks to be more anecdotal. Extending this to an in depth evaluation would improve the paper.


Detailed comments below


# Intro/Methods
- Figure 1/2: . I believe this figure can be improved by assigning each layer space corresponding of "unit 1" on the x-axis. Right now, the decay for the earlier layers (with fewer filters) is hard to see. By normalizing the space used by a single layer this might become easier to read.

- Figure 1 and 2: I am very surprised that in Figure 1 and Figure 2-bottom the saliency averaged over filters is always positive. In contrast it is always negative in Figure 2 top.  Fig 1: states that it is not standardized. Fig 2. does not specify this but it seems to me that there was an inconsistency. 

- Section 2.2/ Eq 3  I did not find later on in the manuscript how the magnitude of the selected filters was increased. Instead of using the cosine distance, one could also try to only pass the gradients through the filters that were deemed most salient. But this might result in a very sparse heatmap.

# Experiments
- Section 3.1/Figure 3:  If I understand it correctly these plots only contain data from initially misclassified samples. While the evaluation is done over all misclassifications, it is the sum of experiments on single images. I.e. when a filter is removed this is done only for a single example and evaluated on this example. In that sense it is similar to an adversarial perturbation in the middle of the network. The fact that this changes the confidence should not be surprising per se.

- Section 3.2/Figure 4/5: The nearest neighbor approach is done by computing the cosine similarity of the saliency maps. Since this is dominated by the higher layers it should not be surprising that this results in conceptual similarity vs input similarity.  This can also explain why the same classes are misclassified since higher level concepts are also higher in the layers. In Fig. 5 similarity of low vs higher level layers is shown. The texts states that this is done to show misbehavior, but it is not clear what is misbehaving in this plot. 


- Section 3.3 Correcting mistakes by updating salient filters is evaluated on all images independently. How was batch norm used in this case for the update? Setting it to training mode could have weird effects.  I am not convinced that it should be surprising that updating the ones with large gradients results in changes at the output. 

- Section 3.4
This section is the most problematic to me since it lacks large scale evaluation and is mostly anecdotal. 

Figure 8 is a set of 
# General
Make the labels of the axes in plots larger. They are hard to read.
summary_of_the_review: I tend to vote for rejection of this paper. The effects described are similar to adversarial attacks and in most but not all cases the influence of a parameter is only investigated on the image where the saliency was computed. 
The evaluation of the proposed approach in terms of providing pixel wise interpretability section 3.4 is limited.

This review was done after the rebuttal period as an emergency review. I had no access to the discussion with the authors which might impact my understanding.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper conducts some basic experiments based on the network parameters which are responsible for erroneous decisions. To support the proposed standpoints, the authors conduct a lot of quantitative and qualitative experiments.
main_review: Strengths
1. Different from the previous papers focusing on the saliency maps, this paper proposes to analyze the network parameters, which is novel and interesting.
2. Quantitative and qualitative experiments are conducted to verify the proposed standpoints.
3. The discoveries in the paper is interesting and inspired.

summary_of_the_review: 
I am not an expert in this field and I did read carefully about this paper. I could not find any weakness in this paper. The contributions proposed in this paper are really interesting and are verified by the proposed experiments.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper introduces a parameter-space saliency map to explore the salient parameters that are responsible for miscalssification. A set of experiments and visulizations are conducted on the salient parameters, leading to several interesting findings, such as, the nearest parameter neighbors share similar semantic information. Besides, the authors are also trying to improve the prediction accuarcy by turning off or fine-tuning the salient parameters. 
main_review: + Though the idea of parameter-wise visualization is not new, and widely studied in other works, this paper provides an interesting perspective on the parameters responsible for misclassification. 
+ Meanwhile, the authors provides the thorough analysis and potential for applying the salient parameters on improving classification accuracy. This makes the paper with pratical use.

- The proposed idea is somehow similar to the parameter-space adversarial attacks. What if use attack methods to filter out the salient parameters that mislead the final classfications? This seems more intuitive.
- For the experiments of parameters pruning, it may be not reasonable for directly turn off the parameters, what if give them a perturbation instead? It can still reveal the sensitivity of the parameters.
summary_of_the_review: The proposed method is new as a parameter-wise visualization, but it becomes somehow similar to the  parameter-space adversarial attacks. The authors would better claim the similarity and difference of these 2 topics. Besides, adversarial attack needs to be reviewed as the related work.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper devises an analytic method for explainability based on the observation of filter-wise parameter saliency distribution, and tests on several models. And several experiments are conducted to deminstrate the conjecture. The motivation is straightforward and easy to understand. 
main_review: [1] Eq. 1 is not clear enough, what's the meanings of K. 

[2] If the proposed analytic method only suits for the image classification task or the test models, the contribution would be limited. 

[3] As stated in Section 3.3, we can correct mistakes by fine-tuning salient filters. Does it mean a model of general performance could become a top-performing model benefiting from the proposed fine-tuning method?

[4] Whether the performance-gain obtained by fine-tuning salient filters can transfer to the independent database, rather than the closed test set. 
summary_of_the_review: The contribution might be not enough for iclr.
review_title: null

Meta-review: For this paper initially the reviews were 6,8,5,5. All the reviewers have provided constructive and substantial feedback. The authors have incorporated changes to address some of these comments and some of the comments could not be addressed. The main criticism of the reviewers have been that the Reviewer tkQp finds two clear limitations in the paper, Reviewer 3o7Z finds that the proposed idea is similar to the parameter-space adversarial attacks and Reviewer sCeW questions the generalisability of the method to other tasks. After the rebuttal the reviewers have reached the consensus that the paper may not be above the acceptance threshold (final scores: 6,6,5,5). Following the reviewers' recommendations, the meta reviewer recommends rejection.
Decision: Reject