Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary: In this work, the authors propose to use Sobol indices for black-box explanations of deep neural networks on computer vision and NLP tasks. The method is based on efficiently estimating Sobol indices and Total Sobol indices with a Quasi-Monte Carlo sampling strategy. The experiments on the tasks of pointing game, deletion, and word deletion show that the proposed method achieves comparable results to state-of-the-art attribution models.

main_review: Strengths:
+ The proposed approach is based on the Sobol indices that have been used for sensitivity analysis in many fields. 
+ The authors propose to use the Quasi-Monte Carlo sampling strategy for efficient estimation of Sobol indices
+ The baselines and experiments are comprehensive.

Weaknesses:
- The technical novelty appears to be limited. The proposed method is the adoption of existing techniques.
- The experimental results do not show that the method has outstanding performance, according to Table 1,2,3.
- I do not see why black-box attribution models are necessary for practice applications. Can authors say some applications with black-box attribution models?
- The experiments have not demonstrated that high-order information is effective in the proposed model. 

limitations_and_societal_impact: The limitation is not clearly explained yet. The potential negative societal impact may include wrong explanation to deep learning models.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper proposes a black-box method to assess the contribution of image pixels and groups of pixels (regions) to the classification decision. The proposed approach is based on Sobol indices, that allow to characterize the influence of an input subset (image pixel/group of pixels or region) as its own variance with respect to the total variance.  
To employ these indices in an efficient way, they are approximated by empirical Sobol indices. Specifically, from an input image, a set of real-valued masks is drawn from a Quasi-Monte-Carlo Sequence to generate perturbed inputs. The Sobol indices and total Sobol indices are estimated empirically from the variance of model predictions of these images. The first-order Sobol indices indicate the importance of each pixel in isolation, the total Sobol  indices also encompass higher-order interactions.
The method is evaluated on standard benchmarks for vision and NLP and shows competitive performance to black box and white box methods.
main_review: The paper proposes a new method to asses the contribution of image pixels and regions to the classification score of an image, of the contribution of words in the case of NLP applications. The proposed paper/approach has several strengths:

- The motivation to consider the relative variance of the prediction is clear and interesting.
- The combination with the occlusion approach yields practically good results and allows to assess positive vs. negative impact to a score.
- The paper is well written and mathematically clear.
- The experimental evaluation for vision and language applications is thorough.
- The efficiency is also properly evaluated

Weaknesses and Questions: 
- A discussion on the discrepancy between the approximated, empirical variance and the actual variance would be interesting. Also, the accuracy of this approximation depends on the number of samples drawn. An ablation study on the needed number of samples would be interesting.
- considering the result in the pointing game, the a version of the proposed approach outperformes RISE in every case. Yet, the signed Sobol variant is not always the best. 
- in the NLP evaluation, the Occlusion method works better than the proposed approach. I think this needs more discussion. For example, what is the role of masking during training in this context?

Overall, the paper proposes an efficient model that can be useful to the community and that is well motivated. 
limitations_and_societal_impact: yes
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes a new method to understand the input-output causal relationship of a deep network. The proposed method is based on input perturbation (black-box), Quasi-Monte Carlo is used to sample two real-value masks for each input. Sobol indices is applied to measure the variance discrepancy between all the feature and a subset of the total. The sampling method for perturbation is efficient, and the sobol indices can capture both first- and high-order information of the variance. The proposed method can better visualise and understand the causal relationship of a learned network.
main_review: I have seen some interpretability studies proposed based on variance or entropy, but they can only show first-order information. The proposed method can capture high-order information. From Fig 1, the high-order information could be different from the first-order info. Given the receptive field of a model is bigger than the input in most cases, the high-order information could play a more significant role in decision making. Thus, this high-order information study is more intuitive and reasonable to me. 

One question in my mind is about the real-value mask. The use of binary mask is intuitive that some of the features(or input) will be completely ignored. When applying a real-value mask, does this actually create new patterns, e.g., a specific pixel or region could have low values due to the mask. Is it safe to consider the low-real-value region is somehow "masked out" for the variance computation, or could the weight will be increased to compensate this mask? I might miss some background for this setup, so I hope I can see some explanations in the rebuttal. If this is not a common setting, it is better to discuss a little in the paper. (L111 did not answer this question.)

Generally, the idea of applying sobol indices is interesting to me.
limitations_and_societal_impact: I personally like the clarity of this draft, the aim of this study is stated clearly in Section 1(a black-box method for causal relationship analysis), and the method is explained in details in Section 3 (sobol indices can deliver high-order information and Quasi_monte carlo can reduce the sampling complexity). And to my knowledge, this study of high-order information is important to better understand which regions(based on their interaction) are used for making a prediction. (other reviewers can complete the literature I missed.) If the rebuttal can explain the question I raised in the last comment, I would like to accept this study.
review_title: null

Meta-review: A method for assessing the influence of image regions on neural network predictions is proposed.  Reviewers n1GW and Szkm are positive about the approach and its reported experimental performance.  In discussion following the author response, Reviewer Xxm2 stated the response's ablation study sufficiently addressed concerns.

Decision: Accept (Poster)