Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper explores the characteristics of the method, Integrated Gradients, as an attribution method, that has been proposed to explain black box models. “Baselines” in analyzing integrated gradients are discussed and the shortcomings of integrated gradients are further evaluated. The paper then proposes Integrated Certainty Gradients and shows its application on data.
main_review: On the theory side:

This paper can been as a discussion on integrated gradients. The paper evaluates different baselines and moves to investigate the (un)fairness of attribution methods that the authors define as “whether the relative attributions of input components are reasonable”. After considering the (un)fairness of attribution methods in this respect, the authors propose their attribution method. The authors then apply their method to data. I did not find enough theory backing the usefulness of the proposed methodology in various aspects. Even though the paper has applied their methodology and has shown practice, the paper lack in theoretical support.

On the application side:

The paper applies their methodology to two scenarios in the main text and also provides the results of some experiments about accuracy on the MNIST dataset in the supplementary materials.

About the results in the main text, take Figure 4 as an example. The behavior of the proposed method is not clear. The method does not show attributions of the dark squares in columns B and C of the ICG row. Why is this the case? The authors have pointed out that repeating the algorithm leads to better results here, but the authors have not explained the reasons of this behavior clearly.
 
About the results of the supplementary materials, it is unclear how the results on the MNIST dataset can be evaluated and compared against other results in the main text. Also, the results in section A.1.3 are not promising.

Overall, the experiments of this paper are not convincing enough.

Looking at the theory and application together, the paper lacks novelty.
summary_of_the_review: Even though this paper approaches an interesting problem, it needs improvements and clarifications.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors analyze the fairness of Integrated Gradient-based attribution methods. They exploit SHAP and BShap, two approaches based on the theory of Shapley Values, as the reference of "fair" methods. 

Specifically, they present an "attribution transfer" phenomenon in which the Integrated Gradients are affected by some sharply fluctuated area across the integration path, thereby deviating from the ''fair'' attribution methods. To avoid the attribution transfer issue, they further propose Integrated Certainty Gradients method, where the integration path does not pass through the original fluctuated input space.  Such an objective can be achieved by training the network with perturbed inputs and corresponding certainty maps. Finally, the gradients integral can be calculated by querying the trained network with fixed inputs and varying certainty. Various purpose-designed experiments are performed to demonstrate the advantages of ICG in avoiding attribution transfer.
main_review: Strengths:
1. The proposed Integrated Certainty Gradients training and attribution framework is novel. It makes sense that the authors utilize the fact that the integrated gradient path will be potentially less fluctuated than passing through the input space when it is only calculated along the direction of certainty change. Therefore, the impact of the previous issues on the IG could be mitigated.

Detailed Comments:
1. In Section 3.1, the example of attribution transfer is somewhat ambiguous. This part could be more readable if the authors elaborate more on how BShap, SHAP, IG gave the exact attribution value, respectively. 

2. In the first paragraph of Section 3.1, it is said that the model must indicate whether either of the two components falls within the range 0.25 - 0.5. However, from Figure 2 and its description, there is a differentiable region where the function value changes continuously (gradually from white to grey) at the edge of this range. I would suggest the authors modify this paragraph to be consistent with their figures.

3. The formal definition of attribution transfer seems unclear. In the third paragraph of Section 3.1, it is said that "We name this phenomenon attribution transfer" while "this" here is ambiguous. Based on the context, I infer that attribution transfer refers to "IG attribution results deviate from fair attribution due to the integration path traverses the fluctuating input space." However, I would like to see a more explicit definition given by the authors.
 
4. In Section 3.2, I think the logical coherence of the paper would be better if the authors could explain how the axiomatic difference is related to the attribution transfer and the fairness of IG.

5. In Section 4, it would be more convincing if the authors elaborate on the training process of ICG. I wonder whether there are changes compared with the ordinary training other than input data, such as loss function and backpropagation. Would ICG training affect the model performance? How and to what extend.

6. Two purpose-designed experiments without any general scenarios are insufficient to justify the use of Integrated Certainty Gradients. I notice that the authors mention their experiments on MINIST with little details in the appendix. I expect the attribution results of this experiment.

7. Since IG requires no modification of the original model while ICG trains the model with damaged data, it naturally makes me curious whether the utility of the ICG trained model decreases. I would suggest the authors show the test results of the ICG trained model and discuss whether there is a trade-off between generalization and attribution ability.
summary_of_the_review: Assessing fairness is important butt non-trivial. It is interesting to exploit SHAP and BShap, two approaches based on the theory of Shapley Values, as the reference of "fair" methods. Many concepts are not clear, and details are missing. Experiments are not comprehensive; please see the detailed comments.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper studies the unfairness of a family of integrated gradients based attribution methods, by the what they call ‘attribution transfer’. To solve the problem, it proposes a integrated certainty gradients attribution by adding ‘noisy’ to the input during the model training. 
main_review: For the discussion in Figure 2. What is the exact mapping, how the numbers in Table 1 were gotten. For path A, shouldn’t the attribution IG is less for x1 because the partial derivation is uniform along the horizontal axis?

For the Figure 2 example, I can’t understand very well what is ‘maximally misleading’ and what is ‘attribution functions’. Why IG is unfair? How to measure fairness.

For my understanding, SHAP and IG are both post-hoc attributions. But the proposed ICG needs to perturb the input by certainty noise (my words) to train the model. If this is correct, what is the performance of IG or SHAP on such a trained model? Any difference with the ICG?

Figure 3 also makes me confused. How to get the damaged input. What is the meaning of ‘sample’ here (bottom left). What do the green bounding box and blue bounding box mean? And blue and green rectangles as well.

I’m also wondering is there any visual difference for ICG and IG in terms of the visualization results on the real image like they show in their paper?

A typo, fourth line from the bottom of page 2, ‘any any’.
summary_of_the_review: The theoretical analysis part (section 3) is somehow hard to understand for me. This lowers my score.
review_title: null

Meta-review: This paper analyzes analyze the fairness of Integrated Gradient based attribution methods. The authors exploit SHAP and BShap, two approaches based on the theory of Shapley Values, as the reference of "fair" methods. Specifically, they present an "attribution transfer" phenomenon in which the Integrated Gradients are affected by some sharply fluctuated area across the integration path, thereby deviating from the ''fair'' attribution methods. To avoid the attribution transfer issue, they further propose Integrated Certainty Gradients (ICG) method, where the integration path does not pass through the original fluctuated input space. Experiments are performed to demonstrate the advantages of ICG in avoiding attribution transfer. While the basic premise of the work is interesting, many conceptual details remain unclear and experimental evaluation can also be improved (please see detailed reviewer comments below). Given this, we are unable to recommend acceptance at this time. We hope the authors find the reviews helpful.
Decision: Reject