Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary: The paper addresses the question of attributing the predictions of deep neural networks to the input features. It proposes to combine a particular restriction on the prediction function: non-negative homogeneity (e.g. a deep neural networks without biases) with the fast gradient x input explanation technique. With this combination, it is possible to produce explanations that satisfy the completeness axiom that can be computed quickly (from a single function evaluation). The paper also argues that non-negative homogeneity makes the model invariant to contrast, which is a useful property in image recognition.
main_review: The paper contributes theoretical results, such as the connection between Integrated Gradients and Gradient x Input for nonnegatively homogeneous functions, the relation between gradient x input and the completeness axiom, as well as the connection between gradient x input and the sensitivity axiom. However, some of these results, in particular, the completeness of gradient x input, are already known and described in earlier papers for the case k=1, which is arguably the most important case since it covers standard deep neural networks. A practical motivation for the higher-order case (k>1) is not provided in the paper.

The paper is mathematically rigorous. However, in Table 1, I'm not sure why Input x Gradient would fail the symmetry and sensitivity(a) tests. I would encourage the authors to provide the corresponding counter-examples in the supplement. Regarding the content of the paper, references need to be added to previous work related to explaining positively homogeneous functions and on the completeness of gradient x input. 

The paper is generally clear. However, Table 1 is misleading: Non-negative homogeneity is not an axiom (or some desirable property of the method) but a modeling constraint. Maybe a more suitable line to add in the table would be "computable in a single forward/backward pass". In Table 3, there is no description of what KPM, KNM, etc. mean (only a reference to the paper introducing them). A brief introduction of these metrics would ensure that this table can be fully understood.

The significance of the work is limited. The theoretical contributions are incremental, and the novel aspects (k>1, sensitivity) are not the focus of the experiments. Regarding the practical applicability of the method, removing biases requires retraining the model (which might offset the benefits of reducing gradient evaluations at test time). If the model is instead meant as a self-explainable model, this should be stated explicitly, and the proposed approach should be compared with other self-explainable models, e.g. bag-of-local-features.

Lastly, the proposed X-gradient approach satisfies the completeness axiom, but it is still based on a single gradient evaluation, which raises the question how it could reliably explain a function that contains multiple scales (or some strong local gradient variation superposed to weaker global variations).
limitations_and_societal_impact: The limitation of methods based on a single gradient evaluation (e.g. for handling functions with multiple scales), as well as the potential cost of having to retrain the model, should be discussed.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The authors propose a class of DNNs, which they call non-negatively homogenous, where the Integrated Gradient (which satisfy certain desirable axioms) for feature attribution can be computed in closed-form using a single gradient evaluation (as opposed to multiple gradients calls for a Riemann sum as in Sundararajan et al 2017). The authors propose instantiating such DNNs in practice by removing the bias term. They conduct experiments on two tasks where they show that removing this bias does not cause a substantial drop in accuracy while still computing IGs efficiently.

main_review: The paper is well-written and easy to follow. The notation is clear and the propositions/definitions are adequately explained informally which makes the paper easy to understand. Additionally, the authors make an effort to anticipate a reader's questions and answer them which makes it easy to track their specific contributions.

The primary novel contributions are proposing a new class of DNNs that can be instantiated in practice by removing a bias term and proving a closed-form IG for this class. The method is efficient because it only requires a single gradient call.

Re “removing bias term”: The authors sufficiently acknowledge this limitation. It was interesting to see that at least for the experiments in the paper, accuracy does not decline too much. I imagine that in practice, a practitioner could conduct a preliminary analysis to see whether removing the bias hurts performance (through a validation set) and use the proposed DNN if it does not. I think some more suggestions for deciding when this method is appropriate (from a practitioner’s standpoint) would be valuable to add to the paper.

Re “efficiency”: The authors claim that attributions methods need to be invoked at each training step. Does that only happen when attribution priors are used for regularization during training (as in Eq. (1))? Without these priors, it seems to me that attribution only needs to be computed after training a network as opposed to every step. Are there other cases where each training step needs attribution computation?

Table 3 and lines 294-296: The results show that the RAM value for Grad (1) is the best (and does reasonably well for the standard AlexNet too). This is not addressed in the discussion. What are the potential reasons for this?

Line 337 “leads to a strong drop inaccuracy”: I found this surprising given that AlexNet is strictly more expressive than the $\chi$-DNN. I would have imagined that some amount of hyperparameter tuning would have prevented this drop in accuracy.

Figure 2 (right): Are the qualitative attributions shown for the cases where both DNNs predict the correct class label? I am wondering if the less interpretable attribution for AlexNet is simply because it is predicting the wrong label due to the drop in accuracy.

Typo in Eq (5): It should be $i \geq 1$.

limitations_and_societal_impact: Yes, there are two main limitations and the authors acknowledge and address both of them: (i) removing the bias term limits expressivity of the DNN and (ii) the attribution needs a baseline of zero.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper first identifies the intensive computational cost in using Integrated Gradient (IG) as an attribution prior in training explainable deep neural networks. To this end the authors propose to switch the network architecture to negatively homogeneous networks, specifically networks without bias term, a.k.a. $\mathcal{X}$-DNN, in this paper, where they prove that $\texttt{input} \times \texttt{gradient}$ is an equivalent attribution method to IG; and therefore can decrease the number of gradient evaluations in computing IG. The paper then conducts several experiments over the ImageNet dataset and several tabular datasets to validate the performance of the proposed method. 
main_review: In general, I find the theoretical results are very interesting and it opens another fresh view of connecting Integrated Gradient with inherent properties of networks, just like the connection between randomized smoothing and SmoothGrad. However, the evaluation part of the paper is confusing to me, which prevents me from giving higher scores. However, I am on the fence and expect the authors’ response to clear my concerns. Please find the detail as follows.

### Originality
The work is a novel combination of well-known techniques. The related work is well-cited.

### Quality: 
I have two major concerns on the quality side of this paper. Please find the detail as follows:
1) Theoretical limitations of no-bias network, a.k.a.$ \mathcal{X}$-DNN. The authors provide empirical evidence that $\mathcal{X}$-DNNs match the Top-5 accuracy of regular networks in Section 4.1; however the experiment is only limited to one data distribution and one network, which is far away from covering all kinds of applications the author believes their work can potentially benefit for, e.g. health care. I would suggest that more theoretical analysis can be provided to convince the reader $\mathcal{X}$-DNNs are as useful as regular networks. An example question is: assuming zero vectors as baselines can decourage the network to use features with values close to 0. For example, it will always consider darker pixels as not useful so black dogs may always have smaller attribution values compared to white dogs. Will the $\mathcal{X}$-DNN end up with becoming a network that is always more sensitive to “bright” features and less sensitive to “darker” features that are actually important?  

2) Evaluations are not well-motivated. My biggest concern is that several proposed evaluations and their metrics are not well-motivated to show the proposed network architecture is promising. 
- Firstly, as$\mathcal{X}$G is not a new method but more like an alias of input x grad for networks without bias and is mathematically equivalent to IG in the proposed architecture, the result of Table 3 is actually comparing existing attributions using existing metrics (KPM, KNM, KAM and RAM). More analysis could be added to justify why Table 3 is important to include and why the result is useful to show the proposed $\mathcal{X}$-DNN is promising.  Besides, the motivations of using metrics from [1] instead of other work, e.g. [2], are unclear to the reader. 

- Secondly, the paper is motivated to accelerate the training with attribution priors but statistics of the timing and memory cost in training $\mathcal{X}$-DNN are missing in Section 4. More numerical results instead of descriptions, e.g. line 328-330, will show exactly how fast and efficient the proposed algorithm is. 

[1] S. M. Lundberg, G. Erion, H. Chen, A. DeGrave, J. M. Prutkin, B. Nair, R. Katz, J. Himmelfarb, N. Bansal, and S.-I. Lee. From local explanations to global understanding with explainable AI for trees. Nature Machine Intelligence. 

[2] Yeh, C., Hsieh, C., Suggala, A.S., Inouye, D.I., & Ravikumar, P. (2019). On the (In)fidelity and Sensitivity of Explanations. NeurIPS.

### Clarity
The motivation is clearly stated: the attribution prior regularization is usually computational expensive in terms of the number of gradient evaluations w.r.t the input; therefore, it is naturally to find another way of computing attributions with less number of gradient evaluations while the attribution used in the training is also faithful; 
2) the paper describes the motivation and the formulation of \mathcal{X}-DNN clearly through multiple definitions and propositions. The equations and proofs are easy to understand. 

### Significance
The results are important and others (researchers or practitioners) are likely to use the ideas if the technical questions can be resolved accordingly. 

### Update from discussion
Thanks for the authors to post the thoughtful responses to my questions. Even though some questions are probably not able to resolve during this discussion time, e.g. the theoretical guarantees about the bias-free networks, I generally find this paper to be interesting and the authors' response is convincing and they are trying to make the changes that can answer my questions in the future revision. I therefore will increase my score from 5 to 7.
limitations_and_societal_impact: The authors adequately addressed the limitations and potential negative societal impact of their work.
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: This paper proposes to use non-negatively homogeneous DNN to perform feature attribution with better computational efficiency at a minimal performance cost. The so-called non-negatively homogeneous DNN is obtained by removing the bias term from most common neural networks and using a Relu/Leaky_Relu activation function. By doing this, the model would be linearly scaled by the input data (F(ax) = a*F(x)). With this property, the integrated Gradient (IG) method can be easily computed with one call of gradient computing, which improves its original computational efficiency. In the experiment, this model shows comparable performance with the IG method but much less computation effort.


main_review: I’m not an expert in feature attribution research but this paper sounds like an interesting discovery based on the existing IG method, which takes advantage of neural networks that are non-negatively homogeneous after removing their "Bias" terms. However, the key idea of this paper that improves the IG method looks very simple and straightforward (for myself, I know the non-negatively homogeneity of DNNs without bias term a long time ago from many papers discussing the property of DNNs. e.g., for papers like "Analytic Marching: An Analytic Meshing Solution from Deep Implicit Surface Networks", they already kind of show these results by analyzing the local linearity of the neural networks). At least, this conclusion is not very novel from my perspective. On the other hand, this discovery could be an interesting application of this "property" on the feature attribution research. Therefore, I think the paper should give a better introduction to the background of this problem (feature attribution) and demonstrate that there is enough contribution based on the efficiency improvement of the IG method. For example, is "IG" the state-of-the-art method for feature attribution while being limited by its computational efficiency? Is there any other feature attribution method that are efficient compared with one-gradient-call IG?

Minor questions:
1. for clarity, can the author provide a more solid example of what the Integrated Gradients method really does?
2. What is the relative attribution distance? (Mentioned in Table 2)

limitations_and_societal_impact: N/A
review_title: null

Meta-review: This paper focuses on feature attribution techniques (which, in a frustrating linguistic turn are often called “explanations”). These techniques are all heuristics chosen for their ability, on the basis of anecdotal evidence to highlight features that the beholder finds important. What if anything they are  “explaining” is seldom explained, and I would encourage the authors to strip the paper of gratuitous uses of the term explanation wherever it is wielded in a quasitechnical form: this includes the title “for Training Neural Networks with Explanations” and in the italicized phrase, as though introducing a technical term: "efficiently explainable”. That said, for better or ill, and the jury is still out, some of these techniques are objects of interest of a large part of the ML community, and while there has been some progress to positing axioms that these attribution maps ought to satisfy, not all methods that satisfy those axioms are known to be efficiently computable.

 In particular, the authors present a new method for computing Integrated Gradients efficiently, showing that when bias terms are removed from neural networks they are efficiently computable. Whether integrated gradients are a useful concept in the first place remains a dubious proposition (what should the baseline be and what basis could anyone possibly have for choosing one?). However, insofar as they are an object of interest and the community is willing to publish lots of papers about them, this paper strikes me as unusually useful and concrete in its offering. The reviewers debated the surprisingness of the finding and whether it was already known to some in the XAI community. However, while I have seen many papers computing integrated gradients, I have no knowledge of anyone computing integrated gradients efficiently via this trick. It seems sufficiently useful, that if it were known, you would have seen it. Questions of the value of the underlying method aside, I do not believe that a finding must require an earthshaking analysis to to be surprising to a great mathematician to be valuable and useful. If that were the case, most useful papers in ML would never have been published. The reviewer’s suggestions that a key result was already known have been rebutted by the authors (who claim that the result is only known for linear models) and the reviewer did not take the opportunity to respond. In this light, I will give the authors the benefit of the doubt. 

The authors had a constructive discussion with Reviewers bst1 and gwR5 and I trust that they will act in good faith and add the discussed modifications, clarifications, and exposition to the final draft.  I will recommend acceptance and hope that the authors will act in good faith and tone down the quasitechnical language about “explanation” and to improve the paper per the promises that arose in the discussion period.
Decision: Accept (Poster)