Paper: LATEC — A benchmark for large-scale attribution & attention evaluation in computer vision (ICLR-2024)

Reviewer comment (Novelty):
The authors missed highly relevant studies that also extensively study a wide range of attribution methods according to different criteria (e.g., Hesse et al., 2023, Rao et al. 2022). A comparative discussion to such works seems necessary, especially as those works explicitly try to avoid the pitfalls raised in W2 (i.e., disentangling model behaviour from the attribution methods).

Author response:
**Answer to W2b:**\
We thank you for highlighting these relevant studies. We've included [1] in the main manuscript. Moreover, in the added metric-analysis, we compare ourselves to their approach for metric aggregation and evaluation of the metric-sensitivity to the model. We would like to highlight that, unlike our real-world study, both approaches include explanation "ground truths". This makes it for example impossible for us to compute a metric like Background Independence from [1].

Reviewer comment (Writing):
The implications and relevance of questions Q1-Q3 as well as their answers are not fully clear to me and I would appreciate if the authors could elaborate.

Author response:
**Answer to W3:**\
Q1: Comprehending the differences between attention and attribution methods is crucial to determine whether new attention methods can be universally applied or if they present specific drawbacks or advantages over traditional methods.\
Q2: While almost all XAI methods are designed for structured or image data, we don’t know if their applicability also transfers to other computer vision modalities, hindering their range of application.\
Q3: Attribution methods are widely employed; however, it remains unclear whether their application to popular Transformer models parallels their use with CNNs. Thus ensuring their applicability with forthcoming DL architectures is crucial.

We have updated the introduction of our work to express the significance of the three questions.

Reviewer comment (Writing):
The manuscript is very dense and not easy to read and I encourage the authors to place a particular focus on making the writing more accessible. For example, using non-standard abbreviations for the various methods makes it tedious to read the main text, as one needs to constantly look up what a particular abbreviation stands for (e.g., LI for LIME or LA for 'LRP Attention', etc.). Further, it is difficult to understand a given table or figure from figure + caption alone, which makes the results much less accessible (this is heavily aggravated by the non-standard abbreviations, see e.g., Fig. 3c).

Author response:
**Answer to W4:**\
We acknowledge that the abbreviations restrict readability. To address this issue, we have created a clean List of abbreviations on the first page of the appendix, which provides a quick and accessible reference for all abbreviations used in the manuscript.

Reviewer comment (Presentation):
The relevance and implications of the discussion of Fig. 3a are unclear to me and the 'distinct clusters' that form seem to be an exaggeration of what is visible in the plots. Without the distinct colourings and the boxes drawn around some of the points, these points seem fairly uniformly distributed to me.

Author response:
**Answer to W5:**\
We agree that the term "distinct clusters" could be misleading and have toned down this statement to convey that similar operating methods tend to group together, which implies a non-random structure to the rankings.

Reviewer comment (Theory):
Is there any interesting interplay between faithfulness, robustness and complexity?

Author response:
We would argue that, theoretically, there can be a "trade-off" between faithfulness and complexity. A very faithful map can be very noisy, thus more complex, and very uncomplex maps such as CAM maps can be evaluated as quite unfaithful as they only highlight a general region of interest, not the exact pixels/features the model uses. Although a trade-off is noticeable for methods like LRP, GC, or EG, we advise against overinterpreting these findings in light of our reservations regarding the complexity metrics.

Reviewer comment (Theory):
It is mentioned that theoretically biased XAI methods are less faithful on Transformers, but it is not clear why (from Sec. 3)? At least can you give some intuitions?

Author response:
Specific architectural choices in Transformer architectures, such as negative activation outputs, matrix multiplication in attention layers, or skip connections can, in theory, affect attribution methods. For example, this breaks the conservation rule for relevance backpropagation methods, (see [1] for a detailed analysis). In response to your valuable comment, we made the argument more prominent, especially in Section 3.

Reviewer comment (Presentation):
Texts in some images, e.g. Fig 2 are a bit small and difficult to read.

Author response:
We agree and increased the text size.

Reviewer comment (Presentation):
The presentation is hard to read and understand. It is an experimental report rather than a well-organized research paper.

Author response:
**Answer to W1:**\
We thank you for your critique regarding the presentation of our manuscript. We would like to point out that our submission aligns with the structure and style that is customary for the benchmark and dataset track of this conference, which should not be confused with the structure of a classical method proposal. Exemplary, we selected five previously accepted and outstanding ranked ICLR benchmark and dataset papers, that have similar structure: Jaeger et al. (Notable Top 5%) [1],  Galil et al. (Notable Top 25%) [2], Vaze et al. (Oral) [3], Zong et al. (Notable Top 25%) [4] and Galil et al. [5]. We followed these successful ICLR papers in their structure: 
- Formulating/Motivating Research Question
- Benchmark Design
- Experiment Evaluation
- Insights and Take-aways.

Notably all other reviewers specifically acknowledged that our paper is “well thought out presentation” (“6LcT”), “well-written and easy to follow” (“FKNb”), “presentation and description of the benchmark [...] is quite clear” (“dkvT”) and “experimental details are clearly described” (“aNQJ”). We would greatly appreciate it if you could specify the sections that you found to be lacking in organization so that we may enhance those areas.

Reviewer comment (Novelty):
As a benchmark and analysis work, the takeaways from this paper are not insightful but rather just some straightforward observations. After reading this paper, I am not able to gain good insights about the good way to evaluate XAI methods.

Author response:
**Answer to W2:**\
We would like to clarify that, while our evaluative strategies may not differ conceptually from existing ones (and were not intended to), the scale of our analysis is unprecedented and the dimensionality is unique. The resulting insights regard the application of XAI methods, not their evaluation, and exemplary show how relevant knowledge can be derived from our benchmarking.

Specifically, our benchmark offers a practical tool for practitioners to test various XAI methods across different metrics suited to their specific problem configurations. For researchers developing new XAI methods or metrics, our recommendations for testing across diverse datasets, modalities, and model architectures ensure robust and meaningful insights. Only because we followed these recommendations, we resolved prior inconsistencies in method evaluations (e.g. ignoring attention methods and modalities besides images) and showcased how benchmarking according to these recommendations is highly relevant. We have amended the manuscript to better highlight these contributions and ensure the insights gained are clear and actionable.

We thank you for pointing out the lack of clarity in this context and believe that our respective changes will avoid future confusion. As we believe to have resolved your comments, please let us know if there are any remaining concerns that could prevent you from recommending acceptance.

[1] Jaeger, P. F. et al. (2022). A call to reflect on evaluation practices for failure detection in image classification.\
[2] Galil, I.  et al. (2023). A framework for benchmarking class-out-of-distribution detection and its application to ImageNet.\
[3] Wiles, O. et al (2021). A fine-grained analysis on distribution shift.\
[4] Zong, Y. et al.  (2022). MEDFAIR: Benchmarking fairness for medical imaging.\
[5] Galil, I. et al. (2023). What can we learn from the selective prediction and uncertainty estimation performance of 523 imagenet classifiers.

Reviewer comment (Evaluation):
Overall, this paper makes solid experiments. However, it fails to reveal convincing explanations and insightful comments.

Author response:
**Answer to W2:**\
We would like to clarify that, while our evaluative strategies may not differ conceptually from existing ones (and were not intended to), the scale of our analysis is unprecedented and the dimensionality is unique. The resulting insights regard the application of XAI methods, not their evaluation, and exemplary show how relevant knowledge can be derived from our benchmarking.

Specifically, our benchmark offers a practical tool for practitioners to test various XAI methods across different metrics suited to their specific problem configurations. For researchers developing new XAI methods or metrics, our recommendations for testing across diverse datasets, modalities, and model architectures ensure robust and meaningful insights. Only because we followed these recommendations, we resolved prior inconsistencies in method evaluations (e.g. ignoring attention methods and modalities besides images) and showcased how benchmarking according to these recommendations is highly relevant. We have amended the manuscript to better highlight these contributions and ensure the insights gained are clear and actionable.

We thank you for pointing out the lack of clarity in this context and believe that our respective changes will avoid future confusion. As we believe to have resolved your comments, please let us know if there are any remaining concerns that could prevent you from recommending acceptance.

[1] Jaeger, P. F. et al. (2022). A call to reflect on evaluation practices for failure detection in image classification.\
[2] Galil, I.  et al. (2023). A framework for benchmarking class-out-of-distribution detection and its application to ImageNet.\
[3] Wiles, O. et al (2021). A fine-grained analysis on distribution shift.\
[4] Zong, Y. et al.  (2022). MEDFAIR: Benchmarking fairness for medical imaging.\
[5] Galil, I. et al. (2023). What can we learn from the selective prediction and uncertainty estimation performance of 523 imagenet classifiers.

Reviewer comment (Presentation):
This paper is an experimental report rather than a well-prepared paper.

Author response:
**Answer to W1:**\
We thank you for your critique regarding the presentation of our manuscript. We would like to point out that our submission aligns with the structure and style that is customary for the benchmark and dataset track of this conference, which should not be confused with the structure of a classical method proposal. Exemplary, we selected five previously accepted and outstanding ranked ICLR benchmark and dataset papers, that have similar structure: Jaeger et al. (Notable Top 5%) [1],  Galil et al. (Notable Top 25%) [2], Vaze et al. (Oral) [3], Zong et al. (Notable Top 25%) [4] and Galil et al. [5]. We followed these successful ICLR papers in their structure: 
- Formulating/Motivating Research Question
- Benchmark Design
- Experiment Evaluation
- Insights and Take-aways.

Notably all other reviewers specifically acknowledged that our paper is “well thought out presentation” (“6LcT”), “well-written and easy to follow” (“FKNb”), “presentation and description of the benchmark [...] is quite clear” (“dkvT”) and “experimental details are clearly described” (“aNQJ”). We would greatly appreciate it if you could specify the sections that you found to be lacking in organization so that we may enhance those areas.

Reviewer comment (Writing):
The citations on the first few sentences in Section 1 aren't directly tied to the citing text. Perhaps better references can be used, or the text can be modified slightly to better link to those cited articles. Please clarify whether those articles provide supporting evidence, focus on subproblems or application domains, or whether their authors simply echoed similar opinions.

Author response:
**Answer to Q1 (Section 1):**\
Thank you for your valuable feedback regarding the citations in Section 1 of our manuscript. Upon reviewing the section, we concur that the initial references were not optimally aligned with the corresponding text. To address this, we have carefully revised the text to more accurately reflect and connect with the cited articles. While most citations reflect the definition of a term or method (e.g. for faithfulness, robustness, or complexity), others give supporting evidence (e.g. (Adebayo et al., 2018) or (Wang et al. (2019); Rosenfeld (2021))), or are examples for applications (e.g. Doshi-Velez& Kim (2017); Ribeiro et al. (2016); Shrikumar et al. (2017)).

Reviewer comment (Writing):
It seems the discussion at the beginning of Section 1 is focused on saliency maps specifically. If so, please clarify the scope at the onset to be focused on XAI for computer vision models, with saliency methods as the primary approach being discussed. General statements about the state-of-the-art in XAI would require an equally general selection of citations.

Author response:
**Answer to Q2 (Section 1):**\
We agree with your observation that the term "XAI methods" was initially defined too broadly. Accordingly, we have refined the manuscript to specify our focus on saliency map methods right from the beginning.

Reviewer comment (Writing):
The authors seem to repeatedly use "XAI research" to solely mean works on computer vision.

Author response:
**Answer to Q3 (Section 1):**\
We appreciate the opportunity to clarify the scope of our research. The shortcomings identified in our work, though applicable across various domains of XAI research, are indeed addressed within the confines of computer vision modalities in this paper. In light of this, we have revised the manuscript to clearly define our focus on computer vision within the broader context of XAI research. We have made the necessary adjustments in the manuscript to ensure that our narrative accurately represents the scope and boundaries of our work.

Reviewer comment (Novelty):
Please clarify why saliency methods were chosen as the primary approach for this study, with a forward reference to a literature review highlighting other approaches.

Author response:
**Answer to Q4 (Section 1):**\
In our manuscript, we posit that saliency maps represent the most prevalent post-hoc XAI methods in computer vision. These maps, while primarily illuminating "what" aspects of the data are influential in a model's decision-making process rather than "why," continue to form the foundational basis of most XAI analyses [1,2,3]. We underscore the criticality of trustworthy saliency maps at this stage, as mistakes can have critical consequences in subsequent analysis, leading to a misdirected emphasis on irrelevant input features. Moreover, we highlight the evolving sophistication of saliency maps, as advancements in attention and concept methods [4] show promise for the future of saliency maps. It is also noteworthy that post-hoc methods such as saliency maps offer ease of integration with existing model training and evaluation workflows, a practical advantage over ante-hoc XAI methods.

In response to the feedback, we have adapted the manuscript to provide a more comprehensive overview of the XAI field, emphasizing the significance of saliency methods within this domain. We have included references to two exemplary approaches, counterfactual examples, and concept testing, and cited an extensive literature review to offer readers a full perspective of the field.

Reviewer comment (Presentation):
In Section 2, please include a reference to Fig. 1 in the main text.

Author response:
**Answer to Q5 (Section 2):**\
Thank you for pointing out the oversight. We have now included a direct reference to Figure 1 in Section 2.

Reviewer comment (Writing):
In S2.1, it would help to include a brief description of each XAI method, e.g., in an appendix, together with references to recent surveys (perhaps in a form similar to Appendix B.2).

Author response:
**Answer to Q6 (Section 2):**\
We agree that this would be beneficial for readers who are new to the field, or just miss the knowledge about a specific method. In accordance with your advice, we have included a new section in the appendix that offers concise summaries of the XAI methods discussed, akin to the structure presented in Appendix B.2. We trust that this addition will enrich the informational value of our work for all readers.

Reviewer comment (Presentation):
In Figures 3 and 4, it seems the legend shows significance indicators that don't actually appear in the figures.

Author response:
**Answer to Q7 (Section 2):**\
Thank you for bringing this to our attention. The significance indicators are indeed present above the figures but are rendered at a size that may compromise their visibility. We acknowledge this oversight and have since adjusted the size of the significance indicators to ensure they are clearly visible within the figures.

Reviewer comment (Writing):
Page 3: "without adaptions" -> should be "adaptation"? (I found 7 or so other occurrences).

Author response:
**Answer to Q8 (Nitpicking):**\
We thank you for the comments and revised the manuscript accordingly.

We appreciate the thoughtful questions you raised, as they have significantly contributed to the improvement in the clarity and explicitness of our manuscript. Please let us know if there are any remaining concerns that could prevent you from increasing your score.