Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes a visual and linguistic bounded explanation method to make part of NN models explainable by adding both attribute-wise and language-wise explanations. Specifically, they add a trainable part between the feature extractor (e.g., pre-trained Resnet) and label embedding, and the added part can be explained. The paper is easy to follow and clear.
strength_and_weaknesses: **Strength**

1 Using multilevel (visual and linguistic) to conduct more intuitive explanations is an interesting and promising idea.

2 The author shows an easy-to-follow method and results.

**Weaknesses** 

1 Generalization. The proposed method could only be used to explain a specific kind of NN, which learn a mapping starting from an extracted feature to an attribute-related label embedding. It is hard to explain general pre-trained neural networks (e.g., ResNet, GNN, Transformers). 

2 Partial explainable. The whole model has three parts, feature extractor, added learnable mapping, and attribute to label mapping; only the middle part can be explained, and the best part still performs as a black box.

3 Fidelity issue. The method treats not the true label Y but an attribute set as a target during training. How to guarantee the GPT-3 described attributes are useful features for prediction? If you directly train a model using true label Y, they use different features during the decision. So the explained attributes do friendly to human understanding but may not follow the original model's logic.

4 Need to improve accuracy results. The final accuracy is lower and has a relatively large gap compared with the original model, even though the proposed method uses more learnable parameters. These results also aligned with point 3 above. The newly learned explainable mapping does not use the same logic as the original model. 

5 Accumulated bias. There may be some bias in the feature extractor model (Resnet) and GPT-3. The new trained mapping can not remove those biases if they are input and target.
clarity,_quality,_novelty_and_reproducibility: The idea is interesting. Writing is easy to follow. Reproducibility is good due to the provided code.
summary_of_the_review: Overall, using a multilevel explanation is interesting, while the reviewer thinks the weakness outweighs the strength.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper proposes a new explainable neural network architecture called Multilevel XAI architecture. The key contribution of the work is to use both visual and language-guided interpretability, where a list of attributes pertaining to a class prediction (e.g., Dog --is furry, ...). Experiments show the model's ability to ground salient attributes in images in addition to providing region-based saliency maps. 
strength_and_weaknesses: **Strengths:**

*[S1:]* Interesting results: The results obtained are certainly interesting; The per-attribute visualization can indeed shed greater light on the decisions of the neural networks compared to "class-level" heat maps. 


**Weaknesses:**

*[W1]:* Inadequate baselines, comparisons, and discussion: The proposed visualizations are interesting, but a huge swath of work in vision and language that already demonstrate very similar ability, or can be trivially extended to produce similar visualization and/or linguistic explainability (E.g., concept bottleneck models https://proceedings.mlr.press/v119/koh20a.html). This is woefully missing in the paper and it is very hard to contextualize why the proposed method is more explainable than any number of other works. I will list a few ways this could be done, but the main issue is that it already feels trivial to do what the paper does with existing tools. What new benefits does the proposed work bring?

W1.1. Visualization of existing V&L models: Almost all of the recently introduced VL pretraining methods can ground image regions for a given word in a caption. (E>g., https://github.com/salesforce/ALBEF). Some methods even work for new objects as long as they are adequately described with attributes (A llama, which is a white tall animal in the picture (e.g., https://github.com/microsoft/GLIP)). Going back, even more, most of the object-based representations are also trained to predict object attributes (https://github.com/peteanderson80/bottom-up-attention). The paper makes a distinction between "class-based attributes" and "image-based attributes" but I see no distinction when the attributes are describing various objects; For a "dog" isn't the "class-based attributes" simply a union of all attributes contained for "dog"? (E.g., https://vawdataset.com/). Almost all methods that work with objects and attributes can visualize their affinity in various image regions to some degree. How does this compare with the proposed method?

W1.2 Trivial modification of existing work (E.g., Grad-CAM) to use additional information used by the paper: The paper makes use of additional attributes to obtain the given visualizations; However, if any network was modified to predict these attributes in addition to the class label, Grad-CAM visualization for making the prediction "has beak", could be used in exactly the same way the paper is using.

Without a comparison, discussion, or adequate comparative analysis with these, the impacts of the paper are hard to justify.

*[W2]:* The effect of components used is not explained: Firstly, the method requires additional annotations. While the paper does mention that may be easy to obtain (e.g., via large language models such as GPT-3), there are no experiments showing whether a lower-quality (noisy) attribute annotations would still work. Similarly, the training of MLP_L is not described well. How sensitive is it? What happens when only a small portion of the attributes are present? What happens when attributes (y_hat) is not predictive of class y? Since having high-quality attribute data seem to be the central requirement 


*[W3]:* Lack of Clarity: The paper feels obfuscated and very hard to follow. However, upon closer inspection, the core concepts are fairly simple. I would urge the authors to streamline the presentation and explain in terms of their function rather than hiding details unnecessarily under hard-to-follow conventions. For example, in Section 2.2 paragraph 2, the first sentence is literally just describing the process to generate predictions from a neural network. It is entirely unnecessary to write that as:

> 8Xj 2 X used for test, let yXj = †(˜⇡( (Xj ))), i.e., the predicted class label of the test image Xj. (Not rendered properly here -- please see pdf.)

The second part of the sentence beginning with i.e. describes the same thing as the stuff before that. This is simply an example in what is an unnecessarily dense symbol used to describe fairly simple underlying processes in Section 2 (Methodology).  

clarity,_quality,_novelty_and_reproducibility: **Clarity:** To me, the clarity is poor and feels deliberately obfuscated, especially in the methods section.

**Quality:** See strengths and weaknesses above.

**Novelty:** There is a limited novelty in both the techniques used and the empirical evidence (and discussion).

**Reproducibility:** The method is built on existing models and a code is also released in an anonymized link, I think reproducibility is excellent. 
summary_of_the_review: I am leaning toward recommending the rejection of this paper. While there certainly are interesting tidbits, the biggest gripe is that the proposed model requires both class label and attribute annotation per class; If we allow other methods to have access to the same (or similar) type of data, they could (and do) produce similar "interpretable" visualizations as shown in the paper.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The authors propose to learn explanations for the predictions from a neural network by employing a multilevel explanation approach. More specifically, the authors propose to leverage attributes for specific images and map the coarse class labels to the fine-grained object attributes during training. They show that this leads to human-like explanations while achieving nearly similar prediction accuracy. The perform experiments on benchmark datasets and show improved explanation ability of the model compared to state of the art.
strength_and_weaknesses: The paper looks at the important problem of explanation in deep neural networks which are of inevitable importance. Their motivation of defining what is learned by a neural network by breaking down the learning process from coarse categories to fine grained attributes is interesting, intuitive and well supported experimentally.

Despite the interesting approach, there is a limitation as the results and the method are only applied for easy datasets / classes and will be very hard to scale for imagenet classes for instance. What will be the approach to scale for instance the attributes and how would the method perform for thousands of dog breeds in imagenet for example ? 

What is the loss function that is used to match the feature and attributes obtained from zero shot literature ? 

It is not very clear what the difference is between the single level (Explainable MLP) versus the multilevel Explainable CNN. How’s explaining using attributes (whether with or without saliency maps) a multilevel approach ? 

Is there any way to measure the attribute prediction accuracy ? 

clarity,_quality,_novelty_and_reproducibility: The paper is well written, original and can be reproduced by the details mentioned in the paper.

summary_of_the_review: The paper is well motivated but the contributions for explainability are marginal and does not bring any significant improvements to the already existing research or outcomes in the field of explainability.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper presents an explainability approach for CNNs. The approach combines class activation maps with the semantic representation learned by the model to produce explanations based on salient attributes and attribute-wise salient maps. The experiments are conducted with the AWA2 and CUB-200-2011 datasets.

The reviewers value the interest of producing explanations that combine visual and semantic information, and the insights observed in the obtained results. However, the reviewers also pointed out the following aspects: (1) The approach is not general enough to be applied to any pre-trained NN; (2) Need to test the proposed approach in larger scale datasets; (3) Novelty with respect to existing methods is limited. The work also requires a better contextualization of the proposed approach with respect to existing methods and more comparisons with related methods. Despite the feedback of the authors the main concerns still remain, making the paper to be not mature enough to be published at ICLR. We encourage the authors to extend the related work adding the references recommended by the reviewers. We also recommend the authors to extend their experiments to larger scale datasets and even to the medical imaging domain, as suggested by the authors in their feedback.

justification_for_why_not_higher_score: The paper follows an interesting direction. However, all the concerns raised by the reviewers suggest that the paper is not mature enough to be published.
Decision: Reject