MULTILEVEL XAI: VISUAL AND LINGUISTIC BONDED EXPLANATIONS
Authors: Halil Ibrahim Aysel
Xiaohao Cai
Adam Prugel-Bennett
Venue: Submitted to ICLR 2023 (2023)
Keywords: Deep neural networks
Black box
Explainable Artificial Intelligence
Saliency maps

Abstract: Applications of deep neural networks are booming in more and more fields but lack transparency due to their black-box nature. Explainable Artificial Intelligence (XAI) is therefore of paramount importance, where strategies are proposed to understand how these black-box models function. The research so far mainly focuses on producing, for example, class-wise saliency maps, highlighting parts of a given image that affect the prediction the most. However, this way does not fully represent the way humans explain their reasoning and, awkwardly, validating these maps is quite complex and generally requires subjective interpretation. In this article, we conduct XAI differently by proposing a new XAI methodology in a multilevel (i.e., visual and linguistic) manner. By leveraging the interplay between the learned representations, i.e., image features and linguistic attributes, the proposed approach can provide salient attributes and attribute-wise saliency maps, which are far more intuitive than the class-wise maps, without requiring per-image ground-truth human explanations. It introduces self-interpretable attributes to overcome the current limitations in XAI and bring the XAI towards human-like level. The proposed architecture is simple in use and can reach surprisingly good performance in both prediction and explainability for deep neural networks thanks to the low-cost per-class attributes.

Under review as a conference paper at ICLR 2023
MULTILEVEL XAI: VISUAL AND LINGUISTIC BONDED EXPLANATIONS
Anonymous authors Paper under double-blind review
ABSTRACT
Applications of deep neural networks booming in more and more fields but lack transparency due to their black-box nature. Explainable Artificial Intelli- gence (XAI) is therefore of paramount importance. where strategies are proposed to understand how these black-box models function. The research so far mainly focuses on producing. for example, class-wise saliency maps, highlighting parts of given image that affect the prediction the most. However, this way does not fully represent the way humans explain their reasoning and, awkwardly, validat- ing these maps is quite complex and generally requires subjective interpretation. In this article, we conduct XAI differently by proposing a new XAI methodology in multilevel (i.e., visual and linguistic) manner. By leveraging the interplay between the learned representations, i.e., image features and linguistic attributes, the proposed approach can provide salient attributes and attribute-wis saliency maps, which are far more intuitive than the class-wise maps, without requiring per-image ground-truth human explanations. tributes to overcome the current limitations in XAI and bring the XAI towards human-like level. The proposed architecture is simple in use and can reach sur- prisingly good performance in both prediction and explainability for deep neural networks thanks to the low-cost per-elass attributed
introduces self-interpretable at-
INTRODUCTION
Exciting developments in computational resources with a significant rise in data size have led deep neural networks (DNNs) to be widely used in various tasks, for example image classification. De- spite their excellent performance in prediction, DNNs are seen as black boxes as their decision pro- cess generally includes huge number of parameters and nonlinearities (Gilpın et al., 2018 [Hagras 2018 Zeiler & Fergus 2014p. The lack of explanation in these black boxes hinders their direct im- plementation in important and sensitive domains such medicine and autonomous driving, where
human life may directly be affected (Loyyla-Gonzallz 2019 Lipton 2018
An example would be the DNNs trained to detect coronavirus. Although many works have been conducted and claimed to have a high predictive performance in detecting COVID-19 19 cases, Turing Institute's 's recent report (Heaven 202T) disappointingly finds that Artificial Intelligence (AI) used to detect coronavirus had little to no benefit and may even be harmful, mainly due to unnoticed biases in the data and its inherent black-box nature (also see e.g. Roberts et al. 202Tb. Another example woman who was hit and killed by autonomous car. An investigation showed that the death was caused by the incapability of the car detecting human unless they are near a crosswalk McCausland] 2019). In addition to these life-related examples, there plenty of others where bias in training data or the model itself causes unwanted discriminations that may immensely affect people's 's lives. Amazon's 's Al-enabled recruitment tool is example of how discriminative these models could be by only recommending men and directly eliminating resumes including the word "'woman"; the company later announced that this tool had never been used recruit people due to the detected bias (Olavrrud] 2022p. These examples clearly show that for machine learning models gain acceptance, it is critical to be able reason why certain decision has been made to prevent any unwanted consequences.
Our code wehpage:
level XA FBBC

---

Under review as a conference paper at ICLR 2023
Attribute-wii saliency mup
Input image
Belly
Bill
Breast
7
Class-wise saliency map by Grad-CAM
Salient aftributes by multilevel XAI:
Striped belly
Rufous bill
Grey breast
Figure I: Explainabilityy of the proposed multilevel XAI model. bird image (middle) from Least Auklet class is predicted correctly by approach, with numan-like multilevel explanations via salient attributes (e.g. "striped belly") and the corresponding attribute-wiss wise saliency maps (right). Result by Grad-CAM (Selvaraju al. 2017) (left) is also given for comparison.
Explanations delivered by explainable AI (XAI) can help machine learning practitioners debug their models by for example investigating the misclassification cases (Adadi & Berrada 2018) and de- tecting bias in data (Tan et al. 2017). There have been a number of works in this context to reveal the reasoning of the black-box models (Simonyan al. 2014 [Springenherg al. 2014 Zhou al. 2016 Chattopadhay et al. 2018 Petsiuk et al. 2018 Ribeiro et al. 2016 2018). However, the most widely used techniques, creating class-wise saliency maps (e.g. see left of Figure I to indicate the areas that contribute to the prediction the most, have severe innate limitations. The first is the val- idation process of these maps, which is mostly qualitative or requires labour intensive object-wise annotations (Goebel et al. 2018 Park et al. 2018). A recent study in (Bearman et al. (016)) showed that a full supervision of object segmentation by humans takes around 78 seconds per instance while higher eиror rate bounding boxes take 10 seconds per instance to produce, which
much more expensive than second per instance image level annotations. Morcover, requiring a higher level of annotation by experts is rather impractical. Another limitation stems from the discrepancy between these maps and human-like explanations. Humans naturally explain their reasoning using discrimi- native words (e.g. domestic vs wild weak strong differentiate cat from lion) together with pointing to where those words lie in the given image if visually permitting (Park et al. 2018 Goebel 2018) (cf. our results on the right of Figure M. To produce human-like explanations, this multilevel (i.e., visual and linguistic) manner is crucial, which also inspires the work in this article.
In this article we propose new methodology called nultilevel XAI to delve into DNNs by leveraging visual and linguistic attributes. Our approach exploits per-class attributes (rather than per-image attributes, which are too expensive and generally impractical) to interpret DNNs e.g. classifying raw images. By creating multilevel explanations, i.e., linguistic salient attributes and attribute- wise saliency maps, our method can achieve towards human- explanations (e.g. see right of Figure D. This is big step forward in XAI and this new methodology does not suffer from the above- mentioned limitations existing current XAI solutions. The proposed setting adds a tiny extra cost to the training set, i.e., per-class attributes. which can be easily obtained if needed using for example online search engines or some autonomous tools (c.g. GPT-3 API Brown et al. 2020), and once acquired they can always be in use since in most cases they time and image invariant.
Our main contributions lie in: i) proposing a multilevel XAI methodology which is easy to use and can achieve towards human-like explanations; ii) implementing extensive experiments both coarse-grained and fine-grained datasets to validate the performance of the proposed approach; and iii) conducting insightful discussions in XAI and future paths.
2 METHODOLOGY OF MULTILEVEL XAI
In this section, we introduce our multilevel XAI methodology, see Figure for its main architecture. It consists of three main components: i) pre-trained feature extraction block generating high level image features from input images (left of Figure : ii) explanapl DNN block bridging the extracted features with linguistic attributes (middle of Figure 2 and iii) language model block
2

---

Under review as a conference paper at ICLR 2023
(being frozen after training) linking the linguistic attributes to the output class labels (right of Figure 2). All of these blocks are important and are well studied in various fields, yet in the XAI regime, their study is rather limited. To the best of our knowledge, this is the first time they have been used to explain neural networks in a multilevel (i.e., visual and linguistic) manner particularly when the per-image attributes are unavailable. Further description is given below.
Feature Extraction
Proposed Architecture
Attributes: y
π
Classes: y
oooo
vb
X
X
MLP/CNN
MLPL
ux
Input
Features
Longuage Model
Figure 2: Multilevel XAT architecture. Image features & are extracted from images X' using feature extraction model vb. Class labels y embedded into class attributes y using language model ø DNN क (e.g. MLP and CNN) is then trained match X' with V. The explainability of DNN is by the obtained salient attributes ux, (linguistic) and attribute-wise saliency maps X (visual).
Preliminary. Let X' be the set of images and y 21,2. ... be the set of C' class labels. Let = {(X,, 3/) X; E X. Yi E V.i = where Yi is the ground truth label of image X; TR. xM with M3 for grey and colour images. Image classification is to learn a function say π : X' -> under a loss y I such that the energy Ex.xX:.. P(y,, T (Xx)) ) minimised.
a training set with N image/label pairs,
and 3 respectively
function p
Given deep neural architecture, the main aim is to find proper weights, say W, for all layers, such that (X;; W) is able to predict the label of v E X accurately through training. If the quality of S is perfect, it is highly likely that X: W) can perform excellently after training by minimising
Σ
ply K (X,, W)).
(X.B)JE
However, it is generally impossible acquire ideal set S, plus the reason why the trained T (X; W) works lacks interpretability, i.e., its so-called black-box nature. This motivated many XAI methods (e.g. Grad-CAM Selvaraju et 2017 with its result Figure D to peek inside this kind of black box.
To improve the performance of its prediction, more complicated models are proposed, e.g. the way of exploiting image- and label-embeddings. Let X' and y respectively be the sets of image features and linguistic class attributes (which can be generalised to other sources of side informa- tion such as taxonomy of classes (Tsochantaridis et al.) 2005)). ). The image- and label-embeddings
are to find proper embedding functions v : - X' and Φ y -> ), respectively. Then the image classification task can be generalised to learn function क X y such that the energy is minimised together with z and é. Note that function क is general. An appropriate form of it may be found by exploiting compatibility functions used by the zero-shot learning community (Akata et al., 2015); see e.g. Frome et al. (2013): Reed et al. (2016); Romera- Paredes Tom (2015) with their own pros and cons.
Ex,xx P()(y:),
In this work are interested in studying neural architecture for न (i.c., DNN 7)) with its weights, W, via minimising the energy
Σ (ES
(((()). W)).
The choice of &b and ő is wide and can also be represented using neural architectures in deep learning. Our proposed architecture in Figure i is to reveal the explainability for utilising language models (for () in multilevel manner.
3

---

Under review as a conference paper at ICLR 2023
Feature extraction. Within our framework there is freedom to choose the feature extraction mod- els v (see the left of Figure 2 To illustrate this flexibility we have used both pre-trained ResNet101 (He et al. 2016 and V (Simonyan & Zisserman 2014) for visual feature extraction.
2.1
CLASS EMBEDDING
A central component in our approach is the introduction of meaningful K-dimensional embedding space, y. We consider a mapping & from class y, E to an embedding vector yi Y, where each component of yj is a linguistic attribute. In practice & would be a probabilistic embedding describing the conditional probability of Ptz I y:); however, obtaining this is difficult. Instead, we start from a matrix A E IR RC** xx* provided by experts (in our case this was conveniently provided by the zero-shot learning community, see Table ! for an example [aappet et al. 2013) which can be interpreted as [ū: y1). In our approach we need the "inverse mapping", Φ (?+) ). giving IP(1: út)- We learn this mapping using a multilayer perceptron (MIP; ML.P, in our model, see the right of Figure 2 where our inputs are noisy vectors y, (i.c., the rows of matrix A) and our targets the classes Vi- This mapping is learned entirely without sceing the training images and is then frozen. Although this is rather simple approach, it is extremely fast to learn and leads to good performance.
2.2 EXPLAINABLE NEURAL NETWORKS
Below we introduce the strategies regarding how the DNN म in our proposed multilevel architecture (middle of Figure D can be explainable. Note that ग (Xx) + Φ(ni), where X; E X' and y: E y. Since o(y:) is not unique, we train स to learn the match between features and attributes in an unsupervised way (regarding 5/) using the training set S (i.e., image/label pairs), with the trained MLP1 and pre-trained v/.
XX, E X' used for test, let (+(((( ()))), i.e., the predicted class label of the test image Xj- Let i (x)())) h be the k-th attribute of मt ((((X)) where E (1. K}. We define ux, E R as the importance of the attributes for the test image X, and evaluate it by taking the gradient of the predicted class label yx, with respect to every attribute of # (xx()))). k.)),ie. i.e.,
dyx, (O (f(b(X)))
dyx,
dyx, Ox()(X)))x)
ux, (ux,; "x,,
x
(3)
Then the top K largest of {ux,, k== will be selected as the salient linguistic attributes. In this sense, 0 therefore can be explained by these salient linguistic terms. As examples, two of the most common neural networks- MLP and CNN are adopted for below.
Explainable MIP. When स represents an MLP, say #MPP consisting of few dense layers, ĀMPP can then be explained by the obtained salient linguistic terms a single level manner. We also call TMLP explainable MLP (X-MLP).
Explainable CNN. When represents a CNN, say 7oNN consisting of convolutional layers with K channels followed by global average pooling layer. 7CNN can then be explained by the obtained salient linguistic terms. Moreover, we also introduce finding out where these salient attributes are related in the given test image X, by exploiting the spatial information reservation property of the CNN structure. For the i-th attribute, i E (1,2,... K). its attribute-wic saliency map (i.e., heat map mask) say X ; can be obtained by the output of the last CNN layer after being upsampled the same size of Xj. In other words, the salient part of X, corresponding to the i-th salient attribute can be shown by x;
In contrast to #MLP, #CNN provides both the salient linguistic terms and the corresponding attribute- wise saliency maps in a multilevel manner with no extra cost. For ease of reference, we call #CNN explainable CNN (X-CNN).
EXPERIMENTS
The proposed methodology is trained and tested on coarse-grained and fine fine-grained benchmark datasets. The data description, implementation setup and results in the XAI regime given below.
4

---

Under review as a conference paper at ICLR 2023
Further details are in the Appendix; in particular, Appendixes B and C are mainly used for further implementation setup and the explainability results, respectively.
Data. Animals with Attributes (AwAl) is well-known dataset used in zero-shot learning Lampert et al. 2013). Due to the absence of raw images and copyright issues, an alternative version of named A2 was introduced in Xian al. (008)). It is medium-scale coarse-grained dataset with 37,322 images from 50 classes collected from public web sources, including 85 attributes per class available. Table | presents a size of 5x7 excerpt of AwA2 (see Appendix for the full attribute- class matrix A), exemplifying the nature between the attributes and different classes. The other benchmark dataset used in this work dataset containing around 11,800 images of 200 different bird classes, including 312 attributes per class available. These linguistic attributes will be exploited to create self-explainable DNNs under our proposed methodology.
LUB-200-2011 (Wah al., 201T). CUB is a fine- fine-grained
Table I: An excerpt of the attribute- class matrix A for the AwA2 dataset. Attribute values are in [0, 100] and are standardised before use.
Attributes
Classes
Gray
Patches
spots
Lean
Tail
Strong
Muscle
Antelope
12.34
16.11
9.19
39.99
40.59
33.56
26.14
Grizzly bear
3.75
1.25
0
0
9.38
78.48
48.89
Killer whale
1.25
68.49
32.69
22.68
41.67
63.35
10.45
Beaver
7.5
0
7.5
8.75
86.56
32.81
24.38
Dalmatian
0
37.08
100
63.68
53.75
34.93
23.75
Implementation setup. I) For the feature extraction model v. pre-trained ResNet101 and VGG16 are respectively used for datasets AwA2 and CUB. The sizes of the extracted features for each image in datasets Aw and CUB respectively 8 x 8 x 2048 and 8 x 8 512. II) The language model MLPL is a few layers wide MLP (here 3 layers are used). To train it, two training sets, T, with size of 5, 000 and 20, 000 respectively for datasets AwA2 and CUB are formed; see Appendix for the details of forming T in Algorithm| || and its accuracy performance in Table B (on dataset AwA2 where different number of attributes was changed randomly per class). MLP1. L.. including the order of the attributes, is frozen after the training completes. III) TMLP is a few layers wide MLP (here 4 layers are used) taking 2048 and 512 features extracted by v and outputting 85 and 312 attributes for datasets Aw A2 and CUB, respectively. #CNN for simplicity is set to one single convolutional layer with the size of x x 85 and 8 8 x 312 for datasets AwA2 and CUB, respectively. A 30/70 split of the data was formed for training/test.. IV) For comparison, fine-funed ResNet101 and VGG16 are obtained by directly using the training set of image/label pairs AwA2 and CUB, respectively.
The Adam optimizer with a learning rate of 0.001 and batch size of 32 is used in all experiments. The number of epochs set to 100 and early stopping is applied (with patience set to based on the validation loss). We stress that the main goal of this work is to make DNNs self-explainable rather than accuracy-driven. It is expected that the prediction performance reported could be improved e.g. with hyper-parameter fine-tuning and/or wiser selection of feature extraction model vh.
Performance in accuracy. Table 2 shows that the proposed architecture in Figure 2 can achieve surprisingly good performance (i.e., over 90% and 50% accuracy for the 50-class and 200-class datasets AwA2 and CUB, re- spectively) in classification accuracy against the fine-tuned neural networks (i.e., ResNet101 and VGG16 trained di- rectly on the labelled data, which lack explainability) on hold-out test set even though this is not the main aim of this work. The neural networks' perfor- mance in accuracy highly depends on the quality of the data acquired. However, most of the data researchers work on, if not all, could be biased, insufficient and/or sensitive. Creating architectures
Table 2: Classification accuracy. X-MLP/X-CNN can achieve comparable performance against the fine-tuned ResNet101 and VGG16, which, however, lack linguistic and visual explainability that X-MLP/X-CN -NN delivers.
Data
Model
Test Accuracy
Explainability
R0sNer101
95.8 + 1.32
N/A
AwA2
X-MLP
90.5 + 0.84
Unilevel
X-CNN
9.11
Multilevel
VGG16
57.2 1.44
N/A
CUB
X-MLP
54.9 + 1.52
Unilevel
X-CNN
44.6 1 1.08
Multilevel
5

---

Under review as a conference paper at ICLR 2023
that explain themselves and simultaneously reach high prediction performance just like the one introduced in this work is arguably the long-term termeteemmm pursuit in machine learning.
Antelope
Zebra
Horms
Group
Figure 3: Explainability of the proposed approach for correct prediction. Human- like multilevel explanations are delivered by the salient at- tributes (by X-MLP/X-CNN) and their saliency maps (by X X-CNN), which are matched well.
Stripes
Quadrupedal
ooo
Horms
Grouu
Salient attributex for explanation
3.1 PERFORMANCE EXPLAINABILITY
Explainablility for correct prediction. For a given image Least Auklet from the CUB dataset, see Figure both the fine-tuned DNN (VGG16 in our case) and our proposed method can easily classify it correctly. However, the fine-tuned DNN gives explanation on why it reaches decision by itself. Post-hoc XAI methods (such as Ribeiro et al. 2016 Selvaraju et al. 2017; Ribeiro et al. 2018) could be employed to whether the classified object the fine-tuned DNN focuses on (i.c., left of Figure D. but this level of explanation is rather limited and is incomplete reflection of human-like explanations discussed throughout the article. In contrast, the attribute-wise level of explanation the proposed multilevel XAI model delivers (i.e., right of Figure is much richer, wider, deeper and self-explainable thanks the linguistic attributes. In detail, some of the most salient attributes that affect the prediction are presented as striped belly, rufous bill and grey breast. Their corresponding saliency maps convincingly highlight the correct part of the image for the mentioned individual attributes. This type of explanation is desirable and is an important indicator of the match between image features and class-wise attributes that are learnt in an unsupervised way by the proposed architecture (unsupervised in the sense that the training images have not been labelled by linguistic attributes and/or salient regions have not been given).
whole in the given image is the main part that
White
Input image
Brown
Manipulated image
Brown
brown
gridlly bear (91%)
Polar
Grizzly
white
pauss
coastal
(828)
(16%)
brown
forest
muscle
fish
pathees
hipedal
salient altribute
Figure 4: Explainability of the proposed approach for incorrect prediction. Left: grizzly bear which is misclassified as polar bear. Top five salient attributes are shown in ellipses for the highest probable class (i.e. polar bear) and second class (i.e. grizzly bear). The most salient attributes (i.e., white and brown) and their saliency maps provide insights of the prediction. Right: A manipulated grizzly bear image (obtained by replacing the area related to the attribute "white" by a brown patch) which is then correctly classified by approach with high confidence (91%).
Figure g demonstrates the power of the proposed model in explainability with more challenging images. Linguistic self-cxplainable attributes stripes and group are outputted as salient for zebra, while guadrupedal and horns outputted for antelope X-MLP and X. X-CNN. Attribute wisb saliency maps for horn and group outputted by X-CNN show the human-like explanation power of approach. Further results provided in Appendix.
Explainabilility for incorrect prediction. Reaching 100% prediction accuracy is not the case for any method given nontrivial task. Therefore, investigating the reason behind wrong predictions is

---

Under review as a conference paper at ICLR 2023
equally important. The left of Figure shows a grizzly bear which is misclassified polar bear and, to understand the reason behind, the top five salient attributes obtained by the proposed approach for both grizzly bear and polar bear classes are presented. These attributes are indeed the ones that differentiate these two classes (cf. the full attribute-class matrix in Appendix). The attribute- wise saliency maps of the most salient attributes (i.e., white and brown) for both classes provide further insights regarding why this misclassification occurred. After occluding the part considered as "white" (by our approach) with a brown patch, the manipulated image is then correctly classified by our approach as grizzly bear with a high confidence, see the right of Figure E furthermore, the saliency map for the most salient attribute, "brown", now indeed highlights both the head of the bear and the brown patch, showing that our approach clearly learns what brown is and considers it as strong indicator of grizzly bear class. Further results are provided in Appendix.
shows
Sensitivity between attributes and features. To further in- vestigate the effectiveness of the linguistic attributes in our method in explainability, we test zebra image and its artifi- cial conversion to a horse using CycleGan (Zhu et al. 2017). i.e., the attribute stripes is removed from the zebra, see Fig- ure 5 Again, all three models (i.e., fine-tuned ResNet101, X-MLP and X-CNN) classified the zebra image as zebra and the artificially generated horse as horse At this point the fine-tuned ResNet101 has no explanation ability to show what changed in the original image that forces it to output "horse". In contrast, our model clearly shows that "stripes" is one of the salient attributes for the original zebra image with the at- tribute value of 2.15 and drops to 0.41 for the artificially generated horse image. To validate the reason behind this visually, the attribute-wise saliency maps, generated by our approach with extra cost, indicate that the X-CNN model focuses on the body of zebra where the "stripes" lie, whereas arbitrary parts of the artificially generated horse image highlighted when asked to show where the stripes are, see the bottom of Figure S
Zehra
Artificial Horse
Longleg
Stripes
Furry
2.15
0.41
Figure 5: Effectiveness of linguistic attributes in our approach. "Stripes" is one of the salient attributes for zebra with value of 2.15 and its saliency map reasonably highlights the body of zebra. For the artificial horse image, the attribute "stripes" is of value 0.41 and its saliency map is meaningfully unrelated.
Class embeddings with shuffled attribute values. Previous experiments conducted using the prior information (e.g.. the attribute-class matrix shown in Table | provided by ex- perts. An interesting and natural question is: what the results will be if the attribute-clas: matrix takes different values? In other words, to what extent, will the prior information in the attribute-class matrix be helpful in interpretability? To inves- tigate this, we firstly shuffled all the columns of the attribute- class matrices for both datasets. In extreme case, say the values of "ground" and "water" for the tiger class may be switched, which apparently would cause a dramatic information loss against the one provided by experts. The shuffled datasets are then used to train the model MLP1- Surprisingly, we found that it converged as fast as using the original data; morcover, the newly trained models X-MLP and X-CNN also reached an accuracy close the ones obtained by using the original data. Figure 6 shows the interpretability results provided by our approach in this attributes shuffling sce- nario. Obviously the linguistic attributes become meaningless; moreover, we also observe that the salient regions no longer appear to be associated with the object being recognised and defy an easy human explanation in contrast to what was observed in Figure D This finding by our approach shows that those pre-determined attribute lists are crucial to explainability. It also suggests that purely relying on the accuracy of DNNs (which might be trained on data with unknown flaws) could be perilous and the corresponding interpretability is essential. Further discussion is in Section
4 DISCUSSION AND LIMITATIONS
To the best of our knowledge, the proposed approach is totally novel to XAL It also raises number of questions that are not commonly addressed in this context. Many of these open research questions we believe are important to the further development of XAI. Some of these are outlined below.
7

---

Under review as a conference paper at ICLR 2023
Attribulewwise saliency map
Figure Explainability of the proposed approach in the scenario of attributes shuffling (ef. Figure D Model accu- racy reaches to the one using the true linguistic attribute val- ues, whereas the attribute-wise saliency maps are meaningless to humans, illustrating the im- portance of prior knowledge of the attribute values.
Input image
Belly
Bill
Breast
Salient attributes by multilevel XAI:
Striped belfy
Raflous bill
Grey breast
Linguistic alignment. By using linguistic prior of what classes are associated linguistic at- tribute, we encourage the network to find visual features that correlate with the linguistic attribute something we call "linguistic alignment", Because we are learning this correlation in an unsuper- vised manner, we are not guaranteed that the visual feature aligns correctly with the linguistic label. This is likely to improve if we were to use more classes. On small database of animals dominated by undulates and fish, it would be easy to confuse the linguistic term "hooves" with "legs". Such confusion is much less likely if the dataset contained other animals, such as dogs or primates.
The linguistic alignment is complicated as a DNN is likely to assess circumstantial or contextual evidence for the existence of an attribute. As hooves are highly correlated with legs, it would not be surprising if the legs were considered highly salient to the presence of hooves in an image. This appears to be common in many of the attributes, where saliency maps appear to take a much larger area than the feature described by the linguistic attribute. At some level, this clearly makes sense. A hoof-like object that is not attached to a leg is unlikely to be a hoof. Similarly, where a hoof occluded, there may be enough context infer that the animal is very likely to be hooved. However, this contextual evidence reduces the linguistic alignment. This is a feature of explainability rather than fault of our approach. However, an important line of research XAI is to separate circumstantial and contextual evidence from direct evidence.
The nature of explainability. Explanations for classifications are not unique. This was shown when we randomised the elements in the matrix, A, between classes and linguistic attributes. In doing seems highly unlikely that any attribute has a simple linguistic description. Yet, we can train our model with these random attributes and still obtain classification levels of around 90%. This seems at first sight counter-intuitive, although given that we have 85 continuous features they have the potential to carry sufficient information separate the classes with high accuracy. What separates, at least, some of the true linguistic attributes from other attributes is the information content of the linguistic attributes; that is the linguistic attributes that have high mutual information in regard to the classes. However, some of these attributes are hard for a DNN to learn.
What attributes DNNs learn. We have shown examples of attributes such "rufous bill" that appear to be well captured by the networks. However, through using an independent test set we find that some of the linguistic attributes appear not have been learned by the network. An example of this is "big", clearly an attribute that would be useful for humans to distinguish an elephant from a squirrel. Because of the nature of the training set, where objects tend be resized fill most of the image, this turns out to carry little information about the classes (see Appendix for more details). However, the lack of success of these attributes very informative regarding how DNNs perform discriminative task. All the linguistic attributes used in the data are chosen by experts because linguistically they carry considerable mutual information about the classes. The failure of DNNs to exploit some of these terms obtained by approach conveys important insights that directly address the issue of what information a neural network is actually learning a core concern of XAI.
Tangible abstract attributes. The linguistic attributes used in this paper (that we inherited from the zero learning community) interestingly incorporate both tangible and abstract attributes. For the tangible attributes such as "homn" we would expect the corresponding saliency map to highlight horn (although as we have argued may highlight areas that are important contextual clues to the presence of horn). The more abstract attributes such "domestie" or "fast" are less easily
8

---

Under review as a conference paper at ICLR 2023
attributed to a particular area of the image. They may however be highly informative for example in differentiating between cat and lion. When these attributes are informative then it is clearly important to understand whereabouts in the image these attribute inferred. Again our approach goes some way towards addressing this issue.
Atomic and compound attributes. In our approach, we have treated all attributes as atomic. Con- sequently, pointy, fluffy and large ears would all be treated as separate linguistic attributes. However, each attribute would correspond the same area of the image, and in many cases it seems more nat- ural to treat attributes as compound entities. We have not attempted to do this, but if we wish to scale up our approach larger datasets, this seems to us to be an important area of future research.
5 RELATED WORK AND CONCLUSION
The complexity of machine learning models generally affects the transporcecyy/exllianniiiy be- cause of the difficulty of following the model prediction process (Adadi & Berrada, 2018). One line of research is where researchers employ inherently explainable models and utilise white-box mod- els such as bayesian rules (Letham et al. 2013) and linear models (Ustun & Rudin 2016) to handle complex problems. These models, generally, struggle to reach the prediction ability of DNNs.
Methodologies in the XAI field mainly aim to propose methods to understand how high performance black-box machine/deep learning models work. The majority of XAI methods introduced ideas to explain pre-trained models in a post-hoc manner, i.c., they are neither interested in the training set- ting nor in changing any of the models' components. These methods could be model-independent requiring only prediction function (Ribeiro et al.) 2016, 2018 or model-dependent that need addi- tional information of the trained model such as feature maps at certain layer (Zoou et al., 2016) or gradients (Selvaraju et al., 2017p. DNNs for visual tasks do not output any textual justification. Modern visual-language models criminative features that cause the prediction (Goebel et al. 2018). Forcing these models to output more discriminative features is one related work proposed in Park et al. (018). It aims to out- put multilevel explanations for vision-language tasks, e.g., visual question answering and activity recognition. Apart from a completely different focus against the work in this article, this method also requires labour intensive per-image annotations during training that are avoided in our work.
effective in describing image content but lack outputting dis-
The rero-shot learning regime is where side information (e-g. attributes and class taxonomies) ex- ploited to classify images of classes that have no labelled samples during training (Xian et al.. 2018) The aim is to match image features with class attributes and then to classify unseen classes thanks to the prior side information. There are various of techniques to find the best match that allows unseen
class predictions (Frome et al. 2013 Palatucci et al. 2009; Akata et al. 2015: Romera-Pareee Paredes & Tom 2015). Although we integrate side information into our training process similar to zero-shot learning we are not interested in unseen classes; instead, the proposed work aims to train self- explainable models in many-shot case. Unlike the majority of XAI methods, our explanations multilevel outputting both linguistic and visual explanations. Finally, different from other extremely limited number of multilevel attempts that specifically work on vision-language models, our training setting is significamtly cheaper and does not require per-image annotations.
Concept bottleneck models Kohh et al. 2020) share ideas with our work and are analogous to X- MLP, but neither of them is multilevel. In particular, unlike the concept bottleneck models, our X- MLP and X-CNN only require class-wise attributes, ensuring our approach is significantly cheaper to implement. Moreover, our main focus is the X- X-CNN, which possesses huge advantages (e.g. see Figure D to distinguish our work from the concept bottleneck models.
High performance DNNs are highly desirable when they can reason about their decisions. We presented new XAI methodology with self-explainable models delivering human-like multilevel explanations alongside the class probabilities. Explaining why certain prediction is made using linguistic terms and attribute-wise saliency maps without requiring per-image ground-truth expla- nations in the training phase makes the proposed technique efficient and inexpensive. The results in explainability demonstrated by the match between image features and class embeddings greatly empower the explainability of DNNs while preserving their prediction ability Given the importance of XAI and the power of the newly introduced approach, we believe this could spark new avenues in XAI and shed light on developing and applying Al in more sensible ways.
reasonable level.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Amina Adadi and Mohammed Berrada. Peeking Inside the Black-box: a Survey on Explainable Artificial Intelligence (XAI). IEEE Access, 6:52138-521 2018.
Zeynep Akata, Florent Perronnin, Zaid Harchaoui, and Cordelia Schmid. Label-embedding for image classification. IEEE transactions on pattern analysis and machine intelligence, 38(7):
1425-1438, 2015.
Amy Bearman, Olga Russakovsky, Vittorio Ferrari, and Li Fei-Fei. What's the point: Semantic segmentation with point supervision. In European conference on computer vision, pp. 549-565. Springer, 2016.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot leamerr. Advances in neural information processing systems, 33:1877-1901, 2020.
Aditya Chattopadhay, Anirban Sarkar, Prantik Howlader, and Vineeth N Balasubramanian. Grad-
cam++: Generalized gradient- -assdd visual explanations for deep convolutional networks. In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 839-847. IEEE, 2.18.
Andrea Frome, Greg S Corrado, Jon Shlens, Samy Bengio, Jeff Dean, Marc' Aurelio Ranzato, and Tomas Mikolov. Devise: A deep Aisual-semantic embedding model. Advances in neural infor- mation processing systems, 26, 2013.
Leilani H Gilpin, David Bau, Ben Z Yuan, Ayesha Bajwa, Michael Specter, and Lalana Kagal. Explaining explanations: An overview interpretability machine learning. In 2018 IEEE 5th International Conference data science and advanced analytics (DSAA), pp. 80-89. IEEE, 2018.
Randy Goebel, Ajay Chander, Katharina Holzinger, Freddy Lecue, Zeynep Akata, Simone Stumpf, Peter Kieseberg, and Andreas Holzinger. Explainable Al: the new 42? In International cross- domain conference for machine learning and knowledge extraction, pp. 395-303. Springer, 2018.
Hani Hagras. Toward humaa--ndeessanddable explainable AI. Computer, 51(9):28-36 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. In Proceedings the the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
Will Douglas Heaven. Hundreds of AI tools have been built to catch covid. None of them helped, 2021.
Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept Bottleneck Models. In International Conference on Machine Learning, pp- PMLR, 2020.
Christoph H Lampert, Hannes Nickisch, and Stefan Harmeling. Attribute-based classification for zero-shot visual object categorization. IEEE transactions on pattern analysis and machine intel- ligence, 36(3):453-465 2013.
Benjamin Letham, Cynthia Rudin, Tyler H McCormick, and David Madigan. Interpretable classi- fiers using rules and bayesian analysis: Building better stroke prediction model. The Annals of Applied Statistics, 93): 2015.
Zachary C Lipton. The Mythos of Model Interpretability: In machine learning, the concept of interpretability is both important and slippery. Quene, 16(3):31-57, 2018.
Octavio Ceyola-Gonzale... box woite-box: Understanding their advantages and weak- nesses from practical point of view. IEEE Access, 7:15599--544111 2019.
Phil McCausland. Self-driving Uber car that hit and killed woman did not recog-
nize that pedestrians jaywalk.
self-driving-ubes car waaa-hittte hat.:-iiiieioooaad-------------- 2019.
10

---

Under review as a conference paper at ICLR 2023
Thor Olavsrud. 7 famous analytics and Al disasters.
https: www. cic. com artmme/ttelee////o 2022.
190888/ 5-
Mark Palatucci, Dean Pomerleau, Geoffrey Hinton, and Tom M Mitchell. Zero-shot learning with semantic output codes. Advances in neural information processing systems, 22, 2009.
Dong Huk Park, Lisa Anne Hendricks, Zeynep Akata, Anna Rohrbach, Bernt Schicle, Trevor Dar- rell, and Marcus Rohrbach. Multimodal explanations: Justifying decisions and pointing to the evidence. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1779-8788 2018.
Vitali Petsiuk, Abir Das, and Kate Saenko. Rise: Randomized input sampling for explanation of black-box models. arXiv preprint arXiv:1806.07421. 2018.
Scott Reed, Zeynep Akata, Honglak Lee, and Bernt Schiele. Learning deep representations of fine-grained visual descriptions. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 49-58, 2016.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Why should i trust you?" Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference 2016.
on Knowledge Discovery and Data Mining, pp.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Anchors: High-precision model-agnostic explanations. In Proceedings of the AAAI Conference on Artificial Intelligence, 2018.
Michael Roberts, Derek Driggs, Matthew Thorpe, Julian Gilbey, Michael Yeung, Stephan Ursprung, Angelica I Rivver, Christian Etmann, Cathal McCague, Lucian Beer, et al. Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID- 19 using chest radiographs and CT scans. Nature Machine Intelligence, 3(3):199-21) 2021.
Bernardino Komera-Paredes and Philip Torr. An embarrassingly simple approach zero-shot learn- ing. In International conference on machine learning, pp. 2152-216 21611 PMLR, 2015.
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-CAM: Visual Explanations From Deep Networks via Gradient-Banl Based Localization. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Oct 2017.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1409. 1556, 2014.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. In Workshop at International Confer- ence on Learning Representations. Citeseer, 2014.
Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. Striving for simplicity: The all convolutional net. arXiv preprint arXiv: 142.6666, 2014.
Sarah Tan, Rich Caruana, Giles Hooker, and Yin Lou. Detecting bias in black-box models using transparent model distillation. arXiv preprint arXiv:1710.061662 2017.
Ioannis I'sochantaridis, Thorsten Joachims, Thomas Hofmann, Yasemin Altun, and Yoram Singer. Large margin methods for structured and interdependent output variables. Journal of machine learning research, 6(9), 2005.
Berk Ustun and Cynthia Rudin. Supersparse linear integer models for optimized medical scoring systems. Machine Learning, 102(3):349-99) 2016.
Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-uesd birds-200-2011 dataset. 2011.
Yongqin Xian, Christoph H Lampert, Bernt Schiele, and Zeynep Akata. Zero-shot learning com- prehensive evaluation of the good, the bad and the ugly. IEEE transactions on pattern analysis and machine intelligence, 2119):2251-2265, 2018.

---

Under review as a conference paper at ICLR 2023
Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In European Conference on Computer Vision, pp. 818- 8333.. Springer, 2014.
Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pp. 2921-2929, 2016.
Jun- Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE international conference on computer vision, pp. 2223-2232, 2017.
12


Reviews:
[{"review_id": "udzbb0J2eL", "reviewer": "Reviewer_4txy", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose to learn explanations for the predictions from a neural network by employing a multilevel explanation approach. More specifically, the authors propose to leverage attributes for specific images and map the coarse class labels to the fine-grained object attributes during training. They show that this leads to human-like explanations while achieving nearly similar prediction accuracy. The perform experiments on benchmark datasets and show improved explanation ability of the model compared to state of the art.", "summary_of_the_review": "The paper is well motivated but the contributions for explainability are marginal and does not bring any significant improvements to the already existing research or outcomes in the field of explainability. ", "strength_and_weaknesses": "The paper looks at the important problem of explanation in deep neural networks which are of inevitable importance. Their motivation of defining what is learned by a neural network by breaking down the learning process from coarse categories to fine grained attributes is interesting, intuitive and well supported experimentally.\n\nDespite the interesting approach, there is a limitation as the results and the method are only applied for easy datasets / classes and will be very hard to scale for imagenet classes for instance. What will be the approach to scale for instance the attributes and how would the method perform for thousands of dog breeds in imagenet for example ? \n\nWhat is the loss function that is used to match the feature and attributes obtained from zero shot literature ? \n\nIt is not very clear what the difference is between the single level (Explainable MLP) versus the multilevel Explainable CNN. How’s explaining using attributes (whether with or without saliency maps) a multilevel approach ? \n\nIs there any way to measure the attribute prediction accuracy ? \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "s6i1nK7Tgv", "reviewer": "Reviewer_iU9V", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a new explainable neural network architecture called Multilevel XAI architecture. The key contribution of the work is to use both visual and language-guided interpretability, where a list of attributes pertaining to a class prediction (e.g., Dog --is furry, ...). Experiments show the model's ability to ground salient attributes in images in addition to providing region-based saliency maps. ", "summary_of_the_review": "I am leaning toward recommending the rejection of this paper. While there certainly are interesting tidbits, the biggest gripe is that the proposed model requires both class label and attribute annotation per class; If we allow other methods to have access to the same (or similar) type of data, they could (and do) produce similar \"interpretable\" visualizations as shown in the paper. ", "strength_and_weaknesses": "**Strengths:**\n\n*[S1:]* Interesting results: The results obtained are certainly interesting; The per-attribute visualization can indeed shed greater light on the decisions of the neural networks compared to \"class-level\" heat maps. \n\n\n**Weaknesses:**\n\n*[W1]:* Inadequate baselines, comparisons, and discussion: The proposed visualizations are interesting, but a huge swath of work in vision and language that already demonstrate very similar ability, or can be trivially extended to produce similar visualization and/or linguistic explainability (E.g., concept bottleneck models https://proceedings.mlr.press/v119/koh20a.html). This is woefully missing in the paper and it is very hard to contextualize why the proposed method is more explainable than any number of other works. I will list a few ways this could be done, but the main issue is that it already feels trivial to do what the paper does with existing tools. What new benefits does the proposed work bring?\n\nW1.1. Visualization of existing V&L models: Almost all of the recently introduced VL pretraining methods can ground image regions for a given word in a caption. (E>g., https://github.com/salesforce/ALBEF). Some methods even work for new objects as long as they are adequately described with attributes (A llama, which is a white tall animal in the picture (e.g., https://github.com/microsoft/GLIP)). Going back, even more, most of the object-based representations are also trained to predict object attributes (https://github.com/peteanderson80/bottom-up-attention). The paper makes a distinction between \"class-based attributes\" and \"image-based attributes\" but I see no distinction when the attributes are describing various objects; For a \"dog\" isn't the \"class-based attributes\" simply a union of all attributes contained for \"dog\"? (E.g., https://vawdataset.com/). Almost all methods that work with objects and attributes can visualize their affinity in various image regions to some degree. How does this compare with the proposed method?\n\nW1.2 Trivial modification of existing work (E.g., Grad-CAM) to use additional information used by the paper: The paper makes use of additional attributes to obtain the given visualizations; However, if any network was modified to predict these attributes in addition to the class label, Grad-CAM visualization for making the prediction \"has beak\", could be used in exactly the same way the paper is using.\n\nWithout a comparison, discussion, or adequate comparative analysis with these, the impacts of the paper are hard to justify.\n\n*[W2]:* The effect of components used is not explained: Firstly, the method requires additional annotations. While the paper does mention that may be easy to obtain (e.g., via large language models such as GPT-3), there are no experiments showing whether a lower-quality (noisy) attribute annotations would still work. Similarly, the training of MLP_L is not described well. How sensitive is it? What happens when only a small portion of the attributes are present? What happens when attributes (y_hat) is not predictive of class y? Since having high-quality attribute data seem to be the central requirement \n\n\n*[W3]:* Lack of Clarity: The paper feels obfuscated and very hard to follow. However, upon closer inspection, the core concepts are fairly simple. I would urge the authors to streamline the presentation and explain in terms of their function rather than hiding details unnecessarily under hard-to-follow conventions. For example, in Section 2.2 paragraph 2, the first sentence is literally just describing the process to generate predictions from a neural network. It is entirely unnecessary to write that as:\n\n> 8Xj 2 X used for test, let yXj = †(˜⇡( (Xj ))), i.e., the predicted class label of the test image Xj. (Not rendered properly here -- please see pdf.)\n\nThe second part of the sentence beginning with i.e. describes the same thing as the stuff before that. This is simply an example in what is an unnecessarily dense symbol used to describe fairly simple underlying processes in Section 2 (Methodology).  \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "KXwmbFhowk", "reviewer": "Reviewer_Snsk", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a visual and linguistic bounded explanation method to make part of NN models explainable by adding both attribute-wise and language-wise explanations. Specifically, they add a trainable part between the feature extractor (e.g., pre-trained Resnet) and label embedding, and the added part can be explained. The paper is easy to follow and clear.", "summary_of_the_review": "Overall, using a multilevel explanation is interesting, while the reviewer thinks the weakness outweighs the strength. ", "strength_and_weaknesses": "**Strength**\n\n1 Using multilevel (visual and linguistic) to conduct more intuitive explanations is an interesting and promising idea.\n\n2 The author shows an easy-to-follow method and results.\n\n**Weaknesses** \n\n1 Generalization. The proposed method could only be used to explain a specific kind of NN, which learn a mapping starting from an extracted feature to an attribute-related label embedding. It is hard to explain general pre-trained neural networks (e.g., ResNet, GNN, Transformers). \n\n2 Partial explainable. The whole model has three parts, feature extractor, added learnable mapping, and attribute to label mapping; only the middle part can be explained, and the best part still performs as a black box.\n\n3 Fidelity issue. The method treats not the true label Y but an attribute set as a target during training. How to guarantee the GPT-3 described attributes are useful features for prediction? If you directly train a model using true label Y, they use different features during the decision. So the explained attributes do friendly to human understanding but may not follow the original model's logic.\n\n4 Need to improve accuracy results. The final accuracy is lower and has a relatively large gap compared with the original model, even though the proposed method uses more learnable parameters. These results also aligned with point 3 above. The newly learned explainable mapping does not use the same logic as the original model. \n\n5 Accumulated bias. There may be some bias in the feature extractor model (Resnet) and GPT-3. The new trained mapping can not remove those biases if they are input and target.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject