SlowGuess/NeurIPS_2023_OCR huggingface dataset:
1. In Figure 10a, our model correctly predicts "Crosswalk". We see that the two concepts assigned highest positive value in the sparse code by IP-OMP are "divided by lines or curbs" and "usually found near an airport". However the latter is not really discriminative for a crosswalk since they are found everywhere, and not necessarily at an airport. This is correctly reflected in the weights of the linear classifier, which assign a much higher weight to the concept "divided by lines or curbs" but a relatively low (the lowest positive weight out of all the 11 concepts in the sparse code) to "usually found near an airport". Negative concepts like "extensive grounds and gardens", "a tile floor", "often found on barns", and "found floating in the ocean" are strong indicators that the scene in the image is not of a crosswalk.
2. In Figure 10b, our model correctly predicts "Wet Bar". Interestingly, the two most important concepts (largest classifier weights) are "cabinets and counter-tops" and "a dishwashing area". The main distinguishing feature between a dry bar and a wet bar is that a wet bar has a washing sink. Other positively identified concepts by IP-OMP are "brochures", "resort" and "a protruding ledge of shelf". However, these are not discriminative for the class and are accordingly assigned low weights by the classifier. For instance, this could be an image of a wet bar in a resort, however wet bars are also present in places that are not a resort.
3. In Figure 10c, our model correctly predicts "Baseball Field". The prediction is explained by the present of positive concepts like "an outfielder", "white bases at the corners" and "a cleat" (which is a type of baseball shoe). IP-OMP also identifies "action" as a positive concept; however, that is very generic and is not specific to a baseball field. Accordingly, the linear classifier assigns the smallest positive weight to it. Like previous examples, many of the negative concepts in the sparse code are strongly negatively correlated with a baseball
field. These are "a producer", "a light, airy feel", "a large, flat expanse of snow", "cars being worked on", and "often found near a beach".
4. In Figure 10d, our model incorrectly predicts "Operating Room". The true label is "Clean Room" which is what an operating room is also called [82]. Thus, we believe this is a perfectly reasonable classification given the positive concepts extracted by the sparse code are present in the image and are salient features of an operating room. These are namely "medical equipment", "a person working", "sterile surfaces", and "a prep area".
5. In Figure 10e, our model correctly predicts "Arena, Hockey". The positive concepts found by IP-OMP are "pucks", "pads", "a scorpion" (the Hanover Scorpions and the New Mexico Scorpions are hockey teams), "a Zamboni" (a machine used in ice skating rinks to clean the surface) and "state" (this is perhaps one concept whose reason for positive attribution in the sparse code and the classifier weights isn't clear).
E.3 CIFAR-10
For this dataset, we set  $\tau = 9$ , that is, the sparse code  $\hat{\beta}_{\mathrm{IP-OMP}}$  for every image has support of size 9. A linear classifier trained on these sparse codes gets about  $91.6\%$  accuracy, which is within 2.4 points of what can be achieved if we run the IP-OMP algorithm long enough. This can be seen from Figure 5, where the test accuracy vs. explanation length curve for CIFAR-10 saturates at about  $94\%$  accuracy for relatively large  $\tau$  (say  $\tau = 30$ ). Results are provided in Figure 11. Unlike ImageNet and Places365, the sparse code is noisier (incorrectly identified positive concepts) for this dataset and CIFAR-100 since these images are much lower resolution and hence out-of-distribution with respect to the distribution of natural images on which CLIP is trained. We do not describe the explanations for each image case-by-case as in the previous two datasets since they are all of the same nature as the previous two datasets. However, we mention some key points. In Figure 11b, IP-OMP assigns a relatively large coefficient to the concept "a large size". However, frogs are typically considered small animals and this is reflected in the classifier weights which assigns the second lowest positive weight (after "reddish-brown coat") to it. In Figure 11c, "antlers", which is a defining feature of a deer, is the most important positively attributed concept, both in the sparse code and in the classifier weights. Similarly, in the same figure, IP-OMP assigns a positive weight to "a halter" which is a headgear for animals like horses. This is probably due to the fact that at such low resolution, the deer looks similar to a horse. However, the linear classifier assigns a relatively negligible weight to this concept indicating that it corrected for this noisy code provided by IP-OMP. Similarly, in Figure 11d, IP-OMP incorrectly assigns a positive weight to "quadruped" which is clearly not what a truck is, but the classifier assigns a negligible weight on that concept, indicating that the "quadruped" concept is not used in its prediction for a "Truck". Finally, in Figure 11e, the concept "a beetle" most likely refers to the car model which looks similar to the car displayed in the picture and not the insect.
E.4 CIFAR-100
For this dataset, we set  $\tau = 9$ , that is, the sparse code  $\hat{\beta}_{\mathrm{IP-OMP}}$  for every image has support of size 11. A linear classifier trained on these sparse codes gets about  $64\%$  accuracy, which is within 7 points of what can be achieved if we run the IP-OMP algorithm long enough. This can be seen from Figure 5, where the test accuracy vs. explanation length curve for CIFAR-100 saturates at about  $71\%$  accuracy for relatively large  $\tau$  (say  $\tau = 30$ ). Results in Figure 12. As explained in §E.3, since the images in CIFAR-100 are of a low resolution (blurry), the IP-OMP sparse codes sometimes make errors by assigning a positive coefficient to a concept that is not related to the object in the image. For example, in Figure 12a "two large, compound eyes" are assigned a positive coefficient in the sparse code, which is unrelated to the class "Forest". Notice the corresponding classifier assigns a negative weight to this concept. In Figure 12c, IP-OMP assigns a positive weight to "baby possums" which is a negative concept for the label of the image, "Rose". Similarly, in Figure 12e, IP-OMP assigns a positive weight to concepts "greenish-brown color" and "a small, narrow neck"; both are incorrect concepts for label "Keyboard". The corresponding classifier assigns a negative and negligibly (the smallest among all concepts in the sparse code) positive weight to them respectively, indicating they are not very important for the classifier's prediction of a Keyboard with 0.87 probability. Figure 12d illustrates an example in which the prediction is completely wrong. The correct label is "Cockroach". The classifier predicts a trout (which is a type of fish) since the sparse code has a relatively large positive weight on "aquatic animal", probably because the blue background looks similar to an ocean
at such low resolutions (the CIFAR-100 images are originally of  $32 \times 32$  resolution, and then upscaled to  $224 \times 224$  resolution). We do not describe the explanations for each image case-by-case as in the previous two datasets since they are all of the same nature as discussed for Imagenet and Places365.
E.5 CUB-200
For this dataset, we set  $\tau = 10$ , that is, the sparse code  $\hat{\beta}_{\mathrm{IP-OMP}}$  for every image has support of size 10. A linear classifier trained on these sparse codes gets about  $43\%$  accuracy, which is within 32 points of what can be achieved if we run the IP-OMP algorithm long enough. More precisely, a relatively large  $\tau$  (say  $\tau = 155$ ) is required on this dataset to achieve an accuracy of about  $75\%$ , which is the accuracy that is obtained if all the atoms in our dictionary were selected, and a linear classifier was trained on that sparse code (these sparse codes are not really sparse since their support size is 208, the size of the dictionary). Results in Figure 13. We do not describe the explanations for each image case-by-case as in the previous two datasets since they are all of the same nature as the previous two datasets. In Figure 13b, the correct prediction is a "Yellow Breasted Chat" which visually looks very similar to a chestnut-sided warbler, which is shown in Figure 8). Both birds have a greenish-yellow back,
a small, sparrow-like body, a white body with gray wings, and black cap and white eyeline. As a result, we conjecture that more concepts are needed to discriminate between the two.
Figure 8: A chestnut-sided warbler, which is the incorrect class that CLIP-IP-OMP predicts for the yellow-breasted chat in Figure 13b.
E.6 Global Explanations for CLIP-IP-OMP
The examples provided so far were for explaining individual predictions made in terms of the coefficients of the sparse code and the corresponding weights of a linear classifier training on top of these sparse codes. In this subsection we show that one can get a global understanding of the key concepts used in predicting different classes by averaging over the sparse codes generated for that class. Specifically, we consider all test images belonging to a particular class, compute the  $\tau$ -length sparse codes for all these images. Consequently, we average over these sparse codes to get the mean coefficient for every atom (averaged over all test images belonging to that class). We show these results for three randomly chosen classes from the ImageNet dataset in Figure 14. We see that the top concepts (especially positive ones) for each class are salient features of the respective classes. For example, "Volcano"s are classified by "cone-shaped mountain", "geological phenomenon", "eruption", "crater", "aflame" and so on. "Peru" and "the Andes" are locations where several volcanic mountains are located. Similarly, the top positive concepts (in terms of their mean coefficient value) for "Tiger" is "striped fur", "prey animal", "savannah" (a habitat where tigers are found), "orange and white stripes" and "a fierce expression".
Figure 9: Example runs from the ImageNet validation set. In each row, we display the test image in the first column, the dictionary atoms with non-zero coefficients in the sparse code along with the value of the coefficients in the second column, and finally the weights of the trained linear classifier in the third column. Below every image we note the predicted class. The prediction probability reported is obtained by taking a softmax over the linear classifiers output. The colour scheme is as follows. In column 1, if the prediction is correct it is reported in green, if it is wrong it is reported in red. In column 2, green bars are for positive coefficients and red bars are for negative coefficients. In column 3, we use the same color for the bar as in column 2. For example, if a coefficient was positive, its corresponding weight in column 3 is shown in green regardless of whether the weight was positive or negative.
(e)
Prediction: Arena, Hockey with 0.86 probability
Figure 10: Example runs from the Places365 small validation set. Refer to Figure 9 for a description of the different columns in each row.
(e)
Prediction: Automobile with 0.98 probability
Figure 11: Example runs from the CIFAR-10 test set. Refer to Figure 9 for a description of the different columns in each row.
(b)
Prediction: Lion with 0.97 probability
(c)
Prediction: Rose with 0.58 probability
(d)
Prediction: Trout with 0.40 probability
(e)
Prediction: Keyboard with 0.87 probability
Figure 12: Example runs from the CIFAR-100 test set. Refer to Figure 9 for a description of the different columns in each row.
Prediction: Chestnut Sided Warbler with 0.5 probability
(c)
(d)
(e)
Figure 13: Example runs from the CUB-200 test set. Refer to Figure 9 for a description of the different columns in each row.
Figure 14: Top 15 concepts identified by CLIP-IP-OMP for different classes from the Imagenet dataset. Each bar shows the average value of coefficients for that concept, averaged over all test images from the same class.