SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We evaluate our hypothesis by measuring the Oracle Impurity Score (OIS) [10] of concept representations learnt by CBMs, CEMs, and IntCEMs across all tasks. This score, between 0 and 1, measures how much extra information, on average, each learnt concept representation captures from other possibly unrelated concepts (with higher scores representing higher impurities and, therefore, more leakage between concepts). Our results, shown in Table A.4, show that not only is there significantly more leakage in CEMs compared to CBMs, as one would expect given their higher capacity, but IntCEM's embeddings are capturing more impurities than CEM's embeddings across all tasks but one, providing evidence towards our hypothesis. This preliminary study suggests that contrary to common assumptions, leakage may not always be undesired and could be a healthy byproduct of more expressive concept representations in models that accommodate such expressivity. Nevertheless, this phenomenon may also potentially lead to IntCEM's concept representations being less interpretable than those in CEMs when such leakage is unaccounted for. Therefore, we encourage future work to explore leakage's positive and negative consequences in embedding-based concept models to design better inductive biases for effective interventions.
A.11 Frequency of Selected Concepts by IntCEM's policy
In Figure A.7 we show the frequencies of concept groups selected by IntCEM's  $\psi$  policy when intervening on the validation set of CUB. To study how much this policy changes across different initialisations and random seeds for the same hyperparameters and models, we show these histograms across multiple instances of identically trained IntCEMs with different random seeds. We notice that although, in most cases, concepts such as "upperparts colour" and "size" seem to be highly preferred in the early steps of intervention, there seems to be a relatively large variance across multiple seeds. This may suggest that there are several equally informative or good concepts one may request at a given time to obtain similar improvements, something that we leave for future work to explore.
A.12 Tabulation of Results
This section presents Tables A.5, A.6, and A.7, a set of tabular summaries of our results introduced in Section 4. These summaries show the precise means and standard deviations (computed over five different random seeds) of test-time accuracies after interventions for all baselines and tasks when deploying various intervention policies. They represent tabular versions of the results presented in Figures 3, 4, and 5, respectively.
Concept Group
Concept Group
Figure A.7: Frequency of selected concept groups by IntCEM's learnt policy  $\psi$  after a given number of interventions in CUB's validation set. Each column represents the distribution of intervened groups after a fixed number of interventions. Similarly, each row represents a different initialisation and seed for IntCEM before training.
Table A.7: Tabular summary of Figure 5. Here, we show task performance when intervening on IntCEMs following test-time policies  $\psi$ ,  $CooP$ ,  $Random$ , and  $BC-Skyline$ . The fraction of concept intervened on for each dataset is shown on the left-hand-side of the table.
Dataset	Random	CooP	Learnt Policy	BC-Skyline	IntCEM no ψ (Random)
25%	MNIST-Add	93.78 ± 0.09	93.16 ± 0.14	94.75 ± 0.28	93.20 ± 0.19	93.81 ± 0.18
MNIST-Add-Incomp	90.88 ± 0.25	90.09 ± 0.34	91.69 ± 0.39	90.18 ± 0.35	90.98 ± 0.18
CUB	88.53 ± 0.23	94.68 ± 0.25	94.10 ± 0.49	88.43 ± 0.29	88.46 ± 0.46
CUB-Incomp	78.34 ± 0.34	82.34 ± 0.24	81.71 ± 0.23	79.06 ± 0.41	78.15 ± 0.30
CelebA	42.25 ± 0.36	50.72 ± 0.56	47.63 ± 1.49	48.47 ± 0.47	33.45 ± 0.58
50%	MNIST-Add	95.70 ± 0.11	94.74 ± 0.27	96.71 ± 0.20	94.56 ± 0.47	95.69 ± 0.11
MNIST-Add-Incomp	92.07 ± 0.21	91.31 ± 0.59	92.98 ± 0.31	91.56 ± 0.43	92.13 ± 0.16
CUB	96.14 ± 0.35	99.27 ± 0.16	99.17 ± 0.14	96.28 ± 0.19	95.89 ± 0.40
CUB-Incomp	85.86 ± 0.29	91.25 ± 0.13	90.69 ± 0.30	86.74 ± 0.41	85.31 ± 0.41
CelebA	52.01 ± 0.45	65.47 ± 0.57	62.01 ± 1.21	59.95 ± 0.84	40.71 ± 0.56
75%	MNIST-Add	97.51 ± 0.08	97.99 ± 0.46	98.22 ± 0.09	96.39 ± 0.72	97.50 ± 0.06
MNIST-Add-Incomp	93.48 ± 0.25	93.62 ± 0.82	94.18 ± 0.24	93.38 ± 0.39	93.54 ± 0.14
CUB	98.98 ± 0.14	99.88 ± 0.04	99.82 ± 0.08	99.67 ± 0.10	98.87 ± 0.26
CUB-Incomp	91.74 ± 0.19	95.37 ± 0.31	94.96 ± 0.26	93.22 ± 0.20	91.53 ± 0.29
CelebA	56.99 ± 0.37	68.54 ± 0.66	66.48 ± 1.10	64.63 ± 0.93	44.37 ± 0.61
100%	MNIST-Add	99.51 ± 0.04	99.51 ± 0.04	99.51 ± 0.04	99.51 ± 0.04	99.47 ± 0.06
MNIST-Add-Incomp	94.99 ± 0.11	94.99 ± 0.11	94.99 ± 0.11	94.99 ± 0.12	95.03 ± 0.09
CUB	99.90 ± 0.04	99.90 ± 0.04	99.90 ± 0.04	99.90 ± 0.01	99.85 ± 0.08
CUB-Incomp	96.52 ± 0.19	96.52 ± 0.19	96.52 ± 0.19	96.56 ± 0.19	96.00 ± 0.21
CelebA	70.02 ± 0.62	70.02 ± 0.62	70.02 ± 0.62	69.93 ± 0.72	53.39 ± 0.80