SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Orthogonality Orthogonality in neural networks has been extensively studied in the literature, with various approaches proposed to enhance model performance and interpretability. X and Y have explored orthogonal regularization techniques, which impose orthogonality constraints on weight matrices or feature representations during training. These methods have been shown to improve generalization and reduce overfitting. Saxe et al. [45] have introduced orthogonal initialization methods, which initialize weight matrices using orthogonal transformations. This initialization strategy has been found to aid training convergence and stabilize the learning dynamics of deep networks. Trockman and Kolter [52] have proposed techniques to enforce orthogonality constraints specifically in convolutional filters of convolutional neural networks (CNNs). By imposing orthogonality on the filters, these methods enhance the representational power and robustness of CNNs. In our work, we introduce orthogonality to concept feature space.
The details regarding datasets and hyperparameters are mentioned below. Our codebase is available at https://github.com/ivaxi0s/coop-cbm and is built upon from open source repos [27, 41].
B.1 Dataset Details
CUB The CUB-200-2011 dataset is a collection of 11788 images that are used for fine-grained visual categorization. There are 312 concept attributes that are binarised following the Koh et al. [27] work. While most existing studies use a subset of these concepts, we have chosen to use the entire concept bank in our models and baselines, addressing the fairness issue of subgrouping concepts as highlighted by [49]. The primary task is to classify 200 different species of birds. We also utilize the meta-data of this dataset to obtain human uncertainty.
TIL [42] dataset contains Tumor-Infiltrating Lymphocytes Maps from TCGA HE Whole Slide Pathology Images. The dataset contains tumor maps from the most common cancer tumor types. We use all 13 subsets of the TCGA dataset, therefore constituting 13 cancer types. Although the popular task for such a dataset is necrosis classification, we modify the task to be a classification for different (here 13) tumor types. The advantage of medical images is that their meta-data is readily available from diagnosis. The metadata includes information such as the origin of the tumor, age, gender, and size of tumor cells, which are converted to concepts. We follow [48] for the dataset pre-processing. We wind up with 185 binary concepts following the pre-processing.
AwA2 Animals with Attributes dataset contains over 37,000 images of 50 different animal species, each labeled with 85 distinctive attributes. These attributes can include various characteristics such as color, shape, or behavior, providing a rich source of information for concept bank.
B.2 Experimental setup
For CUB[55] dataset, we trained using 128 batch size with SGD optimizer with 0.9 momentum and learning rate of  $10^{-2}$ . The feature extractor was InceptionV3[50] as a concept encoder model.
For AwA2[57] dataset, we trained using 128 batch size with Adam optimizer with 0.9 momentum and learning rate of  $5 \times 10^{-3}$ . The feature extractor was VIT[15] as a concept encoder model.
For the medical dataset, we create the task of classification divided on the basis of cancer types for TIL dataset[42]. We generate a concept attributes from the meta-data. We use a traditional  $70\% - 10\% - 20\%$  random split for training, validation, and testing datasets. Additionally, we trained using 64 batch size with SGD optimizer with 0.9 momentum and learning rate of  $10^{-2}$ . The feature extractor was InceptionV3[50] as a concept encoder model.
For m-CelebA[31] dataset, we train using 64 batch size with Adam optimizer with 0.9 momentum and learning rate of  $5 \times 10^{-3}$  for 500 epochs. The feature extractor was InceptionV3[50] as a concept encoder model.
Across all of the models for tasks, we use weight decay of factor of  $5 \times 10^{-5}$  and scale the learning rate by a factor of 0.1 if no improvement has been seen in validation loss for the last 15 epochs during training. We also train using an early stopping mechanism i.e. if the validation loss does not improve for 200 epochs, we stop training.
In this paper, we introduced two auxiliary losses, one to improve the task accuracy using multi-task settings and the other to improve concept representation in latent space, leading to improved concept accuracy. The final loss is a linear combination  $(\alpha, \beta, \gamma)$  are hyperparameters for weighting in Equation 7) of concept and task losses along with immediate and concept orthogonal losses.
$$
\arg \min  \left[ \alpha \mathcal {L} _ {C} (f (x), c) + \beta \mathcal {L} _ {y ^ {\prime}} (h (x), y) + \mathcal {L} _ {y} (g (c), y) + \gamma \mathcal {L} _ {C O L} (q) \right] \tag {8}
$$
For the hyper-parameters of Equations 7, we use  $\alpha$  and  $\beta$  values to 0.01 for all of the experiments.
Model	Task	Concept
α = β = γ = 0.01	84.1	97.3
α = β = γ = 1.0	83.0	96.8
α = β = 0.1,γ = 0.01	83.4	97.4
α = β = 0.01,γ = 0.1	83.2	97.4
α = β = 0.1,γ = 0.1	83.4	97.5
α = 0.1,β = γ = 0.01	84.0	97.3
α = 0.01,β = 0.1,γ = 0.01	84.2	97.0
Table 7: Different weightage - coop-cbm with COL on CUB dataset
CEM hyperparameters We would like to point out that we used the same concept  $\alpha$  weightage hyperparameter for each of the model. In literature, for all [27, 60, 20] of the methods used the same concept weightage and we follow the same convention. Looking at our selected hyperparameter, the most divergent value is for CEM [60]. The original CEM paper selected the  $\alpha = 5$ . Since we could not find an ablation study around this hyperparameter in the original paper, we continued to use the same value as rest of the models. We acknowledge that our results might be skewed due to this reason.
B.3 Resources used
Our codebase was built upon the open codebase of [27]. We trained on Linux-based clusters mainly on V100 GPUs and partially on A100 GPU. The following table includes compute timing for each epoch for every baseline.
Model type	CUB	TIL	AwA2
Standard	57s	33s	272s
CBM	68s	39s	286s
CEM	78s	46s	297s
CBM-AR	87s	50s	313s
Coop-CBM (ours)	61s	41s	289s
Table 8: Compute timings of baseline models on different datasets on a V100 GPU.
C Further experiments on COL
C.1 Disentanglement
Disentangled features allow for a more intuitive understanding of the underlying factors that influence the concepts. Several research efforts have explored the benefits and applications of disentangled representations [23]. One of the notable characteristics of Concept Orthogonal Loss (COL) is its ability to induce disentanglement in the concept space. By incorporating COL into the training process, the model learns to separate and represent concepts in a more distinct and independent manner. This disentanglement is achieved by enforcing an orthogonal relationship among different concept representations. To evaluate disentanglement, we use the Oracle Impurity Score as proposed by Zarlenga et al. [61]. The metric essentially aims to detect for impurities in soft representations of concepts. We use this metric to compare against Joint-CBM. The OIS score for Joint-CBM on the CUB dataset was 0.19 while on coop-CBM with COL 0.14 which shows better disentanglement in concept learning for coop-CBM + COL model showing the benefit of using COL on top of any model ad-hoc.
COL encourages the model to assign orthogonal directions to different concepts, thereby reducing the overlap and correlation between them. As a result, each concept becomes more independent and captures a specific aspect or attribute of the input data. This disentanglement in the concept space enables better interpretability and facilitates a clearer understanding of how different concepts contribute to the model's decision-making process.
Through disentanglement, COL enhances the separability and discriminative power of the learned concept representations. It allows the model to focus on relevant and informative aspects of the data while minimizing the influence of irrelevant or redundant features. This disentanglement not only improves the interpretability of the model but also contributes to its overall performance by reducing concept interference and enhancing the model's ability to generalize to new and unseen data.
By promoting disentanglement in the concept space, COL provides a valuable tool for understanding and analyzing the inner workings of concept-based models. It opens up opportunities for further research and exploration into how disentangled concept representations can be leveraged for various tasks, including transfer learning, domain adaptation, and model debugging.
C.2 When is concept orthogonality relevant
One obvious question and could be an interesting future work could be to devise what explicit concepts must be orthogonal. Our work assumes that every concept must be orthogonal but potentially there could be an application where it could be beneficial to include partial orthogonality. Devising an optimal point for when to use COL could be great future work building on our work. We attempt to provide justification using empirical analysis and some intuition in this section.
Duplicating concepts We consider a scenario where input concepts are intentionally duplicated to create a high degree of concept correlation. In our experiment from Table 9, we duplicated  $10\%$ ,  $25\%$ ,  $50\%$  and  $100\%$  of concepts and added them to the original concept bank. This is a worst-case representation of "similar concepts". From the table, we see that the duplication of concepts does not impact the concept or the task accuracy. Additionally, this experiment contributes to the broader understanding of how COL performs in various scenarios. We observe that the performance is not significantly impacted for concept duplication if we add COL.
10%	25%	50%	100%
Task accuracy CUB	83.9 ±0.4	83.7 ±0.7	83.2 ±0.5	83.4 ±0.2
Task accuracy TIL	54.0 ±0.7	54.1 ±1.1	53.8 ±1.2	53.8 ±0.1.1
Table 9: Evaluating the robustness of COL loss in the presence of concept correlation. We randomly duplicate a percentage of the concept bank and evaluated our model Coop-CBM+COL on the CUB and TIL datasets.
Histogram of concept logits for joint-CBM and coop-CBM To gain further insight into the effect of COL, we computed histograms of the activations of the penultimate layer (to which the loss is applied) and saw that the histogram of CBM+COL is very sparse (with a large peak at 0 and a much smaller one at 1) in contrast to the vanilla CBM.
Figure 3: Concept logits histogram comparison on TIL dataset
Intuition The dissimilarity loss  $d_{2}$  encourages independent concept prediction. This is particularly important to reduce leakage in CBMs and improve the robustness of concept explanations. For the connection between entanglement of concept predictions and leakage, we refer the reader to the introduction section of Havasi et al. [20]. Motivated by this insight, we introduce  $d_{2}$ , to encourage disentanglement of the penultimate layer of features for concept prediction. We agree that this may result in an overcomplete representation, induced by the opposing forces of  $d_{1}$  and  $d_{2}$  for samples with partially overlapping concepts, and multiple feature groups may contribute to the same concept. However, it might be difficult to achieve disentanglement otherwise, and as our experiments show, COL improves the concept representation and accuracy, including in the out-of-domain settings. To gain further insight into the effect of COL, we computed histograms of the activations of the penultimate layer (Figure 3) and see that the histogram of CBM+COL is very sparse (with a large peak at 0 and a much smaller one at 1) in contrast to the vanilla CBM. This suggests that COL may encourage learning of an overcomplete sparse feature space, the elements of which encode various combinations of concepts, and the last layer of  $f$  learns to introduce invariance in the prediction of each concept  $c_{i}$  with respect to specific combinations with other concepts  $c_{j}$  by linear combination. Additionally, it must be noted that this regularization is only applied to the penultimate layer before concept prediction which means low-level features are still free to share weights. Essentially, we believe COL encourages learning a sparse over-complete dictionary of features with concepts still partially entangled in different combinations. The concept prediction layer in  $f$  then learns a linear combination of these specialized features to introduce invariance
with respect to the specific combinations. In fact, when studying histograms of activations of the penultimate layer, we observe that activations with COL are indeed very sparse in contrast to CBMs without COL. Given a sparse dictionary of combinations of concepts as induced by COL, the task of disentanglement of concept prediction would ideally reduce to a linear combination of dictionary elements. This is the intuition behind sparse coding (see [30]). In our case, the sparsity is induced indirectly as a result of the orthogonality-based loss formulation. Our experiments show that this approach significantly facilitates the overall optimization of CBMs, improving concept accuracy and downstream performance.
C.3 Effect of lambda on COL
We experimented with different loss weights for  $\lambda$  in our experiments and the model+COL seemed to be fairly robust with different values of  $\lambda$ . We have put those results in Table 5 of the rebuttal PDF on CUB and TIL datasets. While a fine-tuned value of  $\lambda$  might show good performance, we observe that regardless, the model is still able to beat the performances of other baselines. We observed that 0.05 set as a good tradeoff between performance and uncertainty across datasets.
Dataset	λ=0.05	λ=0.1	λ=0.5	λ=1.0	λ=10.0
CUB	84.1	84.1	83.8	84.0	83.6
±0.2	±0.4	±0.3	±0.5	±0.3
TIL	54.2	54.0	54.3	53.6	54.1
±0.9	±0.8	±0.6	±0.8	±1.0
D Further experiments
D.1 Comparison against models with automated concept acquisition
[38] and [58] used a pre-trained model - CLIP which was trained on a massive corpus of data to obtain concepts. This can potentially introduce inherent biases from pretraining into the concepts. This was also brought up in the Limitations and Conclusion section of [58]. Furthermore, the dissimilarity in the concepts employed in these works adds complexity to establish a fair and meaningful comparison. Moreover, we wish to emphasize that neither of these works directly compare with CBM variants in their main paper, except for [58] which appears in Appendix C. Also it is not possible to compare with realistic medical datasets as CLIP fails to generate meaningful concepts. Regardless, we have compared both the methods with our method on CUB+OOD datasets and our model outperforms the accuracy of [38] and [58], which is lower than the standard model. We believe it will be interesting future to include our model methodology and COL in the respective models.
Table 10: Effect of  $\lambda$  on the coop-CBM+COL model. We observe that the orthogonal loss is fairly robust to hyperparameter selection.
Model type	CUB	OOD-CUB	corr-CUB
Standard	82.3	27.7	63.3
PCBM	78.4	33.4	62.7
PCBM-h	80.9	32.1	62.9
LF-CBM	81.0	33.8	63.9
Coop-CBM	83.6	35.4	64.6
+ COL	84.1	36.2	64.9
Table 11: Comparison of Posthoc CBM and Label-Free CBM with our proposed methodology. OOD-CBM refers to Exp 5.1 relating to spurious correlation generalization. Coor-CBM refers to Exp 5.2 relating to image corruption. Unfortunately, Exp 5.3 could not be conducted due to the different concept bank.
D.2 Coop-CBM and COL in the presence of sparse concept labels
Concept labeling could be a labor-intensive task and hence it is important to understand the most optimal point of operation. We randomly select a subset of concepts and train baselines on the subset. Due to concept and task prediction at the same level, we observe coop-CBM provides inductive bias for the downstream task.
Model	10%	25%	50%	100%
Standard	82.3	82.3	82.3	82.3
±0.4	±0.5	±0.3	±0.2
Independent CBM [27]	33.6	62.0	74.7	76.0
±0.5	±0.3	±0.1	±0.2
Sequential CBM[27]	45.5	62.0	74.3	76.3
±0.3	±0.4	±0.2	±0.2
Joint CBM[27]	75.2	78.5	79.6	80.1
±0.4	±0.5	±0.3	±0.1
CEM [60]	76.1	79.3	79.5	82.5
±0.5	±0.2	±0.1	±0.2
CBM-AR [20]	73.4	77.8	79.7	81.6
±0.4	±0.4	±0.3	±0.4
Coop-CBM (ours)	81.4	82.9	83.7	83.6
±0.3	±0.5	±0.4	±0.3
+ COL	82.4	83.2	83.9	84.1
±0.4	±0.3	±0.5	±0.2
Table 12: Task accuracy CUB with sparse concept annotations (% - fraction of concepts)
Model	10%	25%	50%	100%
Independent CBM [27]	94.2	95.3	96.7	96.6
±0.2	±0.1	±0.3	±0.0
Sequential CBM[27]	94.2	95.3	96.7	96.6
±0.1	±0.3	±0.2	±0.1
Joint CBM[27]	90.7	92.0	93.5	93.2
±0.3	±0.2	±0.1	±0.1
CEM [60]	93.6	93.9	94.1	94.8
±0.2	±0.1	±0.3	±0.2
CBM-AR [20]	93.1	93.7	94.0	94.2
±0.1	±0.3	±0.2	±0.1
Coop-CBM (ours)	92.5	92.9	93.6	93.9
±0.3	±0.2	±0.1	±0.2
+ COL	96.6	96.8	97.1	97.3
±0.2	±0.1	±0.3	±0.2
Table 13: Concept accuracy CUB with sparse concept annotations (% - fraction of concepts)
Model	10%	25%	50%	100%
Standard	51.1	51.1	51.1	51.1
±1.2	±0.9	±0.7	±0.9
Independent CBM [27]	15.4	25.6	43.2	47.4
±1.3	±0.8	±1.1	±1.0
Sequential CBM[27]	19.8	27.9	43.9	47.9
±0.7	±1.4	±1.0	±0.9
Joint CBM[27]	43.5	44.9	46.1	47.6
±1.5	±0.6	±1.2	±0.7
CEM [60]	46.1	47.9	49.2	51.3
±0.9	±1.3	±0.7	±1.3
CBM-AR [20]	46.1	48.8	49.2	49.5
±1.1	±0.8	±1.5	±1.0
Coop-CBM (ours)	50.3	51.7	52.8	53.4
+ COL	±1.4	±0.7	±1.2	±0.8
51.0	52.2	53.5	54.2
±1.0	±1.5	±0.6	±0.9
Table 14: Task accuracy TIL with sparse concept annotations (% - fraction of concepts)
Model	10%	25%	50%	100%
Independent CBM [27]	92.3	94.5	95.7	96.4
±0.5	±0.3	±0.7	±0.2
Sequential CBM[27]	92.3	94.5	95.7	96.4
±0.6	±0.4	±0.8	±0.3
Joint CBM[27]	91.7	92.3	93.2	93.9
±0.7	±0.5	±0.2	±0.6
CEM [60]	93.2	93.8	94.2	94.4
±0.8	±0.6	±0.3	±0.7
CBM-AR [20]	93.6	93.8	94.2	94.2
±0.2	±0.8	±0.5	±0.6
Coop-CBM (ours)	93.2	93.8	93.9	94.2
±0.3	±0.7	±0.4	±0.8
+ COL	96.4	96.6	97.0	97.1
±0.4	±0.2	±0.6	±0.5
Table 15: Concept accuracy TIL with sparse concept annotations (% - fraction of concepts)
Model	10%	25%	50%	100%
Standard	96.2	96.2	96.2	96.2
±0.2	±0.1	±0.6	±0.1
Independent CBM [27]	42.4	67.6	89.4	94.9
±0.4	±0.2	±0.1	±0.3
Sequential CBM[27]	52.6	71.5	91.7	94.6
±0.3	±0.6	±0.2	±0.2
Joint CBM[27]	89.2	91.8	94.0	95.4
±0.1	±0.4	±0.3	±0.1
CEM [60]	89.9	93.1	95.5	96.2
±0.6	±0.3	±0.1	±0.1
CBM-AR [20]	91.0	92.6	93.8	95.9
±0.2	±0.1	±0.4	±0.0
Coop-CBM (ours)	92.6	94.2	96.1	96.6
±0.3	±0.1	±0.2	±0.1
+ COL	92.9	95.4	96.5	97.0
±0.1	±0.1	±0.3	±0.6
Table 16: Task accuracy AwA2 with sparse concept annotations (% - fraction of concepts)
Model	10%	25%	50%	100%
Independent CBM [27]	94.7	95.8	97.3	97.7
±0.2	±0.1	±0.4	±0.3
Sequential CBM[27]	94.7	95.8	97.3	97.7
±0.3	±0.2	±0.1	±0.4
Joint CBM[27]	93.1	94.2	94.8	95.2
±0.4	±0.3	±0.2	±0.1
CEM [60]	92.6	93.2	94.8	95.6
±0.1	±0.4	±0.3	±0.2
CBM-AR [20]	93.5	93.7	95.0	95.4
±0.2	±0.1	±0.4	±0.3
Coop-CBM (ours)	93.5	94.6	95.2	95.7
±0.3	±0.2	±0.1	±0.4
+ COL	96.6	96.8	97.1	98.4
±0.4	±0.3	±0.2	±0.1
Table 17: Concept accuracy AwA2 with sparse concept annotations (% - fraction of concepts)
D.3 Detailed results with image corruptions
Model	TIL
1	2	3	4	5	6	7	Avg
Standard	35.5±1.2	34.7±1.7	39.3±1.4	35.7±1.1	41.6±1.6	39.9±1.3	43.9±1.0	38.4
Independent CBM [27]	32.8±1.5	31.9±1.0	39.5±1.7	34.3±1.2	40.8±1.1	32.9±1.6	43±1.3	36.6
Sequential CBM[27]	33.0±1.3	31.6±1.4	39.9±1.1	34.6±1.6	40.2±1.5	32.4±1.0	42.9±1.7	36.3
Joint CBM[27]	33.3±1.4	32.3±1.3	41.6±1.2	38.2±1.7	41.2±1.0	33.3±1.5	39.8±1.1	37.1
CEM [60]	35.2±1.1	34.8±1.6	40.0±1.5	37.3±1.0	42.1±1.7	33.6±1.2	46.5±1.3	38.5
CBM-AR [20]	35.4±1.7	35.1±1.2	39.7±1.3	38.4±1.4	41.6±1.1	42.8±1.6	25.6±1.5	36.8
Coop-CBM (ours)	36.7±1.0	36.2±1.7	41.4±1.2	37.6±1.3	43.3±1.4	39.9±1.1	49.1±1.6	40.6
+ COL	37.2±1.6	36.5±1.5	41.5±1.0	38.3±1.7	43.0±1.2	44.4±0.9	45.4±1.0	40.9
Table 18: Comparison of concept-based models on image corruptions on TIL datasets
E Further about interventions
Concept Uncertainty Score
A significant part of the intervention selector constitutes Concept Uncertainty Score (CUS) which denotes the model uncertainty for the prediction of concepts. We calculate epistemic uncertainty that arises due to model parameters and lack of training samples. Realistically, labeled medical data is often scarce, encouraging the application of epistemic uncertainty quantification of the predicted concepts. We use Monte-Carlo dropout [16] to model epistemic uncertainty, with a random dropout rate of 0.2. We apply the dropout before the prediction of concepts. For an image  $x_{i}$  predicting concepts  $c_{i}\ldots c_{K}$  where  $K$  is the total number of concepts, we evaluate  $T$  softmax probabilities  $\{p_t\}_{t = 1}^T$  for each concept prediction. We measure the uncertainty for each concept which we refer to as  $\mathcal{H}(\cdot)$ . We compute the entropy-based uncertainty for each concept as the measure of the expectation of the information inherited in the possible outcomes of a random variable. Using the uncertainty metric, we calculate the overall uncertainty concept vector  $\mathcal{H}$ .
$$
\mathcal {H} (\cdot) = - \frac {1}{N} \sum_ {i = 1} ^ {N} \left(\frac {1}{T} \sum_ {t = 1} ^ {T} p _ {g} (c | x _ {i}) \log \frac {1}{T} \sum_ {t = 1} ^ {T} p _ {g} (c | x _ {i})\right) \tag {9}
$$
Concept Weightage Score
The second part of the intervention selector score is signified by Concept Weightage Score (CWS), which accounts for the importance of a concept in the final downstream prediction task. Using CWS, the intervention selector is able to prioritize the concept for intervention that is deemed to change the final prediction significantly. We define the weightage score as  $\beta$ . The  $f(y|c)$  is a linear one-layer Multi-Layer Perceptron (MLP) which helps in defining  $\beta$ .
$$
\beta_ {i} = c _ {i} \sum_ {j = 1} ^ {N} \left| w _ {i j} \right| \tag {10}
$$
Supervisor Confidence Score
Finally, in the intervention selector, we consider the reliability of the annotator via Supervisor Confidence Score (SCS). For example, while a histopathologist can identify diseases across human tissues and organs, they often have more specialized and nuanced areas of focus. It is therefore
beneficial for the model to request additional information from histopathologists from their expert knowledge. This in practice prevents ambiguous or inaccurate concept correction. The SCS is a variable across each annotator and is represented by  $\gamma$ .
Finally, bringing the three desiderata of concept selection for intervention, the final intervention selector score is a linear combination of all of the scores.
$$
A I S e l e c t = k _ {1} * \left(\frac {1}{N} \sum_ {i = 1} ^ {N} \left(\frac {1}{T} \sum_ {t = 1} ^ {T} p _ {g} (c | x _ {i}) \log \frac {1}{T} \sum_ {t = 1} ^ {T} p _ {g} (c | x _ {i})\right)\right) + k _ {2} * \sum_ {i = 1} ^ {K} c _ {i} \sum_ {j = 1} ^ {N} \left| w _ {i j} \right| + k _ {3} * \gamma \tag {11}
$$
where  $k_{1}, k_{2}, k_{3}$  are hyperparameters for importance weightage on each of the scores.
Intervention allows the model to query the most significant concepts. In our case, it is hypothesized that the most uncertain concepts will build a symbiotic relationship between the human and the model. The supervisor decides the threshold  $\mathcal{I}_{th}$  to correct the concept prediction. Realistically, in medical scenarios, due to the professional's limited availability, we would like to optimize the number of concepts to intervene. Setting a lower threshold is a trade-off decided by the user. In contrast to other works that perform group interventions [27] by intervening on a group of similar concepts, we perform single interventions. Group interventions require clustering of concepts on the basis of their similarity $^2$ , which is not realistic as such information is not always available. Therefore, single interventions are performed to minimize the dependence on human priors.