Learning Concept Credible Models for Mitigating Shortcuts
Authors: Jiaxuan Wang
Sarah Jabbour
Maggie Makar
Michael Sjoding
Jenna Wiens
Venue: NeurIPS 2022 Accept (2022)
Keywords: shortcuts
spurious correlations
deep learning

Abstract: During training, models can exploit spurious correlations as shortcuts, resulting in poor generalization performance when shortcuts do not persist. In this work, assuming access to a representation based on domain knowledge (i.e., known concepts) that is invariant to shortcuts, we aim to learn robust and accurate models from biased training data. In contrast to previous work, we do not rely solely on known concepts, but allow the model to also learn unknown concepts. We propose two approaches for mitigating shortcuts that incorporate domain knowledge, while accounting for potentially important yet unknown concepts. The first approach is two-staged. After fitting a model using known concepts, it accounts for the residual using unknown concepts. While flexible, we show that this approach is vulnerable when shortcuts are correlated with the unknown concepts. This limitation is addressed by our second approach that extends a recently proposed regularization penalty. Applied to two real-world datasets, we demonstrate that both approaches can successfully mitigate shortcut learning.

Learning Concept Credible Models for Mitigating Shortcuts
Jiaxuan Wang'
Sarah Jabbour'
Maggie Makar'
Michael Sjoding"
Jenna Wiens'
Division of Computer Science & Engineering, University of Michigan, Ann Arbor, MI, USA Division of Pulmonary and Critical Care, Michigan Medicine, Ann Arbor, MI, USA Correspondence to: Wiensj) @umich.edu
Abstract
During training, models can exploit spurious correlations as shortcuts, resulting in poor generalization performance when shortcuts do not persist. In this work, assuming access a representation based on domain knowledge (i.e., known concepts) that is invariant to shortcuts, we aim to learn robust and accurate models from biased training data. In contrast to previous work, we do not rely solely on known concepts, but allow the model to also learn unknown concepts. We propose two approaches for mitigating shortcuts that incorporate domain knowledge, while accounting for potentially important yet unknown concepts. The first approach is two-staged. After fitting a model using known concepts, accounts for the residual using unknown concepts. While flexible, we show that this approach is vulnerable when shortcuts are correlated with the unknown concepts. This limitation is addressed by our second approach that extends recently proposed regularization penalty. Applied to two real-world datasets, we demonstrate that both approaches can successfully mitigate shortcut learning.
Introduction
In practice, machine learning (ML) models often fail to generalize under distribution shift 14 due shortcut learning 3 4. "Shortcut learning occurs when a predictor relies on input features that are easy to represent (i.e., shortcuts) and are predictive of the outcome in the training data, but do not remain predictive when the distribution of inputs changes" [5]. For example, consider building ML model to predict the severity of knee osteoarthritis from X-ray images (6. If people with mobility problems in the training set are more likely have an X-ray using a particular type of mobile X-ray scanner, the model may learn to rely on features related to the scanner type to make a prediction, resulting failure generalize when X-rays are captured from different scanner.
More formally, consider the causal graph in Figure I where Y is the target of interest (e.g., diag- nosis), S is the shorteut (e.g., scanner type). X is the input (e.g., X-ray image), and C and U are representations that can be inferred from X but are not causally affected by S. Here, the dashed bidirectional arrow denotes spurious correlation that holds during training but not at test time. Solid arrows denote causal relationships that are robust to changes. Note that S only affects the part of the input that is irrelevant for the diagnosis (X'), making causally irrelevant for the prediction. We will consider both settings where S is and is not correlated with U. In our example, C' could be known radiological risk factors, and U could be unknown radiological risk factors for the disease.
To mitigate a model's reliance on S, one can use existing tools S is observed (e.g., through model interpretation) [5] 7. However, these methods do not apply when shorteuts unknown prior to the occurrence of distribution shifts. Moreover, such approaches fail when the spurious correlation strong (i.e., more convineing shortcuts). In such scenarios, we need additional guardrails. Our
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Label (egg, Diagnosis)
Y
Known concepts (e.g., radiological findings)
Unknown concepts
U
S
Shortcut
(e.g. scanner type)
Causally relevant (e.g., bone spurs)
Causally irrelevant
(e.g., artifact on image)
X
Input (e.g., radiograph)
We formalize shorteut learning with a causal graph: Y is the label (e.g., disease diagnosis) and X is the input (e.g., radiograph). X can be decomposed into causally relevant and irrelevant features (X* and X'), ), i.e., changing X changes the label whereas changing X' does not. X can be further decomposed into known and unknown relevant concepts (C' and U). The node surrounding U and C' abstracts their interaction (e.g.. they can be correlated). A shortcut variable S changes X' and correlated with U and C. Observed variables are gray. Dashed/solid edges represent correlation that is broken/unaffected under distribution shifts. We aim to eliminate model dependence on S.
Figure 1:
approach considers the setting in which we do not have direct knowledge of S, but have access to a representation, C,, that is invariant to S (formally defined in Section 3.1.1). It is true that However, practice, we
without knowing S. we cannot truly confirm whether C' is invariant can rely on established domain knowledge such as risk factors for disease to not encode shortcuts. By exploiting this representation C, we mitigate the reliance on shortcuts.
Where does C' come from? C' arises from domain knowledge and can be elicited in a number of different ways. For example, C' may be elicited using transfer learning. Using domain knowledge, experts can identify a related source task. Predictive features (i.e., learned representation) from the related source task (i.e., C) can be shared to predict Y 3.. Alternatively, if one has auxiliary concept labels, one can train a model to predict the presence of concepts and use these predictions as C' [6]. In fact, both Koh et al. a standard model (i.e., state of the art model) in the presence of shortcuts.
[6] and Jabbour et al. [3] have shown that relying on C' alone can outperform
However, depending solely on C, referred to concept bottleneck model (CBM) 66. ignores potentially unknown concepts (i.e., U). When U contains additional useful information, relying solely C' results in inferior predictive performance. E.g., when learning to diagnose the etiology of acute respiratory failure (ARF), clinicians are only accurate about 70% of the time [H12) Recently ML models have been shown to improve on this accuracy and have also been shown to generalize across hospitals, suggesting that ML models are using additional features beyond those that humans rely on [13]. We need U because while experts have lots of knowledge about what might be relevant to a specific clinical setting, they certainly do not know everything, or may not be able to always appreciate what exactly relevant in each setting. For example. Jabbour al. [13.. found that their model learned that a 'saber sheath trachea' was relevant for identifying patients with a COPD exacerbation. When asking clinicians to come up with the set of features a model should use to identify COPD, most would provide things like 'lung hyperinflation', and miss 'saber sheath trachea,' despite its relevance [4]. More generally, there are likely findings out there that have yet to be recognized as important; new medical discoveries are made all the time. But even when all relevant risk factors are known a priori, sometimes the universe of possibly relevant risk factors is large, such that collecting and labeling all these features for particular problem is unrealistic. In such cases, the flcxibility yielded by including U helps. In some cases, U may turn out to be something known before, e.g., saber sheath trachea, whereas other times it might be a brand new finding.
To tackle the decrease in accuracy when using C alone, we propose two approaches based on concept credible models (CCM). The first approach, CCM RES, while simple, is susceptible to particular failure case when U is correlated with S. The second approach, CCM EYE, extends the EYE penalty from Wang et al. [15] to address those issues. The EYE penalty was proposed for linear models as a way to increase the model's alignment with expert knowledge (in our case, C) without sacrificing predictive performance. Here, we hypothesize that the same idea can mitigate the use of shorteuts. We thus extend the EYE penalty to work with non-linear models by applying to the learned representatioonccnneqp space. This is a nontrivial application of the EYE penalty since here

---

the concept space is not equivalent to the input space. Moreover, we identify the conditions in which CCM RES and CCM EYE mitigate learning shortcuts. In summary. our contributions include
We propose the idea of learning concept credible models (CCM), in which C' is not required to be directly represented in the input space. We demonstrate that CCMs are more robust to shortcuts compared to existing approaches.
Unlike previous work on shortcut learning, we show that our approaches still apply when shortcuts are perfectly correlated with other features of C, and address the limitation of existing methods that rely solely on C' in making predictions.
Theoretically, we identify the sufficient conditions under which CCM can eliminate shorteuts for the setting considered in Figure [
Empirically, we demonstrate that our approach can still help mitigate shortcuts even when these conditions identified above are violated.
2 Related work
The idea of "concept credible models" is connected to multiple fields in ML, explained below.
Connection to shorteut learning. Shortcut learning is a particular failure mode that arises due to distribution shifts 16]. However, to date, researchers have typically assumed shortcuts are known a priori. Under such settings, one can augment the dataset to decorrelate shortcuts with data 21 regularize model parameters to not rely shortcuts 77 S [22]. In contrast, we do not assume that we know S. This change makes approaches such as IRM [66 and REx [23] no longer applicable, because without knowing S, it is hard to specify the family of distributions to which model should be robust. The above methods also will not work when shortcuts and robust features are perfectly correlated because without prior knowledge, they cannot be separated apart.
Connection to concept bottleneck models. The concept bottleneck model (CBM) was proposed in Koh et al. [6] with the goal of making model's decision more transparent by only using C for prediction. While this can mitigate shortcuts since the model is forced to rely on instead of spurious correlations, it ignores unknown concepts, often resulting in lower accuracy compared to standard model [24]. We address this problem by adding a channel that takes X as input, in addition to predicting Y from C. This added channel. along with carefully chosen regularizer, enable CCM better accuracy. We note that the added channel design also appears in a concurrent work [25] However, unlike CCM, (25 does not have a mechanism mitigate learning shortcuts. Instead, it makes the added concepts interpretable, which is complementary to our work.
to learn U, resulting
Connection to credible learning. Credible models are trained by regularizing model's feature attribution to be close to expert identified features (i.e., features known to be relevant for the prediction), in addition to being accurate [ While credible learning has been shown to work well in a transfer learning setup within natural language processing [26), we are the first to study its applicability to mitigate the effects of shortcut learning. However, unlike previous work, do not require domain knowledge (i.e., concepts) to be expressed directly in the input space. This provides with greater flexibility in exploring different types of inputs in which it may be difficult to collect domain expertise (e.g., the pixel value of images). As a result, our approach does not require models to be linear.
3 Proposed approach
We formalize the problem and assumptions, and propose methods to learn concept credible models.
3.1 Preliminaries
To simplify the exposition, we illustrate the setup for a regression problem. The setting, however, is easily adaptable to multi-class classification. We capitalize random variables and bold vectors. For example, xc denotes instance of the random vector X. We denote the Pearson correlation between two random variables as corr(...).
3.1.1 Setup & assumptions
Y is the target of interest (e.g., diagnosis), S is the shortcut (e.g., seanner type). X is the input (e.g., X- ray image). C and U are representations that can be inferred from X but are not causally affected by S.
3

---

Given dataset D = (xx(1) E y E ER) 1 of n samples generated according to Figure 1 and a function fo IR + R' such that C == fc(X), we aim to learn an accurate prediction from the input variable X to the target variable Y (f R + R) that is invariant to the unknown shorteut S. Here, and d are dimensionality for C' and X respectively. We also assume that there exists unknown concepts U == (x) E R" that are invariant to 5 with dimensionality u.
Although we assumed that fe is given in this setup, we can also learn it from related dataset (e.g., predictive features for this related task). Note that this does not require direct knowledge of S. E.g.. it is sufficient to know that the shortcut for the target task is unlikely shorteut for the related task.
In this paper, invariance refers to counterfactual invariance from Veitch et al. [22 (Definition 1.1). Adapting their notation, let X (s) denote the counterfactual X we would have seen had S been set to s, leaving all else fixed, f counterfactually invariant to S if (s)) = (s')) almost everywhere, for all 8, s' in the sample space of S. This invariance ensures generalization of the model regardless of the shortcut's 's distribution. Note that invariance is not the same as independence. E.g., scanner type does not cause the diagnosis (i.e., diagnosis is invariant to scanner type) yet they can be correlated.
We require assumptions about C. A1 is implied by the causal graph, while A2 not.
A1: C is counterfactually invariant to S. E.g., changing the scanner type does not change the occurrence of bone spur in an X-ray image. Thus the presence of bone spur is invariant to the scanner type. Without this assumption, even a model that only uses C' may indirectly depend on S.
A2: S is redundant given C (i.e., Y L SIC). E.g., given bone spur and other radiological findings from an X-ray image, the type of scanner irrelevant in predicting arthritis severity. Without this assumption, including S improves accuracy.
Since both A1 and A2 are testable without knowing S, in experiments, we test methods' sensitivity to each assumption empirically. Note that we do not make any assumption regarding the correlation between S and U.
3.1.2 Existing approaches
We formally introduce common methods that are typically used in this prediction setting.
Standard Model: Standard model refers to the task specific state-of-the-an the- model trained with loss L ()))))))))))))))))))))))))))()) ()). Such model cannot distinguish among C. U, and S. Thus, the model is vulnerable to relying on shortcuts for prediction.
with empirical risk minimization: arg min jon Eig(,..... [JSTTD (x)rt (=)..,
Concept bottleneck model (CBM): Our proposed approach builds on CBM (6) CBM's prediction can be written as fCBM (X) (x))). Here, fy maps from C' Y and trained using em- pirical risk minimization: arg miny, Exep,.....) ([ffcu(((()), ()) When U contains additional information useful in predicting Y given C, CBM less accurate than a standard model.
3.1.3 A motivating example
To build intuition, consider the following linear regression example in which C and are perfectly
correlated during training (i.e., C = S in D) while C and U are not. Given X [C, S, U] and
Y C + U, least squares linear regression solution gives prediction of Ý (1 - -)- + + tS (derived in A.2). The free parameter E IR results from the spurious correlation between C and S.
The minimum Ta2 norm solution of this problem results in (= 0.5 and will fail to generalize when the correlation between and C no longer holds test time. contrast, if we only use C' for prediction (i.e., CBM), the solution will not achieve loss of 0 since it ignores U. Furthermore, in cases where C' and U are correlated, CBM is asymptotically biased due to omitting the variable U [27]. This means that models that ignore U, such as CBM, cannot recover the true regression coefficients even as the training set size approaches infinity.
3.2 Proposed approaches: concept credible models
We introduce two approaches to learn a concept credible model with the goal of mitigating shorteuts: CCM RES and CCM EYE.

---

3.2.1 CCM RES
The limitation of CBM stems from its inability to infer U from X. To address this limitation, we design two stage approach, CCM RES, that first fits a CBM on the dataset, and then fits a residual model fa based on the difference between Y and the output of the CBM. This idea is similar in spirit to boosting methods. La enables CCM RES to learn U. CCM RES obtains its prediction by adding the output from fa to the output of the CBM: [RES(#) achieves small training loss (e.g., the difference between Y and CBM's prediction is small), CCM RES does not have to rely information other than C. Otherwise, CCM RES relies on fo to make up for what C' alone cannot learn. We learn CCM RES with empirical risk minimization: fuss = arg si Eiep,.... mn] L(RRES( (x)))
[CBM(#) + [=( When CBM
Applied to Example when U is independent from S in D, the resulting model not only achieves 0 empirical loss, but also has 0 reliance on S, achieving our goal of accuracy without relying shortcuts. We generalize this motivating example to all linear models below.
To highlight the strength of CCM RES compared to previous approaches, we consider the worst case scenario where |corr(C, S)] = In this case, the system under-specified (i.e., allowing multiple minimum loss solutions), and standard model may not be consistent with the causal DAG. Previous approaches fail in such scenarios since they cannot distinguish C' from
Consistency of CCM RES. If SRES is linear (both fcuM and Ja. are linear), X = Y aC + BU 4 with a, E R and zero mean error e, |corr(C, recovers the true parameters without relying
on the training distribution
Pxy. and U IL S, arg minffunua fus Exyy-Pxy fRES on S (i.e., weight for C, for U, and 0 for S).
Proof. We first show that the residual is independent of S, thus fitting to the residual will not use S. From JL and (corr(C,S)
1, we know 1 C. Fitting on infinite data with squared loss simplifies SCRM to CY = B(CC + bU +c(C) = aC + bB(UIC) aC + bB(U). The residual, Y EEY[C) b(U B(U)) + e, is independent of because U 1 S. Thus the prediction is
aC + bU, recovering the true parameters.
Remark: As the |corr(S, C decreases, the system may no longer be under-specified, and both CCM RES and a standard model will be consistent. This happens when S is not a linear combination of C and U, in which case the minimum loss solution is unique. If, however, S is a function of U, we cannot distinguish S from U and thus cannot guarantee the consistency of CCM RES.
This result shows that a linear CCM RES, unlike a linear CBM, is consistent estimator. However, while CCM RES enables learning unknown concepts, it fails when S and U are correlated because the residual can be estimated as a linear combination of and S, making fa vulnerable to encoding shortcut. We address this problem with second approach.
3.2.2
In our second approach, we utilize the EYE regularization from Wang al. 15] to learn a concept credible model (CCM EYE). The EYE penalty penalizes reliance on features that are correlated with C but not in C. We propose to apply EYE regularization on the concept space (i.e., the learned representation space) as follows: (ÆYE(x) f (x) + ff (h)reer fe computes the known relevant representation C' and fz computes representation from the last layer of a standard model. This transformation from (z) allows the model to be non-linear. 0 and O. coefficients for (z) and fc(z) respectively. We then apply the EYE regularization on those parameters:
fEYE = arg min
Σ
I(FExE(x()) (0 + 0.))
0. H0 [E(
Here, J([0z, 0.)) = 100 + 0z 10ell the EYE regularization applied our setting and
A E R20 is hyperparameter that controls the trade-off between regularization and loss. The EYE penalty more strictly penalizes O, compared to 0z, allowing the norm of 0, to be larger and hence encouraging the model to rely on C. Conversely, J discourages the use of X, which include both U and S. If U is in fact important in predicting Y, the minimization of the loss encourages the use of U more than S because of A2: U has more predictive power compared to S given C.
5

---

We choose À such that 0₂ is strictly regularized without sacrificing in-distribution performance (i.e., performance under the biased training distribution). We do so by picking the largest such that the model's accuracy on the validation set is not statistically worse than that of standard model, where statistically worse performance measured in terms of empirical 95% bootstrapped confidence intervals. This ensures that CCM EYE maximizes the use of C' without sacrificing predictive performance.
Similar to CCM RES, CCM EYE is a consistent estimator even when |corr(C, = 1. The remark for CCM RES applies to CCM EYE as well.
Consistency of CCM EYE. If fEYE is linear, X = CC,, aC + bU + c with a, b E R and zero mean emror e, |corr(C,U) * 1 and |corr(C,S)| =1 on the training distribution Pxy. arg min feys Exy (y- - [[vE(r))² + XJ((2, 0z)) recovers the true parameters (i.e., weight a for C, for U. and ( for S) with standardized input, where À is chosen as described before from Px,y.
Proof. With infinite data, the empirical loss is the in-distribution generalization loss, therefore is chosen such that the generalization loss is minimized. In the worst case scenario, the perfect correlation between S and C makes this linear problem underspecified (i.e., multiple solutions), which means À is non-zero because it is set to be the largest value such that model performance is not statistically worse than a standard model trained on Px,y. Fixing the same loss, the EYE penalty places zero weight on standardized features (features normalized to zero mean and unit variance) that are perfectly correlated with expert identified features [T]. Treating C as the expert identified feature, the coefficient for S is thus O. Combined with the fact that C' and U are not perfectly correlated, to achieve the minimum loss, the coefficients for C' and U must be and b respectively.
Unlike CCM RES, the consistency of CCM EYE does not require U 1 S. Intuitively, EYE can separate U from S because of A2: U is needed in addition to C' to be accurate, yet S is not needed given C. Note that |corr(S, C)] = 1 does not imply U 1. S because U can be correlated with C, which in turn is correlated with S.
Remark:
Our consistency results not limited to the regression setting. One replace the mean squared loss in the consistency proof with the logistic loss or the negative log likelihood. Under those losses, the same results hold and the proposed approaches do not rely on S.
For our theoretical results, we focused on the linear case. In this setting, perfect correlation is required to create model underspecification. When features are linearly independent, all methods are consistent. Encouraged by experiments with non-linear models (Section 4 we believe these results could be further expanded upon (e.g., non-linear setting) in future work. While our theoretical results on linear models are restrictive, they serve as a sanity check and hint at what we might expect with more complex models as their last layers are often linear (e.g. neural networks). We also note that the additive structure of both CCM RES and CCM EYE does not restrict their expressive power as fx can be arbitrarily complex, capturing the interactions between C' and U.
4 Experiments & results
In this section, we verify CCM's robustness to spurious correlations three classification tasks using publicly available datasets. The first is an image classification task similar to the one examined by Koh et al. (6] This task demonstrates the superior performance of CCM when C is complex and non-linear in X. The second task is the prediction of pulmonary edema from chest radiographs. It demonstrates CCM's effectiveness in critical domain where accuracy and robustness are needed. We include an additional task to predict in-hospital mortality in Appendix AA66] To demonstrate that the shortcuts used in our experiments are not easily observable, we include results of feature attribution of the baselines as well as our proposed approaches on both image datasets in Appendix A.7
Concept Acquisition: In this paper, we explored two practical settings to obtain C, one using auxiliary labels (the bird attributes) and another using a transfer learning setup (transfer from the cardiomegaly task to the edema task). In both settings, we show that our approaches are effective at mitigatinng shortcuts. While there are more ways to obtain C, we defer a comparison of the benefits/tradeoffs of various approaches to future work.
Note that we not compare the approaches on common computer vision datasets such as CIFAR 10/100 28 because C' is not readily defined. Instead, we relied on benchmark datascts that have been

---

used in past work involving C [6]. We also note that in contrast with the theoretical results, which focused on linear models, the experiments do not make those assumptions. For example, the edema experiment uses sex as the shorteut, which is not a linear concept in terms of the input (e.g., pixels
We start with setting in which our assumptions hold (Section 3.1.1, and then relax our assumptions to stress test our methods. We evaluate on both biased and unbiased/clean data (defined in the evaluation section of each task) to explore the effects of distribution shift caused by the shortcut. All models are trained and selected using only the biased dataset (e.g., the dataset where corr Y) o 0).
Baselines. We compare CCM with the following methods:
STD(X) is model trained end-to-end on the biased dataset 29 using X to predict Y. We expect it to learn because there a backdoor path from S to in Figure
Concept bottleneck model (CBM) removes STD(X)'s reliance on S by only fitting on C' [6]. However, CBM lacks the ability infer U and thus may sacrifice discriminative performance before and after shortcut induced distribution shifts. Following Koh et al. [6. we train a CBM by fitting logistic regression model on top of C.
STD(C, X) is a standard model that conditions both on X and C' for prediction. On the one hand, we expect this baseline to be more robust than STD(X) when S breaks because it has an easy access to C. On the other hand, conditioning on X gives the baseline the ability to infer U, unlike CBM. However, when S is highly correlated with C, this baseline prediction. While there are many ways to implement STD(C, X), we implement this baseline as a special case of CCM EYE with O. This allows us clearly demonstrate the effect of EYE regularization on model robustness.
still rely on S to make
4.1 Experiments on the CUB dataset
The Caltech-UCSI Birds-200-2011 dataset (CUB) consists of 11, 788 images of birds [30] each belonging to one of 200 species (Y), In addition to the images, the dataset also contains 312 binary attributes/concepts (e.g., beak color) describing birds in each image. Following Koh et al. [6]. we filter out concepts with noisy annotations.
Concept Acquisition: We train a standard model to predict those concepts from X, with random Gaussian noise (O, O. 12) added to the image. The resulting prediction treated as C. Note that no shorteut is introduced in obtaining C, in order to satisfy A1. Later, we will break this assumption.
Shortcut:
The shortcut consider here is the level of noise, a, in an image. We correlate with bird species to mimic setting where some birds have noisier photos than others because they are harder to observe in the wild. Ideally, a model should be able to classify the birds regardless of the noise level. To introduce a as a shortcut, we correlate it with bird classes in the training data. However, we do not correlate a with Y directly because it violates A2 (a setting we explore in Appendix A.4). Instead, we correlate a to Y through C, using the BIAS function described in Appendix A.3
Method
Test Acc (biased)
Test Ace (clean)
Table 1: On the CUB dataset, when A1 and A2 hold, CCM is worse than baselines on the biased data (column 1), and is better than baselines on the clean data (column 2). Empirical 95% CI are in parentheses.
CCM EYE
(75.0, 77.2)
75.2 (74.1, 76.5)
CCM RES
75.6 (74.2, 76.9)
76.0 (74.8, 77.2)
STD(X)
75.7 (74.7, 77.0)
55.8 (7.37,
CBM
71.6 (70.4, 72.9)
72.8 (71.2, 73.9)
STD(C.))
76.0 (74.7, 77.2)
69.7 (68.6, 70.8)
Model Training: Following (6). all methods use Inception V3 architecture 29 initialized using the Imagenet dataset (31], We divide the training set predefined in the CUB dataset into train and validation set with 80/20 random split, and use the predefined test set for evaluation. We report the performance on this test set as the result for unbiased/clean dataset. We then add class dependent noise described earlier to the train, validation and the test set form the biased dataset. All methods are trained on the training set of this biased dataset using SGD with learning rate of 0.01, momentum of 0.9, and batch size of 32. We apply 10- weight decay each model and decay the learning rate every 15 epochs. For CCM EYE, we tune À in the range of [10 10 10-4, 10-5, 10-6).
'Code is available at /tetts://gitiln eecs. http://wwaaleeeeeaaaa///eeeeeeee

---

test acc (clean)
Test acc (clean)
85
CCM EYE
CCM EYE
16
CCM
CCM RES
STD(C,X)
STD(C,X)
CBM
CBM
STD(X)
STD(X)
00
02
04
06
00
100
60 number of C's dimensions swapped with noise
100
(a) Test Accuracy (clean) violating A1
(b) Test Accuracy (clean) violating A2
Figure 2: (a) Model performance under the clean test set when violating A1. When is learned using biased dataset (sweeping I' on the horizontal axis), violate A1. Unless C' is extremely corrupted (e.g., T 1). CCM performs relatively well. (b) Model performance under the clean test set when violating A2. C becomes less informative when replaced with noise, presenting an advantage to using S and violating assumption A2. Despite this, CCM still performs well, even when large portions of C' are irrelevant for the prediction.
Evaluation: Recall that our goal is to learn an accurate model without using S. We generate the biased and the clean test described in the model training section above. Evaluating the biased test set demonstrates how the model performs when S is correlated with Y. Evaluating on the clean test set demonstrates how the model performs without image noise (i.e., the shortcut no longer exists). A model that does not rely on S should perform similarly on both datasets. We measure performance on the CUB dataset using accuracy (ACC) bird classes balanced. Empirical 95% confidence intervals are reported based on bootstrapped samples from the test set.
Results: We examine the results when A1 and A2 hold/break. We also explore varying À in Appendix A.4 to justify its choice.
Q1: How does CCM perform when A1 and A2 are satisfied?
worse than baselines when tested the biased dataset (Table column), fllumn) but are significantly better than baselines when is removed In contrast, STD(X) performs well the biased dataset but underperforms
Both CCM RES and CCM EYE
second column). the clean dataset, indicating its reliance on S. As expected, CBM does not rely on the shortcut as its performance is stable with and without S. However, its inability utilize U results in a drop accuracy compared to others. Finally, STD(C, X) is accurate on the biased dataset and improves over STD(X) on the clean dataset because encourages the model to use However, CCM EYE dominates, suggesting that conditioning on both C' and X is not enough to remove model reliance on S.
Q2: How does CCM perform when A1 is violated?
We relax A1 by learning C' on biased dataset. Specifically, we use the BIAS function introduced in Section A.3 but vary the probability of correlating 5 with Y (i.e., varying the parameter T). Figure 2a shows the results varying T on the test accuracy the clean data. As before, CCM dominates the other baselines except at T = 1, indicating that unless C' is extremely corrupted with S, CCM performs well compared to the other models. We note that STD(X) does not change with T because it does not C. In Appendix A.4 show that all methods except CBM perform similarly well on the biased dataset.
Q3: How does CCM perform when A2 is violated?
We relax A2 in two ways: a) S contains information outside of C but within U (i.e., 1 SIC,U), and b) S contains information outside of C' and U (i.e., Y L S(C,U).
To violate A2, can no longer redundant given To achieve this, we first introduce a correlation between S and C as before (to satisfy A1) and then we randomly replace columns in C' with Gaussian noise (0, 1), but keep the same. This procedure correlate with because the swapped out information becomes unknown concepts based on which S is generated. The more concepts swapped

---

for noise, the less informative C' becomes. increasing the relative value of S in predicting Y. For example, when 100 random dimensions of are replaced with noise, a linear model trained with (C. S) significantly outperforms model based on just C.
This concept swapping greatly affects CBM because it relies solely C, which is corrupted. When S is removed (Figure[b). both CCM EYE and CCM RES outperform baselines until all concepts are corrupted (in which case CCM performs similarly to a standard model). The performance of STDD( is constant across settings because it does not rely on C. This experiment also shows that not all dimensions of C need be relevant to the prediction for CCM to work. This is a desirable property as expert knowledge with respect to relevant concepts could be flawed.
In the case where S contains information outside of C' and U (i.e., Y L S|C, U), our findings similar (Appendix A.4). All methods except CBM perform well on the biased dataset.
4.2 Experiments on the MIMIC dataset
The MIMIC-CXR dataset 32
33 consists of chest X-rays and corresponding radiology reports.
These data can be linked MIMIC-IV 34 331, which contains de-identified clinical data. Each chest X-ray is associated with 14 text-mined radiology report labels corresponding to 14 different radiological findings. Among these findings, we aim predict a diagnosis of edema (excess fluid in the lungs).
Concept Acquisition: Oftentimes in healthcare, there exist related tasks from which one can draw concepts. Thus, here, we explore a transfer learning setup to extract concepts. Based on domain knowledge, chose cardiomegaly (enlarged heart) as the source task learn C. Patients with cardiac dysfunction are more likely to develop heart failure, and pulmonary edema can develop as consequence of heart failure [55 3) Thus, we expect that predictive features of cardiomegaly are useful concepts in diagnosing edema. After excluding images without labels the cardiomegaly/edeee tasks contained 108, 785/107, 510 X-rays, respectively. To obtain C, we trained an Inception V3 network pretrained on the ImageNet dataset to predict cardiomegaly. Then we used the last layer representation of the network as C' (dimension 2048).
We introduce a realistic shortcut based on patient sex. We increased the correlation between male and edema by dropping T proportion of females/males with/without positive label. Prior to resampling, male was only mildly correlated with cardiomegaly (corr. coefficient of -0.025; Empirical 95% bootstrapped CI of (-0.031, -0.01)).
Shortcut:
Model Training: Similar to the CUB experiment, we used the Inception V3 network initialized using the ImageNet dataset as the prediction model. We divided the chest X-ray datasets into train, validation, and test sets with a 64/16/20 random split. Then, we resampled the edema dataset such that male and edema are correlated. All methods were trained on this biased edema dataset. The hyperparameter search range was the same as the CUB experiments.
Evaluation: Since both S and Y are binary, we can resample the test set to vary the correlation between S and Y to stress test our model under different testing distributions. In particular, we swept the correlation between sex and edema from -1 (reversing the training correlation) to 1 (extremely biased distribution). A model robust to the sex shortcut should do well in all settings.
1.00
Figure 3: Result of the MIMIC- CXR experiment The model is trained on a biased dataset where S and Y has a correlation of 0.65 and tested on subsampled dataset with different correlation. The result shows that CCM EYE is consistently better than baselines. The error bars are the 95% confi- dence intervals bootstrapped on the test set.
0.90
test auc
0.85
0.50
CCM EYE
CCM RES
STD(C,X)
065
0.60
STD(X)
75.5
-030
-0.25
0.0
025
850
675
1.00
test: corr[Male, Edema)
Results: The performance of all methods is significantly affected when the test correlation is decreased to the point of reversal with the correlation in the training set (Fi This is inevitable as the shortcut provides information to predict Y given C., violating A2. However, across the range

---

of test correlation settings, when trained on the biased distribution (correlation between male and edema is 0.65), CCM EYE performs consistently better than baselines. Compared to CBM, CCM EYE is more effective when the testing correlation is similar to the training correlation. Compared to other baselines, CCM EYE most effective when the shorteut negatively correlated with the outcome, demonstrating the robustness of CCM EYE against the sex shortcut. Similar trends hold when we vary the training correlation between male and edema (Appendix AA.5
Discussion & conclusion
In this work, proposed two approaches that domain knowledge C to learn accurate model while mitigating the use of shortcuts. Our methods do not assume C' to be sufficient to make an accurate prediction and apply even scenarios where |corr(S, C'I = 1, settings previous work have not addressed. Between our proposed methods, we recommend using CCM EYE as a) it does not rely on U S, and b) empirically outperformed CCM RES on the clinical datasets. Applied to two datasets, we show that CCMs successfully reduce shortcut learning without sacrificing accuracy, even when our assumptions that C' is invariant to S and S is redundant given C do not hold.
Our work is not without limitation. First, we rely on the ability to obtain a good C. In our experiments, explore two different approaches extract C, with some success. However, there may be other approaches that apply as well. Second, other regularizers may further increase credibility (e-g., (1).. Future work could consider the task of finding the optimal credible regularizer mitigate using shortcuts. Third, we did not explore grounding/imterpreting the implicitly learned features (U), which is important for the practical use of CCM. Indeed, while CCM is capable of ruling out shortcuts that are redundant given C, it does not replace the need to carefully interpret and examine concepts picked up by the model (i.e., U), as some shortcuts contain more information given C. This requires examining what U encodes and having domain experts validate the learned concepts. Fortunately, this an active area of research and there are already ways close the interpretation gap (i.e., interpret learned concepts), both supervised (36) and unsupervised [37 25 Complementing CCM with the interpretation of U is an interesting future direction that could work further mitigate shortcut learning. Nonetheless, we expect CCM to be a step towards building trustworthy systems that can be safely applied in practice.
Acknowledgments
This work and the dissemination of this work was supported by the National Science Foundation (NSF; award IIS- 1553146 and grant 2153083) and the National Heart Lung and Blood Institute of the National Institutes of Health (NHLBI; grant RO1HL158626). The views and conclusions in this document are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of the National Science Foundation, nor of the National Institutes of Health.
References
[1] Alexander D' Amour, Katherine Heller, Dan Moldovan, Ben Adlam, Babak Alipanahi, Alex Beutel, Christina Chen, Jonathan Deaton, Jacob Eisenstein, Matthew Hoffman, al. Under- specification presents challenges for credibility in modern machine learning. arXiv preprint arXiv:2011.03395, 2020.
[2] Adarsh Subbaswamy, Peter Schulam, and Suchi Saria. Preventing failures due to dataset shift: Learning predictive models that transport. In The 22nd International Conference Artificial Intelligence and Statistics, pages 3118-3127, PMLR, 2019.
[3] Sarah Jabbour, David Fouhey, Ella Kazerooni, Michael W Sjoding, and Jenna Wiens. Deep learning applied to chest x-rays: Exploiting and preventing shorteuts. In Machine Learning for Healthcare Conference, pages 750-782, PMLR, 2020.
[4] Robert Geirhos, Carlos R Medina Temme, Jonas Rauber, Heiko H Schütt, Matthias Bethge, and Felix A Wichmann. Generalisation in humans and deep neural networks, arXiv preprint arXiv:1808.0815 08750, 2018
10

---

[5] Maggie Makar, Ben Packer, Dan Moldovan, Davis Blalock, Yoni Halpem, and Alexan- der D' Amour. Causally- motiivted shorteut removal using auxiliary labels. arXiv preprint arXiv:2 100.0622,, 2021.
[6] Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In International Conference on Machine Learning, pages 5338-5344 PMLR. 2020.
[7] Andrew Slavin Ross, Michael C Hughes, and Finale Doshi-Velez. Right for the right rea- sons: Training differentiable models by constraining their explanations. arXiv preprint arXiv:1703.03717. 2017.
[8] Hardeep Singh, Ashley ND Meyer, and Eric Thomas. The frequency of diagnostic errors in outpatient care: estimations from three large observational studies involving adult populations. BMJ quality & safery, 23(9):277-331, 2014.
[9] Yuya Matsue, Kevin Damman, Adriaan A Voors, Nobuyuki Kagiyama, Tetsuo Yamaguchi, Shunsuke Kuroda, Takahiro Okumura, Keisuke Kida, Atsushi Mizuno, Shogo Oishi, al. Time-to-furosemide treatment and mortality in patients hospitalized with acute heart failure. Journal of the American College of Cardiology, 69(25):3042-3053 2017.
[10] Kumar Dharmarajan, Kelly M Strait, Mary E Tinetti, Tara Lagu, Peter K Lindenauer, Joanne Lynn, Michelle R Krukas, Frank R Ernst, Shu-Xia Xia Li, and Harlan M Krumholz. Treatment for multiple acute cardiopulmonary conditions older adults hospitalized with pneumonia, chronic obstructive pulmonary disease, or heart failure. Journal of the American Geriatries Society, 64 (8):1574-1582, 2016.
[11] Patrick Ray, Sophie Birolleau, Yannick Lefort, Marie-Hélène Becquemin, Catherine Beigelman, Richard Isnard, Antonio Teixeira, Martine Arthaud, Bruno Riou, and Jacques Boddaert. Acute respiratory failure in the elderly: etiology, emergency diagnosis and prognosis. Critical care, 10(3):1-12, 2006.
[12] Laura Zwaan, Abel Thijs, Cordula Wagner, Gerrit van der Wal, and Daniëlle RM Timmermans. Relating faults in diagnostic reasoning with diagnostic enrors and patient harm. Academic Medicine, 87(2):149-156 2012.
[13] Sarah Jabbour, David Fouhey, Ella Kazerooni, Jenna Wiens, and Michael W Sjoding. Combining chest x-rays and electronic health record (ehr) data using machine learning to diagnose acute respiratory failure. Journal of the American Medical Informatics Association, 1966):1000-1068 2022.
[14] Federica Ciccarese, Antonio Poerio, Silvia Stagni, Domenico Attinà, Luca Fasano, Paolo Carbonara, Maria Letizia Bacchi Reggiani, and Maurizio Zompatori. Saber-sheath trachea as marker of severe airflow obstruction chronic obstructive pulmonary disease. La radiologia medica, 119(2):90-96. 2014.
[15] Jiaxuan Wang, Jeeheh Oh, Haozhu Wang, and Jenna Wiens. Learning credible models. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2417-2426. 2018.
[16] Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk mini- mization. arXiv preprint arXiv: 1907.0283, 2019.
[17] Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V Le. Autoaugment: Learning augmentation policies from data. arXiv preprint arXiv:1805.095500 2018.
[18] Dong Yin, Raphael Gontijo Lopes, Jonathon Shlens, Ekin D Cubuk, and Justin Gilmer. A fourier perspective on model robustness computer vision. arXiv preprint UrXiv:1906.08900 08988. 2019.
[19] Raphael Gontijo Lopes, Dong Yin, Ben Poole, Justin Gilmer, and Ekin D Cubuk. Improving robustness without sacrificing accuracy with patch gaussian augmentation. arXiv preprint 11X11 2019.

---

[20] Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: critical analysis of out-of-distribution generalization. arXiv preprint arXiv:2006. 16241, 2020.
[21] Meike Nauta, Ricky Walsh, Adam Dubowski, and Christin Seifert. Uncovering and correcting shortcut learning in machine learning models for skin cancer diagnosis. Diagnostics, 12(1):40, 2022.
[22] Victor Veitch, Alexander D' D'Amour, Steve Yadlowsky, and Jacob Eisenstein. Counterfac- tual invariance to spurious correlations: Why and how to pass stress tests. arXiv preprint arXiv:2/06.00545 2021.
[23] David Krueger, Ethan Caballero, Joem--eerik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. Out-of duttrf--dsstiibution generalization via risk extrap- olation (rex). In International Conference on Machine Learning, pages 5815-5826. PMLR, 2021.
[24] Chih-Kuan Yeh, Been Kim, Sercan O Arik, Chun-Liang Li, Tomas Pfister, and Pradeep Raviku- mar. On completeness-aware concept-based explanations in deep neural networks. arXiv preprint 19X.::19 2019.
[25] Yoshihide Sawada and Keigo Nakamura. Concept bottleneck model with additional unsuper- vised concepts. IEEE Access, 10:41758-41700) 2022.
[26] Mengnan Du, Ninghao Liu, Fan Yang, and Xia Hu. Learning credible deep neural networks with rationale regularization. 2019 IEEE International Conference on Data Mining (ICDM), pages 150-159. IEEE, 2019.
[27] Jeffrey M Wooldridge. Introductory econometrics: A modern approach. Cengage learning, 2015.
[28] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[29] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Re- thinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2818 1128226, 2016.
[30] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-uesd birds-200-2011 dataset. 2011.
[31] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. 2009 IEEE conference on computer vision and pattern recognition, pages 248-28 285.255.r Ieee, 2009.
[32] Alistair EW Johnson, Tom J Pollard, Seth J Berkowitz, Nathaniel Greenbaum, Matthew Lungren, Chih-ying Deng, Roger G Mark, and Steven Horng. Mimic-exr, a de-identified publicly available database of chest radiographs with free-text reports. Scientific data, 6(1):1-8, 2019.
[33] Ary Goldberger, Luis AN Amaral, Leon Glass, Jeffrey M Hausdorff, Plamen Ch Ivanov, Roger G Mark, Joseph E Mietus, George B Moody, Chung- Kang Peng, and H Eugene Stanley, Physiobank, physiotoolkit, and physionet: components of new research resource for complex physiologic signals. circulation, 101(23):e215=e222, 2000.
[34] Alistair Johnson, Lucas Bulgarelli, Tom Pollard, Steven Horn, Leo Anthony Celi, and Roger Mark. Mimic-iv, 2020. URL http:tttts::/hhyioonnet http:///wwsoonttt...rrr//cotte/s atgg/contten//mimcccv//....
[35] Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Silviana Ciurea-Ileus, Chris Chute, Henrik Marklund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, et al. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In Proceedings of the AAAI conference artificial intelligence, volume 33, pages 590-597, 2019.
12

---

[36] Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In International conference machine learning, pages 2668 2677. PMLR, 2018.
[37] Chih- Yeh, Been Kim, Sercan Arik, Chun-Liang Li, Tomas Pfister, and Pradeep Ravikumar. On completeness-awe aware concept- explanations in deep neural networks. Advances in Neural Information Processing Systems, 33:20554-2000 20565,, 2020.
[38] Ikaro Silva, George Moody, Daniel J Scott, Leo Celi, and Roger G Mark. Predicting in- hospital mortality of icu patients: The physionet/comppting in cardiology challenge 2012. In Computing in Cardiology, 2012, pages 248.248. IEEE, 2012.
[39] A. L. Goldberger, L. N. Amaral, Glass, J. M. Hausdorff, P. Ch. Ivanov, R. G. Mark, J. E. Mietus, G. B. Moody, C.-K. Peng, and H. E. Stanley. Phys- ioBank, PhysioToolkii, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation, 101(23)=e15-<2220 2000. Circulation PMID:1085218; doi:
Electronic Pages: http://cire.ahaane www.a//uraallloooooooo////////// 1T1.://00..IR.01..3.....
[40] Jean-Roger Le Gall, Philippe Loirat, Annick Alperovitch, Paul Glaser, Claude Granthil, Daniel Mathieu, Philippe Mercier, Remi Thomas, and Daniel Villers. A simplified acute physiology score for icu patients. Critical care medicine, 12(11):975-977, 1984.
[41] Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization (2014). Interna- tional Conference on Learning Representations, 15, 2015.
[42] Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International Conference on Machine Learning, pages 3319-3328. PMLR, 2017.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section 5 Discussion &
Conclusion. We also discuss the limitations of our theoretical analyses in Section 3.2.2
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See
Appendix[A.I
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yss] See Appendix A.
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] See Section
(b) Did you include complete proofs of all theoretical results? [Yes] See Section 3.2 and Appendix A.2
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as URLY? [No] All data are pub-
licly available, and we included citations in Section 4 Our code for all experiments is available at httth::t//// /ttp//ggtaaab.../. eecs.
eecs. umi ch. edu/ edu/ m1t3mmm33/mmmm htts:/Conneptrredbllleee
(b) Did you specify all the training details (e.g.. data splits, hyperparameters, how they were chosen)? [Yes] See Section 4. 4.2 and Appendix [A.4]
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes] See Table and all Figures.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [No] Since we are not comparing the efficiency of the proposed approaches, this did not seem particularly relevant especially given the space constraints.
13

---

4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] We use existing datasets: CUB and MIMIC, both cited in Sections 4.
and 4.2 respectively.
(b) Did you mention the license of the assets? [No] Please refer to the website of the creator of MIMIC-III and the CUB datasets https::/physioneee org/ cot//ent///////////// mimiciii/1.4 4/ /attps://authon: catth www.cclltccccccccco edu/27452/
(c) Did you include any new assets either in the supplemental material or as URL? [No]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [Yes] Section 4.2 MIMIC data are de-identified.
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "zlyZZTHamB", "reviewer": "Reviewer_HAYL", "summary": "This paper suggests learning unknown concepts besides the known ones (prior knowledge) in other to better tackle shortcut learning.  The authors have tested the method on the CUB (birds) dataset and edema prediction from x-ray images.", "questions": "I would like the authors to define what U refers to in practice, particularly in a clinical setting.", "limitations": "The limitation of the work can be discussed in more detail in the conclusion section. ", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:\n\n- This paper is in the direction of incorporating prior knowledge into the machine learning models which is important for addressing various problems including mitigating shortcut learning.\n- Having access to domain knowledge is not easy/cheap but I agree that without prior knowledge it is difficult to mitigate spurious correlations\n- The writing is clear and flows well.\n- The linear analysis is interesting.\n\n\nCons:\n\n- The method's novelty is in forcing the learner to learn unknown (U) concepts in addition to known ones (C). I'm not sure what U stands for in practice or why we need it. Could the authors provide specific real-world examples? Consider C as the known concept in a simple classification setup, such as how a cat looks in terms of body shape, texture/colour of skin, facial attributes, and so on. So, if we encourage the model to learn these features, what is U then? Do you mean the C's that we missed for some reason? For example, C* = {C,U} and we want to learn C*. If so, learning those should be difficult, especially in clinical settings where one must constantly check whether the newly learned features (U) make sense to include in the final prediction. Given an x-ray, if a doctor knows that a crack (the known concept, C) in a bone indicates that the bone is broken, what else do we need to learn as U?\n\n- How strong is the added noise to the samples in the CUB experiment? Shouldn't training and selecting checkpoints on noisy data but testing on clean data result in poor performance if the model hasn't seen any clean images during training?\n\n- Transfer learning appears to be unusual for learning concept features when clear connections between the proxy and man tasks are difficult to establish. Could the authors elaborate on the reasoning behind the transfer learning in experiment 4.2 in the paper, i.e., can they confirm that such an experiment makes clinical sense? Did the authors discuss with clinicians the possibility that training a model on cardiomegaly (the proxy task) would teach it the concept features C on diagnosing edema (the main task)?\n\n- I recommend that the authors remove experiment 4.2 from the main paper if they are unable to confirm (with clinicians) that the transfer learning component makes sense (cardiomegaly —> edema). The authors can substitute the Physionet Challenge experiment in the appendix for this experiment.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "wnO-LhKmDH3", "reviewer": "Reviewer_LUM8", "summary": "The authors proposed a new method for mitigating shortcuts. They built upon Concept Bottleneck Models (CBM) to also learn the unknown concepts. The proposed approach is simple and easy to understand, with the detailed analysis presented for linear models. Experiments on the CUB dataset show that the method is able to mitigate the shortcuts. \n", "questions": "1. Why does CBM perform worse than STD(X) in figure 3. for T=0?\n", "limitations": "Yes, the authors have discussed the limitations and potential negative societal impact.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### **Stregnths**\n1. The paper is well-written and easy to follow. The paper attempts to solve an important problem, i.e., mitigating shortcuts.\n2. The method tries to solve the limitations of CBMs and use both known and unknown concepts for learning. \n3. Detailed analysis of the method and assumption is presented for the linear model.\n4. Experiments on CUB dataset show clear performance gains.\n\n### **Weakness**\n1. Experiment on the MIMIC-CXR dataset doesn’t show significant performance gains.\n2. Only two experiments are included in the paper. More experiments should be conducted (CelebA, etc.).\n3. Analysis presented in section 3 is for linear models. It would be interesting to see a similar analysis for non-linear models.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "9JHMYWL1Pj", "reviewer": "Reviewer_UnPr", "summary": "The paper proposes concept credible models (CCMs) to mitigate spurious correlations/shortcuts $S$ by assuming access to some known concepts ($C$) (e.g., attribute annotations/transferred features) and then learning remaining unknown concepts ($U$). It first proposes: CCM RES, which fits a residual model to account for the unknown concepts relevant for the task. To account for the case when shortcuts are correlated with $U$, it further proposes CCM EYE which under some assumptions, mitigates reliance on $S$, even when they are correlated with $U$.", "questions": "Please address the points mentioned in the weaknesses section.\nWith attribute-based CBMs, the predictions are grounded on semantically meaningful factors of the data. But, using transferred features for $C$ and with residual features $U$, the implicitly learned features likely do not have this property. Could you elaborate more on the implications and the potential ways to close the interpretability gap?\n", "limitations": "I think the section on ethical considerations is adequate.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "$\\textbf{Strengths}$\n\n\n1. This work addresses an important weakness of the original Concept Bottleneck Model (CBM), which assumes that all the concepts required for the task are known in advance and the known set can be used to perform the task. CBM, in its original formulation, is incapable of learning additional concepts and shows inferior results if other concepts are indeed necessary. The proposed CCM RES/EYE models on the other hand, are capable of learning unknown concepts too, while still remaining robust to the shortcuts, so this has practical value.\n\n2. Leveraging domain knowledge (e.g., extra labels or features that represent task-relevant concepts) is a realistic way of using deep learning models robustly in practice. Robustness can be increased by relying on those task-relevant features instead of the shortcuts. One major advantage of the proposed approach is that this is done without actually accessing information about the shortcuts. This is different from many existing works which use information about spurious factors e.g., to divide the dataset into multiple environment (e.g., as done in IRM).\n\n3. CCM EYE/CCM RES obtain comparable accuracies on both biased and clean test accuracies for CUB, showing that it is relying on the core features instead of the spurious ones. Adaptation of EYE regularization on the concept space improves alignment with extra information to mitigate shortcuts. This is logically sound.\n\n4. Both assumptions $A1$ and $A2$ seem reasonable even for practical scenarios. I like the studies performed when the assumptions are violated. When violating $A1$ ($C$ is counterfactually invariant to $S$), the method remains robust, unless $C$ is heavily impacted by $S$. Violation of $A2$ ($S$ is redundant given $C$) shows that it is not necessary to have perfect $C$ i.e., not all dimensions need to be relevant leaving room for even the experts to be imperfect.\n\n\n$\\textbf{Weaknesses}$\n\n1. I have a question in regards to model interpretability. With attribute-based CBMs, the predictions are grounded on semantically meaningful factors of the data. But, using transferred features for $C$ and with residual features $U$, the implicitly learned features likely do not have this property. Could you elaborate more on the implications and the potential ways to close the interpretability gap?\n\n2. While the method compares accuracies on biased/clean test sets to provide evidence that the model is relying on concepts instead of shortcuts, the study does not provide qualitative visualizations/quantitative analyses to confirm that. I wish it was studied on other datasets where one could perform visual grounding analysis (compute segmentation metrics) to show it is not looking at the spurious regions of images.\n\n3. It would have been impactful to test on WILDS [1] and other more realistic benchmarks instead of introducing artificial shortcuts in CUB. The distribution shift/shortcuts should be reflective of the real world challenges. Furthermore, it would also be interesting to see how one could inject known concepts/domain expertise for such cases.\n\n[1] Koh, Pang Wei, et al. \"Wilds: A benchmark of in-the-wild distribution shifts.\" International Conference on Machine Learning. PMLR, 2021.\n\nMissing reference:\nLearning additional concepts for CBMs was explored in [2], so it can be cited.\n\n[2] Sawada, Yoshihide, and Keigo Nakamura. \"Concept Bottleneck Model With Additional Unsupervised Concepts.\" IEEE Access 10 (2022): 41758-41765.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept