Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper suggests learning unknown concepts besides the known ones (prior knowledge) in other to better tackle shortcut learning.  The authors have tested the method on the CUB (birds) dataset and edema prediction from x-ray images.
strengths_and_weaknesses: Pros:

- This paper is in the direction of incorporating prior knowledge into the machine learning models which is important for addressing various problems including mitigating shortcut learning.
- Having access to domain knowledge is not easy/cheap but I agree that without prior knowledge it is difficult to mitigate spurious correlations
- The writing is clear and flows well.
- The linear analysis is interesting.


Cons:

- The method's novelty is in forcing the learner to learn unknown (U) concepts in addition to known ones (C). I'm not sure what U stands for in practice or why we need it. Could the authors provide specific real-world examples? Consider C as the known concept in a simple classification setup, such as how a cat looks in terms of body shape, texture/colour of skin, facial attributes, and so on. So, if we encourage the model to learn these features, what is U then? Do you mean the C's that we missed for some reason? For example, C* = {C,U} and we want to learn C*. If so, learning those should be difficult, especially in clinical settings where one must constantly check whether the newly learned features (U) make sense to include in the final prediction. Given an x-ray, if a doctor knows that a crack (the known concept, C) in a bone indicates that the bone is broken, what else do we need to learn as U?

- How strong is the added noise to the samples in the CUB experiment? Shouldn't training and selecting checkpoints on noisy data but testing on clean data result in poor performance if the model hasn't seen any clean images during training?

- Transfer learning appears to be unusual for learning concept features when clear connections between the proxy and man tasks are difficult to establish. Could the authors elaborate on the reasoning behind the transfer learning in experiment 4.2 in the paper, i.e., can they confirm that such an experiment makes clinical sense? Did the authors discuss with clinicians the possibility that training a model on cardiomegaly (the proxy task) would teach it the concept features C on diagnosing edema (the main task)?

- I recommend that the authors remove experiment 4.2 from the main paper if they are unable to confirm (with clinicians) that the transfer learning component makes sense (cardiomegaly —> edema). The authors can substitute the Physionet Challenge experiment in the appendix for this experiment.
questions: I would like the authors to define what U refers to in practice, particularly in a clinical setting.
limitations: The limitation of the work can be discussed in more detail in the conclusion section.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors proposed a new method for mitigating shortcuts. They built upon Concept Bottleneck Models (CBM) to also learn the unknown concepts. The proposed approach is simple and easy to understand, with the detailed analysis presented for linear models. Experiments on the CUB dataset show that the method is able to mitigate the shortcuts. 

strengths_and_weaknesses: ### **Stregnths**
1. The paper is well-written and easy to follow. The paper attempts to solve an important problem, i.e., mitigating shortcuts.
2. The method tries to solve the limitations of CBMs and use both known and unknown concepts for learning. 
3. Detailed analysis of the method and assumption is presented for the linear model.
4. Experiments on CUB dataset show clear performance gains.

### **Weakness**
1. Experiment on the MIMIC-CXR dataset doesn’t show significant performance gains.
2. Only two experiments are included in the paper. More experiments should be conducted (CelebA, etc.).
3. Analysis presented in section 3 is for linear models. It would be interesting to see a similar analysis for non-linear models.

questions: 1. Why does CBM perform worse than STD(X) in figure 3. for T=0?

limitations: Yes, the authors have discussed the limitations and potential negative societal impact.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper proposes concept credible models (CCMs) to mitigate spurious correlations/shortcuts $S$ by assuming access to some known concepts ($C$) (e.g., attribute annotations/transferred features) and then learning remaining unknown concepts ($U$). It first proposes: CCM RES, which fits a residual model to account for the unknown concepts relevant for the task. To account for the case when shortcuts are correlated with $U$, it further proposes CCM EYE which under some assumptions, mitigates reliance on $S$, even when they are correlated with $U$.
strengths_and_weaknesses: $\textbf{Strengths}$


1. This work addresses an important weakness of the original Concept Bottleneck Model (CBM), which assumes that all the concepts required for the task are known in advance and the known set can be used to perform the task. CBM, in its original formulation, is incapable of learning additional concepts and shows inferior results if other concepts are indeed necessary. The proposed CCM RES/EYE models on the other hand, are capable of learning unknown concepts too, while still remaining robust to the shortcuts, so this has practical value.

2. Leveraging domain knowledge (e.g., extra labels or features that represent task-relevant concepts) is a realistic way of using deep learning models robustly in practice. Robustness can be increased by relying on those task-relevant features instead of the shortcuts. One major advantage of the proposed approach is that this is done without actually accessing information about the shortcuts. This is different from many existing works which use information about spurious factors e.g., to divide the dataset into multiple environment (e.g., as done in IRM).

3. CCM EYE/CCM RES obtain comparable accuracies on both biased and clean test accuracies for CUB, showing that it is relying on the core features instead of the spurious ones. Adaptation of EYE regularization on the concept space improves alignment with extra information to mitigate shortcuts. This is logically sound.

4. Both assumptions $A1$ and $A2$ seem reasonable even for practical scenarios. I like the studies performed when the assumptions are violated. When violating $A1$ ($C$ is counterfactually invariant to $S$), the method remains robust, unless $C$ is heavily impacted by $S$. Violation of $A2$ ($S$ is redundant given $C$) shows that it is not necessary to have perfect $C$ i.e., not all dimensions need to be relevant leaving room for even the experts to be imperfect.


$\textbf{Weaknesses}$

1. I have a question in regards to model interpretability. With attribute-based CBMs, the predictions are grounded on semantically meaningful factors of the data. But, using transferred features for $C$ and with residual features $U$, the implicitly learned features likely do not have this property. Could you elaborate more on the implications and the potential ways to close the interpretability gap?

2. While the method compares accuracies on biased/clean test sets to provide evidence that the model is relying on concepts instead of shortcuts, the study does not provide qualitative visualizations/quantitative analyses to confirm that. I wish it was studied on other datasets where one could perform visual grounding analysis (compute segmentation metrics) to show it is not looking at the spurious regions of images.

3. It would have been impactful to test on WILDS [1] and other more realistic benchmarks instead of introducing artificial shortcuts in CUB. The distribution shift/shortcuts should be reflective of the real world challenges. Furthermore, it would also be interesting to see how one could inject known concepts/domain expertise for such cases.

[1] Koh, Pang Wei, et al. "Wilds: A benchmark of in-the-wild distribution shifts." International Conference on Machine Learning. PMLR, 2021.

Missing reference:
Learning additional concepts for CBMs was explored in [2], so it can be cited.

[2] Sawada, Yoshihide, and Keigo Nakamura. "Concept Bottleneck Model With Additional Unsupervised Concepts." IEEE Access 10 (2022): 41758-41765.

questions: Please address the points mentioned in the weaknesses section.
With attribute-based CBMs, the predictions are grounded on semantically meaningful factors of the data. But, using transferred features for $C$ and with residual features $U$, the implicitly learned features likely do not have this property. Could you elaborate more on the implications and the potential ways to close the interpretability gap?

limitations: I think the section on ethical considerations is adequate.
review_title: null

Meta-review: metareview: The paper considers the important problem of shortcut learning and the existing solutions and suggests learning unknown concepts besides the known ones (prior knowledge) in other to better tackle shortcut learning. They built upon Concept Bottleneck Models (CBM) to also learn the unknown concepts. The proposed approach is simple and easy to understand, with the detailed analysis presented for linear models.The authors have tested the method on the CUB (birds) dataset and edema prediction from x-ray images.
The considered problem is important and the experiment results validate the effectiveness of the proposed approach. Moreover, the authors were able to address reviewers questions and concerns during the rebuttal period. Therefore, I suggest the paper to get accepted.  
confidence: Certain
Decision: Accept