Formalizing Spuriousness of Biased Datasets using Partial Information Decomposition

Authors: Barproda Halder, Faisal Hamman, Pasan Dissanayake, Qiuyi Zhang, Ilia Sucholutsky, Sanghamitra Dutta
Primary area: interpretability and explainable AI
Keywords: Explainability Framework, Spuriousness, Partial Information Decomposition, Blackwell Sufficiency, Auto-encoder, Worst-group Accuracy

Abstract:
Spuriousness arises when there is an association between two or more variables in a dataset that are not causally related. Left unchecked, they can mislead a machine learning model into using the undesirable spurious features in decision-making over the core features, hindering generalization. In this work, we propose a novel explainability framework to disentangle the nature of such spurious associations, i.e., how the information about a target variable is distributed among the spurious and core features. Our framework leverages a body of work in information theory called Partial Information Decomposition (PID) to first decompose the total information about the target into four non-negative quantities namely unique information (in core and spurious features respectively), redundant information, and synergistic information. Next, we leverage this decomposition to propose a novel measure of the spuriousness of a dataset that steers models into choosing the spurious features over the core. We arrive at this measure systematically by examining several candidate measures, and demonstrating what they capture and miss through intuitive canonical examples and counterexamples. Our proposed explainability framework Spurious Disentangler consists of segmentation, dimensionality reduction, and estimation modules, with capabilities to specifically handle high dimensional image data efficiently. Finally, we also conduct empirical evaluation to demonstrate the trends of unique, redundant, and synergistic information, as well as our proposed spuriousness measure across several benchmark datasets under various settings. Interestingly, we observe a novel tradeoff between our measure of dataset spuriousness and empirical model generalization metrics such as worst-group accuracy, further supporting our proposition.

Decision: reject
Reviewer scores: 3, 5, 6, 5, 8

Reviews and discussion:
<thread>
creation date:
1730181609882

thread_type:
official_review

<thread_object>
creator: reviewer_wncj

<content>
summary:
This paper introduces a novel framework called spurious disentangler that uses Partial Information Decomposition (PID) to analyze and quantify spurious correlations in datasets. The authors propose a new measure $M_{sp}$ that assesses how likely a dataset will lead models to rely on spurious features over core features, implementing this through a three-module system of segmentation, dimensionality reduction, and PID estimation. Through experiments on multiple datasets, they demonstrate a consistent negative correlation between their spuriousness measure and model generalization metrics.

strengths:
1. This paper clearly explains why simpler measures are insufficient and develops a novel spuriousness measure through examples and counterexamples. 

2. This paper proposes a novel and complete framework called spuriousness disentangler for handling high-dimensional image data.

3. This paper provides extensive experimental results. It tests on multiple benchmark datasets, examines different types of sampling biases, and provides Grad-CAM visualizations. The experimental results well support their claims.

4. I think this research is of great significance. It can help identify problematic datasets before expensive model training.

weaknesses:
1. This works requires manual identification of core and spurious features, which significantly limits its applicability since this might requires human-expert knowledge.

2. This paper focuses on the image classification task, it might be better if the authors can validate their framework on some NLP tasks.

questions:
N/A
</content>
</thread_object>

</thread>
<thread>
creation date:
1730631063858

thread_type:
official_review

<thread_object>
creator: reviewer_qqlo

<content>
summary:
This work focuses on the problem of spurious correlation in the data-driven models. It leverages the partial information decomposition (PID) to decompose the total information into four quantities such as the unique information of core and spurious features, the redundant information that is shared by two features, and the synergistic information that arises due to the collaboration of the two features. Based on this decomposition, the authors propose a framework called Spurious Disentangler to empirically evaluate the “spuriousness” of image data.

strengths:
1. The paper focuses on an important task that evaluates the degree of “spuriousness” of a dataset.
2. The idea of decomposing the total information into the aforementioned four values to study the spurious correlation problem is very interesting.
3. The paper is well-written and easy to follow.

weaknesses:
1. My main concern is the contribution and practicality of the proposed method. Although the idea of using information theory to quantify spuriousness is interesting, the actual use case of the proposed framework is limited and not properly discussed. Most results only show that the framework “is consistent with existing knowledge” (e.g. Theorem 1, experimental observations).

2. The proposed framework, “spuriousness disentangler”, relies heavily on segmentations. This greatly reduces its application scenarios. Datasets where spurious and core features can be explicitly separated as object & background are limited.

3. The requirement of the existence of a pre-trained semantic segmentation model is problematic. This is equivalent to requiring a much larger and more general dataset or a much more powerful model where the spurious correlation problems are already mitigated to a good extent. Such a “Deus Ex Machina” approach is questionable in practice.

4. The experiment section lacks insights and does not highlight the contribution of the proposed method.
    - The experiments are repeated on four datasets. However, the observations are all descriptive yet the contribution and the superiority of the proposed method are limited. For example, in L427-L429, the authors conclude from Fig. 7 that $M_{sp}$ is a good measure because it is consistent with worst-group accuracy. This claim treats worst-group accuracy as the standard for evaluating “spuriousness”. The contribution of PID is completely missing here. 
    - The qualitative visualization of Figure 8 is only one sample. In L430, it is concluded from Figure 8 that “when the dataset is balanced or mixed background, the model emphasizes \textbf{more} on the core features (the red regions)”. This justification is insufficient. To justify that the model focuses “more” on the core feature. A score such as the IoU should be computed over the entire dataset to support this claim.


[Minor]:

1. L76, L78: “We first” appears twice.
2. L82-L83: The meaning of $A$ is not specified in $\mathrm{Syn}(Y:A,B)$. Is it supposed to be $F$? The definitions of $A$ and $F$ overlap throughout the manuscript and create unnecessary difficulty for the audience. The authors may consider unifying them for a clearer presentation.
3. In L147-148, $\mathcal{X}$ isn’t defined above.
4. In Figure 5, the location of the text “Encoder”, and “Decoder” and the curly brackets are misplaced.
5. The spacing after Figure 8 is completely missing.
6. It’s better to add captions for the subfigures in Figure 8 to indicate the five variants.

questions:
1. Regarding the main concern, can the authors elaborate more on the use case of the framework where it is a better choice than existing measures such as worst-group accuracy?
2. In L53, the authors mentioned, “this notion of spuriousness in any given dataset has classically lacked a formal definition. To address this gap,…”. There are similar works discussing the quantification of spuriousness. e.g. [1,2]. Can the authors elaborate more on how the contribution of this work differs from this existing work? 
3. Is it possible to generalize the proposed “spuriousness disentangler” framework to datasets where the segmentation of two features is infeasible? For example, in tabular data, the spurious features can be sensitive attributes such as gender, race, etc. These features cannot be segmented. 
4. In counterexample 1, the authors refer to canonical example 1 and claim that this scenario should be considered as having “no spuriousness”. However, in canonical example 1, since $B = Y+N_B$, $F = Y+N_F$ with i.i.d. noise $N_B, N_F$, the spurious feature $B$ is equally connected to the label $Y$ compared with the core feature. Shouldn’t this be the “most spurious” scenario?


[Reference]

[1] Ye, H., Zou, J., & Zhang, L. (2023, April). Freeze then train: Towards provable representation learning under spurious correlations and feature noise. In *International Conference on Artificial Intelligence and Statistics* (pp. 8968-8990). PMLR.

[2] Wang, Y., & Wang, X. (2024, April). On the Effect of Key Factors in Spurious Correlation: A theoretical Perspective. In *International Conference on Artificial Intelligence and Statistics* (pp. 3745-3753). PMLR.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730651331880

thread_type:
official_review

<thread_object>
creator: reviewer_scou

<content>
summary:
The framework builds on a foundation in information theory known as Partial Information Decomposition (PID) to break down the total information about the target variable into four distinct, non-negative components: unique information (within both core and spurious features), redundant information, and synergistic information. Using this decomposition, we introduce a novel metric for assessing the spuriousness of a dataset, guiding models to prioritize spurious features over core features.

strengths:
this paper is based on a sound foundation: abstract / line 96 - line 125
the provided experimental evaluation doesn't include the statistical ratios (mean + std)

weaknesses:
poor writing quality

questions:
-
</content>
</thread_object>

</thread>
<thread>
creation date:
1730704732485

thread_type:
official_review

<thread_object>
creator: reviewer_pw3s

<content>
summary:
In this paper, the authors propose a novel measure of spuriousness by utilizing Partial Information Decomposition (PID) and an explainability framework consisting of segmentation, dimensionality reduction, and estimation modules to specifically handle high dimensional image data efficiently. In general, the proposed measure of spuriousness is interesting.

strengths:
The proposed methods are novel and the experiments are extensive.

weaknesses:
The writing in some places is a bit unclear and some implementation details are lacking.

questions:
1, In the proposed autoencoder-based explainability framework, it seems that we need to select a non-negative constant $\gamma$ in dimensionality reduction phase, the readers may want to know how to select the value of $\gamma$. It will be helpful if the authors can give some guidance about the selection of $\gamma$.

2, In this paper, the authors propose a novel metric of spuriousness, then how can we identify one feature as a spurious one? It seems that we need the threshold?

3, It seems that there is a typo in "a the" in line 266.

4, In line 182, what is "Z_3 \bigoplus N$?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730710285309

thread_type:
official_review

<thread_object>
creator: reviewer_2nsi

<content>
summary:
This paper presents a novel framework to quantify dataset spuriousness, addressing a gap in formalizing how spurious correlations between non-causal features and the label affect model generalization. The measure is calculated based on unique information and synergistic information values obtained from partial information decomposition. Experiments show negative correlation between the values of this measure and generalization metrics under distribution shift.

strengths:
1)The problem is important and relevant to OOD Generalization.

2)The proposed measure is novel.

3)The experiments consider a range of datasets and somewhat empirically support the claims of the paper.

weaknesses:
1)The measure relies on the assumption that causal and spurious features can be separated in the image as foreground and background. However, this assumption may not hold universally or even in most of the cases; for instance, spurious features like rotation or color affect all pixels rather than specific regions. In fact, disentangling causal and spurious features in a major challenge for many OOD tasks.

2)In the experiments, standard deviations or error bars are not provided, making it difficult to assess the scientific significance of the results.

3)There is no theoretical proof for why a higher value of the proposed measure would correspond to worse OOD performance. 

4)Related to above, this paper lacks novel theoretical contribution. The theory presented is straightforward from partial information decomposition theory. Methodologically too, the main contribution comes from Bertschinger et al., (2014) which is used to calculate PID.

questions:
1)During dimensionality reduction, how are the number of clusters chosen? Why do we need to approximate the distribution in a discrete way and what do we lose by doing so?

2)Why are other measures like I(Y;B) etc. not clearly reported in the results? This would help us compare the proposed measure with other measures.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732684052362

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
SUMMARY OF AUTHOR RESPONSE

comment:
We thank the AC and reviewers for reviewing this paper. We appreciate that the reviewers have found our contributions novel and interesting. We have made several edits to the main paper (marked in blue), including reorganizing the Main Results, including a new Fig. 3 to highlight the nuanced insights from Partial Information Decomposition (PID), and performing **additional experiments for automatic segmentation (Appendix A), tabular datasets (Appendix B), and other findings for the previous experiments (Appendix C.1).**

**On Novelty:** We would like to reiterate that the problem of **mathematically explaining the nature of spurious correlations for dataset evaluation** has received limited attention. Existing techniques such as data reweighting or alternate ways of model training have often shown empirical success in avoiding the spurious features; however, *a fundamental understanding of the nature of spurious patterns in the dataset itself (in a “pre-emptive” or “anticipative” manner) prior to actual model training is less explored.*

Our main *novelty* lies in isolating four different types of statistical dependencies in a dataset involving the core ($F$) and spurious features ($B$), bringing in the nuanced framework of Partial Information Decomposition (PID) that has not been explored in this context before. We can identify when $F$ or $B$ is indispensable, or when either $F$ or $B$ can suffice (ambiguous), or when both are needed jointly, which goes beyond simply saying whether a model is relying on spurious features or not (also **see the new Fig. 3 in the main paper and other reorganizations of the Main Results into concrete lemmas**). 

**On Separating Core and Spurious Features:**  There is limited work on automatically identifying core and spurious features without using any other information.  We now demonstrate applicability of our framework *even in scenarios where the core and spurious feature split is not known apriori.* 

•	For instance, in **Appendix A**, we show how one can leverage automatic segmentation such as CLIPseg to at least identify various objects and then choose some of the most relevant objects as core, e.g., in supervised classification tasks on Waterbirds, one might know the “bird” object is core. The remaining image can be chosen as spurious for dataset quality evaluation. 

•	In another use-case in **Appendix B**, we show how this framework can also be applied for explainability on tabular datasets, e.g., to understand the dependencies of one feature such as gender with respect to another subset of core features. 

**Clarification on Experiments:** Since our measure is a *pre-emptive* dataset evaluation measure prior to training, the **goal of our experiments** is to see how our anticipations from dataset agree with post-training model performance metrics like worst-group accuracy, SHAP, IoU, etc. *Notably, they are not mathematically the same thing – our measures anticipate spuriousness from dataset based on how the Bayes optimal classifier(s) **should** behave, and the empirical measures show how specific models actually behave (when trained on that dataset without doing anything specific for spuriousness).*  Nonetheless, we observe an interesting correlation with our anticipative dataset quality measure and post-training model performance metrics on a broad range of experimental setups, further validating the efficacy of our approach.

We have also included the suggested references and are committed to making any other edits or answering any other questions. We look forward to the responses to our rebuttal.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper proposes a explanability framework to disentangle spuriousness by using partial information decomposition (PID). The paper's method can isolate four different types of statistical dependencies in a dataset involving the core and spurious features. The paper proposes a framework that has segmentation, dimensionality reduction, and estimation modules. Experiments show a tradeoff between the proposed measure of spuriousness and empirical model generalization metrics such as worst-group accuracy. The reviewers mention that the problem of spuriousness is an important topic, the proposed decomposition idea is interesting and novel, and experiments support the paper's claims.

There were two reviewers that gave high scores (6 and 8), but both of them privately communicated with the area chair that the area is outside their expertise and that their understanding of the paper is limited. I assigned less weight to the two compared to the other reviews.

The remaining three reviews gave lower scores (5, 5, 3). Reviewer pW3s mentioned writing is unclear and implementation details are missing. The authors provided further information. pW3s did not provide a final response but the rebuttal addresses the original concerns. Reviewer 2NSi mentioned a few weaknesses such as strong assumption about the spuriousness, lack of error bars in experiments, lack of theory of relationship between OOD performance, and lack of methodological novelty. The paper was updated with more experiments and error bars on some of the experiments. Although disagreements remained about the novelty, the discussions addressed the original concern to some extent and the novelty is clarified. Reviewer qqLo raised many important weaknesses. After some discussions, the reviewer raised the score from 3 to 5, but the reviewer's concerns regarding practicality and validity remained.

Based on the reviews and discussions and the summary above, I would like to recommend rejection this time, but hope the authors can further discuss practicality and validity, revise the paper, and resubmit in the future (A minor comment: I also recommend to include a Conclusion section. Without it, it would be logically strange and difficult for the reader to grasp the takeaways of this paper.)