Towards Formalizing Spuriousness of Biased Datasets Using Partial Information Decomposition
Authors: Barproda Halder
Faisal Hamman
Pasan Dissanayake
Qiuyi Zhang
Ilia Sucholutsky
Sanghamitra Dutta
Venue: Accepted by TMLR ()

Abstract: Spuriousness arises when there is an association between two or more variables in a dataset that are not causally related. In this work, we propose an explainability framework to preemptively disentangle the nature of such spurious associations in a dataset before model training. We leverage a body of work in information theory called Partial Information Decomposition (PID) to decompose the total information about the target into four non-negative quantities, namely unique information (in core and spurious features, respectively), redundant information, and synergistic information. Our framework helps anticipate when the core or spurious feature is indispensable, when either suffices, and when both are jointly needed for an optimal classifier trained on the dataset. Next, we leverage this decomposition to propose a novel measure of the spuriousness of a dataset. We arrive at this measure systematically by examining several candidate measures, and demonstrating what they capture and miss through intuitive canonical examples and counterexamples. Our framework Spurious Disentangler consists of segmentation, dimensionality reduction, and estimation modules, with capabilities to specifically handle high-dimensional image data efficiently. Finally, we also perform empirical evaluation to demonstrate the trends of unique, redundant, and synergistic information, as well as our proposed spuriousness measure across $6$ benchmark datasets under various experimental settings. We observe an agreement between our preemptive measure of dataset spuriousness and post-training model generalization metrics such as worst-group accuracy, further supporting our proposition. The code is available at https://github.com/Barproda/spuriousness-disentangler.

Published in Transactions on Machine Learning Research (11/2025)
Towards Formalizing Spuriousness of Biased Datasets Using Partial Information Decomposition
Barproda Halder
bhalder@umd.dd eduu
Deportment of Electrical and Computer Engineering University of Moryland, College Park
Faisal Hamman
fhammaan Grumd. edu
Department of Electrical and Computer Engineering University of Morylond, College Park
Pasan Dissanayake
pasand@umd. edu
Department of Electrical and Computer Engineering University of Moryland, College Park
Qiuyi Zhang
giuyiz@google.com com com
Google Research
Ilia Sucholutsky
is2961@princtonn.com edu
Department of Computer Science
Princeton University
Sanghamitra Dutta
sanghamd@umd.cm edu
Department of Electrical and Computer Engineering University of Moryland, College Park
Reviewed OpenReview:
Abstract
Spuriousness arises when there is an association between two or more variables in a dataset that are not causally related. In this work, we propose an explainability framework to pre- emptively disentangle the nature of such spurious associations in a dataset before model training. We leverage a body of work information theory called Partial Information Decomposition (PID) decompose the total information about the target into four non- negative quantities, namely unique information (in core and spurious features, respectively), redundant information, and symeryislic information. Our framework helps anticipate when the core or spurious feature is indispensable, when either suffices, and when both are jointly needed for an optimal classifier trained on the dataset. Next, we leverage this decomposi- tion to propose a novel measure the spuriousness of a dataset. We arrive at this measure systematically by examining several candidate measures, and demonstrating what they cap- ture and miss through intuitive canonical examples and counterexamples. Our framework Spurious Disentangler consists of segmentation, dimensionality reduction, and estimation modules, with capabilities to specifically handle digh-ddiennioonl image data efficiently. Fi- nally, we also perform empirical evaluation to demonstrate the trends of unique, redundant, and synergistic information, as well as our proposed spuriousness measure across 6 bench- mark datasets under various experimental settings. We observe an agreement between our preemptive measure of dataset spuriousness and post-training model generalization metrics such as gorst-group accuracy, further supporting our proposition. The code is available at https://gitthu http:////aa.ppa./ooooooosssraaaa

---

Published in Transactions on Machine Learning Research (11/2025)
1 Introduction
The success of machine learning is heavily determined by the quality of datasets
BIRDTYPE
used for training or fine-tuning, Spurious patterns Haigg 2003) arise when two more variables are associated in a datasel even though they do not have causal relation, e.g., image classifiers on Waterbird dataset (Wah et al., 2011) learn to use the background rather than the foreground for classification, because most waterbirds are photographed on a water background (see Fig. D This pattern in the dataset misleads a classifier into learning an undesirable spurious link between the target label (bird type) and background ("spurious" feature) as opposed to the foreground (core feature). Spuriousness in datascts may result in deceptively high performance on in-distribution datasets but significantly hinders generalization on out-of-distribute datasets, e.g., accuracy on minority groups like waterbirds with
WATTESBIO
SPURIUUS SEATUREEEEAESS
BACCGROO MANCAGROOUNO MOUND
MAIORITY
MINORITY
MINORITY
MAIORITY
Figure 1: Spuriousness in Waterbird dataset due to sampling bias.
land background is low (Lynch et al. 2023 Sagawa et al. 2019 Puli et al. 2023
Despite advances in dataset-based and model-trriningg-basee approaches to mitigate such spurious pat- terns (Ye ct al. 2024 Srivastava 2023 Ghouse et al. 2024), the notion of spuriousness in any given datasct has classically lacked formal definition. To address this gap, in this work, we ask the following question: Given dataset and split of core and spurious features, can we preemptively quantify the spuriousness of the dataset before training! In essence, our goal is to arrive at a framework that would help anticipate the feature preferences of an optimal classifier prior to training.
To this end, we provide an information-theoreee explainability framework to disentangle the nature of spu- rious associations in a datasct, i.c., how the information about the target variable distributed among the spurious and core features. We leverage body of work in information theory called Partial Information Decomposition (PID) (Bertschinger et al.} 2014 Banerjee al., 2018), which has its roots in statistical decision theory. We note that classical informationn-theoree measures such mutual information (Cover & Thomas 2012) capture the entire statistical dependency between two random variables but fail to capture how this dependency is distributed among those variables, i.e., the structure of the multivariate informa- tion. Partial Information Decomposition (PID) addresses this nuanced issue by providing a formal way of disentangling the joint information content between the core and spurious features into unique, redundant, or synergistic information. We leverage this decomposition to systematically arrive at a novel measure of dataset spuriousness with empirical evaluation on high-dimensional image datasets. This work provides a more nuanced understanding of the interplay between spurious and core fcatures in dataset that can better inform dataset quality assessment. Our main contributions can be sumnarized as follows:
We leverage PID to disentangle the total information about a target (Y) in the core (F) and spurions (B) features into four non-negative terms: unique information (in core and spurious fealures respeclively), redundand information, and syneryislic in- formation (see Proposition D We elucidate four types of statistical dependencies captured by the PID terms (scc Fig. 3) providing proemptive insights when an optimal classifier might find a spurious fea- ture more informative useful than the core features. We establish how unique information quantifies the informativeness of feature over another for predicting Y (see Theorem D. Then, redundant information turns out to be the common information that can be obtained from either the spurious or core features, allowing a classifier to potentially choose either without any preference. An interesting term is synergy that captures scenarios when both spurious and core features are jointly informative about the target Y but not individually (classificr likely to use both spurious and corc).
Unraveling nature of spurious associations leveraging PID:
Novel measure of dataset spuriousness: Though many works attempt to prevent model from learning spurious patterns, there is limited theoretical understanding of how to quantify the spuriousness of a dataset given a choice of core and spurious features. In this work, we leverage PID to propose a novel mcasure of the undesirable spuriousness of a dataset (Map) that steers predictors into choosing the spurious features over the core (see Proposition 2 We arrive at this measure systematically by examining several candidate measures, and demonstrating what they capture and miss through intuitive canonical examples and counterexamples. Our measure provides a fundamental understanding of feature informativeness for classification task, enabling dataset quality assessment and interpretability.
2

---

Published in Transactions on Machine Learning Research (11/2025)
Spuriousness Disentangler: We propose an autoencoder-basee explainability framework that we call Spuriousness Disentangler - to obtain the four PID values as well as our spuriousness measure Map for high- dimensional image data. The framework consists of three modules: (i) Segmentation: This module performs segmentation to separate the foroground (core features F) and background (spurions features B) for every image, either using pre-trained semantic segmentation models (Lin el al., 2017) or CLIPSeg (Lüddecke & Semantic Segmentation model if necessary; (ii) Dimensionality Reduction: An autoencoder converts high-dimensional images into lower-dimensional, discrete feature rep- resentations. The dimensionality reduction and clustering are performed jointly through minimization of a joint loss function, drawing inspiration from Guo et al. (2017). (iii) Estimation: The final step includes the estimation of the joint probability distribution of the acquired lower-dimensional representation, followed by
Ecker 2022), which is an
optimization (James et al. 2018
Liang el al. 2023 to compule PID values and Map.
Since our proposed framework is a preemptive dataset explainability framework, the goal of our experiments is to show broad agreement between our anticipations from the dataset before training and the post-training behavior of the models for various experimental setups. We examine four experimental setups: i) Both core and spurious features are available; (ii) Either core or spurious is available; (iii) Segmentation to obtain core and spurions features; and (iv) Non-spatial spuriousness. Our evaluation spans six datasets: Waterbird (Wah et al. 2011), Adult (Becker & Kohavi 1996). CelebA (Lee et al.) 2020). Dominoes (Shah et al. 2020), Spawrious (Lynch et al. 2023). and Colored MNIST (Arjovsky et al. 2019). We
Empirical results:
observe a negative correlation between our proposed measure of dataset spuriousness Map and post-training model generalization metries, such as the worst-group accuracy for each experimental setting. We also study
Grad-CAM (Selvaraju et al. 2017] visualizations and intersection-over-miion (IoU) metric (Rezatofighi et al.,
2019 to further confirm which features are actually being emphasized by the model.
A framework for dataset explainability provides an alternative to combating spuriousness during training by providing preemptive insights to inform the training process (analogous to "nutrition labels" Yang et al. (2018) or "datashects for datasets" Gebru et al. (2021)) ) By enabling dataset quality check and cleansing prior to training, it can bypass expensive adversarial training, oflen used to avoid spurious patterns. Having clean datasets for fine-tuning is particularly valuable in the era of large foundation models when one has limited control over the training process.
Related Works: There are several perspectives on spurious correlation (see Haig 2003 1 Kirichenko et al. Izmailov et al.
[2 |s
; Wu et al. (2023): Ye et al. (2023); Liu et al. (2023): Stromberg et al. (2024)
Singla & Feizi [2021] b Moayeri et al. (2023 Lynch et al. (2023) and the references therein; also see surveys Ye
ct al. (2024): Srivastava (2023): Ghouse et al. (2024)). ). Spuriousness mitigation techniques are broadly divided
into two groups: (i) Dataset- based techniques GGel et al. 2020 Kirichenko et al.] 2022
Wu et al.]
2023
Moayeri et al.
2023 Liu et al.
2021 and (ii) Learning- based techniques (Liu al.) 2023
Yang et al. 2023
Ye et al. 2023 Zhang et al.)
2022). Among mataset-based techniques, Kirichenko et al.
(2022) shows that
last-layer fine-tuning of a pre- pre-trained model with a group- balaneb subset of data is sufficient to mitigate spuriousness. Wu et al. (2023) proposes a concept-aware spurious correlation mitigation technique. There are also some works that try to separate spurious and core features in the feature space of deep neural networks using external feedback (Sohoni et al., 2020 Kattakinda et al. 2022). ) Recent work Waag & Wang (2024) looks into the problem through the mathematical lens of separability of the spurious and core features under a mixture of Gaussian assumptions (also assuming a split between core and spurious). Ye et al. (2023) discusses how the noise in the core feature plays a role in a model's reliance on it. Our novelty lies in investigating the problem of spurious patterns through the lens of PID, rooted in statistical decision theory, focusing on quantifying the spuriousness of dataset for interpretability and quality assessment. Our work isolates four specific types of statistical dependencies in the dataset, providing a more nuanced understanding (see Fig. going beyond solely identifying a model's reliance on a specific feature.
Partial Information Decomposition (PID) (Williams& Beer 2010
Bertschinger et al. 2014 is an active area
of research, beginning to be used in different domains of neuroscience and machine learning Tax et al. 2017
Dutta et al., 2020; Hamman & Dutta, 2024 Ehrlich et al.
2022
Liang et al. 2024 Wollstadt et al., 2023
Mohamadi et al. 20223 Venkatesh et al. 2024 Dutta et al. 2021 Dissanayake et al.] 2024). We also refer to
survey (Dutta& & Hamman 2023). However, interpreting spuriousness in datasets through the lens of PID is
unexplored. Additionally, there is limited work on calculating PID values for high digh-dimensional multivariate
3

---

Published in Transactions on Machine Learning Research (11/2025)
continuous data. Some existing works (Dutta et al. 2021 Venkatesh et al. 2024) handle continuous data with Gaussian assumptions while (Pakman et al. b considers one-dimensional multivariate case. Hence, estimating PID for high-dimensional data through proper dimensionality reduction and discretization is also
fairly open. For dimensionality reduction, different learning based mothods cxist (Hotelling 1933 Law &
Jain 2006 Lee & Verleysen 2005; Wang et al.) 2015 2014 Sadeghi & Armanfard] 2023). Similarly. y, for
discretization, different. clustering algorithms exist, c.g., k-means clustering (MacQueen et al. 1967 Bradley
et al., 2000), deep embedded clustering (Xie et al., 2016). In this work, we train an autoencoder jointly
learn a good lower-dimensional representation of the input image data in self-supervised manner (with
additional bottleneck structure) while also clustering simultancously to deal with the challenge of high- dimensional real-valued image data.
2 Preliminaries
Let X = XX,.. X22 be the random variable denoting the input (e.g., an image) where each X; E X' de- notes a finite set of values that each feature can take. The core features (e.g,, the foreground) will be denoted by F - X, and the spurions features (e.g., the background) will be denoted by B = F.F We typically use the notation B and F to denote the range of values for the spurious and core features. Let Y denote the target random variable, e.g., the true labels which lie in the set V/ and the model predictions are given by y = f (X) (parameterized by 0). Generally, we use the notation PA to denote the distribution of random variable A, and PAIB a to denote the conditional distribution of random variable A conditioned on B. Depending on the context, we also use more than one random variable as subscript, e.g., PABY denotes the joint distribution of (A, B). Whenever necessary, we also use the notation Q^ to denote an alternate distribution on the random variable A that is different from PA. We also use the notation PAIB o PBCC to denote a composition of two conditional distributions given by: PAIS o (alc) = Eeer PA Ppjc A, B, and C denote the range of values that can be taken by random variables A, B, and C.
Va E A,, c C, where
We provide a brief background on PID that would be relevant for the rest of the paper (also see Fig. 2) The classical information-theorrti quantification of the total information that two random variables A and B together hold about Y is given by mutual information I(Y; B) (see (Cover & TThomas 2012) for a background on mutual information). Mutual information T(Y; A, B) is defined as the KL divergence (Cover & Thomas 2012) between the joint distribution PY and the product of the marginal distributions Py 8 PAR and would go lo zero if and only if (A, B) is independent of Y. Intuilively, this mutual information captures the total predictive power about Y that is present jointly in (A, B) together, i.e., how well one can learn. Y from (A, B) together. However, I(Y; A, B) only captures the total information content about Y jointly in B) and does not unravel what is unique or shared between A and B.
Background PID:
PID [Bertcchinger et al. 2014) provides a mathematical framework that decom- poscs the total information content AY B) into four non-negative terms:
RY:
Synergr Syn(Y:
I B)
((Y:))
B) = Uni(Y:BIA) + Uni(Y:A|B) +Red(Y:A,,)) Syn(Y:A, B).
Unique To In Both B Uni(Y: Red) Red(() BYU BIAY
Redundant
Unique
Here, UnI) denotes the unique information about Y that is only in A but not in B and : denotes the unique information about Y that is only in B but not in A. Next, AAA, B) denotes redundant information (common knowledge) about Y in both A and B. Lastly, Syn(Y:A, B) is an interesting term that denotes the synergistie information that is present only jointly in A, B but not in any one of them individually, e.g., public and private key can jointly reveal information not in any of them alone.
Figure 2: I(Y; A, B) is de- composed into four non- negative terms.
Example to Undersland PID: Let 2- Zz, Za) with each Z, i.i.d. Bern(1/2). Let A = (Z1, Z2, Za + N), B = NN, and N ~ Bern(1/2) which is independent of Z. Here, A( = bits. The unique information about Z that is contained only in A and not in B is effectively in Z,. Thus, Uni(Z:A|B) = I(Z; Zz) 1 bit. Redundant information about Z that contained in both and B effectively in Zz and is given by Re,(Z:A, B)= = T(Z; 72) bit. Synergistic information about Z that is not contained in either
4

---

Published in Transactions on Machine Learning Research (11/2025)
F
B
B
B
B
Uni B\
Uni(Y:B\P)
Red F)
is dominant Syn(Y:F,B)
is dominant
is dominant
dominant
Figure Canonical examples distilling four types of statistical dependencics involving core and spurious features when any one PID term is dominant and its effect on the Bayes optimal classifier. In the first two cases, unique information in either F or B is dominant, and they indispensable to the optimal classifier. When redundant information is dominant, the optimal classifier pick either F or B without preference, The fourth scenario is interesting, where B is independent of the label Y, and yet it contributes to the optimal classifier along with F.
A or B alone, but is contained in both of them together is effectively in the tuple (Zg e NN,,,, and is given by Syn(Z:A, (Z; (Zg @ N)) = bit. This accounts for the 3 bits in I(Z; A.
Defining any one of the PID terms suffices for obtaining the others. This is because of another relationship
among the PID terms as follows (Bertschinger et al. 2014 [(Y;A) = AB
:A, B). Essentially
Red B) is viewed as the sub-volume between T(Y: and I(Y; B) (see Fig. 2 Hence, Red(Y: (Y:A, B)
I(Y; - Uni(Y:AAp) :A)B). Lastly, Syn(Y: :A, I(Y: ; A, B) Uni(Y:A|B))
Uni(Y:BJA) Red(Y :A, B) (can
be obtained once both unique and rodundant information has been obtained). Here, include popular definition of Uni(Y:A|B) from (Bertschinger et al.) 2014) which is computable using convex optimization.
Definition 1 (Unique Information (Bertschinger et al. 2014)). Let A be the set of all joint distributions on (Y, B) and Ap be the set of joint distributions with same marginals on (Y,A) and (Y, as the true distribution AB, i.e. Ap = =QQYABEA: QY = PY and Qy = PYB). Then,
Uni(Y:A min QEAp
AB).
Here (Y; ABB denotes the conditional mutual information when (Y,A,B) have joint distribution Qy AB rother than Ay
3 Theoretical Contributions
3.1 Unraveling the nature of spurious associations with PID
Proposition (Proposed Disentanglement). For a given data distribution, the total predictive power of the spurious features B and core features about the target variable Y can be decomposed into four non-negative components:
I(; = Uni(Y: UnilY:BB Reel::/P))BRRRR((.....
Red(Y: Syn(Y:F,
For each term in Proposition we now explain their muanced role for any given dataset.
Interpreting Unique Information Uni(Y:B(F) and Uni(Y:FIBB) Unique information cap- tures information that is unique in one feature and cannot be obtained from another. To explain the role of unique information in interpreting spuriousness, draw upon concept in statistical decision theory called Blackwell Sufficiency (Black- well 1953) which investigates when random variable is "more informative" (or less noisy") Ihan another for inference (also relates to stochastic degradation of channels (Venkatesh et al., 2023 Raginsky 2011)). Let us first discuss this notion intuitively when trying infer Y using two random variables and B. Suppose there exists transformation on F to give a new random variable B' which is always equivalent to B for predicting Y (similar predictive power). We note that B' and B do not necessarily
P PFIY =F
Y
unique
Pajy BB Information
Figure 4: Blackwell
Sufficiency
5

---

Published in Transactions on Machine Learning Research (11/2025)
have to be the same since we only care about inferring Y. In fact, B and B' can have additional irrelevant information that do not pertain to Y, but solely for the purpose of inferring Y, they need to be equivalent. Then, F' will be regarded as "sufficient" with respect to B for predicting Y since F' can itself provide all the information that B has about Y (see Fig. and first two cases of Fig. 3
Definition 2 (Blackwell Sufficiency (Blcckwell 1953)). A conditional distribution Ppl is Blackwell sufficient with respect to another conditional distribution PRIY if and only if there crists a stochastic transformation (equivalenlly another conditional distribulion PB with bolh B and B' E B) such that PBIP Priy = PBIY-
In fact, the unique information Un)(Y:B|F) is 0 if and only if Priy is Blackwell sufficient with respect the proof is given in the Appendix C
PBJY (sce Theorem
Theorem (Interpretability Insights from Unique Information). The following properlies hold:
Un)(Y:B(F) s B) and goes to 0 if the spurious feature B is independent of the target Y. However, Uni(Y:BIF) may be 0 even if I(Y: B) > 0.
Uni(Y:B|F) = if and only if Pryy is Blackwell sufficient with respect to PAY.
Uni(Y:B|F) < Uni(Y:B'(F") i.e., it is non-decreasing if some features from the core set are moved to the spurious set, i.e., B' = BUBW W and F = F.W.
Since unique information Fn = 0 if and only if PF is Blackwell Sufficient with respect to PBy,, we note that Uni(Y:B(E) > 0 captures the "departure" from Blackwell Sufficiency, and thus quantifies relative informativeness. Intuitively, what this means is that for a data distribution, there is no such transformation on core feature F that is equivalent the spurious feature B for the purpose of predicting Y. This essentially makes spurious feature B indispensable for predicting Y, forcing model to emphasize it in decision-making A similar argument can be made for Uni(Y:F(B). Furthermore, Uni(Y:B) also satisfies an intuitive property that as more features get categorized as spurious instead of core, the unique information in the spurious set would keep increasing.
Interpreting Redundant Information Red(Y:F, B): Redundant information about the target variable Y is the information that can be obtained from either the spurious features or the core features F without any preference towards either. We consider the following canonical example to interpret the role of redundant information Red(Y:F.Bl for predicting the target variable Y (third case of Fig. 3
Lemma 1 (Redundancy). Let B = N F= Ne where noise Ng and Ny are Gaussian such that NB = NN = N ~ (0, and N I1 Y. In this case, (i) an optimal predictor Y can either utilize B or F' with neither being indispensable, i.e., y = F(B) or f(F) or f(B, F); and (ti B and F will only have redundant information with the other PID terms being O.
Interpreting Synergistic Information: Synergistic information Syn(Y:F,BB is interesting term that emerges when spurious features B and core features F together reveal more about the target variable Y than what can be revealed by either of them alone. In essence, it is the "extra" or "emergent" information that arises only when multiple features interact, rather than when they are considered separately.
Lemma 2 (Synergy). Let B-N, F-Y+N where Y~Bern(1/2) N~N o) N JLL and 03 Then, (i) optimal predictor Ý B)F F B (uses both F and B); and (i) 1(Y; B) and I(Y; 0 but I(Y; F) is still significant due to synergistic information Syn(Y:B.F).
For this example (fourth case in Fig. 3 both F and B alone will have limited predictive power when N has high variance. However, using F and B logether, one can perfectly predict Y, e.g., an optimal predictor is Y = B(F F B. Here I(Y; B) = 0, and we also show that I(Y; ss 0 (see Lemma 8 in Appendix C However, the synergistic information Fy B) still significant. Since I(Y; O, we contend that here B essentially denoises the core feature F, enhancing its predictive power. Thus, synergistic information captures interesting nuanced interplay between core and spurious, not captured by the other PID terms.
6

---

Published in Transactions on Machine Learning Research (11/2025)
B: Background
Dimensionality Reduction
Unl(Y:
Distribution P(F - L,8 -.Y
Corvex Optimizatioon Uni(Y: BIF) min (o(F: gedp
Sogmentation Aa(x))
RYd F.
Sym(V:P,8)
JII
Dimensionality Reduction
Map = Uni(Y: F(BB) Syn
Faroground
Figure 5: Spuriousness Disentangler: An autoencoder- based explainability framework to handle high dimen- sional contimuous image data with modules: (i) Segmentation of images into background (spurious features) and foreground (core features); (ii) Dimensionality reduction involving an autoencoder with bottleneck and clustering; and (iii) Estimation of the joint distribution followed by the computation of PID values through convex optimization and computing Map.
3.2
A novel measure of dataset spuriousness
Our objective is quantify dataset's spuriousness, which steers machine learning models towards the spurious features the core features. To this end, we will examine of spuriousness through examples and counterexamples and systematically arrive al measure that meets our requirements. Since we are trying to capture spuriousness which arises when the target variable Y is associated with the spurious features B, we might first consider the mutual information I(Y; B) as a candidate measure for spuriousness since captures the dependence between Y and B.
candidate measures (Map)
Candidate Measure 1. Map IB:
Counterexample 1. We refer to the example in Lemma [ where Uni(Y:B|F) = O. Hence, I(Y; = Uni(Y:B + Red(Y:F, B)- = Re,(Y:F, B). Here, our candidate measure Map = T(Y; B) is positive, which would indicate "spuriousness," i..,, undesirable steering towards B. However, in this case, the model can use either spurious features or core features (see Lemina without any preference. Thus, I(Y; B) is not well suited be a measure of undesirable spuriousness.
Since redundant information can lead to the utilization of either spurious core features, another candidate measure of spuriousness might be obtained by subtracting the desirable dependence I(Y; F) from the unde- sirable dependence I(Y; B), i.e. Map I(Y; B) - I(Y; For the example in Lemma this new Map = 0, indicating no preference towards spurious or core features.
Lemma 3. Let B = Nn. = Np where noise NB and Ne are standard Gaussian noises with NB N(0,oNn): NF ~ NNO0 (Ne) and NB JL NF 11 Y. Now if oN > own (i) the optimal classifier relies strongly on spurious feature B; and (ii) Uni(Y:B(F) > 0.
If oN, >> N.. then T(Y; B) > F),; Le. Map > 0 (see Lemma in Appendix C Here, the output of a model more likely to be y f(B) and the model might be more prone utilizing the spurious features B (see Fig. 3) On the other hand, if oxp & Ne: then F)Y; T(Y; B), i.e., Map 0 In this case, the output of the model is also more likely to be Y = S(F), and the model might lean towards the core features F. Hence, Map = I(Y; B) 10---Y I(Y;F)) might seem like a suitable measure to quantify spuriousness, i.e, steering models towards B over F.
Candidate Measure 2. Map I(Y; B) I(Y: F) Uni(Y:B(F) - Uni(Y:F(B).
Counterexample 2. Consider Lemma 2 where the optimal predictor & = F-B utilizing both the spurious features B and core features F. Here, this Map x 0 (Lemma 2). However, for this particular example, since the prediction is jointly influenced by both core features F and spurious features B, we contend that a measure of spuriousness should not be O. The measure should therefore include a term that considers the joint contribution both of these features, capturing the fact that here B simply helps in denoising and enhancing the predictive capabilities the core features F. This aspect is precisely captured by synergistic information Syn(Y:F, B). Hence, we also include it in Maps leading to the following proposed measure.
7

---

Published in Transactions on Machine Learning Research (11/2025)
Proposition 2 (Measure of Spuriousness Map). Our proposed measure of spuriousness is given by:
Ma = Uni(Y:
- Sy
(2)
4 Methodology: Spuriousness Disentangler
We propose an autoencoder based explainability framework - that we call Spurionsness Disentangler to disentangle the PID values and compute the measure Map (see Fig. and Algorithm| I for a given dataset. The framework consists three modules: segmentation, dimensionality roduction, and estimation.
Algorithm 1: Spuriousness Disentangler: An Autoencoder--Bssed Explainability Framework
Input Encoder input or B, decoder output F' or B', autoencoder parameters 6, cluster centers Py (parameters of the clustering layer), embedded point =; (output of the clustering layer), soft label qi (output of the clustering layer), hyperparameter % pretrain epochs Ep, maximum epochs 6max: update interval T, batch number b, threshold 6, target variable Y.
Step 1: Segmentation;
Perform segmentation to separate the core features F and spurious features B if necessary;
Step 2: Dimensionality Reduction (Autoencoder Training):
for e = Cp do
compute Ly + || ;
// Reconstruction loss (MSE)
compute gradient Vo and update 0 via gradient descent end
Initialize Mj performing k-means clustering and Pij +- for e == to Emax do
(0)
(0)
Σ. (:5((() (0)
(0)
compute L f Ly + !co! where, Tue < KT(PIIQ) = E. E, Pig log Py and (i((e) + 2z(e)
(1+ -
Σ -a;l==(t))=_(ee]lp)))
if (b-1) mod T == 0 and (b + and + 1) then
update Pij +- q.((e)² aill) E3 (y(()²
as(e) preds + arg max (i)(e) and Stabel
Σ
(preds new Apreds prede
if òzabel < d then break;
end
preds +- preds
end
compute gradient VoL and update via gradient descent
end
Use discrete values from clusters of latent representations for F and B;
Step 3: Estimation;
Estimate P(F = i, B j, k); Compute FF | B) = mingear Fo(YFFIB) according to Eq, I // Iterative optimization
// Joint distribution estimation
Calculate Map using 2q
Output: Measure of spuriousness Mapi
Segmentation:
This step involves separating the core (F) from the spurious (B) features. Publicly avail- able segmentation masks used where available. For datasets without explicit core spurious feature information, masks can be generated using pre-trained semantic. segmentation models (Lin et al. 2017). Another possibility is to use CLiPSeg (Lüddecke & Ecker 2022), an Open- Vocabulary Semantic Segmenta- tion model, to automatically isolate various objects in the foreground to approximately obtain the core and spurious features. Experiments for various scenarios are provided in Section
Dimensionality Reduction: Since we are dealing with high-dimensional image data, our next module compresses them into lover-dinensional discrete vectors. We propose to use autoencoder, deep neural
8

---

Published in Transactions on Machine Learning Research (11/2025)
network consisting of an encoder and a decoder, shown in Fig. 6 to jointly do dimensionality reduction
and clustering. We incorporate a bottleneck structure from (Sadeghi & Armanfard] 2023) with convolutional autoencoders of Guo et al. (2017) to obtain more informative lower-dimensional representation of the input image. Along the lines of Guo et al. (0177), we obtain the clusters of the low-dimensional data by optimizing a joint loss function defined as T,p + the where L, is the representation loss, Le the clustering loss, and y is a non-negative constant. See Appendix C.4 for more details.
The final step includes the estimation of the joint distribution and the PID values, also leading to the proposed measure Map- The joint distribution is obtained by comput- ing the normalized 3D histogram of the discrete clusters of the foreground, background, and binary target variable. Then, the PID values are estimated from the joint distribution using the DIT package (James et al.) 2018), which is a Python pack- age for discrete information theory. We use TBROJA developed in [Bertschinger et al., 2014) to compute PID which solves the convex optimization problem in Definition [ and results in four non-negative terms, namely, Un) Uni(Y:FIB): FF,(Y:F, B), and Syn(Y:F, B). We use them to caleulate the measure Map. In case of a multiclass classification task (more than two classes), we use the PID estimator proposed by (Liang al. 2023).
Estimation:
Deçoder
x: Backkrooudd /Fareground
q: lower dineessinnl representation of x,
chustering
optimized by mindmizing the loss,
Encoder
layer
L yhe
Figure 6: Dimensionality reduction mod-
ule: Autoencoder with clustering to have discrete lower- dime--iinniionall embedding.
5 Experimental Results
Since our proposition is a preemptive dataset explainability framework, the objective of our experiments is to see how our anticipations from datasct before training agrec with post-training model generalization metrics like worst-group accuracy, SHAP, IoU, elc. In particular, we consider four setups: (i) Both core and spurious features are available; (ii) Either core or spurious is available; (iii) Segmentation to obtain core and spurious features; and (iv) Non-spatial spuriousness. We conduct experiments on six datasets: Waterbird (Wah et al. 2011), Adult [Becker & Kohavi 1996). CelebA (Lee et al. 2020). Dominoes {Shah) et al. 2020 Spawrious Tynch et al. 2023), and Colored MNIST (Arjovsky et al. 2019). We begin with using our explainability framework, namely Spuriousness Disentangler, on each dataset (often with dataset- specific sampling biases and variations) to compute the PID values and Map. We fine-tune the pre- pra-trei ResNet-50 (He et al.] 2016) model and calculate the worst-group accuracy over all groups for the Waterbird, CelebA, Dominoes, Spawrious, and Colored MNIST datasots. For the tabular dataset Adult, we train the XGBoost (Chen & Guestrin 2016) model and calculate the worst-group accuracy. More details of the experiments are provided in the Appendix D along with comparison of our proposed measure Msp with other measures in Table 2 Appendix D.2
Unbalanced W.G.ter-15.22111
Class balanced W.G. Acc. 76.910555
Group balanced
Addition Act. 00.19_2.37%
Concatenation W.G. 52.60.0.399%
W.G. Accuracy (%) 13
W.G. Acc. 05872007755
10
Unirg UHRLF byn
Uni WHHLF
5ye
R
UWALB UHRL7
Sye
UHIL5 UHA
Spn
9pn
-0.05 0.00 0.08
0.38
Figure 7: Bar-plot showing the redundant information (R), unique information in background (Uniq-B) and foreground (Uniq-F), and Synergistic information (Syn) for different variants (essentially differrnt sampling biases) of the Waterbird dataset. Observe that the Uniq-B decreases and Uniq-F increases for group- balanced, addition, and concatenation datasets compared to those of the unbalanced dataset. Also, observe a negative trend between Msp and W.G. Acc. Note that the y-axis of the first five subplots is in log scale.
1. Both core and spurious features available: For the Waterbird, Dominoes, and Adult datasets, core and spurious features are well-defined and accessible. In Waterbird, the bird's pixels serve as core features, while the background is spurious. In the synthetic Dominoes dataset, car or truck images are core, and
9

---

Published in Transactions on Machine Learning Research (11/2025)
digits zero spurious. For the tabular Adult dataset, gender is considered spurious, while age, educatiin-num, and hours-per-week are chosen as core.
W.f.koe M.0.c....0........
W.G. 90.19-1.23%
W.G. Roc. 96.2202249
Mita Acc. 20.06-1.3355
W.GG 10..c.778... 28.78..
WIG Acc. 88 09:1.12%
W.G. Acc. 86.741122%
W.G. Acc. 30.72
200
0944
IHKE
6166
1964
0.0
0.4
Figure Bar-plot showing the redundant information (R), unique information in background (Uniq-B) and foreground (Uniq-F), and Synergistic information (Syn) for Dominoes dataset. The Uniq-B decreases group-balanced and background mixed datasets, and the Uniq-F increases for background mixed datasets compared to the unbalanced dataset. Also observe negative trend between the Map and W.G. W.G. Acc.
Unbalanced WGG.Act. 10.63%
Group balanced
W.G. Accuracy (%)
10*
V.S.
3073
Map
Figure 9: Bar-plot showing the PID values for the tabular dataset: Adult. The last plot shows negative relationship between the worst-group accuracy and the measure of spuriousness Msp. Note that the y-axis of the first two subplots in log scale.
Unbalanced
Class Balanced
Group Balanced
Addition
Concatenation
Figure 10: Examples of Grad-CAM images of Waterbird: For the unbalanced dataset, the model adds more emphasis (red regions) the background, while the class-balanced, group-balanced, addition, and concatenation versions, the foreground gets more emphasis.
Observations: Fig. (Waterbird), Fig. 8 (Dominoes), and Fig. 9 (Adult) show that the unique information in the background decreases and the unique information in the foreground increases when the dataset is modified from unbalanced to other variants. We also observe negative correlation between the measure of spuriousness M and the worst-group accuracy (see the Pearson correlation coefficient (r) with corresponding p-value in Fig. 70. The negative correlation between our proposed dataset spuriousness measure Map and the model generalization metric worsl-group accuracy indicales that Map can serve as an indicalor of datasel quality before training. Fig. [0 and Fig [8 in Appendix D.4.1 show that when the dataset is balanced or mixod background, the Grad-CAM (Selvaraju al.) (017)) emphasizes more on the core features (the rod regions) while in the unbalanced dataset, the background is more emphasized, which results in poor worst- group accuracy. See Table 1 and Table in Appendix D.2 which show how Intersection-over-mnioo (ToU)
10

---

Published in Transactions on Machine Learning Research (11/2025)
between the ground truth masks and Grad-CAM masks changes over different variants of the Waterbird dataset and the variation of PID values for different numbers of clusters, respectively.
2. Only core features available: For the CelebA dataset, the core features are the pixels corresponding to hair. However, the spurious one not given directly. We consider everything exchuding hair as spurious.
Unbalanced 72.41 -0.tt
Class balanced
Group balanced
W. Accuracy (%)
W.S. 85.2942 9ats.
W.G. 68.3641 6ot
Mag
Figure 11: CelebA: Observe that the Uniq-F and Synergy increase for class-balanced and group-balancee datasets compared to that of the unbalanced dataset, and a trade-off between Map and W.G. Acc.
Observations: Fig. [I shows the PID values for unbalanced, datascts. Firstly, the unique information in the foreground is the most prominent one among all other PID values. Observe that. the Uniq-B is almost negligible. The Uniq-F increases while the dataset is class- balanced or group-balanced, along with the increasing worst-group accuracy. There is negative trend between worst-group accuracy and the measure of spuriousness Map. Secondly, the Grad-CAM images (see Fig. 21 in Appendix. D.4.2p show that the model focuses on the hair for the balanced dataset, but for the unbalanced dataset, it emphasizes more on the face.
and group-balanced CelebA
Impur Image
Sugneenttion musk
Fonegroudd
Background
Figure 12: The segmentation mask obtained by zero-shot image segmentation using CT.IP [Lüddecke & Ecker 2022). We get the foreground by multiplying the input image with the mask and background by multiplying mask).
3. Segmentation to obtain features: When explicit information about the core and spurious features is unavailable, we need to perform segmentation. We perform experiments with the Waterbird dataset assuming the unavailability of the segmentation masks. We leverage CLIPSeg (Lüddocke & Ecker 2022) an Open-Vocabulary Semantic Segmentation (OVSS) model, to generate the mask for the bird object with the prompt "bird" (details Appendix D.I), thus utilizing only partial knowledge about the target object. The generated masks are applied the original images to separate the foreground and background (see Fig. 12 Fig. 13a reveals negative correlation between the worst-group accuracy and incrcasing values of Map: sp. calculated using the obtained background and foreground.
Alternatively, for the Spawrious dataset, we use a pre-trained segmentation model Lin et al, 2017 to generate the mask of the dog and separate the foreground "dog" from the background. Fig. 13b shows that the redundancy and unique information the background decrease while the unique information the foreground and synergy increase when the datasct group-balancod. We also observe a negative trend between Map and the worst-group accuracy, showing the effectiveness of the measure.
4. Non-spatial spuriousness: We apply our framework to the Colored MNIST (Arjovsky et al. 2019 dataset, which has non-spatial biases. In this sctting, we define the grayscalc digit images as the core features and the colored digit images as the spurious features B, leveraging partial knowledge that the target variable should correspond to the actual digits rather than their color. We perform two experiments: (i) a 10-class digit classification and (ii) binary classification, where digits < are class 0, and 2 are class 1. There is negative correlation between the measure of spuriousness Map and worst-group accuracy (W.G. G. Acc.) (see the Pearson correlation coefficient. (r) with corresponding p-value), consistent. with observations
11

---

Published in Transactions on Machine Learning Research (11/2025)
Unbalanced
Group balanced
W.G. Accuracy (%6
100
Unbalanced W.S. 51 9im1 94%
Group balanced
w. Accuracy (%)
20a
W.S. Acc. 95.2400288
10"13
BA44, UN4 9/0
0.00
0:45
9.18
-0015
(a) Waterbird with CLIPSeg
(b) Spawrious
Figure 13: (a,b) The first two plots show the change in redundancy, unique information, and the synergistic information. The last plot shows negative relationship between the W.G. Acc. and the measure of spuriousness Map Note that the y-axis of the first two subplots is in log scale.
from other datasets, validating the broad applicability of our proposed measure (see Fig. 14 by extending our framework to the multiclass setting, we demonstrate its enhanced versatility.
Furthermore,
Flip
Flip
Accuracy
W.G. Acc. 96.09%
Art. 95.73.0.46%
W.G. Act, 42.0.20%
0.22 2.600
Map
(a) Binary CMNIST
Flip 15%
Flip
W.G. Accuracy
W.G. Acc. 06 9.777777%
V.GG Arc 94 04.08.0 37%
V.G. Arc. 95.22.0.55%
0.08, pvell- 0.40
18-1
Unit_
Unit,
Unig.
Untt.
twie.F
0.0
Ng
(b) Multiclass CMNIST
Figure 14: (a,b) The first three plots show the change in rodundancy, unique information, and the synergistic information. The last plot shows a negalive relationship between the W.G. Acc.(%) and the measure of spuriousness Map. Note that the y-axis of the first three subplots is in log scale.
6 Conclusion
This work brings in a novel perspective on spuriousness by identifying four types of statistical dependencies in a dataset, leveraging the PID framework: when F or B is indispensable (unique information dominant), when either F or suffices (redundant information dominant), and when both F and are jointly needed (synergistic information dominant). This leads us to propose a measure of dataset spuriousness as an efficient way to assess dataset quality before performing the actual training fine-tuning, which can be computationally intensive, particularly in the era of foundational models. We also perform experiments several datasets to check if our anticipations from data agree with post-training model generalization metrics such as worst-group accuracy. Notably, worst-group accuracy and our measure of spuriousness are not malhematically the same thing: our measures anticipate how the Bayes optimal classifier(s) should behave, and the empirical measures show how specific models actually behave (when trained on that dataset without doing anything specific for spuriousness). Nonetheless, we observe an interesting correlation across broad range of experimental setups, further validating the efficacy of our measure. Our implementation is able to handle high-dimensional image data and estimate the PID terms (Broader Impacts in Appendix B
12

---

Published in Transactions on Machine Learning Research (11/2025)
References
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXio:1907.02893 2019.
Pradcop Kr Banerjec, Johannes Rauh, and Guido Montúfar. Computing the unique information. In JEEE International Symposium on Information Theory, pp. 141-145, 2018.
Barry Becker and Ronny Kohavi.
Adult.
UCI Machine Learning Repository, 1996.
DOI:
Mohamed Tshmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Devon Hjelm. Mutual information neural estimation. In International conference on machine learning. pp. 531 540. PMLR, 2018.
Nils Bertschinger, Johannes Rauh, Eckehard Olbrich, Jürgen Jost, and Nihat Ay. Quantifying unique infor- mation. Entropy, 16( 16(4):2161-2183, 2014.
David Blackwell. Equivalent comparisons of experiments. The amnals of mathemalioal statistics, pp. 265 272, 1953.
Paul S Bradley, Kristin P Bennett, and Ayhan Demiriz. Constrained k-means clustering, Microsoft Research, Redmond, 20(0):0, 2000.
Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pp. 785-794, 2016.
Thomas M Cover and Joy A Thomas. Elements of Information Theory. John Wiley & Sons, 2012.
Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6)1141-142, 2012.
Pasan Dissanayake, Faisal Hamman, Barproda Halder, Ilia Sucholutsky, Qiuyi Zhang, and Sanghamitra
Dulla. Quantifying knowledge distillation using partial information decompasition. In International Conference on Artificial Intelligence and Statistics, 2024.
Sanghamitra Dutta and Faisal Hamman. A review of partial information decomposition in algorithmic fairness and explainability. Entropy, 25(5):795, 2023.
Sanghamitra Dutta, Praveen Venkatesh, Piotr Mardziel, Anupam Datta, and Pulkit Grover, An information- theoretic quantification of discrimination with exempt features. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 8825-3833. 2020.
Sanghamitra Dutta, Praveen Venkatesh, Piotr Mardziel, Amupam Datta, and Pulkit Grover. Fairness un- der feature exemptions: Counterfactual and observational measures. IEFE Transactions on Information Theory, 67(10):6675 6710, 2021.
David A Ehrlich, Andreas Schneider, Michael Wibral, Viola Pricscmann, and Abdullah Makkch. Partial in- formation decomposition reveals the structure of neural representations. arXiv preprint or arxiv:2209. 10438, 2022.
Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé lii, and Kate Crawford. Datasheets for datasets. Communications of the ACM, 64(12): 86-92, 2021.
Ghulam Ghouse, Atiq Ur Rehman, and Muhammad Ishaq Bhatti. Understanding of causes of spurious associations: Problems and prospects. Journal of Statistical Theory and Applications, 23(1):44 66, 2024.
Karan Goel, Albert Gu, Yixuan Li, and Christopher Ré. Model patching: Closing the subgroup performance gap with data augmentation. XiXio preprint or orx::20008.00 06700, 2020.
13

---

Published in Transactions on Machine Learning Research (11/2025)
André FC Gomes and Mário AT Figueiredo. A measure of synergy based on union information. Entropy. 26(3):271, 2024.
Chaitanya Goswami and Amanda Merkley, Analytically deriving partial information decomposition for affine systems of stable and convolation-closed distributions. In Neural Information Processing Systems, 2024.
Xifeng Guo, Xinwang Liu, En Zhu, and Jianping Yin. Deep clustering with convolutional autoencoders. In Newral Information Processing: 24th International Conference, ICONIP 2017, Guangzhou, China, November 14-18, 2017, Proceedings, Part II 24, pp. 373-382. Springer, 2017.
Brian D Haig. What is a spurious correlation? Understanding Statistics: Statistical Issues in Psychology, Education, and the Social Sciences, 2(2):125-132, 2003.
Faisal Hamman and Sanghamitra Dutta. Demystifying local and global fairness trade-offs in federated lcarning using information theory. In International Conference on Learning Representations (ICLR), 2024.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition.
In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770 700,778 2016. doi: N0.1109/CVPR.200
Harold Hotelling. Analysis of a complex of statistical variables into principal components. Journal of educalional psychology, 24(6):417, 1933.
Pavel Izmailov, Polina Kirichenko, Nate Gruver, and Andrew G Wilson. On feature learning in the presence of spurious correlations. Advances in Newral Information. Processing Systems, 35:38516 --5532, 2022.
R. G. James, J. Ellison, and J. P. Crutchfield. dit: a Python package for discrete information theory. The Journal of Open Source Software, 3(25):738, 2018. doi:
00738.
Priyatham Kattakinda, Alexander Levine, and Soheil Feizi. Invariant learning via diffusion dreamed distri- bution shifts. arXiv preprint arXfv:2211.10370, 2022.
Polina Kirichenko, Pavel Iznailov, and Andrew Gordon Wilson. Last layer re-training is sufficient for robustness to spurious correlations. ar preprint arXiv:2204. 02937, 022.7777, 2022.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Martin HC Law and Anil K Jain. Incremental nonlinear dimensionality reduction by manifold learning. IEEE transactions on pattern analysis and machine intelligence, 28(3):377 -91,, 2006.
Cheng-Han Lee, Ziwei Liu, Lingyun Wu, and Ping Luo. Maskgan: Towards diverse and interactive facial image manipulation. In JEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
John Aldo Lee and Michel Verleysen. Nonlinear dimensionality reduction of data manifolds with essential loops. Neurocomputingg 67:29-53. 2005.
Paul Pu Liang, Yun Cheng, Xiang Fan, Chun Kai Ling, Suzanne Nie, Richard Chen, Zihao Deng, Nicholas Allen, Randy Auerbach, Faisal Mahmood, et al. Quantifying & modeling multimodal interactions: An information decomposition framework. Advances in Neural Information Processing Systems, 36:27351 27393, 2023.
Paul Pu Liang, Chun Kai Ling, Yun Cheng, Alex Obolenskiy, Yudong Liu, Rohan Pandey, Alex Wilf, Louis- Philippe Morency, and Ruslan Salakhutdinov. Multimodal learning without labeled multimodal data: Guarantees and applications. International Conference on Learning Representations (ICLR), 2024.
Tsung-Yi Yi Lin, Piotr Dollár, Ross Girshick, Kaiming Hc, Bharath Hariharan, and Serge Belongic. Feature pyramid networks for object detection. In 2017 IEEE Conference on Computer Vision and Pattern Recog- nition (CVPR), pp. 936 944, 2017. doi: D0.1109/CVPR.2011
14

---

Published in Transactions on Machine Learning Research (11/2025)
Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. Tn International Conference on Machine Learning, pp. 0781-6792. PMLR, 2021.
Shong Liu, Xu Zhang, Nitesh Sckhar, Yue Wu, Pratock Singhal, and Carlos Femandoz-Grandd.. Avoiding spurious correlations via logit correction. The Eleventh International Conference on Learning Repre- sentations, 2023. URL https://openrevieee http:////pppaoooeeee.oeooorrrrrs wot.:/oooounndddBBaFaa555
Xiaoyu Läu, Paiheng Xu, Junda Wu, Jiaxin Yuan, Yifan Yang, Yuhang Zhou, Fuxiao Liu, Tianrui Guan, Haoliang Wang, Tong Yu, et al. Large language models and causal inference in collaboration: A compre- hensive survey, Xix preprint or arxiv:2403. 09606, 2024.
Timo Lüddecke and Alexander Ecker. Image segmentation using text and image prompls. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pp- 7086-7096, 2022.
Aengus Lynch, Gbětondji JS Dovonon, Jean Kaddour, and Ricardo Silva. Spawrious: benchmark for fine control of spurious correlation biases. arXiv preprint (EK::03030054 2023.
Aobo Lyu, Andrew Clark, and Netanel Raviv. Explicit formula for partial information decomposition. In 2024 IEEE International Symposium Information Theory (ISIT), pp. 2329-2334, 2024. doi: 10.1109/ N11T57864.20241106195
Andrzej Maćkiewicz and Waldemar Ratajezak. Principal components analysis (pca). Computers & Geo- sciences, 19(3):303-34) 342, 1993.
James MacQueen et al. Some methods for classification and analysis of multivariate observations. In Proceedings of the fifth Berkeley sympostum on mathematical stotistics and probability, volume 1, pp. 281-297 Oakland, CA, USA, 1967.
David McAllester and Karl Stratos. Formal limitations the measurement of mutual information. In International Conference on Artificial Intelligence and Slalistics, pp. 875 884. PMLR, 2020.
Pedro AM Mediano, Fernando E Rosas, Andrea T Luppi, Robin L. Carhart-Harris, Daniel Bor, Anil K Seth, and Adam Barrett. Toward a unified taxonomy of information dynamics via integrated information decomposition. Proceedings of the National Academy of Sciences, 123(3999)22233227222 2025.
Mazda Moayeri, Wenxiao Wang, Sahil Singla, and Soheil Feizi. Spuriosity rankings: sorting data to measure and mitigate biases. Advances in Neural Information Processing Systems, 0111522241600 2023.
Salman Mohamadi, Gianfranco Doretto, and Donald A Adjeroh. More synergy, less redundancy: Exploiting joint mutual information for self-supervised learning, arXiv preprint arXiv:2307.00551, 2023.
Ari Pakman, Amin Nejatbakhsh, Dar Gilboa, Abdullah Makkeh, Luca Mazzucato, Michael Wibral, and Elad Schneidman. Estimating the unique information of continuous variables. Advances in neural information processing systems, 34:20295 203007 2021.
Aahlad Manas Puli, Lily Zhang, Yoav Wald, and Rajesh Ranganath. Don't blame dataset shift! shorteut learning due to gradients and cross entropy. Advances in Neural Information Processing Systems, 36: 71874-71910, 2023.
Maxim Raginsky. Shannon meets blackwell and le cam: Channels, codes, and statistical experiments. In 2011 IEEE International Symposium on Information Theory Proceedings, pp. 1220-1222 IEEE, 2011.
Hamid Rezatofighi, Nathan Tsoi, Jun Gwak, Amir Sadeghian, Ian Reid, and Silvio Savarese. Gener- alized intersection over union. June 2019.
Mohammadreza Sadeghi and Narges Armanfard. Deep clustering with superssunoii using pairwise data similarities. Authorea Preprints, 2023.
15

---

Published in Transactions on Machine Learning Research (11/2025)
Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Perey Liang. Distributionally robust neural networks for group shifts: shifts:: On the importance of regularization for worst-case generalization. arxiv preprinl wrXXu::911.08791, 2019.
Ramprasaath Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from doep networks via gradient-based localization. In Proceedings of the IEEE international conference computer vision, pp. 218-626, 2017.
Harshay Shah, Kanstav Tammly, Aditi Raghmathan, Prateek Jain, and Pranceth Netrapalli. The pitfalls of simplicity bias neural networks. Advances Neural Information Processing Systems, 33, 2020.
Sahil Singla and Soheil Feizi. Salient imagenet: How to discover spurious features in deep learning? ar preprint Xr:::221 04301, 2021.
Nimit Sohoni, Jared Dunnmon, Geoffrey Angus, Albert Gu, and Christopher Ré. No subclass left behind: Fine-grained robustness in coarse-grained classification problems. Advances in Neural Information Pro- cessing Systems, 33:19339-19352, 2020.
Mashrin Srivastava. Addressing spurious correlations in machine learning models: comprehensive review. OSF Prepr, 2023.
Nathan Stromberg, Rohan Ayyagari, Monica Welfert, Sanmi Koyejo, and Lalitha Sankar. Robustness to subpopulation shift with domain label noise via regularized annotation of domains. orXiv preprint arXiv:2402.1100 2024.
Tycho Tax, Pedro Mediano, and Murray Shanahan. The partial information decomposition of generative neural network models. Entropy, 19(9):474, 2017.
Praveen Venkatesh, Keerthana Gurushankar, and Gabriel Schamberg. Capturing and interpreting unique information. In 2023 IEEE International Symposium on Information Theory (ISIT), pp. 2023.
IEEE,
Praveen Venkatesh, Corbett Bennett, Sam Gale, Tamina Ramirez, Greggory Heller, Severine Durand, Shawn Olsen, and Stefan Mihalas. Gaussian partial information decomposition: Bias correction and application to high-dimensional data. Advances in Neural Information Processing Systems, 36, 2024.
C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. The caltech-uesd birds-200-2011 dataset. Technical Report CNS-TR-2011-005 California Institute of Technology, 2011.
Wei Wang, Yan Huang, Yizhou Wang, and Liang Wang. Generalized autoencoder: A neural network framework for dimensionality reduction. In 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 496 503, 2014. doi: 10.1109/CI 1119//VPRWWW// 22014.79.
Yasi Wang, Hongxun Yao, Sicheng Zhao, and Ying Zheng. Dimensionality reduction strategy based on aulo-encoder. In Proceedings of the 7th International Conference on Internet Multimedia Computing and Service pp. 1 1-4, 2015.
Yipei Wang and Xiaoqian Wang. On the effect of key factors in spurious correlation: A theoretical perspec- tive. In Sanjoy Dasgupta, Stephan Mandt, and Yingzhen Li (ods.). Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, volume 238 of Proceedings of Machine Learning Re- search, pp. 3745 3753. PMLR, 02 04 May 2024. URL httplittts:///ttt htttps://proceedingg. mlr pttpp//eee///aaeeneeaa wang24j. html
Paul L Williams and Randall D Beer. Nonnegative decomposition of multivariate information. arXiv preprint arXiv:1004.2550 2010.
Patricia Wollstadt, Scbastian Schmitt, and Michael Wibral. A rigorous information-fhcorrticc definition of redundancy and relevancy in feature selection based on (partial) information decomposition. J. Mach. Learn. Res., 24:131-1, 2023.
16

---

Published in Transactions on Machine Learning Research (11/2025)
Shirley Wu, Mert Yuksekgomul, Linjun Zhang, and James Zou. Discover and cure: concept-aware mitiga- tion of spurious correlation. In Proceedings of the 40th. International Conference on Machine Learning, 1CML'23. JMLR.org, 2023.
Junyuan Xie, Ross Girshick, and Ali Farhadi. Unsupervised deep embedding for clustering analysis. Tn International conference machine learning, pp. 478-487. PMLR, 2016.
Ke Yang, Julia Stoyanovich, Abolfazl Asudeh, Bill Howe, HV Jagadish, and Gerome Miklau. A nutritional label for rankings. In Proceedings of the 2018 international conference on management of data, pp. 1773- 1776, 2018.
Yao-Yuan Yang, Chi-Ning Chou, and Kamalika Chaudhuri. Understanding rare spurious correlations in neural networks, 2023. URL https::/openrevierr met:/forumiid--rzx--tt-
Haotian Ye, James Zou, and Linjun Zhang. Freeze then train: Towards provable representation learning under spurious correlations and feature noise. Proceedings of Machine Learning Research, 206:8968 8990, 2023. ISSN 2640-3499.. Publisher Copyright: Copyright @ 2023 by the author(s); 26th International Conference on Artificial Intelligence and Statistics, AISTATS 2023 Conference date: 25-04-2023 Through 27-04-2023.
Wenqian Ye, Guanglao Zheng, Xu Cao, Yunsheng Ma, Xia Hu, and Aidong Zhang, Spurious correlations in machine learning: A survey. arXiv preprint Xix:22402 12715, 2024.
Alessio Zanga, Elif Ozkirimli, and Fabio Stella. A survey on causal discovery: theory and practice. Interna- tional Journal of Approrimate Reasoning, 151:101-129, 2022.
Michael Zhang, Nimit S Sohoni, Hongyang R Zhang, Chelsea Finn, and Christopher Ré. Co contrastive approach for improving robustness to spurious correlations. Xr preprint or 220::2200..1111, 2022.
17

---

Published in Transactions on Machine Learning Research (11/2025)
A
Limitations
(i) Identifying spurious features and core features of given dataset automatically is not always straightfor- ward. Future work will look into alternate techniques, such as causal discovery (Zanga et al. 2022 (recently using LLMs (Liu et al. 2024)) ) as well as validation on NLP datasets. (ii) Separation of spurious and core features often relies on manual annotations, heuristic segmentation, or prior domain knowledge, potentially introducing subjective bias in feature calegorization. (iii) The estimation is highly data-dependent. A small change in the dataset can greatly affect the PID values. In fact, estimating iiformation-thereeee quantities such mutual information is inherently difficult in high-dinensional settings (Belghazi et al. 2018), includ- ing formal limits on sample complexity for all estimators (McAllester & Stratos 2020), and PID estimation would also inherit these challenges. Moreover, PID requires solving an additional optimization problem beyond mutual information computation, which can introduce additional difficulties. Future work will look into estimation error analysis and alternate tochniques [Lyu al. 2024] Goswami Mcrkley 2024 Gomes & Figueiredo 2024 Mediano al.) 2025). (iv) The efliciency and robustness of the Spurious Disentangler also be improved. (v) Additionally, there be groups of spurious features rather than just. one, which can have nuanced interplay among them, which is another interesting direction.
B Broader Impact
Quantifying spuriousness has significant broader impacts across multiple domains. Quantification of dataset spuriousness might improve the trustworthiness of AI in several high-stakes and safety-critical applications, such as healthcare, which can directly impact people's lives. Spurious patterns often lead to biased predic- tions, particularly in sensitive domains such as hiring, lending, or criminal sentencing. Going beyond existing works, our research paves the way for improved understanding of the nature of spurious relationships, en- abling interpretability, which could also have significant implications in auditing.
A framework for dataset explainability provides an alternative to combating spuriousness during training by providing preemptive insights to inform the training process (analogous to "nutrition labels" Yang et al. (2018) or "datasheets for datasets" Gebru et al. (2021)) ) By enabling dataset quality check and cleansing prior to training, can bypass expensive adversarial training, often used to avoid spurious patterns. Having clean datasets for fine-tuning is particularly valuable in the era of large foundation models when one has limited control over the training process.
C Appendix to Theoretical Results
C.1 Relevant Mathematical Results
PID [Bertschinger et al. 2014 Banerjee et al. 2018 provides mathematical framework that decomposes
the total information content I(YA, into four non-negative terms:
T(Y;A, Uni(Y:B|A) Uni(Y:A(B))
Red(Y:A, Syn(Y:A,
In addition to this equation, the PID terms also satisfy the following relationships (Bertschinger et al. 2014
Banerjee al. 2018):
A(Y; = Uni(Y:4|B) + Red(Y:A,
(4)
I(;; A(B) Uni(Y:A|B) Syn(Y::n
(5)
Now, defining any one of the PID terms is sufficient to obtain all four by using these relationships. In this
work, we use a popular definition of unique information from (Bertschinger et al. 2014 Banerjee et. al. 2018)
as defined in Definition [ which can be computed by solving a convex optimization problem (Bertschinger
let al. 2014 Banerjce et al. 2018
One of the most desirable properties of this definition is that all four PID terms are non-negative.
18

---

Published in Transactions on Machine Learning Research (11/2025)
Lemma 4 (Nonnegativity of PID). All four PID terms Uni(Y:BJA), Uni(Y:AlB), Red(Y:A, and Syn(Y:A, B) are nonnegotive. per Definitiion 1
This result is proved in Bertschinger 20114 Lemma 5).
Lemma 5 (Monotonicity under local operations on B). Let B = f(B)) where f(-) is a deterministic functton.
Then, we have:
Uni(Y:B[A) & Uni(Y:B'(A).
This result is derived in Banerjee et al. (2018 Lemma 31).
Lemma 6 (Monotonicity under adversarial side information). For all A.B,A,W), have:
Uni(Y:B\A,,) Uni(Y:B|A).
This result is derived in Banerjec et al. 20188 Lemma 32).
Lemma Uni(Y:B|F) = if and only if there erists a row-stochastic matrir T E [0, F|x||1] such that: PYB(Y = B. B = b) = EEEr PYr(Y = y, = f)T(f(b) for all E y and E B.
Proof. This result is from Bertschinger et al.
2014). Here, we include proof for completeness.
If Uni(Y:B|F) 0, then we have: mingear [Y; = 0 where Ap (QEA: QyF = y, F f) = PY = y, f) and Qy = y, B =) b) Py = y, B = b)). Thus, there exists distribution Q Ap such that and B are independent given F' under the joint distribution Q. Then, we have
PYB(Y = =, =- b) = QrblY y, =
(6)
Σ CYpB(YY= = =U F-F f.
(7)
JEF
Σ @BIYF(( b/Y = F =-\rr(YY y. F=
(8)
JEF
Σ QBYF (B = b/y F, D)PPoY = y, F=
(9)
Σ QB = = F,
(10)
FEF
(
Σ (X = =
(11)
FEF
= QyF for all Q E Ap. (b) holds because under joint distribution Q, variables Y and B are independent given F, and (c) simply chooses T(J, = ( = b/F = f) which is a function of (f,b) and will lead to row-stochastic matrix T since Eoes D(f.b) = Q = 1.
Here, (a) holds because Py
Next, prove the converse. Suppose, such row-stochastic matrix T exists such that:
PY = y, y Σ (( yy
JEF
Now, we can define a joint distribution Q" such that:
Q'(Y y, F= -) Pyr(Y PY F (T(,,))
(12)
19

---

Published in Transactions on Machine Learning Research (11/2025)
We can show that Q" is valid probability distribution since T is row stochastic.
y, f, B ΣΣΣ Q' 40
ΣEΣ yEy DEB DEB
-
yEyBEB JEF
ΣΣ yEY FEr
Prp(y =y F Σ T((DD))
BEB
Σ Prp(Y
= yy
(13)
VEy fET
Also, can show that Q Ap since:
QY = y, B = b) =
Σ F, PY = V, B = =b),
(14)
JEF
which holds since such row-stochastic matrix T exists. Also, we have:
Ovr(Y y
PYP y. F)T(F.b) Pyp(Y = =y. F = f),
(15)
BEB
which holds since T is row-stochastic.
Then, Uni(Y:B minqEa,
= -.
C.2 Proof of Theorem I
For the first claim, notice that Uni(Y:B|F) = =[(V; B) Re:(Y:B, F) (from equation 4 and Red(Y:B, F) 2 0 (non-negativity of PID, see Lemma Thus,
Uni(Y:B = B).
For the socond claim, we will use Lemma Uni(Y:B|F) = and only there exists a row-stochastic
matrix T [0, 1| such that: PY v, = b) = Σrer Py y,, = ()T(1, for all and b E B. The existence of such row-stochastic matrix is equivalent to Blackwell Sufficiency as per Definition 2
from [Blackwel] 1953)
For the third claim, first observe that if B' = BUW, then B can be written as local operation on B', i.e., B = F(B'. Thus, from Lemma we have:
Uni(Y:B|F) (n)(
(16)
Next, observe that since F² F\W, then from Lemma we have:
Uni(Y:B'(P) Uni(Y:B')F', W) s Uni(Y:B'(F))
(17)
Combining equation
16 and equation 17 we have the claim
Uni(Y:BJP) < Bn)
C.3 Proof of Additional Results
C.3.1 Proof of Lemma 1
Proof of Lemma 1 Here, B and F Y + N where Y and N are independent. Any optimal
predictor is function of the inputs F and B, i.., γ f(F, B). Since F == B, this function can always be rewritten function of B alone F alone.
20

---

Published in Transactions on Machine Learning Research (11/2025)
Next, will show that only the redundant information Red(Y:B, is positive and all other PID terms Uni(Y:BIF) Uni(Y:FIB), and Syn(Y:F. B) are zero.
Here I\P = 1(;FFBB 0 since B = P.
I(:: B|F) (() ((B))))H(BBB,FF)) 0.
According to the Definition and non-negativity of PTD terms, Uni(Y:B|F) T(Y; (|F Syn(Y:F, s I(Y; (|) 0.
Similarly, we have, Uni(Y:F]B) < I(Y:FBB) 0.
Then, Syn(Y:F, B) = [(; Uni(Y::i( (from equation 5 is also 0. Now, Red(Y:B, P) = I(Y; - -n)(Y-))))))110 is a significant dependence between Y and B.
HYY) - BY which is positive as long as there
C.3.2 Proof of Lemma
We first include another lemma that will be useful in proving main result.
Lemma 8 (Noisy Feature). Let A Y + N where Y Bern(1/2) is random vartable taking values +1 -1 and the noise N NNO0, xx) is Gaussian random variable independent of Y. Then, the matual information
I(Y; S loga 1+
N
Proof.
T(Y; A) = H(A) H(A()) = H(Y + N) H(Y + NIV)
(18)
H(Y + N) H(NIV)
(19)
HYY + +N) ((N). since N TT Y
(20)
(0)
logg 2 (1 oon)
logg 2ne (o%)
(21)
2
2
log2
1+
(22)
2 NN
Here (a) holds because the entropy of is bounded by 1 log, 27ee (1 + o (proved in Cover Thomas
2012 Theorem 8.6.5)). We also refer to Cover & Thomas 2012 Chapter 9) for discussion Gaussian channels.
If we keep the distribution of fixed and vary the noise variance oy, then we will observe decreasing trend of I(Y; B) with increasing o.y. Fig 15 shows the exact trend where Y is Bernoulli random variable.
Proof of Lemma 2 Here B N and F Y N where Y Bern(1/2) takes values +1 or -,, and the noise N ~ (O, v) with N 11 Y and of > 1.
First observe that the predictor & f(B F) F B = Y. Thus, it perfectly predictive of Y, and an optimal predictor.
Now, we will compute the values of the PID terms and show that Syn(Y:B, F) > 0 and all the other three PID terms are negligible.
Since Y, we have (Y;B)) =
Since = Y N, we use Lemma to first show that: I(Y; < 1/ logz (1+ ) Now, as the variance o becomes high, have T(Y; F)
21

---

Published in Transactions on Machine Learning Research (11/2025)
14
infermation
24
22
3.0
0.8
14
Irlue Lmanl (Stanttent (eciniion)
Figure 15: Mutual Information vs. Noisc Level (Y is Bernoulli)
Since N IL Y, we have I(Y; 0. Now, from equation we have
I(Y: B) Uni(Y:B|F) + Fc) = 0.
(23)
According to Lemma 4 Uni(Y:B)P" and RY:(Y:B, F) nonnegative. As their summation is 0, each term should be 0 as well, i.e., Uni(Y:B)F) = 0 and Red(Y:B,F) = 0.
Again, since has a high variance, we have (from Lemma
I(Y;F) S logg (+ oN
0.
(24)
This leads to Uni(Y:F = I(Y; Ro:((Y < loga
1
oi = 0.
However, B) = B) H(Y F) = H(YIN) HYYY + N) H(Y) which positive and
significant. This holds because H(Y + N) since Y is completely determined by Y + N and together.
Now,
1 o
Syn(Y:B,P) = F|B; Uni(Y:F|B) > H(Y)
logg
1 H(Y).
(25)
C.3.3
Proof of Lemma 3
Proof of Lemma
Here, the input feature
X = (F, Observe that we have the following conditional
), and X ~ 1], 2
distributions:
X|y= ~ N[(O0)
~
NNn ). For simplicity, assume
Nn.
0
P(Y 0) = P(Y = 1). We let Σ
o Ne
22

---

Published in Transactions on Machine Learning Research (11/2025)
For the Bayes optimal classifier at the decision boundary, have:
P(X P
loog logg( ( 0)) =
=))
X = -- (X
'x
1 (x -X (
2
|
|B
1F
B B
NF
NN
NF
Nu
F
B
0 Ny
20 Nr
Nr
20 NB
This is the decision boundary for the Bayes optimal classifier. Thus, we can show that when NN > dNp. the boundary relics heavily on core feature F. Similarly, when 2xp > oNg. the boundary relics hcavily on spurious feature B. Also refer to Fig. (first two cases) for a pictorial illustration of how the optimal classifier behaves.
Next, observe that when Nr >> o No. we have I( I(Y:F) with strict equality (see Lemma From the definition of PID, T(Y; B) = Since I(Y; B) > I(Y;F), we therefore have:
F) and T(Y; F) =
F).
Uni(Y:B[F) + (Bed(YB,,)) Uni(Y:F|B)- + Red(Y:B,
This leads to Uni(Y:B|F)> Uni(Y:F| 2 since each PID term nonnegative.
C.4 Additional Details on Dimensionality Reduction
The representation loss is the mean square error between the input the encoder x and the output of the decoder x' defined as I,, = || - 1'112. The cluster centers {ur} (trainable weights of clustering layer) and embedded point 2; (output of the encoder) are used calculate the soft label Øj = E, (1+12= P2-1
[1+ where
di is the jth entry of the soft label 4; denoting the probability of 2; belonging to cluster μy- The clustering loss L. is the KL divergence between the soft assignments (q:) and an auxiliary distribution (p.). First, the autoencoder is pre-trained using only Lyp to initialize the auxiliary distribution, and the cluster centers are initialized by performing k-means on the embeddings all images. After pretaining, the cluster centers and autoencoder weights are updated with the joint loss iteratively while the auxiliary distribution only updated after T iterations.
Appendix to Experiments
This section includes additional results and figures for a more comprehensive understanding,
D.1 Additional Details on Automatic Segmentation of Features
Segmentation, a component of our Spurious Disentangler, plays pivotal role in identifying core features from spurious ones. Identifying spurious features (pixels) without any additional information is challenging in image datasets, particularly if they lack group labels. However, in supervised classification tasks, the availability of target labels corresponding directly the goal of the classification task (and hence some partial knowledge of what the core features should be, if not the exact pixels) often offers practical workaround. Specifically, one leverage automatic segmentation to at least. perform object detection and
23

---

Published in Transactions on Machine Learning Research (11/2025)
choose the most relevant objects the "core". Then, the regions of an image not associated with the "core" objects can often be considered subset of spurious features.
Advances in Open- Vocabulary Semantic Segmentation (OVSS) have significantly reduced the dependence on task-specific training by enabling generalization to unseen categories without requiring labeled data. To leverage these adnncemenss, we employ CLIPSeg (Lüddecke Eoker 2022), a state-of-the-art OVSS model, to generate masks for various objects in zero-shot manner using partial knowledge of the classification task in mind. For instance, in the Waterbird dataset, we specify the prompt "bird' to obtain mask for the bird object. This approach utilizes publicly available fine-grained weights, enabling efficient and accurate segmentation without additional labeled data. The generated mask is applied to the original image to extract the foreground, while the background is obtained by multiplying the original image by 1 - mask, as illustrated in Fig. 12
Thus, our proposed technique of datasct evaluation can be applied in conjunction with such automatic segmentation methods to any image dataset where the group information is not available, enabling to first identify approximation of the core features using partial knowledge of the target objects for the classification task, and then explain the nature of spurious patterns.
D.2 Additional Results
Our explainability framework preemptive or anticipative of spuriousness using just the dataset before training the model. The goal of our experiments is to show broad agreement between our anticipations from the dataset before training any model and the post-training behavior of actual models (when trained regularly to optimize performance without doing anything else specifically targeted towards avoiding spurious features). Apart from Worst- Group Accuracy, we also observe the Brad-CAM visualizations to check if the model demonstrates stronger emphasis on the relevant core features or not (see Fig. 10 21 18
To further
justify this, we calculate the intersection-over-uiio (IoU) metric (Rezatofighi et al. 2019) over the entire test Waterbird datasel. Table D shows that when the dataset is modified from unbalanced to the other variants, the IoU score increases. The IoU score is calculated using the ground-truti segmentation masks of birds and the masks obtained from the Grad-CAM explanation.
Table 1: Intersection over Union (IoU) between ground truth and Grad-CAM masks on the WATERBIRD dataset.
Test Group
Unbalanced
Class Balanced Group Balanced
Addition
Concatenation
Minority Group
0.22
0.29
0.24
0.28
0.32
All Groups
0.19
0.23
0.22
0.29
0.30
Table 2 shows a comparison between our proposed measure of spuriousness Map and other possible measures.
D.3 Additional Details on Clustering
At the dimensionalitt reduction stage, we must choose an appropriate number of clusters. We compute PID values for cluster counts of 5, 10, and 20. As shown in Table 3 the PID components and consequently the spuriousness measure Map are sensitive to the number of clusters. However, when moving from the unbalanced the blasnebalanced setting, Map consistently decreases across all cluster counts, indicating that essential information is retained despite dimensionality reduction. A similar sensitivity analysis on the CMNIST dataset (see Fig. 16 confirms this pattern. We observe that for CMNIST with two target classes, the computational time increases significantly - from approximately 6.8 seconds (cluster size 5) to 470.63 seconds (cluster size 50) when the flip rate is set to 0% (see Table 4). However, most importantly, we observe that for more than 10 clusters, the convex optimization for PID calculation sometimes fails to converge even after 50,000 iterations. Based on these results and previous work (Guo et al. (017), we choose 10 clusters as trade-off between preserving sufficient information and ensuring computational efficiency and faster convergence.
24

---

Published in Transactions on Machine Learning Research (11/2025)
Table 2: Comparison of the proposed spurionsness measure (Map) with altornative metrics across dataset
variants
Dataset
Measure
Unbalanced
Class Balanced
Group Balanced
Addition
Concatenation
I(;; B)
0.1726
0.0315
0.0028
0.0005
0.0002
WATERBIRD
B) (T;)--1022 I():F)
0.1669
0.0298
-0.0089
-0.0052
-0.0054
Proposed Map
0.1486
0.0185
-0.0322
-0.0208
-0.0195
I(: B)
0,1882
0,0005
0.0010
0.0010
DOMINORS 1.0
I(;; B) - 1)
0.1728
-0.0010
-0.0203
-0.0144
Proposed Map
0.1660
-0.0165
-0.0279
-0.0207
I(Y:
0.5913
0.2610
0.0002
0.0001
DOMINOES 2.0
I(Y: B) F(
0.5619
0.2462
-0.0426
-0.0477
Proposed Map
0.5357
0.2237
-0.0501
-0.0574
I); I(Y: B) 1(:FF)
0.0238
0.0005
0.0151
CELEBA
0.3038
-0.3713
-0.4051
Proposed Map
-0.3091
-0.3775
-0.4797
T(Y: B) I(;; B) - F)
0.0437
0.0096
SPAWRIOUS
0.0012
-0.0056
Proposed Map
-0.0007
-0.0176
Table 3: PID components for the Waterbird dataset across varying cluster sizes under unbalanced and class- balanced conditions.
Unbalanced
Clusters
Red(Y: : B)
Uni(Y:B F²)
Uni(Y:FI B)
Syn(Y:F, B)
Map
5
0.0065
0.1220
0.0000
0.0085
0.1135
10
0.0057
0,1669
0.0000
0.0184
0.1485
20
0.0025
0.1736
0.0000
0.0163
0.1573
Class Balanced
Clusters
Red(Y:F, B)
Uni(Y:B F)
Uni(Y=F B)
Syn(Y:F, B)
Map
5
0.0008
0.0221
0.0000
0.0012
0.0209
10
0.0016
0.0300
0.0001
0.0114
0.0185
20
0.0008
0.0128
0.0000
0.0097
0.0031
Table 4: PID estimation time with varying number of clusters for CMNIST (flip rate 0%).
Number of Clusters
Time (seconds)
5
6.80
10
18.69
20
65.12
30
153.29
50
470.63
25

---

Published in Transactions on Machine Learning Research (11/2025)
Msp number of clusters for different flip rates
0.5
Flip
Flip 15%
Flip 40%
0.4
0.3
0.2
MSP
0.1
0.0
-0.1
0
10 20
40
of Clusters
Figure 16: Map versus number of clusters for different label flip percentages. As the number of clusters increases, Map tends to decrease, and higher flip rates result in lower Map values, indicating decreased spuriousness.
D.4 Additional Details on Datasets
Table 5: Summary of the datasets
Dataset
Split
Group 00
Group 01
Group 10
Group 11
Train
3,498
184
56
1,057
WATERBIRD
Validation
467
466
133
133
Test
2,255
2,255
642
642
Train
3,750
1,250
1,250
3,750
DOMINOLS 1.0
Test
473
507
507
473
Train
3,000
500
1,250
300
DOMINOES 2.0
Test
245
490
245
490
Train
10,116
15,930
1,214
6,929
ADULT
Test
4,307
6,802
555
2,989
Train
11,111
8,305
4,003
188
CELEBA
Test
1,391
997
525
18
Train
3,072
2,275
175
1,056
SPAWRIOUS
Test.
96
893
2,993
2,112
26

---

Published in Transactions on Machine Learning Research (11/2025)
Table 6: PID values across dataset variants
Dataset
Variation
Red(Y:F, Uni(Y:B|F)
Uni(Y:F(B)
Syn(Y:F, B)
Map
Unbalanced
0.0057
0.1669
0.0000
0.0184
0,1486
Class Balanced
0.0016
0.0300
0.0001
0.0114
0.0185
WATERBIRD
Group Balanced
0.0026
0.0001
0.0091
0.0233
-0.0322
Addition
0.0004
0.0001
0.0053
0.0156
-0.0208
Concatenation
0.0002
0.0001
0.0055
0.0140
-0.0195
Unbalanced
0.0374
0.0000
0.0661
0.0267
-0.0928
ADULT
Group Balanced
0
0
0.1163
0.0090
-0. 0.1222
Unbalanced
0.0154
0.1728
0.0000
0.0068
0.1660
Group Balanced
0.0003
0.0002
0.0013
0.0155
-0.0165
DOMINOES 1.0
Addition
0.0009
0.0000
0.0203
0.0076
-0.0279
Concatenation
0.0009
0.0000
0.0144
0.0063
-0.0207
Unbalanced
0.0294
0.5619
0.0000
0.0061
0.5557
Group Balanced
0.0148
0.2462
0.0000
0.0225
0.2237
DOMINOES 2.0
Addition
0.0001
0.0000
0.0426
0.0075
-0.0501
Concatenation
0.0001
0.0000
0.0477
0.0096
-0.0574
Unbalanced
0.0238
0.0000
0.3038
0.0053
-0.3091
CELEBA
Class Balanced
0.0000
0.0000
0.3713
0.0063
-0.3775
Group Balanced
0.0151
0.0000
0,4051
0.0746
-0.4797
Unbalanced
0.0396
0.0041
0.0029
0.0019
-0.0007
SPAWRIOUS
Group Balanced
0.0089
0.0007
0.0063
0.0121
-0.0176
D.4.1 Both core and spurious features available
WATERBIRD: The Waterbird dataset (Wah et al. 2011 is popular spurious correlation benchmark. The task is to classify the type of bird (waterbird = 1, landbird = 0). However, there exists a spurious correlation between the backgrounds (water = 1, land = 0) and the labels (bird type). The two types of backgrounds and foregrounds result. in total of four groups. A summary of the Waterbird datasel is given in Table 5 and see Fig. 17 for examples of the dataset. We additionally consider four random training splits with group counts [2096 212 102 2385]. [3493 213 58 1031], [2632 420 150 1593], and [1363 942 1469 1021] to observe the correlation between the measure of spuriousness Map and W.G. Acc. (%) (see Fig 7 in diverse settings. We use Spurious Disentangler to calculate the PID values. The hyperparameters are as follows: batch size of 64, a learning rate of 0.001, Cosine scheduler, an Adam optimizer with a weight decay of 0.0001, 50 pretraining epochs, followed by 100 epochs of additional training. When fine-tuning ResNet-50 we use the following hyperparameters: batch size of 64, learning rate of 0.0001, Cosine Annealing.. scheduler, stochastic gradient descent (SGD) optimizer with weight decay of 0.0001, binary cross-entropy as the loss function, and 100 epochs. See Table 6 for the details of the PID values, and Table 7 for the worst-group accuracies of different variants of the Waterbird dataset.
DOMINOES: Dominocs is a synthetic dataset created by combining handwritten digits (zero and onc) from MNIST (Deng 2012) and images of cars and trucks from CIFARIO (Krizhevsky et al. 2009) (digit 0 or 1 at the top, car (= 0) or truck (= 1) at the bottom of an image). We make two versions of this synthetic datasot, namely Dominocs 1.0 and Dominoes 2.0, inducing different degrees of sampling biases. The task is classify whether the image contains a car or a truck; hence, the car or truck corresponds to the core features (foreground). On the other hand, the digits considered as the spurious features (background).
27

---

Published in Transactions on Machine Learning Research (11/2025)
Background
F: Foreground
Whole Image
Figure 17: Samples of Waterbird dataset (original, concatenation, and addition).
Table 7: Worst-group accuracy (%) across dataset variants
Dataset
Unbalanced
Class Balanced
Group Balanced
Addition
Concatenation
WATERBIRD
45.92 i 1.43
74.49 0.58
85.82 0.71
88.18 2.17
92.60 0.39
DOMINOES 1.0
86.29 + 4.44
90.19 = 1.23
94.42 = 0.24
96.06 0.39
DOMINOES 2.0
78.78 + 1.02
88.06 = 1.12
86.74 + 1.22
90.72 + 3.37
CELEBA
71.41 1 0.81
85.29 1 2.94
98.34 1 1.66
SPAWRIOUS
91.91 + 1.94
95.24 0.28
The summary of Dominoes 1.0 and Dominoes 2.0 given in Table 5 Fig. 19 shows the examples of original, addition, and concatenation variants of the dataset. For PID calculation, the hyperparameters are follows: batch size of 8, learning rate of 0.001, CoosineAneallngg decay of 0.0001, 100 pretraining epochs, followed by 50 epochs of additional training. See Table 6 for the details of PID values and Map-
scheduler, an Adam optimizer with weight
The applicability our proposed framework goes beyond images and can also be applied for explainability on tabular datascts. We perform an experiment on the Adult (Becker & Kohavi, 1996) dataset. The task is to predict whether the annual income of an individual exceeds $50k per year not (> 50k 1,<= 50k 0). Here consider "gender" and "age", "oducation-mum"" "hours-per-woeke jointly as core feature matrix. After performing k-mcans clustering, we use the estimation module (DIT package) to calculate PID values with core features, spurious features, and the target label. Table shows the values for redundancy, unique information, and synergy.
TABULAR DATASET: ADUET:
spurious feature vector (male female 0)
We train the XGBoost (Chen Guestrin 2016) model for the prediction task and calculate the worst- group accuracy which corresponds the accuracy of the minority group (see Table minority group 10 corresponds to female individuals with >50k income.).
28

---

Published in Transactions on Machine Learning Research (11/2025)
Figure 18: Examples of Grad-CAM images Dominoes dataset: Observe that for the unbalanced dataset (Ist from left), the model adds more emphasis (red regions) to the digits (background) while in the group- balanced, addition and concatenation versions (2nd, 3rd, and 4th from left), the car (foreground) is more emphasized.
J
Figure 19: Samples of Dominoes dataset (original, concatenation, and addition).
D.4.2 Only features available
CELEBA: CelebA is another popular dataset for spurious correlation benchmarking, which consists of images of male-female celebrities. We use subset of this dataset, namely CelebAMask-HOO, {Lee et al. 2020), to utilize the segmentation mask of the hair while calculating the PID values. The objective is to identify blonde (= and non-blonde (= 0) hair. However, there exists a spurious correlation between gender (men (= 1), women (= 0)) and the label, which makes the model focus on the face rather than the hair to classify the color of the hair (Moayeri et al., 2023). We consider hair as the foreground and anything but hair as the background. We do not perform background mixing for this datasct since it is not practical to add concatenate two faces randomly. The summary of the CelebA (Lee al.) 2020) dataset is given in Table
The steps and hyperparameters for calculating PIDs are the same in the Waterbird dataset. See Fig. 20 for
29

---

Published in Transactions on Machine Learning Research (11/2025)
the examples of dataset samples. The details of PID values and worst-group accuracies for several variations of this dataset are shown in Table G and Table 7 respectively.
B: Background
Foreground
Whole Image
Figure 20: Samples of the CelebA dataset.
Figure 21: Examples of Grad-CAM images CelebA dataset: Observe that for the unbalanced dataset (1st from left), the model adds more emphasis (red regions) the face (background) while in the Lass-balanced and group-balanced (2nd and 3rd), the hair (foreground) is more emphasized.
D.4.3 Segmentation to obtain features
SPAWRIOUS: Spawrious (Lynch et al. 2023) is synthetic image dataset created by employing text-to- image model. We use a subset of this datasct where we classify dog breeds dachshund (= 0) and labrador (= We select the subset so that most of the dachshunds are on beach (= 0) backgrounds and the rest are on desert 1= backgrounds. The summary of the subset of the Spawrious dataset (Lynch et al.
2023 that
we use for our experiment is given in Table 5 The samples of this dataset are shown in Fig. 22
We use a
segmentation model, namely Feature Pyramid Network (FPN) (Lin al.) 2017] with ResNet-34 He al., 2016) encoder pre-trained with Oxford-IIIT Pet. Dataset to create the segmentation mask of the dogs of our dataset. Using this mask, separate the foreground "dog" from the background. After having background and foreground,
use principal component analysis (PCA) (Mackiewicz & Ratajezak 1993) followed by
k-means clustering to have a discrete dwwer-dimeniional representation. We do not use our autoencoder module since, for this dataset, a simpler dimensionality reduction also seems to have a low reconstruction loss. Then we use our estimation module to calculate the PID values and Map- Tables and Table show all PID values along with the measure and the goost-group accuracy, respectively.
D.4.4 Non-spatial spuriousness
Colored MNIST (CMNIST) is a synthetic datasct derived from MNIST. Whereas MNIST images are grayscale, each image in CMNIST is colored eit red or green in a way that correlates spuriously with the class label. We define two environments (one training, one test) from MNIST, transforming each example as follows: first, assign preliminary binary label y to the image based on the digit: y = for digits Second, sample the color id z by flipping y with probability Pe (flip probability).
COLORED MNIST:
0-4 4 and y = 1 for In the test one, the flip probability is kept 0.9. Finally, color the image red if
green if (see
30

---

Published in Transactions on Machine Learning Research (11/2025)
B: Background
F: Foreground
Whole Image
Figure 22: Samples of the subset of the Spawrious dataset we use in this work.
0
0
5
9
2
10
10
10
10
10
10
20
20
20
20
10
20
10
20
10
20
10
20
20
10
20
Figure 23: Samples of the CMNIST dataset used in this work.
Table 8: Comparison of spuriousness measure Map and worst-group accuracy (%) for multiclass and binary classification on CMNIST under different flip probabilities.
Flip (%)
Binary
Multiclass
Map
W.G. Acc. (%)
Map
W.G. Ace. (%)
0.0
0.58581
0.13++0.15
0.40220
0.19=00.14
2.5
0.45287
78.77=1.18
0.27035
83.88+1.666
5.0
0.38470
87.96-10.37
0.17788
86.96
7.5
0.30933
92.9440.54
0.07443
93.1110.88
10.0
0.24979
94.97+0.85
0.03366
93.33
12.5
0.19956
95.00+10.43
-0.00945
94.46:1:0880
15.0
0.14296
95.73+0.44
-0.06050
94.98+00.37
17.5
0.10727
96.13:0..37
-0.11422
94.83110.43
20.0
0.07481
96.49+0.58
-0.14608
95.66+0.39
22.5
0.04553
97.21+0.28
-0.19066
95.99+0.68
25.0
0.02400
97.62+0.33
-0.20659
96.2410.05
27.5
-0.01175
97.73+00.38
-0.22717
95.87110.20
30.0
-0.01043
97.78+0.22
-0.50008
96.05110.52
32.5
-0.03003
98.04+0.25
-0.27048
96.55++0331
35.0
-0.05213
98.1720.22
-0.28146
96.86110.53
37.5
0.06233
98.3440.36
-0.86255
96.11=0.22
40.0
-0.63881
98.4240.26
-0.30888
96.2210.55
42.5
-0.06186
98.57+0.30
-0.33389
96.65+0.54
45.0
-0.06421
98.42+0.31
-0.30355
96.89110.08
47.5
-0.06254
98.5640.13
-0.30862
96.8140.31
50.0
-0.07099
98.75+00.01
-0.30195
96.27+ 0.00
See Table g for details of the spuriousness measure Map and the worst gorst-group accuracy. After performing k-means clustering, the estimation module to calculate the PID values.
Fig. 23
31

---

Published in Transactions on Machine Learning Research (11/2025)
All experiments executed NVIDIA RTX A4500. For the Waterbird dataset, the dimensionality reduction step takes around 50 minutes each for the background and foreground, and for the PTD estimation with Liang et al. (2023) it takes < seconds, and with James et al. (2018) takes around 50 seconds. For CMNIST, the whole process takes around 7 - 20 seconds for the binary task and 15-50 seconds for multiclass classification. Note that for the CMNIST dataset, we do not use an autoencoder for dimensionality reduction; rather, we directly perform k-means clustering followed by PTD estimation due to the simplistic nature of this dataset.
D.5 PID Estimator Analysis
Liang et al. Dit
0.6
Msp
0.4
0.2
0.0
30
40
Flip (%)
Figure 24: Map for different PTD estimators versus flip probability (%)
Fig. 24 presents a systematic comparison of the two PID estimators for the binary classification task on the CMNIST dataset across multiple flip probabilities. We observe that the estimated spuriousness measure Map is largely consistent across estimators, with both following the same trend. These results suggest that our spuriousness measure can tolerate variations in the choice of the PID estimator and support the reliability of our main findings.
32


Reviews:
[{"review_id": "0dpiAtukzq", "reviewer": "Reviewer_rHRs", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The method builds on Partial Information Decomposition. It connects this with Blackwell sufficiency from decision theory. This gives a structured way to separate unique, redundant, and synergistic information. It also explains when a feature is indispensable or interchangeable.\n\n- The spuriousness metric is carefully derived. The authors use theory and counterexamples to rule out weaker alternatives like plain mutual information. The final measure captures when spurious features dominate, when core features dominate, and when both act together.\n\n- The framework is tested on six datasets with varied bias types. Results consistently show a negative link between the measure and worst-group accuracy. Visual tools like Grad-CAM and IoU confirm whether the model focuses on the right features. This strengthens trust in the results.\n\nWeaknesses:\n- Despite being mentioned by the authors, I am still worried about the PID estimation being highly data-dependent. Small sampling changes or noise in the dataset can shift the values a lot. This makes results less stable for small datasets or those with high variability.\n- The pipeline depends on clustering latent features, then estimating PID on a discretized joint. The spuriousness score moves with the number of clusters, runtime grows quickly, and the PID optimizer sometimes fails to converge. This makes the method brittle to design choices and scale.\n- The method fixes a particular PID estimator for most experiments. It switches to a different estimator for multiclass. Different valid PID formalisms can partition information differently so that the score may inherit estimator-specific behavior. The paper does not study this dependence.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose an information-theoretic framework for preemptively quantifying dataset spuriousness by using Partial Information Decomposition to break down predictive information into unique, redundant, and synergistic components. They introduce a spuriousness measure that captures the tendency of models to rely on spurious rather than core features, and present the Spuriousness Disentangler, an autoencoder-based pipeline combining segmentation, dimensionality reduction, and joint distribution estimation to compute these components for high-dimensional data. Experiments on six benchmark datasets under varied bias conditions show a consistent negative correlation between the proposed measure and post-training worst-group accuracy, supported by visual evidence from Grad-CAM and IoU analyses, demonstrating the method’s effectiveness for dataset quality assessment and interpretability.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I wish the authors would address or give more discussion on the mentioned weaknesses.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Ud1UDVhFYs", "reviewer": "Reviewer_4T6E", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The method is novel and grounded in solid information-theoretic principles. \n\nStrengths:\n- The use of PID is innovative and well grounded in information theory, which I really appreciate. \n- The experimental results are strong and supportive of the theoretical claims, being compared across six benchmark datasets.\n\nWeaknesses: \n- I find the paper structure very confusing. While I appreciate the preliminaries section being grounded in the theoretical framework for the method, there is  no real Method section to describe the algorithm step by step, which I find very confusing to follow. Even worse, the architecture is presented in the Main Results section, which I find even more confusing. Besides, a lot of details about the overall framework are given in the introduction, which caused me to go back and forth through the paper to understand the methods. \n- From my understanding, the framework requires to know the separation between core and spurious features beforehand. However, this may not always be possible to provide as an input.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Spurious correlations are non-causal associations in datasets that mislead machine learning models, degrading generalization. The paper introduces a new framework based on Partial Information Decomposition (PID) to quantify such spurious correlations in datasets before the training of the models. The spurious disentangler is the key innovation, where core features (e.g. foreground) are separated by spurious ones (background) by leveraging an auto encoder architecture, and are then further analysed by PID. \n\nThe method in this paper is a novel contribution towards addressing spurious correlations, with potential to improve the reliability and fairness of machine learning models across diverse applications.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please revise the structure of the paper. I suggest shortening the introduction, renaming Main Results into Material and methods, and moving in the methods the description of the models, data and compute used for the experiments to make the paper easier to follow than its current version.", "broader_impact_concerns": "No concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "5k2I93V40R", "reviewer": "Reviewer_L5Yk", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. I like the idea of trying to quantify spuriousness of a dataset. While previous works just provide a handwavy idea of what is \"spurious\" the authors here try to systematically build on the notion of spuriousness and try to measure it, and also show how it correlates with accuracy measures. \n2. The many datasets used in the empirical evaluation shows the reliability and consistency and effectiveness of the method. \n3. Well written and easy to understand paper. \n4. The connection they make to statistical decision theory is very interesting!\n\nWeakness:\n1. This framework definitely depends on apriori knowing or defining what constitutes core versus spurious features, which may limit the applicability in scenarios where such distinctions are not known apriori. Please throw a little more light on such scenarios and what the authors suggest or propose in those cases, maybe I didn't fully understand the scope of this paper. I'm also concerned about introducing subjective bias in feature categorization; the authors can suggest a few things in this regard. \n2. The dimensionality reduction and clustering techniques surely may lead to loss of critical information. I want to know how this affects the reliability of PD estimates, some suggestions/solutions to overcome this, and a sensitivity analysis with-respect-to the discretization parameters is really helpful I feel. \n3. The method overall is feasible, but I want to know the computational complexity of the algorithm, and the procedure involved, particularly the iterative autoencoder training, and the convex optimization protocol for the PID estimation. Maybe this is in the supplementary and I missed it, if so please point me to it. \n4. Please include statistical tests for the correlations reported than just visual trends that are just observational.\n5. Please release the code or point me to a github link. Maybe I missed it in the paper. That'll clarify a lot of other minor doubts I have about the methodology and implementation.\n6. It would have also been very nice if there would have been some theoretical analysis connecting the PID measure to generalization bounds, would have greatly strengthened the claim. Have the authors thought about this and have any suggestions ideas in this regard.\n\nDespite these weaknesses, I still feel the positives of the paper outweight the negatives, and I'm happy to accept the paper only if my questions and concerns raised above are convincingly addressed.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors present a novel information-theoretic framework for preemptively quantifying spuriousness in datasets using a technique known as Partial Information Decomposition (PID). The PID can be leveraged to decompose total information about target variable into four non negative quantities: unique information in core / spurious features, redundant, and synergestic information. Using these a systematic derivation of a spuriousness measure is provided. It quantifies spuriousness in the dataset so that one may take preemptive measures to avoid them during training/fine-tuning. A three-step procedure: segmentation, dimensionality reduction via autoencoders, and PID estimation is proposed to achieve this. A comprehensive evaluation is done across six benchmark datasets to show the utility and effectiveness of the proposed framework. The authors empirically observe a negative correlation between proposed spuriousness measure and the post-training worst-group accuracy.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please check the weaknesses section above.", "broader_impact_concerns": "I do not see any obvious potential concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)