Identifying General Mechanism Shifts in Linear Causal Representations

Authors: tianyu  chen
kevin  bello
francesco  locatello
bryon  aragam
pradeep  ravikumar
Venue: NeurIPS_cc_2024_Conference
Published: 2024-04-23 23:23:26

Abstract: We consider the linear causal representation learning setting where we observe a linear mixing of $d$ unknown latent factors, which follow a linear structural causal model. 
Recent work has shown that it is possible to recover the latent factors as well as the underlying structural causal model over them, up to permutation and scaling, provided that we have at least $d$ environments, each of which corresponds to perfect interventions on a single latent node (factor). 
After this powerful result, a key open problem faced by the community has been to relax these conditions: allow for coarser than perfect single-node interventions, and allow for fewer than $d$ of them, since the number of latent factors $d$ could be very large. 
In this work, we consider precisely such a setting, where we allow a smaller than $d$ number of environments, and also allow for very coarse interventions that can very coarsely \textit{change the entire causal graph over the latent factors}. 
On the flip side, we relax what we wish to extract to simply the \textit{list of nodes that have shifted between one or more environments}. 
We provide a surprising identifiability result that it is indeed possible, under some very mild standard assumptions, to identify the set of shifted nodes. 
Our identifiability proof moreover is a constructive one: we explicitly provide necessary and sufficient conditions for a node to be a shifted node, and show that we can check these conditions given observed data. 
Our algorithm lends itself very naturally to the sample setting where instead of just interventional distributions, we are provided datasets of samples from each of these distributions. 
We corroborate our results on both synthetic experiments as well as an interesting psychometric dataset. The code can be found at https://github.com/TianyuCodings/iLCS.

Main paper: 
 introduction: 
 The objective of learning disentangled representations is to separate the different factors that contribute to the variation in the observed data, resulting in a representation that is easier to understand and manipulate [3] . Traditional methods for disentanglement [e.g., 19, 20, 7, 9, 26] aim to make the latent variables independent of each other.Consider the setting of linear independent component analysis (ICA) [19] , that is, the observed variables X ∈ R p are generated through the process X = GZ, where Z ∈ R d are latent factors, and G ∈ R p×d is an unknown "mixing" matrix. Under the key assumption that Z has statistically independent components, and under some additional mild assumptions, landmark results in linear ICA show that it is possible to recover the latent variables Z up to permutation and scaling [13, 19] .However, what if instead of independent sources Z we have a structural causal model (SCM, [37, 38] ) over them? For instance, if the latent factors correspond to biomarkers in a biology context, or root causes in a root cause analysis context, then we expect there to be rich associations between them. Indeed, this question is central in the burgeoning field of causal representation learning (CRL) [39, 51] , where we are interested in extracting the latent factors and causal associations between them given raw data.Let us look at the simplest CRL setting where the latent variables Z follow a linear SCM, that is, Z = AZ + Ω 1 /2 ϵ, where A ∈ R d×d encodes a directed acyclic graph (DAG), Ω is a diagonal matrix that controls the scale of noise variances, and ϵ is some noise vector with zero-mean and unit-variance independent components. In such a case, Z is a linear mixing of independent components ϵ, that is, Z = B -1 ϵ, where B = Ω -1 /2 (I d -A) succinctly encodes the SCM and I d is the identity matrix of dimension R d×d . We then have X = GB -1 ϵ so that ICA can only recover BG † up to permutation and scaling, which does not suffice to recover the SCM B since the mixing function G is unknown.Recently, Seigal et al. [40] showed that given the interventional distributions arising from perfect interventions on each latent variable in Z, we can recover the SCM over Z up to permutation. But there are two caveats to this: (a) it is difficult to obtain perfect single-node interventions that only intervene on a single factor in Z; and (b) it is difficult to obtain d number of such perfect interventional distributions or environments.We are interested in the setting where we do not have perfect interventions: we allow for far more general interventions that can quite coarsely change the SCM, namely, soft and hard interventions, interventions targeting single or multiple nodes, as well as interventions capable of adding or removing parent nodes and reversing edges. Moreover, we do not need as many as d of these.Our goal, however, is not to recover the entire SCM over Z but simply to recover those nodes Z that have incurred shifts or changes between the different interventional distributions. This is closely related to root cause analysis [5, 6, 21, 33] , which aims to identify the origins of the observed changes in a joint distribution. In addition, understanding the sources of distribution shifts-that is, localizing invariant/shifted conditional distributions-can benefit downstream tasks such as domain adaptation [30] , and domain generalization [36, 55] .Our work sits at the intersection of linear CRL [40, 23] and direct estimation of causal mechanism shifts [52, 14] . The key contribution of this work is to show that it is possible to identify the latent sources of distribution shifts in multiple datasets while bypassing the estimation of the mixing function G and the SCM B over the latent variables, under very general types of interventions. More concretely, we make the following set of contributions:1. Identifiability: We show that we can identify the shifted latent factors even under more general types of interventions. (Section 4.1). 2. Algorithm: We also provide an scalable algorithm that implements our identifiability result to infer such shifted latent factors even in the practical scenarios where we are not given the entire coarse interventional distributions but merely finite samples from each (Section 4.2). 3. Experiments: We corroborate our results on both synthetic experiments (Section 5.1) as well as an interesting psychometric dataset (Section 5.2). 
related work: 
 Causal representation learning. In contrast to our setting, which focuses on identifying shifted nodes in the latent representation, existing methods in CRL aim to recover both the latent causal graph and the mixing function. Previous works have studied identifiability in various settings, such as latent linear SEMs with linear mixing [40] , and with nonlinear mixing [4] ; latent nonlinear SEMs with finite degree polynomial mixing [1] , and with linear mixing [48] ; and nonlinear SEMs with nonlinear mixing [50, 49, 23, 22] . Although these studies ensure the identifiability of causal graphs (up to permutation and scaling ambiguities), they generally rely on the assumption that each latent variable is intervened upon in at least one environment, necessitating access to at least d interventional distributions. Moreover, the aforementioned works assume specific types of interventions, such as hard/soft interventions and single-node interventions, and restrict changes in interventional distributions, disallowing edge reversals or the addition of new edges. The most recent work [23] enables causal representation learning under general interventions in latent linear SEMs with linear mixing. However, this approach still requires the assumption that the number of environments K is at least equal to the number of latent nodes d and that there are at least Θ(d 2 ) interventions. If the objective is to detect variables with general mechanism changes across multiple environments-environments that may lack a consistent topological order and sufficient interventions or environments-using existing CRL methods to recover each latent graph becomes overly restrictive or even infeasible. In contrast, we present a more flexible approach, enabling the identification of shifted variables without assuming restrictive interventions per environment or a consistent topological order of the latent graphs.Direct estimation of mechanism shifts. The problem of directly estimating causal mechanism changes without estimating the causal graphs has also been explored in various settings in the regime in which the causal variables are observable. Wang et al. [52] and Ghoshal et al. [14] have focused on identifying structural differences, assuming linear SEMs as environments, and proposing methods that take advantage of variations in the precision matrices. More recently, Chen et al. [10] studied this problem for nonlinear additive noise models, assuming that the environments originate from soft/hard interventions and leverage recent work in causal discovery via score matching. Finally, the concept of detecting/localizing feature shifts between two distributions has also been discussed in [27] , although from a non-causal perspective. To our knowledge, there is a gap in the literature regarding the study of these objectives when considering latent causal variables. We address this gap by proposing a novel approach for directly detecting mechanism shifts within the latent SCMs.Independent component analysis. The application of independent component analysis (ICA) [12] in the realm of causal discovery has seen significant developments. Linear ICA [19] and its nonlinear counterpart [20] have been instrumental in causal discovery [35, 44, 53] and more recently in causal latent discovery [23] . Beyond these established applications, our work uncovers a novel use of ICA, namely, identifying shifted nodes within the latent linear SCMs.Given the relevance of ICA for our approach, we briefly recap it next. ICA considers the following setting:X = W ϵ where X ∈ R p , ϵ ∈ R d , p ≥ d.A key assumption in ICA is that each component of ϵ is independent. Given only observations of X, the goal of ICA is to estimate both W and ϵ. The objective function typically aims to maximize negentropy or non-Gaussianity, with further details given in [19] . The identifiability results of ICA can be summarized as follows.Theorem 1 (Theorems 3,4 in [13] ). If every component of ϵ is independent and at most one component is Gaussian distributed, with W being full column rank, then ICA can estimate W up to a permutation and scaling of each column, and ϵ can be recovered for some permutation up to scaling for each component. Furthermore, as noted in [19] , ifE[ϵ 2 i ] = 1, ∀i ∈ [d],the estimated W and ϵ will have ambiguities only in permutation and sign. Formally, this meansX = W ϵ = (W P T D)(DP ϵ),where P is a permutation matrix and D is a diagonal matrix with diagonal entries ±1. Then, the best estimate given by ICA is W P T D and DP ϵ. 
problem setting: 
 Consider a random vector X in R p that is a linear mixing of d latent variables Z = (Z 1 , . . . , Z d ):X = GZ.Here the latent variables in Z follows a linear SCM [37, 38] , that is,Z = AZ + Ω 1 /2 ϵwhere A ∈ R d×d corresponds to a DAG G such that A jk ̸ = 0 iff there exists an edge j → k in the DAG G; Ω ∈ R d×d is a diagonal matrix with positive entries, and ϵ ∈ R d is a random vector with independent components with mean zero and variance one, i.e., that Cov(ϵ) = I d . Denoting B = Ω -1 /2 (I d -A), we have that:Z = B -1 ϵ. Z 1 Z 4 Z 2 Z 3 Z 5 Z 1 Z 4 Z 2 Z 3 Z 5 ⋯ X 50 X 1 X 2 X 3 X 4 X 5 X 49 X 48 X 47 X 46 United States of America ℙ (US) (X 1 , X 2 , …, X 50 ) ℙ (US) (Z ) = ℙ(Z 1 | Z 2 , Z 3 , Z 4 )ℙ(Z 2 | Z 5 )ℙ(Z 3 | Z 5 )ℙ(Z 4 )ℙ(Z 5 ) Z 1 Z 4 Z 2 Z 3 Z 5 G ⋯ X 50 X 1 X 2 X 3 X 4 X 5 X 49 X 48 X 47 X 46 United Kingdom ℙ (UK) (X 1 , X 2 , …, X 50 ) ℙ (UK) (Z ) = P (Z 1 | Z 2 , Z 3 )ℙ(Z 2 | Z 5 )ℙ(Z 3 | Z 5 )ℙ(Z 4 )ℙ(Z 5 ) Z 1 Z 4 Z 2 Z 3 Z 5 G ⋯ X 50 X 1 X 2 X 3 X 4 X 5 X 49 X 48 X 47 X 46 Australia ℙ (AU) (X 1 , X 2 , …, X 50 ) ℙ (AU) (Z ) = ℙ(Z 1 | Z 2 , Z 3 , Z 4 ) P (Z 2 | Z 3 ) P (Z 3 )ℙ(Z 4 ) P (Z 5 | Z 3 ) G Figure 1:We have 5 latent variables Z which in this case relate to personality concepts, and the observations X represent the scores of 50 questions from a psychometric personality test. The latent variables Z follow a linear SCM, while the unknown shared linear mixing is a full-rank matrix G ∈ R 50×5 . Then, for environment k = {US, UK, AU}, the observables are generated through X (k) = GZ (k) . Here, P (US) is taken as the "observational" (reference) distribution, and the distribution shifts in P (UK) and P (AU) are due to changes in the causal mechanisms of {Z 1 } and {Z 2 , Z 3 , Z 5 }, respectively. Finally, the types of interventions are general; for UK, the edge Z 4 → Z 1 is removed and the dashed red lines indicate changes in the edge weights to Z 1 ; for AU, Z 2 was intervened by removing Z 5 → Z 2 and adding Z 3 → Z 2 , while the edge Z 5 → Z 3 was reversed, thus changing the mechanisms of Z 3 and Z 5 . Thus, we aim to identify {Z 1 } and {Z 2 , Z 3 , Z 5 }.We assume that we observe K ≥ 2 generalized interventional distributions that keep the mixing map G fixed but allow for generalized interventions to Z. That is, for environment k ∈ [K] we have,X (k) = GZ (k) ,whereZ (k) = A (k) Z (k) + (Ω (k) ) 1 /2 ϵ (k) . Similarly, we have Z (k) = (B (k) ) -1 ϵ (k) , where B (k) = (Ω (k) ) -1 /2 (I d -A (k) ).Notably, we allow generalized interventions that allow for A (k) to be arbitrary, which includes soft and hard interventions, interventions targeting single or multiple nodes, as well as interventions capable of adding or removing parent nodes and reversing edges. This contrasts with the existing literature on CRL, where single-node soft/hard interventions are the standard assumption [50, 40, 4, 1] . See Figure 1 , for a toy example of what we aim to estimate. Remark 1. Since we allow for general types of interventions, we can take any of the given environments as the canonical "observational" distribution with respect to which we observe interventions, or simply that we observe k interventions of an unknown observational distribution. This is a clear distinction from the standard setting in CRL [1, 50, 48, 23] which requires to know which environment is a suitable observational distribution.To develop our identifiability result and algorithm, we will make additional assumptions on the noise distributions of the linear SEMs. Assumption A (Noise Assumptions). For any environment k ∈[K], let ϵ (k) = (ϵ (k) 1 , . . . , ϵ (k)d ) be the vector of d independent noises with Cov(ϵ (k) ) = I d . We have:1. Identically distributed across environments:P(ϵ (k) ) = P(ϵ (k ′ ) ), for all k ′ ̸ = k. 2. Non-Gaussianity: At most one noise component ϵ (k) i is Gaussian distributed.3. Pairwise differences: For any i ̸ = j, we have P(ϵ(k) i ) ̸ = P(ϵ (k) j ) and P(ϵ (k) i ) ̸ = P(-ϵ (k) j ).Assumption A.1 is usually assumed for learning causal models from multiple environments [31, 4] . Assumption A.2 is typically made in causal discovery methods, as detailed in seminal works such as [43, 42, 19, 45] and is considered a more realistic assumption [34] . Assumption A.3 is generally satisfied in a generic sense; that is, when probability distributions on the real line are randomly selected, they are pairwise different with probability one. This assumption is also adopted in [47, 23] . Assumption B (Test Function). We assume access to a test function ψ that maps each noise r.v. to R s.t. ψ(ϵ(k) i ) = ψ(-ϵ (k)i ), and ψ(ϵ(k) i ) ̸ = ψ(ϵ (k) j ) if ϵ (k) i and ϵ (k) j are not identically distributed.This assumption states that we can access a test function that can help differentiate the noise components. One coarse example is ψ(y) = P(|y| ≤ 1). This assumption is introduced to better understand our method workflow in Section 4, but it is not completely necessary. We discuss how to relax this assumption in Appendix C. Next, we formally define a mechanism shift. P(Z (k) i | PA(Z (k) i )) ̸ = P(Z (k ′ ) i | PA(Z (k ′ ) i)). Remark 2. Following Definition 1, Z i is a latent shifted node between environments k and k ′ if: (1) The i-th rows of A (k) andA (k ′ ) are different; (2) Ω (k) ii ̸ = Ω (k ′ )ii ; or (3) both. Definition 1 aligns with those previously discussed in [52, 14, 10] , with the key difference that we consider changes in the causal mechanisms of the latent causal variables. However, note that our results also contribute to the setting in which causal variables are observable considering that the mixing function is the identity matrix, that is, G = I d . 
identifying shifts in latent causal mechanisms: 
 Following the setup outlined in the previous section, our focus now turns to developing an algorithm to identify latent shifted nodes, given data from multiple environments. First, note that we can write the overall model as a linear ICA problem, where, for any environment k, the observation X (k) is a linear combination of independent components ϵ (k) . Specifically, we haveX (k) = GZ (k) = G(B (k) ) -1 ϵ (k)Under the mild conditions given in Assumption A, from classical ICA identifiability results stated in Theorem 1, we can identify G(B (k) ) -1 up to permutation and sign flip. Let M (k) = B (k) H where H = G † . Then, we can only identify M (k) up to permutation and sign flip, which does not suffice to identify the latent SCM encoded in B (k) . In sum, what we can only obtain from ICA isM (k) = P (k) D (k) B (k) Hwhere P (k) is a permutation matrix, and D (k) is a diagonal matrix with -1 or +1 on its diagonal. As Seigal et al. [40] points out, it is not possible to identify B (k) further given generalized interventions.Our first result is that our present mild assumptions suffice to infer shifted nodes. Theorem 2 (Identifiability). Given access to K ≥ 2 environments, assume that A and B hold for all environments. Then, all latent shifted nodes are identifiable.An interesting facet of our identifiability result is that it is constructive. In the next subsection we will provide an explicit algorithm to infer the shifted nodes and prove the main theorem above.Consider ϵ (k) = B (k) HX (k) and ϵ (k) = M (k) X (k) = P (k) D (k) B (k) HX (k) = P (k) D (k) ϵ (k), where ϵ (k) and M (k) are the output of ICA, which contain the permutation and sign flip ambiguities given by P (k) D (k) .Obtaining a consistent ordering of the noise components across all environments is equivalent to finding P (k) . Under Assumption B, and without loss of generality, we consider that (ϵ(k) 1 , . . . , ϵ (k) d )are in increasing order with respect to their ψ values. Since ψ is invariant to sign flip, we can calculate ψ(ϵ(k) i ) for all i ∈ [d]and sort the calculated ψ values in increasing order. Let P (k) denote the sorting permutation with respect to ψ, so that post-sorting, we get P (k) ϵ (k) . Remark 3. In Appendix C, we discuss how to relax the assumption on the test function ψ. Proposition 1. P (k) = P (k) T , i.e., P (k) is the inverse permutation of the ICA scrambling.From Proposition 1, we thus find that we can unscramble the permutation P (k) by sorting with respect to ψ. We getP (k) ϵ (k) = P k P (k) D (k) ϵ (k) = D (k) ϵ (k) from the above proposition. In other words, we can extract ϵ (k) = D (k) ϵ (k) via M (k) = P (k) M (k) = D (k) B (k) H = D (k) M (k)after ICA and sorting by ψ.Proposition 2. Given access to K ≥ 2 environments, assume that A holds. Then, Z i is identified as a latent non-shifted node between environments k and k ′ if and only ifM (k) i = M (k ′ ) i , where M (k) i represents the i-th row of M (k) , and M (k) = B (k) H.All formal proofs are given in Appendix E. Our next result shows the identifiability of shifted nodes in the unscrambled matrix M (k) . Theorem 3. Z i is identified as a non-shifted node if and only ifM (k) i = M (k ′ ) i or M (k) i = -M (k ′ ) i .We can summarize this in the following algorithm, which proves Theorem 2:• Perform ICA to obtain M (k) and ϵ (k) with input X (k) .• Sort by ψ to get the permutation P (k) and computeM (k) = P (k) M (k) and ϵ (k) = P (k) ϵ (k) . • Check the condition on { M (k) i : k ∈ [K]} to detect if Z i is a shifted node, as prescribed by Theorem 3.Thus far, we have considered the population setting where we are given the entire interventional distributions. In practice, we are given samples from each of these interventional distributions, so that we have K datasets, one for each of the interventional distributions. The overall algorithm is given next in Alg. 1 (see illustration in Appendix B) with detailed explanations following the algorithm.Algorithm 1 iLCS: Identifying Latent Causal Mechanisms Shifts Require: Datasets {X (k) } K k=1 and threshold α (e.g., 0.5) Calculate covariance matrix Σ (k) from X (k) for all k d = max k=1,...,K rank(Σ (k) ) for k = 1, . . . , K do //Step 1: ϵ (k) is samples from ϵ (k) ϵ (k) , M (k) ← ICA(X (k) , d) Calculate ψ(ϵ (k) ) = [ ψ(ϵ (k) 1 ), ψ(ϵ (k)2 ), . . . , ψ(ϵ(k) d )] //Step 2 sorted_idx ← argsort( ψ(ϵ (k) )) M (k) ← M (k) [sorted_idx, :] Initialize S (k,k ′ ) = ∅, for all k ̸ = k ′ for i = 1, . . . , d do for k ̸ = k ′ do Calculate L k,k ′ i // Step 3 if L k,k ′ i > α then S (k,k ′ ) ← S (k,k ′ ) ∪ {i} Ensure: All latent shifted nodes S = (S (k,k ′ ) ) k,k ′Step 1: We perform ICA with samples from X (k) to extract M (k) and samples from ϵ (k) . Remark 4 (Estimation of d.). One missing component in using ICA in practice is that, along with samples from X (k) , we need to input the number of latent nodes d, which need to be estimated from samples. Define Σ(k) = E[X (k) X (k) T ] = G(B (k) ) -1 (B (k) ) -T G T .Since all matrices are full rank, it follows that d = rank(Σ (k) ), where Σ (k) can be estimated by the sample covariance matrix. Thus, d can also be estimated by the rank of the sample covariance matrix.Step 2: We compute the empirical expectation of ψ on samples from ϵ (k) , which by law of large number arguments, converges to its population expectation, which is ψ(ϵ (k) ). We use the sorted order of the empirical expectations to sort the noise components, unscrambling the noise components as earlier, to get M (k) and samples from ϵ (k) .Step 3: Here, we explicitly construct a test statistic to check the condition on { M(k) i : k ∈ [K]} to detect if Z iis a shifted node. Note that from our Theorem 3, there is a non-shift node Z i between environments k and k ′ if and only ifM (k) i = ± M (k ′ ) i. Accordingly, we define a test statistic:L k,k ′ i = min{∥ M (k) i ± M (k ′ ) i ∥ 1 ∥ M (k) i ∥ 1 + ∥ M (k ′ ) i ∥ 1 It can be seen that L k,k ′ i = 0 if and only if M (k) i = ± M (k ′ ) i, which implies node Z i is not shifted between environments k and k ′ . Thus, in step three of the algorithm above, for each coordinatei ∈ [d], we check if there exists k ̸ = k ′ such that L k,k ′ i > α for a given threshold α. If such a k ̸ = k ′ exists, we include i in the list of shifted nodes.Algorithm 1 is consistent with the ground truth set of shifted nodes as n approaches infinity. Empirical evidence supporting this claim is presented in Figure 2 , which shows that with a sufficiently large sample size, all shifted nodes are correctly identified, and the F1 score reaches 1. Further theoretical discussion on the sample complexity of our method can be found in Appendix D.  
experiments: 
 In this section, we investigate the performance of our method in synthetic and real-world data.In our setup, each noise component ϵ i is sampled from a generalized normal distribution with the probability density function given by p(ϵ i ) ∝ exp{-|ϵ i | i }, where i = 1, 2, . . . , d. In this noise generation process, the noise vector ϵ adheres to the condition ψ(ϵ i ) < ψ(ϵ j ) for all i < j if we choose ψ(y) = P(|y| ≤ 1). Following the methodology similar to that in [40] , we start by sampling either an Erdős-Rényi (ER) or Scale-Free (SF) graph with d nodes and an expected edge count of md, where m ∈ {2, 4, 6}, denoted as ERm or SF m. The observed space dimension p is set to 2d. For each graph, the weights are independently sampled from Unif ± [0.25, 1] and the diagonal entries of Ω from Unif [2, 4] . In each environment k, 15% of the nodes are randomly selected for shifting. The new weights A (k) ifor the shifted node i, and the new entries of Ω (k) , specificallyΩ (k)ii , are independently sampled from Unif [6, 8] . The mixing function G is independently generated from Unif[-0.25, 0.25].Empirically, we have observed that the following formulation of L k,k ′ i leads to improved results: i . We will utilized the new formula of L k,k ′ i to detect shifts in the following experiment. Then we explore sample sizes n from 500 to 10 6 , using the observed samples X (k) as input. The parameter α is set to 0.2 for d ≤ 10 and 0.5 for higher dimensions, reflecting the increased complexity in estimating larger dimensional latent graphs and thus necessitating a higher tolerance for L 1 norm differences in detecting shifted nodes. For each setting, we independently generate 10 datasets and take the average of the metrics. The results for n = 10 6 are shown in Table 1 , and the asymptotic consistency results for specific p values are illustrated in Figure 2 . In addition to the causal representative setting, our method can also directly identify mechanism shifts in a fully observed setting, where G = I. We further compare our method's results in this fully observed setting against the baseline DCI [52] , which addresses direct mechanism shifts in linear settings. The results of this comparison are provided in Appendix F, demonstrating that our method outperforms DCI in most settings.L k,k ′ i = ∥| M (k) i | -| M (k ′ ) i |∥ 1 ∥ M (k) i ∥ 1 + ∥ M (k ′ ) i ∥ 1 ,We evaluate our method using a dataset related to the Five Factor Model, also known as the Big Five personality traits [16, 15, 32] . This model is a widely accepted framework, comprising five broad dimensions that encapsulate the diversity of human personality traits. These dimensions are Openness to Experience, Conscientiousness, Extraversion, Agreeableness, and Neuroticism.The dataset utilized in our study was gathered through an interactive online personality test available on OpenPsychometrics.org, a nonprofit endeavor aimed at educating the public about psychology while collecting data for psychological research 1 . This dataset encompasses responses to 50 questions, with 10 questions dedicated to each of the five personality dimensions. Participants responded to each question on a scale from 1 to 5. Additionally, the dataset includes demographic information, such as race, age, gender, and country, comprising a total of 19,719 observations. Question formalization and data processing. In this study, we hypothesize the existence of 5 latent nodes, each representing one of the five personality dimensions, believed to be causally related. The score responses to the 50 questions form our observed space. Our main goal is to determine whether variations in personality dimensions can be observed across genders, thus treating gender as one environment (K = 2). Additionally, we investigate potential personality shifts across countries, selecting the US and UK for analysis due to they have the most observations in our dataset. The only preprocessing step undertaken involves the removal of observations with missing values and the normalization of data to fit within the [0, 1] range, achieved by adjusting according to the maximum and minimum values observed. The research question we have formalized in this study is not derived from any data competition. It aligns with interests explored in existing psychological literature [25, 8, 46, 29] , yet our investigation is distinguished by a unique analytical framework.Labeling latent nodes. Prior to detecting shifted nodes, it is essential to assign semantics to each node. This process involves conducting interventions on each component of the noise vector to aid in labeling the latent nodes. Given that the noise components are distinct for each latent node, labeling the noise effectively equates to labeling the latent nodes.Initially, we apply ICA to the data for males, followed by getting post-sorting M male and ϵ male as outlined in our methodology. Subsequently, we perform interventions on each noise component, setting each to 0 sequentially, and then re-mixing the intervened noise vector using ( M male ) † . By examining the impact of these interventions on the observation space -specifically, identifying which question scores undergo significant changes -we can assign appropriate semantic labels to each latent node index. For instance, nullifying the first column of ϵ and remixing the intervened noise with ( M male ) † alters the score distribution in a manner that reveals the semantic domain affected by the first noise component. An example of assigning the label Agreeableness to a latent node is depicted in Figure 3 . By applying the same process to all noise components, we are able to assign semantic labels Openness, Conscientiousness, Extraversion, and Neuroticism to the remaining latent nodes. More detailed experiment results are shown in Section G. Shifted nodes detection. To identify shifted personality dimensions across gender, we computed L male,f emale i for each latent node, obtaining values of {0.074, 0.0497, 0.078, 0.638, 0.633}. Setting a tolerance threshold α = 0.5 to accommodate real data estimation variances, we observed that the last two nodes exhibit significantly higher L male,f emale i scores, surpassing α, and thus are considered shifted. These nodes correspond to the labels Neuroticism and Extraversion. Consistent with existing psychological literature, women have been found to score higher in Neuroticism than men [25, 8, 46, 29] , while men scored higher in the Activity subcomponent of Extraversion [8] . This discovery aligns with the findings in psychology literature. To further validate our method's effectiveness, a similar analysis was conducted across countries, comparing the UK and the US, which have the most observations in our dataset. The computed L U S,U K i for each latent node was {0.302, 0.258, 0.109, 0.189, 0.088}. All values fell below α, indicating no latent node shifts between these two countries. This finding is also in agreement with existing studies that personality exhibits stability across countries and cultures [25, 24, 11] . 
concluding remarks: 
 In this study, we demonstrated that latent mechanism shifts are identifiable, up to a permutation, within the framework of linear latent causal structures and linear mixing functions. Furthermore, we introduced an algorithm, grounded in ICA, designed to detect these shifts. Our method offers a broader applicability to various types of interventions compared to CRL framework. Unlike shift detection methods where node variables are directly observable, our approach extends to scenarios where latent variables remain unobserved. A promising future direction consists of adapting our methodology to nonlinear transformations, which could address more complex, practical challenges, such as identifying latent mechanism shifts in real-world image data.Limitations of this work include the need to relax the noise assumption and to consider similar settings under nonlinear mixing functions. These are promising directions to explore in the CRL field. The broader impact of this work is that CRL methods can be used to identify mechanism shifts and determine root causes, which can be utilized in the biological field to find disease genes or biomarkers. Currently, the negative impacts of this method are not clear.X (1) ∈ ℝ n 1 ×p X (2) ∈ ℝ n 2 ×p ICA ϵ (1) ∈ ℝ n 1 ×d ϵ (2) ∈ ℝ n 2 ×dM (1) ∈ ℝ d×p M (2) ∈ ℝ d×p ψ(ϵ (1) ) ψ(ϵ (2) ) Rearrange Sort ψ(ϵ (1) ) ψ(ϵ (2) ) ICA M (1) M (2) Sort Compare Rearrange Figure 4 : Overview of our method: For each context k, given the data X (k) , our method involves three main steps. First, we apply ICA to each dataset to estimate ϵ (k) and M (k) . Second, we calculate ψ(ϵ (k) ) = {ψ(ϵ(k) 1 ), ψ(ϵ (k)2 ), . . . , ψ(ϵ (k) d )} for each noise component, sort these components in increasing order, and correspondingly arrange the rows of M (k) . Third, we compare the sorted rows of M (k) to identify the shifted nodes.In Assumption B, we assume that there exists a test function ψ and that we can access it. Here we discuss ways to relax it. Recall that in Section 4.1, ψ is utilized to sort the noise component ϵ (k) to ensure that the post-sorting noise vector ϵ (k) has a consistent order across all environments.An alternative approach to achieve this is to use distribution matching. We take the noise vector in the first environment as a reference and align all other noise vectors post-sorting with the reference vector. To do this, we can use a distribution distance metric D. First, define a signed permutation space asS d = {S = P D | P is a permutation matrix, D is a diagonal matrix with D ii ∈ {-1, 1}}Then, solve the optimization problem:min S∈S d D(ϵ (1) , Sϵ (k) )where D can be any distribution distance, such as Kullback-Leibler divergence. In Assumption A, we assume pair-wise different noise component, thus the optimization questions have minimums value 0 if and only if each noise component of ϵ (1) and Sϵ (k) have the same distribution, thus help us align the noise component order. We solve this optimization problem for each environment k ≥ 2, thus obtaining P (k) . All following steps in our algorithm remain the same when using this alternative approach.One small gap remains: even though all post-sorting noise vectors have a consistent order with ϵ (1) , ϵ (1) is not the ground truth order of ϵ (1) . This ambiguity cannot be eliminated, consistent with the nature of the CRL method, and is the same with other CRL methods, such as [40, 28] . Fortunately, the ground truth order is not so important in practice. What people mainly care about is the semantic label for each latent node. Some CRL generative models, such as [54] , may be helpful for performing fake interventions and manually assigning semantic labels. However, this is beyond the scope of this paper, and we will not discuss it further.Even though the distribution matching optimization method offers greater flexibility, it is computationally expensive. First, note that the cardinality|S d | = d! × 2 d ,which represents a vast search space when d is large. Furthermore, calculating D(•, •) is generally computationally intensive. For example, the KL method requires density estimation, and the Maximum Mean Discrepancy (MMD) method necessitates the computation of pairwise distances among samples. These challenges render this alternative difficult to implement. Consequently, we opt to use the ψ function to facilitate efficient sorting, but it may need carefully design.The sample complexity of our method must be considered from two perspectives: one involves using ICA to estimate ε(k) and M (k) , and the other pertains to utilizing ε(k) and a test function to sort the rows of M (k) . Since the sorting step depends on the choice of test function, we assume for simplicity that M (k) is already sorted by the ground truth order. Thus, we only focus on the asymptotic behavior of M (k) , which closely relates to the properties of the ICA estimator.There are various algorithms for solving ICA [18, 17, 41] ; each algorithm exhibits different asymptotic statistical properties. If we apply the findings in Auddy and Yuan [2] , we assume that the estimated ICA unmixing function has the following statistical accuracy: Theorem 4. If the sample size n ≥ g(d, δ), then with probability at least 1 -h(n, d, δ, ϵ), we have:l( M (k) i -M (k) i ) ≤ C • p(d, n)f (δ), where M (k) irepresents the i-th row of the estimated unmixing function M (k) , C is a constant, and p, f , g, and h are known functions. For instance, in Auddy and Yuan [2] , p(d, n) = d/n and f (δ) = log(1/δ). Here, l denotes the loss function, and the L 2 norm can serve as an option.Under this theorem, for two environments k and k ′ , if node i does not shift, we have:|| M k i -M k ′ i || 2 ≤ || M k i -M i || 2 + || M k ′ i -M i || 2 ≤ 2 • C • p(d, n)f (δ)with a probability of at least 1 -2h(n, d, δ, ϵ). Thus, by setting the threshold α as 2 • C • p(d, n)f (δ), we can control the false discovery rate to be at most 2h(n, d, δ, ϵ). A similar sample complexity theorem can be extended to cases involving more than two environments, as long as the statistical properties of the ICA solution are known. Proof. Given that G possesses full column rank, it follows that H = G † has full row rank. Consequently, the null space of H T is {0}. Therefore, if x T H = y T H, it implies H T (x -y) = 0. This leads to the conclusion that x -y = 0, which in turn implies x = y.B (k) = (Ω k ) -1 2 (I d -A (k) ). Since A (k)is a weighted adjacency matrix, its diagonal entries are zero. Thus,B (k) ij = -Ω (k) ii -1 2 A (k) ij if i ̸ = j, B (k) ii = Ω (k) ii -1 2 if i = j. Under Definition 1, if node Z i is shifted, it implies either 1) Ω (k) ii ̸ = Ω (k ′ ) ii , 2) A (k) i ̸ = A (k ′ ) i, or 3) both conditions hold. In scenarios 1) and 3),B (k) ii ̸ = B (k ′ ) ii , resulting in B (k) i ̸ = B (k ′ ) i . In scenario 2), while Ω (k) ii = Ω (k ′ ) ii , there exists a j ∈ [d] such that A (k) ij ̸ = A (k ′ ) ij , leading to B (k) i ̸ = B (k ′ ) i . If node Z i is not shifted, then A (k) i = A (k ′ ) i and Ω (k) ii = Ω (k ′ ) ii , implying B (k) i = B (k ′ ) i . Therefore, Z i is shifted if and only if B (k) i ̸ = B (k ′ ) i . According to Lemma 1, B (k) i ̸ = B (k ′ ) i if and only if B (k) i H ̸ = B (k ′ ) i H, which is equivalent to M (k) i ̸ = M (k ′ ) i . In conclusion, Z i is shifted if and only if M (k) i ̸ = M (k ′ ) i .Lemma 2. Under problem setting, it is not possible for an intervention on the latent nodeZ i to result in M (k) i = -M (k ′ ) i .Proof. We prove this by contradiction. Suppose thatM (k) i = -M (k ′ ) i . According to Lemma 1, this would imply B (k) i = -B (k ′ ) i. However, we knowB (k) = (Ω (k) ) -1 (I d -A (k) ) where A (k) is the weight matrix for a DAG. Since A (k) ii = 0, it follows that B (k) ii = (Ω (k) ) -1 ii . Therefore, both B (k) ii and B (k ′ ) ii are positive. It is impossible for B (k) i to be equal to -B (k ′ ) i. Consequently, the scenario whereM (k) i = -M (k ′ ) i cannot occur.Proof of Theorem 3. Recall from the data generation process that k) . Without loss of generality, we assume that ϵ (k) is ordered increasingly with respect to ψ. Thus, post sorting with respect to ψ, we eliminate the ambiguity of P (k) , and we getM (k) X (k) = ϵ (k) . When input X (k) to ICA, we have M (k) = P (k) D (k) M (k) and ϵ (k) = P (k) D (k) ϵ (M (k) = D (k) M (k) and ϵ (k) = D (k) ϵ (k) .We are now ready to prove that Z i is not shifted if and only ifM (k) = ± M (k ′ ) . This immediately implies that if Z i is not shifted, then M (k) i = M (k ′ ) i , thus satisfying M (k) = ± M (k ′ ) . If M (k) = ± M (k ′ ) , there are two cases: M (k) i = M (k ′ ) i or M (k) i = -M (k ′ ) i . We prove in Lemma 2 that the scenario M (k) i = -M (k ′ ) i cannot exist. The only surviving situation is M (k) i = M (k ′ ) i , which indicates that Z i is not shifted.As described in Section 5.1, instead of generating the mixing function G from Unif[-0.25, 0.25], we set G = I, such that X = Z and Z can be directly observed. In this setup, finding general interventions in linear causal representations reduces to identifying general interventions in linear SEM, a setting for which the existing method DCI [52] is designed. Table 2 presents the performance comparison between our method and DCI under these conditions, demonstrating that our method outperforms DCI in most cases. This section provides detailed information on the procedures employed in analyzing the real dataset.Preprocessing The initial dataset comprised 19,719 observations, which can be downloaded from https://www.kaggle.com/datasets/lucasgreenwell/ ocean-five-factor-personality-test-responses/data.In the preprocessing phase, any observation with a missing value in any column was excluded, leaving a total of 19,710 observations for further analysis. Subsequently, we applied max-min value normalization to the scores of each question, ensuring that all scores were normalized to fall within the range [0, 1]. This normalization step is crucial for achieving uniformity in the data scale, thereby facilitating accurate analysis and comparison across the dataset.Labeling the Noise To derive meaningful psychological insights, it is crucial to assign semantic labels to all latent nodes. Given that the noise components are pairwise distinct and unique to the latent node Z i , we consider intervening on each noise component, then remixing and observing the changes in the observational space. This approach enables us to assign semantic labels to both the noise components and their corresponding latent nodes. We utilize observations from the male dataset as the reference context, which comprises 7,603 observations. Following the initial step of our method, we obtain the sorted M male and ϵ male . The mixing function G is derived from ( M male ) † .To identify the semantic label for the first component of ϵ, we set its corresponding noise vector component to 0, effectively nullifying the first component of ϵ male . This intervention yields an estimated noise matrix samples from ϵ male inv , denoted as ϵ male inv . The intervened reconstruction, X male inv = G( ϵ male intv ) T , and the original score distribution, X male = G( ϵ male ) T , allow us to compare question scores pre-and post-intervention. Figure 7 plots these distributions, revealing significant shifts for questions pertaining to the Agreeableness dimension, with minimal impact on other scores, thereby identifying the first noise component as Agreeableness. This process is replicated for the second through fifth columns of ϵ male , with results illustrated in Figures 9, 8, 5, and 6. Each plot demonstrates that interventions result in significant distribution changes for questions related to a single personality dimension, with negligible effects on others. Consequently, we label these noise components as Openness, Conscientiousness, Extraversion, and Neuroticism, respectively. These labels will be used for all the following analysis.We then applied our method to data from the male and female contexts. The calculated L male,f emale i values are {0.074, 0.0497, 0.078, 0.638, 0.633}. Based on these results, we identify shifts in the last two personality dimensions, specifically labeled as Extraversion and Neuroticism. Additionally, we conducted a comparative analysis of personality dimensions between the US and UK, which have 8,753 and 1,531 observations, respectively. The computed L U S,U K i values are {0.302, 0.258, 0.109, 0.189, 0.088}, indicating that no latent node is considered as having undergone shifts between these two countries. Answer Guidelines:Agreeableness Conscientiousness Openness• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: The paper has no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes]Justification: One dataset is generated by simulation, and the source of the real data has been explicitly mentioned.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.• If this information is not available online, the authors are encouraged to reach out to the asset's creators. 13. New Assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [Yes] Justification: The primary new asset of the paper is the code, and the code is published in https://github.com/TianyuCodings/iLCS. Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.Emails: tianyuchen@utexas.edu, kbello@cs.cmu.edu 38th Conference on Neural Information Processing Systems (NeurIPS2024).The data can be downloaded via the link: https://www.kaggle.com/datasets/ lucasgreenwell/ocean-five-factor-personality-test-responses/data 
 
 Appendix:  


Decision: This work considers linear causal representation learning and causal mechanism shifts. Reviewers praised the novelty of the work, and generally viewed it as a significant and innovative contribution to the field. I therefore recommend acceptance.

Reviews: review: paper_summary: paper_summary: This work studies the problem of detecting the mechanism shifts in a novel way by considering the latent nodes. The authors prove identifiability results based on assumptions softer from prior identifiability results for causal representation learning. Their method is based on ICA and is evaluated empirically on synthetic data and a real-world psychometric dataset.
strength_weakness: strength_weakness: This work has some possible weaknesses

**Significance of contribution** The proposed work solves a simpler problem than causal representation learning, which doesn't require learning the whole mixing matrix $B$, but rather only the distribution shifts. I am concerned that this simplification of the problem might make it easier solvable and I wonder whether other methods of CRL could be transformed easily so they could perform well on this task. 

**Experiments** Following my previous concern, I wonder why you did not compare against prior CRL methods or methods for causal mechanism shifts. Such a comparison would enhance the experimental results. Can you adapt the CRL methods or the causal mechanism shift techniques for observable variables to your setting?

**Sample complexity** From Figure 2 it seems that indeed for a large (towards infinity) number of samples, your method can accurately detect the mechanism shifts. I am concerned, however, that the sample complexity is high which makes nodes with over hundreds of nodes out of reach. This can also be seen from Table 1 where your performance drops quite early, even from graphs with 60 nodes. From a theoretical perspective, it would be interesting to compute how many samples are required regarding the number of nodes (sample complexity result - either theoretical or experimental study). Such a theorem would be appreciated by the community.

**Real experiment: size of changes** The results are very interesting and agree with psychological findings. However, the number of nodes and changes are very small. A larger number of nodes (up to 100) would be more sufficient to show that your algorithm is valid in practice. 

**Limitations** You should explain either why the application to a large number of nodes is not needed in practice or add it to the limitations.
questions: questions: Line 56: Do you think that the problem you solve is relevant to identifying the locations of the root causes of a linear SEM (as in [1])?
Figure 1 caption, 7th line: Typo, should be $Z_4\to Z_1$ 
Line 143: What are the implications of this assumption? Is it also present in prior identifiability results?
Line 231: What is the effect of the observed space dimension $p$ being larger or smaller with respect to the problem you solve (does it become easier or harder)?
Figure 3: the font in the legend and xticks must be larger.
Line 293-294: Here I got a bit confused. This is different from what you show in the example of Fig. 1, right? Because in Fig. 1 you show interventions (shifts) across different countries.
Line 297: Can you briefly explain how your methodology can be generalized to a nonlinear data-generating process?
Line 303: What would distribution shift imply for image data? 

[1] Misiakos, P., Wendler, C., & Püschel, M. (2024). Learning DAGs from data with few root causes. Advances in Neural Information Processing Systems, 36.
limitations: limitations: The authors have discussed some limitations in the appendix. However, a significant limitation of not being applicable to a large number of nodes is not included.
paper_summary: paper_summary: This paper considers the setting of linear causal representation learning (CRL) with possibly multi-node interventions. Instead of focusing on the task of identifying the causal structure, which is recently shown to be impossible, the authors instead focus on the task of identifying mechanism shifts i.e. which nodes are intervened in each environment. The authors show that this identification task is actually possible, and design an identification algorithm to achieve it. Lastly, the authors empirically demonstrate the effectiveness of their approach.
strength_weakness: strength_weakness: 1. It seems that step 3 in Sec. 4.2 is stated without any proof of why it works. Is it just a heuristic or that it can provably lead to identification?

2. It seems to me that the main results of this paper is very closely related to [1], but the authors do not discuss in detail about this issue. (Please see the "Question part" for more details)

[1] Jin, Jikai, and Vasilis Syrgkanis. "Learning causal representations from general environments: Identifiability and intrinsic ambiguity." arXiv preprint arXiv:2311.12267 (2023).
questions: questions: In the paper [1], the authors consider linear CRL (the same setting as this paper) and design an identification algorithm that works for general environments (i.e. multi-node interventions). Their algorithm can fully recover the causal graph, as well as recover the mixing matrix up to a surrounding node ambiguity (SNA). They show that SNA is am intrinsic barrier in this setting.

What I'm wondering is that, given their identification algorithm, if it is the case that the task of "identifying mechanism shift" can be straightforwardly resolved. Because if you can recover the mixing matrix, then you can also recover the noise-to-latent matrix (i.e. $B^{(k)}$) in the current paper). Then you can identify the mechanism shifts simply by comparing the entries of different $B^{(k)}$'s. Of course, the mixing matrix is actually recovered with some ambuguities. However, given that such ambiguities are inevitable as shown in [1], I suspect that such ambiguities do not affect the task of identifying mechanism shift.

*I am happy to raise my score if the above concern is appropriately addressed.*
limitations: limitations: N/A
paper_summary: paper_summary: This work studies the nontrivial problem of causal representation learning from the perspective of mechanism shifts within the latent SCM. Specifically, the authors relax existing restrictive assumptions in interventional causal representation learning, such as data generated from single-node perfect interventions and the number of environments necessary to facilitate identifiability and show that it is possible to identify the latent nodes that shift between environments/distributions from more general soft/hard and add/reverse interventions given access to fewer environments than the number of causal variables. Furthermore, the authors develop a practical algorithm to recover the latent sources attributable to the distribution shift and evaluate their method on synthetic data and a psychometric dataset. The main contribution in this work seems to be the use of a test function to score the noise factors learned from ICA to construct a sorted permutation matrix, which is used to construct the new unmixing matrix for scrambling the independent SCM noise variables.
strength_weakness: strength_weakness: - This work considers linearity in both the mixing and the SCM, which can be somewhat of a restricting assumption in practice.
- Proposition 2 and Theorem 3 seem to contradict each other. By the current logic, the two statements imply that both shifted and non-shifted nodes require the same condition of the row corresponding to the variable index in the unmixing matrix to be invariant across environments (w/ sign flip). This should also be made clear in Section 4.2 in Step 3.
- Minor points
    - In the caption of Figure 1, for the UK environment, it should be the edge Z_4 → Z_1 is removed instead of Z_5 → Z_1 removed.
questions: questions: - Proposition 2 and Theorem 3 are contradicting. What are the criteria for a node to be identified as a shifted node? Do you mean that if the ith row of the unmixing matrix is **different between environments**, then node i is a shifted node? This seems to be the case when looking at the proof of Proposition 2 in the appendix. I would appreciate it if the authors could provide some clarification on this.
- Do the authors have any intuition about the setting where noise factors are correlated and the iid assumption of noise across environments is violated? In this scenario, we would no longer have the standard linear ICA result to build off of.
limitations: limitations: Limitations are discussed in the appendix.
paper_summary: paper_summary: Recently, causal representation learning has drawn a lot of attention in the representation learning area, where it considers a causal relation among the latent generative factors. This work is under the setting of linear causal representation learning. Data $X$ comes from a linear mixing $X=GZ$ of the unknown latent factors $Z$, and latent factors $Z$ follow a linear SCM. They are trying to estimate the causal mechanism's shifting (causally changed) node, under the more general/relax intervention. They claim that ‘it is possible to identify the latent sources of distribution shifts while bypassing the estimation of the mixing function and the SCM (causal relation matrix). The experiments were provided to justify their setting.
strength_weakness: strength_weakness: Cons:
1. The first issue is the main motivation of this work. Their idea is to bypass the estimation of the mixing function $G$ and the SCM $B$ (causal relation matrix). However, the reviewer believes this is the main motivation for causal representation learning. Why would we want a shift node, if we can't even identify the causal relationship in CRL? Could the author please clarify the point here?

2. The reviewer believes the strong assumption limits the significance of this work. The `access to a test function (Assumption B)' is a really strong assumption that seems not realistic. Since you got the permutation ambiguity fixed. And I believe you also fixed the sign ambiguity in your assumption. Combined with the bounded variance you basically got an ICA with only scaling ambiguity. What is the difference between this work and the classical linear ICA result?

3. The reviewer would like to see the clarification of the motivation and connection between linear ICA and this work. For linear ICA, we can get mixing function $M$ and the independent sources $\epsilon$ in an unsupervised way, with an identifiability guarantee. But for causal representation cases, the $M$ decomposes into $G$ and $B$. The natural/intuitive idea is to identify those factors with certain assumptions. Could the author provide any intuition for this?

The reviewer would love to raise the score if these questions were resolved in the discussion session.
questions: questions: Besides the three questions I mentioned in the cons section, I do have another question here, but it does not lower the marks of this work, just discussion.
1. This question is not an issue for this work since it seems many reference works use interventional settings. But it would be interesting if the author could discuss this. Could the author clarify why we must use interventional data? Is there any alternative way since the interventional data is kind of a semi-supervised strong assumption? I know there is a paper about the 'Challenging Common Assumptions in unsupervised CRL', but is there any other way we can avoid using interventional data?
limitations: limitations: N/A

Comments: title: Response to the author
Response to the author
Thank you.
comment: The reviewer thanks the author for the clarification. While I'm not a big fan of the scope of identifying mechanism shifts compared to the classical task of identifying causal relation graphs in CRL, I'm raising my score to 5 considering the potential value in corresponding downstream applications in their setting.
Dear reviewer, thanks for your response. We are glad we were able to solve your concern about the assumption on the test function. Next we address your remaining questions.

**W1**

We are not sure why the reviewer is concerned about our setting not being aligned with CRL? Nowhere in our paper we claim that our goal is to solve the CRL problem. Our paper considers the CRL setting in the sense that our model of the data generating process follows that of the linear CRL setting, that is, the latent variables follow an structural causal model, but we explicitly mention the goal of our work in our contributions and the paragraph before (Lines 55-72). 

**W3**

Thank you for your question. Indeed, several papers discuss the strict lower bound on the number of interventions needed for causal structure recovery. For example, in [1], Theorem 2 demonstrates that with perfect interventions on each single node across different environments, the causal structure can be estimated up to a permutation. Moreover, Proposition 5 indicates that if the number of interventions is fewer than $d$ (where $d$ is the number of latent nodes), the causal structure becomes non-identifiable. Additionally, Appendix B of the same paper shows that if the interventions are soft rather than perfect, Theorem 2 may no longer hold. The most recent advancement in this area is found in [2], which relaxes the hard intervention assumption but still requires at least $d$ environments and $\Theta(d)$ soft interventions.\
These identifiability limitations in CRL w.r.t. the number of environments and the type of interventions are precisely what motivate the search for alternative goals that are still useful in practice, such as identifying mechanism shifts, all while bypassing full identification of the causal structures.


[1] Squires, Chandler, et al. "Linear causal disentanglement via interventions." International Conference on Machine Learning. PMLR, 2023.

[2] Jin, Jikai, and Vasilis Syrgkanis. "Learning causal representations from general environments: Identifiability and intrinsic ambiguity." arXiv preprint arXiv:2311.12267 (2023).
W1 The reviewer thanks the author for the clarification and agrees that it is of certain value for finding the shifted nodes. however, it does not fully resolve my question. The reviewer respectfully argues that the shift node setting is not aligned with the fundamental motivation of causal representation learning, which is finding the causal graph/mechanism.


W2 The author answered my question about the strong assumption of the test function. The reviewer was satisfied with their explanation about relaxation.

W3 For the motivation, the author referenced paper [1] is probably the closest linear model setting. It showed an interventional-based linear model. They showed the number of interventions needed for their method but that seems not a strict lower bound. The reviewer respectfully disagrees with the author's claim 'in the "inadequate" intervention data setting, estimating $G$ and $B$ from $M$ is impossible'. Please correct me if I was wrong, and I would also like to know if there is any reference proving the strict lower bound of intervention needed for identifiability.
Dear reviewer, thank you for participating during this discussion period, your questions are greatly appreciated. We next address them:

1. We would like to offer a different perspective on the contributions of our work. Our contributions are in explicitly *formulating, proving, and demonstrating through experiments* that one can effectively and efficiently solve the problem of identifying mechanism shifts for linear latent causal variables. Given the assumptions in the paper, we found that there is a simple yet elegant solution for this problem and we firmly consider this to be a strength, not a weakness. \
Regarding the comparison to [1], it's correct that [1] also uses ICA as a first step; however, the same can be said about [2], which also uses ICA as a first step and then a couple of extra steps to identify the causal order in the fully observable setting. In fact, any other application of ICA ranging from predicting stock market prices to working with EEG data would all share with our algorithm the same step of applying ICA. This is to highlight the importance of the problem formulation, which Reviewers tbD8 and 33zq kindly appreciated from our paper.

2. Regarding the real-world implications. We believe our problem setting is more realistic, in the sense that it could be applied more widely, partly because the objective is less ambitious than estimating the full causal graph and we require less assumptions. Indeed, one of the key motivations for **directly** learning differences of causal graphs given in [3,4,5] is that learning full causal graphs is in general impractical (mainly due to strong assumptions and being sample inefficient) and, in many cases, scientists are simply interested in understanding changes among populations/distributions. Note that the latter, i.e. comparing distributions, is a fundamental question in statistics, and our setting is concerned with identifying the *latent* sources of distribution changes, which is closely related to root cause analysis, as stated in Lines 55-60.\
To conclude, for example, in our real-world study in Section 5.2, the research question is whether there exists significant variations in personality traits (latent variables) across populations male/female and US/UK based on psychological tests (observed measurements). Here, our algorithm was able to answer consistently with existing psychological literature (Lines 265-266).


[2] Shimizu, S., et al. (2006). A linear non-Gaussian acyclic model for causal discovery. JMLR.

[3] Wang, Y., et al. (2018). Direct estimation of differences in causal graphs. NeurIPS.

[4] Chen, T., et al. (2024). iSCAN: identifying causal mechanism shifts among nonlinear additive noise models. NeurIPS.

[5] Malik, V., et al. (2024). "Identifying Causal Changes Between Linear Structural Equation Models." UAI.


We hope these comments are helpful, we appreciate your participation during this discussion period. Please let us know any follow-up questions or concerns.
Dear reviewer, thank you for taking the time to respond. We are happy to see our answers were helpful and made you feel more positive about our work. Thanks a lot for your efforts!
Dear reviewer, thank you for participating during this discussion period. We are happy to see our answers addressed your concerns and we are grateful for your support to our work. Once again, thank you.
I thank the authors for the clarifying response. The authors have done a good job of answering my questions. The problem addressed is quite interesting with significant implications in downstream distribution shift generalization. Furthermore, I believe the theoretical identifiability results and practical algorithm are of interest to the CRL community. Since this work helps to bridge the gap between the theory and practice of CRL, I am increasing my score.
I would like to thank the authors for the reply. If I understand it correctly, the authors' point is that the task of this paper is different (and strictly easier) than the one in [1], since the goal here is only to identify the shifted nodes rather than the full causal model. As a result, the assumptions required in this paper is also weaker than [1].

Given the above interpretation of the main contributions of this paper, I would say that the task this paper considers may be of interest on its own and the results are novel. However, I'm still concerned that the contributions of this paper is a bit too close to [1]. Because although [1] requires $d$ environments (or equivalently, $d$ interventions for each node). Indeed, after completing the step 1 in their algorithm, one can directly compare the corresponding rows of $M_k$ to determine whether the node is shifted or not. Although this is not explicitly done in that paper (because their task is causal graph discovery), this deduction seems too simple to be the main result of a NeurIPS paper. Actually the main ideas underlying their algorithms seem to be the same i.e., the $i$-th row of $M_k$ implicitly encodes any information of node $i$ that is invariant under linear transformations.

I'm also concerned about the real-world implications of the mechanism shift identification task. Indeed, without recovering the true latent nodes and the causal graph, it does not seem to be extremely useful to identify a shift at a certain node, because we have no other information of this node. How can we utilize the mechanism shift result to solve downstream tasks?
After your rebuttal, I am very positive that this is great work and I am accordingly increasing my score. 
Particularly I appreciate the novelty of your work, by combining two distinct problems (causal representation learning and causal mechanism shifts).