Identifying General Mechanism Shifts in Linear Causal Representations
Authors: Tianyu Chen
Kevin Bello
Francesco Locatello
Bryon Aragam
Pradeep Kumar Ravikumar
Venue: NeurIPS 2024 poster (2024)
Keywords: latent variable modeling
distribution shifts
causal representation learning
heterogeneous data
root cause analysis
Primary area: causal_inference

Abstract: We consider the linear causal representation learning setting where we observe a linear mixing of $d$ unknown latent factors, which follow a linear structural causal model. 
Recent work has shown that it is possible to recover the latent factors as well as the underlying structural causal model over them, up to permutation and scaling, provided that we have at least $d$ environments, each of which corresponds to perfect interventions on a single latent node (factor). 
After this powerful result, a key open problem faced by the community has been to relax these conditions: allow for coarser than perfect single-node interventions, and allow for fewer than $d$ of them, since the number of latent factors $d$ could be very large. 
In this work, we consider precisely such a setting, where we allow a smaller than $d$ number of environments, and also allow for very coarse interventions that can very coarsely \textit{change the entire causal graph over the latent factors}. 
On the flip side, we relax what we wish to extract to simply the \textit{list of nodes that have shifted between one or more environments}. 
We provide a surprising identifiability result that it is indeed possible, under some very mild standard assumptions, to identify the set of shifted nodes. 
Our identifiability proof moreover is a constructive one: we explicitly provide necessary and sufficient conditions for a node to be a shifted node, and show that we can check these conditions given observed data. 
Our algorithm lends itself very naturally to the sample setting where instead of just interventional distributions, we are provided datasets of samples from each of these distributions. 
We corroborate our results on both synthetic experiments as well as an interesting psychometric dataset. The code can be found at https://github.com/TianyuCodings/iLCS.

Identifying General Mechanism Shifts in Linear Causal Representations
Tianyu Chen" Kevin Bello Francesco Locatello Bryon Aragam' Pradeep Ravikumar!
*Department of Statistics and Data Sciences, University of Texas at Austin
Booth School of Business, University of Chicago Machine Learning Department, Carnegie Mellon University Institute of Science and Technology Austria
Abstract
We consider the linear causal representation learning setting where we observe a linear mixing of unknown latent factors, which follow a linear structural causal model. Recent work has shown that it is possible to recover the latent factors as well the underlying structural causal model over them, up to permutation and scaling, provided that we have at least d environments, each which corresponds to perfect interventions single latent node (factor). After this powerful result, key open problem faced by the community has been to relax these conditions: allow for coarser than perfect single-node interventions, and allow for fewer than d of them, since the number of latent factors could be very large. In this work, we consider precisely such setting, where allow smaller than d number of environments, and also allow for very coarse interventions that can very coarsely change the entire causal graph over the latent factors. On the flip side, we relax what we wish to extract to simply the list of nodes that have shifted between one or more environments. We provide surprising identifiability result that is indeed possible, under some very mild standard assumptions, to identify the set of shifted nodes. Our identifiability proof moreover is constructive one: we explicitly provide necessary and sufficient conditions for node to be a shifted node, and show that we can check these conditions given observed data. Our algorithm lends itself very naturally to the sample setting where instead of just interventional distributions, we are provided datasets of samples from each of these distributions. We corroborate our results on both synthetic experiments as well as an interesting psychometric dataset. The code can be found https: //tpthubbbtt///gg/gg tiu......... com/ TianyuCodings iLCS.
Introduction
The objective of learning disentangled representations to separate the different factors that contribute the variation in the observed data, resulting in representation that is easier to understand and manipulate [3]. Traditional methods for disentanglement [e.g.. 19, 20, 7, 9, 26] aim to make the latent variables independent of each other.
Consider the setting of linear independent component analysis (ICA) [19], that is, the observed variables X E Rº are generated through the process X GZ, where Z Ed are latent factors, and G E gexd is an unknown "mixing" matrix. Under the key assumption that Z has statistically independent components, and under some additional mild assumptions, landmark results in linear ICA show that it is possible to recover the latent variables Z up to permutation and scaling [13, 19].
Emails: tianyuchen eutexas.edu. kbello@es.cmm
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

However, what if instead of independent sources Z we have structural causal model (SCM, [37, 38]) over them? For instance, if the latent factors correspond to biomarkers causes in a root cause analysis context, then we expect there to be rich associations between them. Indeed, this question is central the burgeoning field of causal representation learning (CRL.) [39, 51], where we are interested in extracting the latent factors and causal associations between them given raw data.
biology context, or root
Let us look at the simplest CRL setting where the latent variables Z follow a linear SCM, that is, 2 = AZ + 22/ where A E Rdxd encodes a directed acyclic graph (DAG), Q is a diagonal matrix that controls the scale of noise variances, and is some noise vector with zero-mean and unit-variance independent components. In such case, Z is a linear mixing of independent components c, that is, Z = B--- E, where B = Ω 1/2 (Ia - A) succinctly encodes the SCM and Ia is the identity matrix of dimension RRdxd, We then have = GB- so that ICA can only recover BG up to permutation and scaling, which does not suffice recover the SCM B since the mixing function G is unknown.
Recently, Seigal et al. [40] showed that given the interventional distributions arising from perfect interventions on each latent variable in Z, we can recover the SCM over Z up to permutation. But there are two caveats to this: (a) is difficult obtain perfect single-node interventions that only intervene on single factor in Z; and (b) it is difficult to obtain d number of such perfect interventional distributions or environments.
We are interested in the setting where we do not have perfect interventions: we allow for far more general interventions that can quite coarsely change the SCM, namely, soft and hard interventions, interventions targeting single or multiple nodes, as well as interventions capable of adding or removing parent nodes and reversing edges. Moreover, we do not need as many as d of these.
Our goal, however, is not to recover the entire SCM over Z but simply to recover those nodes Z that have incurred shifts or changes between the different interventional distributions. This is closely related to root cause analysis [5, 6, 21, 33], which aims to identify the origins of the observed changes in joint distribution. In addition, understanding the sources of distribution shifts-thar is, localizing invariant/shifted conditional distributions can benefit downstream tasks such as domain adaptation [30], and domain generalization [36, 55].
Contributions. Our work sits at the intersection of linear CRL [40, 23] and direct estimation of causal mechanism shifts [52, 14]. The key contribution of this work is to show that it is possible to identify the latent sources of distribution shifts in multiple datasets while bypassing the estimation of the mixing function G and the SCM B over the latent variables, under very general types of interventions. More concretely, we make the following set of contributions:
1. Identifiability: We show that we can identify the shifted latent factors even under more general types of interventions. (Section 4.1).
2. Algorithm:
We also provide an scalable algorithm that implements our identifiability result to infer such shifted latent factors even in the practical scenarios where we are not given the entire coarse interventional distributions but merely finite samples from each (Section 4.2).
3.
Experiments: We corroborate our results on both synthetic experiments (Section 5.1) as well an interesting psychometric dataset (Section 5.2).
2 Related Work
Causal representation learning. In contrast to our setting, which focuses on identifying shifted nodes in the latent representation, existing methods in CRL aim to recover both the latent causal graph and the mixing function. Previous works have studied identifiability in various settings, such as latent linear SEMs with linear mixing [40], and with nonlinear mixing [4]; latent nonlinear SEMs with finite degree polynomial mixing [1], and with linear mixing [48]; and nonlinear SEMs with nonlinear mixing [50, 49, 23, 22]. Although these studies ensure the identifiability of causal graphs (up to permutation and scaling ambiguities), they generally rely on the assumption that each latent vuriable is intervened upon in at least one environment, necessitating access to at least d interventional distributions. Moreover, the aforementioned works assume specific types of interventions, such as hard/soft interventions and single-node interventions, and restrict changes in interventional distributions, disallowing edge reversals or the addition of new edges. The most
2

---

recent work [23] enables causal representation learning under general interventions in latent linear SEMs with linear mixing. However, this approach still requires the assumption that the number of environments K is at least equal to the number of latent nodes d and that there are at least (dd)) interventions. If the objective to detect variables with general mechanism changes across multiple enniineett-eevviommentt that may lack consistent topological order and sufficient interventions or environments- overly restrictive or even infeasible. In contrast, we present more flexible approach, enabling the identification of shifted variables without assuming restrictive interventions per environment or a consistent topological order of the latent graphs.
existing CRL methods to recover each latent graph becomes
Direct estimation of mechanism shifts. The problem of directly estimating causal mechanism changes without estimating the causal graphs has also been explored in various settings in the regime in which the causal variables are observable. Wang et al. [52] and Ghoshal et al. [14] have focused on identifying structural differences, assuming linear SEMs as environments, and proposing methods that take advantage of variations in the precision matrices. More recently, Chen et al. [10] studied this problem for nonlinear additive noise models, assuming that the environments originate from soft/hard interventions and leverage recent work in causal discovery via score matching. Finally, the concept of detecting/loccllzing feature shifts between two distributions has also been discussed in [27], although from non-causal perspective. To our knowledge, there is gap in the literature regarding the study of these objectives when considering latent causal variables. We address this gap by proposing novel approach for directly detecting mechanism shifts within the latent SCMs.
Independent component analysis. The application of independent component analysis (ICA) [12] in the realm of causal discovery has seen significant developments. Linear ICA [19] and its nonlinear counterpart [20] have been instrumental in causal discovery [35, 44, 53] and more recently in causal latent discovery [23]. Beyond these established applications, our work uncovers a novel use of ICA, namely, identifying shifted nodes within the latent linear SCMs.
Given the relevance of ICA for our approach, we briefly recap next. ICA considers the following setting: X = We where X E Re, E IR,,, y 2 d. A key assumption in ICA is that each component of is independent. Given only observations of X, the goal of ICA is to estimate both W and e. The objective function typically aims maximize negentropy non- non-Gaussianity with further details given in [19]. The identifiabiltty results of ICA can be summarized as follows.
Theorem 1 (Theorems 3,4 in [13]. eveeyy component of e independent and at most one component is Gaussian distributed, with W being full column rank, then ICA can estimate W up to permutation and scaling of each column, and e can be recovered for some permutation up to scaling for each
component. Furthermore, noted in [19]. if B(T] = 1, Vi E [d], the estimated W and c will have X Wc
ambiguities only in permutation and sign. Formally, this means
where is permutation matrix and D is a diagonal matrix with diagonal entries 1 Then, the best estimate given by ICA is W PT D and DPe.
3
Problem Setting
Consider random vector X in Rp that is linear mixing of latent variables Z = (Zı,...; Za):
x= GZ.
Here the latent variables in Z follows a linear SCM [37, 38], that is,
Z = AZ + 2/
where A E Rdd corresponds to DAG g such that Aςk + 0 iff there exists an edge + k in the DAG 9; Ω gaxd is diagonal matrix with positive entries, and E Id is random vector with independent components with mean zero and variance one, i.c., that Cov(e) = Ia. Denoting B = Q (1 - A), we have that:
= B E.
3

---

United States of America (atlinz - 12,,, 2, 2,1i2,, 12, 22P22,
United Kingdom
Australia
pi (2) Piz i2 P.ZZZ (.
(2)) PIG I2, Z. 1Z
Figure 1: We have 5 latent variables Z which this case relate to personality concepts, and the observations X represent the scores of 50 questions from psychometric personality test. The latent variables Z follow a linear SCM, while the unknown shared linear mixing is a full-rank matrix G E R50x5_ Then, for environment k {US, UK, AU). the observables are generated through (())- GZ Here, IPUUS) is taken as the "observational" (reference) distribution, and the distribution shifts in P(UK) and p(AU) are due to changes in the causal mechanisms of { [Z,] and {Zz, Zz. Zs}. respectively. Finally, the types of interventions are general; for UK, the edge Z4 - Zi is removed and the dashed red lines indicate changes in the edge weights to Z1: for AU, Za was intervened by removing Z5 + Z2 and adding Za- + Z2, while the edge Zs + Z3 was reversed, thus changing the mechanisms of Z3 and Z5. Thus, we aim to identify (21) and { (Z2, Z3, Zs).
We assume that we observe K 2 2 generalized interventional distributions that keep the mixing map G fixed but allow for generalized interventions to Z. That is, for environment & E [K] we have,
(k) = (Z))),
where Z(0) A(4) + (*) Similarly, we have 7(k) (B())))'e(() where ((k) (D2()) (Vaaa
Notably, we allow generalized interventions that allow for ( to be arbitrary, which includes soft and hard interventions, interventions targeting single or multiple nodes, as well as interventions capable of adding or removing parent nodes and reversing edges. This contrasts with the existing literature on CRL, where single-node soft/hard interventions are the standard assumption [50, 40, 4, I]. See Figure 1, for toy example of what aim to estimate.
Remark 1. Since we allow for general types of interventions, we can take any of the given environ- ments as the canonical "observational" distribution with respect to which we observe interventions, or simply that we observe l interventions of an unknown observational distribution. This is a clear distinction from the standard setting in CRL [1, 50, 48, 23] which requires to know which environment is suitable observational distribution.
To develop our identifiability result and algorithm, we will make additional assumptions on the noise distributions of the linear SEMs.
Assumption A (Noise Assumptions). For any environment k E [K], let (x) = (c1 ..... .'d (a)) be the vector of d independent noises with ())( Ia- We have:
1. Identically distributed across environments: P(((k)) = PP(()))), for all k' $ k.
2. Non-Gaussianity: At most one noise component & (k) is Gaussian distributed.
PC- 3. Pairwise differences: For any + j. we have (c(( + P((5))) and PP(C)))))) (K0)
Assumption A. is usually assumed for learning causal models from multiple environments [31, 4]. Assumption A.2 is typically made in causal discovery methods, as detailed seminal works such as [43, 42, 19, 45] and is considered a more realistic assumption [34]. Assumption A.3 is generally satisfied generic sense; that is, when probability distributions on the real line are randomly selected, they are pairwise different with probability one. This assumption is also adopted in [47, 23].
Assumption B (Test Function). We assume access to a test function &b that maps each noise E.v. to RR s.t. (b((() (k) v(-e,a)) (k) and (e( (k) # v(e5)) ) (k)
(k) (b)) if ife and E,
not identically distributed.

---

This assumption states that we can access a test function that can help differentiate the noise components. One coarse example is (by) = p S 1). This assumption introduced better understand our method workflow in Section 4, but it is not completely necessary. We discuss how to relax this assumption in Appendix C. Next, we formally define a mechanism shift.
Definition 1 (Latent Mechanism Shifts), Let PA(Z, denote the set of parents of 2,4h A latent variable Z is called latent shifted node within environments k. and k', if and only if:
P(Z (k) (A((Z()))) & (P(2 [PA[Z, (P))
Remark Following Definition Ze is latent shifted node between environments and kf if: (1) The i-th rows of ( and Alk are different; (2) (2) s2. * _2!²; ; or (3) both.
Definition aligns with those previously discussed in [52, 14, 10), with the key difference that we consider changes in the causal mechanisms of the latent causal variables. However, note that our results also contribute to the setting in which causal variables are observable considering that the mixing function is the identity matrix, that is, G Id-
Identifying Shifts in Latent Causal Mechanisms
Following the setup outlined in the previous section, our focus now turns to developing an algorithm identify latent shifted nodes, given data from multiple environments. First, note that we can write the overall model as linear ICA problem, where, for any environment k, the observation ( is linear combination of independent components (k), Specifically, we have
( GZ G(B(())
Under the mild conditions given in Assumption A, from classical ICA identifiability results stated in Theorem 1, we can identify ((B(k))-2 up to permutation and sign flip. Let ( B(k) H where H G Then, we can only identify A/ up to permutation and sign flip, which does not suffice to identify the latent SCM encoded in ((k), In sum, what we can only obtain from ICA
M p(k) (R)) ((k)
where p(k) is permutation matrix, and (*)) is a diagonal matrix with - or +1 on its diagonal. As Seigal et al. [40] points out, it is not possible to identify (kk) further given generalized interventions. Our first result is that our present mild assumptions suffice to infer shifted nodes.
Theorem (Identifiability). Given access to K 2 2 environments, assume that A and B hold for all environments. Then, all latent shifted nodes are identifiable.
An interesting facet of our identifiability result is that it is constructive. In the next subsection will provide an explicit algorithm to infer the shifted nodes and prove the main theorem above.
4.1 Constructive identifiability
Consider c(k) = (x((() and g(k) = x(() = p(k) D(b) (xx = p()) (())
where g(k) and (k( are the output of ICA, which contain the permutation and sign flip ambiguities
given by p(k) (N))
Obtaining a consistent ordering the noise components across all environments is equivalent to finding P())_ Under Assumption B, and without loss of generality, we consider that (s)
(k)
are in increasing order with respect their & values. Since is invariant to sign flip, we can calculate (E( for all i E [d] and sort the calculated zi values in increasing order. Let ((k) denote the sorting permutation with respect to 1,, so that post-sorting, we get (kk)
Remark 3. In Appendix
discuss how relax the assumption on the test function th.
Proposition p(k) (P(K)) ie.. p(k) is the inverse permutation of the ICA scrambling.
From Proposition we thus find that we can unseramble the permutation p(k) by sorting with respect v.. We get ((k) pk p(k) (R)) ( from the above proposition. In other words, we can extract z(k) (K) via (K) p(k) (*) (1)) B(k) H (kk) (k) after ICA and sorting by v.
5

---

Proposition 2. Given access to K 2 2 environments, assume that A holds. Then, Z₁ is identified as a latent non-shifted node between environments i and k if and only if M,()) M, where M( represents the i-th row of M(),, and M (B() H.
All formal proofs are given in Appendix E. Our next result shows the identifiability of shifted nodes in the unscrambled matrix M (K)
Theorem 3. Z; identified as a non-shifted node if and only if M M, or Tr!h) MM,
We can summarize this in the following algorithm, which proves Theorem 2:
Perform ICA to obtain ( and ε(k) with input x(*),
Sort by & to get the permutation p(k) and compute M = P(k) M and (1 = p()))(k)
Check the condition on (M, k E [K]} to detect if Z; is a shifted node, as prescribed by Theorem 3.
4.2 Finite-sample algorithm
Thus far, we have considered the population setting where we are given the entire interventional distributions. In practice, we are given samples from each of these interventional distributions, so that we have K datasets, one for each of the interventional distributions. The overall algorithm given next in Alg. (see illustration in Appendix B) with detailed explanations following the algorithm.
Algorithm iLCS: Identifying Latent Causal Mechanisms Shifts
Require: Datasets {x
( and threshold a (e.g.. 0.5) ===1
Calculate covariance matrix (() from (k) for all k
max (a))( A k=I...,K
for A 1 K do
//Step 1: ((k) is samples from z(k)
<(k), .T((k) +- (C) d)
Calculate ())(()) (5(e) (K) ) (5 (2)
)..., (() (M)
//Step
sorted idx + argsort($ (c))))
(1 + M [ idx, :]
Initialize (kk,k') 0, for all is # k
for 1, l.... do
for 1 + W do
Calculate L. k.K // Step 3
L. kkk' > a then S(k,') ((k,k)) U {i} Ensure: All latent shifted nodes S (S(K,) )K
Step 1: We perform ICA with samples from (k) extract (M) and samples from e(k)
Remark (Estimation d.). One missing component in using ICA in practice is that, along with samples from (k), need to input the number of latent nodes d, which need to be estimated from samples. Define (k) = Exx ( = G G(B(R))- (B(K)) Since all matrices are full rank, it follows that = (k)), where {{) can be estimated by the sample covariance matrix. Thus, d can also be estimated by the rank of the sample covariance matrix.
We compute the empirical expectation of v on samples from ((k), which by law of large number arguments, converges to its population expectation, which ((()k)). We use the sorted order of the empirical expectations to sort the noise components, unscrambling the noise components as earlier, to get M and samples from (*))
Step 2:

---

Step 3: Here, we explicitly construct test statistic to check the condition on (M, k E [K]] to detect if Z, is a shifted node. Note that from our Theorem 3, there is a non-shift node Z, between environments lo and k' if and only if M +M Accordingly, we define test statistic:
(
k.kk min M, + M,
111
M |11 M 11
It can be that I. O if and only if M, M, which implies node Z; is not shifted between environments & and k'. Thus, in step three of the algorithm above, for each coordinate i [d]. check if there cxists & ş k' such that k.k for given threshold If such a k k/ exists, we include in the list of shifted nodes.
(k))
Algorithm I is consistent with the ground truth set of shifted nodes as approaches infinity. Empirical evidence supporting this claim presented in Figure 2, which shows that with a sufficiently large sample size, all shifted nodes are correctly identified, and the Fl score reaches 1. Further theoretical discussion on the sample complexity of our method can be found in Appendix D.
d= 5
d 10
20
1.0
0.8
0.6
0.4
p=
p
p
20
60
= 40
60
p 100
2
25
50
75
100
0
25
50
75 100
x10
N
x10
x10*
Figure 2: Illustration of the efficacy of our method in accurately identifying latent shifted nodes as the sample size increases, for ER2 graphs. In the first subplot, for a latent graph with = 5 nodes, we examine scenarios with observed dimensions p 10, 20, 40 and plot their corresponding F1 scores against the number of samples n. is observed that the Fl score approaches 1 with a sufficiently large sample size. Detailed experimental procedures and results are discussed in Section 5.
5
Experiments
In this section, we investigate the performance of our method in synthetic and real-world data.
5.1 Synthetic Data
In our setup, each noise component e; is sampled from a generalized normal distribution with the probability density function given by p(c,) х exp(-le,]") where i = 1, ...,.... d. generation process, the noise vector adheres the condition ve())) eeE,) for all i < j if we choose (vy) = P(yy < 1). Following the methodology similar to that in [40], we start by sampling either an Erdös-Rényi (ER) or Scale-Free (SF) graph with d nodes and an expected edge count of md, where m E {2, 4, 6). denoted as E. or SF.. The observed space dimension p is set to 2d. For each graph, the weights are independently sampled from Unif + [0.25, 1] and the diagonal entries of Ω from Unif 4]. In each environment k, 15% of the nodes are randomly selected for shifting. The new weights (AA) for the shifted node i, and the new entries of (2()), specifically (2(A)
In this noise
are independently sampled from Unif[6, 8). The mixing function G' is independently generated from Uni5[-0.25, 0.25].
Empirically, we have observed that the following formulation of L, k.k' leads to improved results:
M 1T. 17.
k.K'
1 1 +
(k)
7

---

Table Performance metrics for shifted node detection across various graph configurations, sample
sizes
10
Graph Type
Precision
Recall
Score
Time (s)
ER2
10
1.000
1.000
1,000
1.23
20
1.000
1,000
1,000
3.84
40
0.933
0.833
0.873
10.34
60
30
0.680
0.700
0.689
20.06
80
40
0.610
0.600
0,605
30,59
FR4
20
10
1.000
1.000
1,000
3.89
40
20
0,933
0.933
0.933
9.39
60
30
0.617
0.600
0.607
30.83
80
40
0.610
0.617
0.613
32.08
SF2
10
0.900
0.900
0,900
1.64
20
1.000
1,000
1,000
3.84
40
0.807
0.833
0.817
15.85
60
0.730
0.750
0.739
22.12
80
40
0.667
0.667
0.667
30.29
SF4
20
1.000
1,000
1,000
3.13
40
20
0.967
0.900
0.927
15.12
60
30
0.725
0.700
0.711
29.79
80
40
0.539
0.533
0.535
30.84
where (M(A) denotes the element-wise absolute value of the vector M/K We will utilized the new formula of L. to detect shifts in the following experiment. Then we explore sample sizes from 500 to 10 using the observed samples ( as input. The parameter is set to 0.2 for d s 10 and 0.5 for higher dimensions, reflecting the increased complexity in estimating larger dimensional latent graphs and thus necessitating a higher tolerance for Lı norm differences in detecting shifted nodes. For each setting, we independently generate 10 datasets and take the average of the metrics. The results for 10" are shown in Table and the asymptotic consistency results for specific p values are illustrated Figure 2. In addition to the causal representative setting, our method can also directly identify mechanism shifts in fully observed setting, where G = I. We further compare our method's 's results in this fully observed setting against the baseline DCI [52]. which addresses direct mechanism shifts in linear settings. The results of this comparison are provided in Appendix F, demonstrating that our method outperforms DCI in most settings.
5.2 Psychometrics Data
We evaluate method using dataset related to the Five Factor Model, also known the Big Five personality traits [16, 15, 32]. This model is a widely accepted framework, comprising five broad dimensions that encapsulate the diversity of human personality traits. These dimensions Openness to Experience, Conscientiousness, Extraversion, Agreeableness, and Neuroticism.
The dataset utilized in study was gathered through an interactive online personality test available on seenPsychometris..oo a nonprofit endeavor aimed at educating the public about psychology while collecting data for psychological research's This dataset encompasses responses 50 questions, with 10 questions dedicated to each of the five personality dimensions. Participants responded to each question scale from to 5. Additionally, the dataset includes demographic information, such as race, age, gender, and country, comprising a total of 19,719 observations.
Question formalization and data processing. In this study, we hypothesize the existence of latent nodes, each representing one of the five personality dimensions, believed to be causally related. The score responses to the 50 questions form our observed space. Our main goal is to determine whether variations in personality dimensions can be observed across genders, thus treating gender as one environment (K- = 2). Additionally, we investigate potential personality shifts across countries, selecting the US and UK for analysis due to they have the most observations in our dataset. The only preprocessing step undertaken involves the removal observations with missing values and the
'The data can be downloaded via the link: https: / www. www.kaglle.. 1e com/ dat asets/ Lucasgreenwell/eeeee Live- waevee--attrrre wot.-e--ooooo---oooooooa-------- test- rewponseesses /data

---

normalization of data to fit within the [0, 1] range, achieved by adjusting according to the maximum and minimum values observed. The research question we have formalized in this study is not derived from any data competition. It aligns with interests explored in existing psychological literature [25, 8, 46, 29], yet our investigation distinguished by a unique analytical framework.
Labeling latent nodes. Prior to detecting shifted nodes, it is essential to assign semantics to each node. This process involves conducting interventions on each component of the noise vector aid in labeling the latent nodes. Given that the noise components are distinct for each latent node, labeling the noise effectively equates to labeling the latent nodes.
Initially, we apply ICA to the data for males, followed by getting post-sorting M and Emate as outlined in our methodology. Subsequently, we perform interventions on each noise component, setting each to 0 sequentially, and then re-mixing the intervened noise vector using (Mmale)t. By examining the impact of these interventions on the observation space - specifically, identifying assign appropriate semantic labels to each latent node index. For instance, nullifying the first column of é and remixing the intervened noise with (Mmaaeet alters the score distribution in manner that reveals the semantic domain affected by the first noise component. An example of assigning the label Agreeableness to latent node is depicted in Figure 3. By applying the same process to all noise components, are able to assign semantic labels Openness, Conscientiousness, Extraversion, and Neuroticism to the remaining latent nodes. More detailed experiment results are shown in Section G.
which question scores undergo significant changes
Lie cerheaatoo
AINORE DOG
Agyonsath Noraaso
Openness
Intervention
We apply an intervention the first column of e and then use (Mmale)t for remixing. The first row of the resulting histograms represents scores for 5 out of the 10 questions related to the Extraversion personality dimension. Subsequent rows display histograms for questions from each of the other four personality dimensions, as indicated at the right end of each row. The red distribution represents the scores before the intervention on the noise, while the blue distribution corresponds to scores after the intervention. Overlapping areas are shown in purple. Notably, the intervention on the first column of e alters the distribution in the observed space, specifically affecting the scores for questions related to the Agreeableness personality dimension, whereas distributions for other dimensions remain unchanged. Consequently, we can label the first noise component as corresponding to Agreeableness.
Figure 3:
Shifted nodes detection. To identify shifted personality dimensions across gender, we computed L male. for each latent node, obtaining values of (0.074, 0.0497, 0.078, 0.638, 0.633). Setting a tolerance threshold a = 0.5 to accommodate real data estimation variances, we observed that the last two nodes exhibit significantly higher I. maate.femat. scores, surpassing o, and thus are considered shifted. These nodes correspond to the labels Neuroticism and Extraversion. Consistent with existing psychological literature, women have been found to score higher in Neuroticism than men [25, 46, 29], while men scored higher in the Activity subcomponent of Extraversion [8]. This discovery aligns with the findings in psychology literature. To further validate our method's effectiveness, a similar analysis was conducted across countries, comparing the UK and the US, which have the most observations in our dataset. The computed r. T.USSUK for each latent node was (0.302, 0.258, 009, 0819,, 0.088). All values fell below a, indicating no latent node shifts between

---

these two countries. This finding is also in agreement with existing studies that personality exhibits stability across countries and cultures [25, 24, 11].
Concluding Remarks
In this study, demonstrated that latent mechanism shifts identifiable, up permutation, within the framework of linear latent causal structures and linear mixing functions. Furthermore, introduced algorithm, grounded in ICA, designed to detect these shifts. Our method offers broader applicability to various types of interventions compared to CRL framework. Unlike shift detection methods where node variables are directly observable, our approach extends scenarios where latent variables remain unobserved. A promising future direction consists of adapting our methodology to nonlinear transformations, which could address more complex, practical challenges, such as identifying latent mechanism shifts in real- world image data.
References
[1] Ahuja, K., Mahajan, D., Wang, Y. and Bengio, Y. [2023], Interventional causal representation learning, in 'International conference on machine learning'. PMLR, pp. 372-407.
[2] Auddy, A. and Yuan, M. [2023]. 'Large dimensional independent component analysis: Statistical optimality and computational tractability', arXiv preprint 3rXivv2303.18156
[3] Bengio, Y., Courville, A. and Vincent, P. [2013], 'Representation learning: A review and new perspectives', IEEE transactions on patlern analysis and machine intelligence 35(8), 1798- 1828.
[4] Buchholz, S., Rajendran, G., Rosenfeld, E., Aragam, B., Schölkopf, B. and Ravikumar, P. [2023], 'Learning linear causal representations from interventions under general nonlinear mixing', arXiv preprint 2.Xi::3306.022355
[5] Budhathoki, K., Janzing, D., Blocbaum, P. and Ng. H. [2021], Why did the distribution change?, in 'Iterrnational Conference Artificial Intelligence and Statistics', PMLR, pp. 1666-1674.
[6] Budhathoki, K., Minorics, L., Blöbaum, P. and Janzing, D. [2022]. Causal structure-basee root cause analysis of outliers, in 'Iteeraaiionl Conference on Machine Learning'. PMLR,
pp. 2357-2369.
[7] Burgess, C. P.. Higgins, I., Pal, A., Matthey, L., Watters, N., Desjardins, G. and Lerchner, A. [2018], "Understanding disentangling in 3-vac', arXiv preprint arXiv:1804.03520
[8] Chapman, P., Duberstein, P. R., Sörensen, and Lyness, J. M. [2007], 'Gender differences in five factor model personality traits in an elderly cohort', Personality and individual differences 43(6), 1594-1603.
[9] Chen, T., Li, X., Grosse, and Duvenaud, K. [2018], 'Isolating sources of disen- tanglement in variational autoencoders', Advances in neural information processing systems 31.
[10] Chen, T., Bello, K., Aragam, B. and Ravikumar, P. [2023], 'iSCAN: Identifying Causal Mech- anism Shifts among Nonlinear Additive Noise Models', Advances in Neural Information Processing Systems.
[11] Cohen, H. and Deuling, K. [2014], 'Structural analysis of the abridged big five eircumplex: A comparison among gender and ethnic groups', Bulletin of Sociological Methodology/Bulleein de Méthodologie Sociologique 122(1), 63-86.
[12] Comon, P. [1994], 'Independent component analysis, new concept?', Signal processing 36(3), 287-314.
[13] Eriksson, J. and Koivunen, V. [2004], 'Identifiability. separability, and uniqueness of linear ica models', IEEE signal processing letters 11(7), 601-604
[14] Ghoshal, A., Bello, K. and Honorio, J. [2019], 'Direct learning with guarantees of the difference dag between structural equation models', arXiv preprint arXiv: 12024.
[15] Goldberg, I.. R. [1992], 'The development markers for the big-fiee factor structure'', Psycho- logical assessment 4(1), 26.
10

---

[16] Goldberg, L. R. [2013], An alternative "description of personality": The big- five factor structure, in 'Personality and Personality Disorders', Routledge, pp. 34-47,
[17] Hyvarinen, A. [1999], 'Fast and robust fixed-point algorithms for independent component analysis', IEEE transactions on Neural Networks 10(3), 626-634.
[18] Hyvärinen, A., Hurri, J., Hoyer, P. O., Hyvärinen, A.. Hurri, J. and Hoyer, P. O. [2009]. Independent component analysis, Springer.
[19] Hyvärinen, A. and Oja. E. [2000], 'Independent component analysis: algorithms and applica- tions', Neural networks 13(4-5), 411-430.
[20] Hyvärinen, A. and Pajumen, P. [1999], 'Nonlinear independent component analysis: Existence and uniqueness results', Neural networks 12(3). 429-439.
[21] Ikram, A., Chakrabortyy S., Mitra, S., Saini, S., Bagchi, S. and Kocaoglu, M. [2022], 'Root cause
analysis of failures in microservices through causal discovery'. Advances in Neural Information Processing Systems 35, 118-311177.
[22] Jiang, Y. and Aragam, B. [2023], 'Learning nonparametric latent causal graphs with unknown interventions', arXiv preprint arXiv:2306.0288
[23] Jin, J. and Syrgkanis, V. [2023], 'Learning causal representations from general environments:
Identifiability and intrinsic ambiguity', arXiv preprint arXiv:2311.12267.
[24] Jolijn Hendriks, A., Perugini, M., Angleiterr, A., Ostendorf, F., Johnson, J. A., De Fruyt, F., Hřebíčková, M., Kreitler, S., Murakami, T., Bratko, D. et al. [2003], "The five-factor personality inventory: cross-cultural generalizability across 13 countries', European journal of personality 17(5). 347-373.
[25] Kajonius, P. and Mac Giolla, E. [2017], "Personality traits across countries: Support for similarities rather than differences', PloS one 12(6), e0179646.
[26] Kim, H. and Mnih, A. [2018], Disentangling by factorising, in 'International Conference on Machine Learning', PMLR, pp. 2649-2658.
[27] Kulinski, S., Bagchi, S. and Inouye, D. I. [2020]. 'Feature shift detection: Localizing which fea- tures have shifted via conditional distribution tests', Advances in neural information processing
systems 33, 19523-19533.
[28] Li, C., Shen, X. and Pan, W. [2023], 'Nonlinear causal discovery with confounders' Journal of the American Statistical Association pp. 1-10.
[29] Löckenhoff, C. E., Chan, W., McCrae, R. R., De Fruyt, F., Jussim, L., De Bolle, M., Costa Jr, T., Sutin, A. R., Realo, A., Allik, J. et al. [2014], 'Gender stereotypes of personality: Universal and accurate?", Journal of cross-cultural psychology 45(5), 675-694.
[30] Magliacane, S., Van Ommen, T., Claassen, T., Bongers, S., Versteeg, P. and Mooij, J. M. (2018), DDomain adaptation by using causal inference to predict invariant conditional distributions', Advances in neural information processing systems 31.
[31] Mameche, S., Kaltenpoth, D. and Vreeken, J. [2024], 'Learning causal models under indepen- dent changes'. Advances in Neural Information Processing Systems 36.
[32]
Matthews, G., Deary, I. J. and Whiteman, M. C. [2003), Personality traits, Cambridge University Press.
[33] Misiakos, P., Wendler, C. and Püschel,, M. [2024], 'Learning dags from data with few root causes', Advances in Neural Information Processing Systems 36.
[34] Montagna, F., Noceti, N., Rosasco, L., Zhang, K. and Locatello, F. [2023], 'Causal discovery with score matching on additive models with arbitrary noise', (exXi:2304.03266
[35] Monti, R. P., Zhang, K. and Hyvärinen, A. [2020], Causal discovery with general non-linear relationships using non-linear ica, in 'Uncertainty in Artificial Intelligence' PMLR, pp. 186- 195.
[36] Muandet, K., Balduzzi, D. and Schölkopf, B. [2013], Domain generalization via invariant feature representation, in 'International conference on machine learning' PMLR, pp. 10-18.
[37] Pearl, J. [2009], CAUSALITY: Models, Reasoning, and Inference, 2nd edn, Cambridge University Press.
11

---

[38] Peters, J., Janzing, D. and Schölkopf. B. [2017], Elements of causal inference: foundations and learning algorithms, The MIT Press.
[39] Schölkopf, B., Locatello, F., Bauer, S., Ke, N. R., Kalchbrenner, N., Goyal, A. and Bengio, Y.
[2021], 'Toward causal representation learning', Proceedings of the IEEE 109(5), 612-634.
[40] Seigal, A., Squires, C. and Uhler, C. [2022], 'Linear causal disentanglement via interventions', arXiv preprint arXhv:2211.166677
[41] Shen, H., Jegelka, S. and Gretton, A. [2009], 'Fast Germel-based independent component analysis', IEEE Trunsactions on Signal Processing 57(9), 3498-3511.
[42] Shimizu, S., Hoyer, P. O. and Hyvärinen, A. [2009], 'Estimation of linear non-gaussian acyelic models for latent factors', Neurocompuling 72(7-9), 2024-2027.
[43] Shimizu, S., Hoyer, P. O., Hyvärinen, A., Kerminen, A. and Jordan, M. [2006), 'A linear non non-gaussian acyclic model for causal discovery.', Journal of Machine Learning Research 7(10).
[44] Shimizu, S., Hyvarinen, A., Kano, and Hoyer, P. O. [2012], 'Discovery of non-gaussian linear causal models using ica', arXiv preprint arXiv: /207. 1413.
[45] Silva, R., Scheines, R., Glymour; C., Spirtes, P. and Chickering, D. M. [2006], 'Learning the structure of linear latent variable models.' Journal of Machine Learning Research 7(2).
[46] Soto, C. J., John, O. P., Gosling, S. D. and Potter. J. [2011], 'Age differences in personality traits from 10 to 65: Big five domains and facets a large cross-scctional sample." Journal of personality and social psychology 100(2), 330.
[47] Sturma, N., Squires, C., Drton, M. and Uhler, C. [2023], 'Unpaired multi-domain causal representation learning', arrivv preprint arXiv:2302.00999
[48] Varici, B., Acarturk, E., Shanmugam, K., Kumar, A. and Tajer, A. [2023]. 'Score-based causal representation learning with interventions', arXiv preprint arXiv:2301.082:10
[49] VancI, B.. Acartürk, E., Shanmugam, K. and Tajer. A. [2023], 'General identifiability and achievability for causal representation learning', arXiv preprint arXiv:2310. 15450.
[50] von Kügelgen, J., Besserve, M., Wendong, L.,, Gresele, L.,, Kekić, A., Bareinboim, E., Blei,
D. M. and Schölkopf, B. [2023], 'Nonparametric identifiability of causal representations from unknown interventions", arXiv preprint arXivv23366.00542
[51] Wang, Y. and Jordan, M. [2021], 'Desiderata for representation learning: A causal perspective', arXiv preprint arXiv:2109.0373
[52] Wang, Y., Squires, C., Belyaeva, A. and Uhler, C. [2018], 'Direct estimation of differences in causal graphs', Advances in neural information processing systems 31.
[53] Wu, and Fukumizu, K. [2020], Causal mosaic: Cause-effect inference via nonlinear ica and ensemble method, in 'International Conference Artificial Intelligence and Statistics', PMLR. pp. 1157-1167.
[54] Yang. M., Liu, F., Chen, Z., Shen, X., Hao, and Wang, [2021], Causalvae: Disentangled representation learning via neural structural causal models, in 'Proceedings of the IEEE/CVF conference on computer vision and pattern recognition', pp. 9593-9602.
[55] Zhang, H., Zhang, Y.-F., Liu, W., Weller, A., Schölkopf, B. and Xing, E. P. [2022], Towards prin- cipled disentanglement for domain generalization, in *Proceddings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition', pp. 8024-8034.
12

---

SUPPLEMENTARY MATERIAL
Identifying General Mechanism Shifts in Linear Causal Representations
A Limitations and Broader Impacts
Limitations of this work include the need relax the noise assumption and consider similar settings under nonlinear mixing functions. These are promising directions to explore in the CRL field. The broader impact of this work is that CRL methods can be used identify mechanism shifts and determine root causes, which can be utilized in the biological field to find disease genes or biomarkers. Currently, the negative impacts of this method are clear.
B Illustration of our algorithm
X
Ra
e Rudd
ICA
M O
Sort
Rearrange
M
Compare
vite
(2e
M
Rearrange
ICA
2 E Rabe
Sort
Figure 4: Overview of our method: For each context k; given the data X (k) our method involves three main steps. First, we apply ICA each dataset to estimate (kk) and (k). Second, we calculate (e((k)) [v( (e (k) ( ...., (b((()) for each noise component, sort these components in
increasing order, and correspondingly arrange the rows of (k), Third, compare the sorted rows of (() (k) to identify the shifted nodes.
C Discussion on Test Function
In Assumption B, assume that there exists function & and that can access it. Here discuss ways relax it. Recall that in Section 4.1, vé is utilized to sort the noise component ((k) to ensure that the post-sorting noise vector ((k) has consistent order across all environments.

---

An alternative approach to achieve this is to use distribution matching. We take the noise vector in the first environment reference and align all other noise vectors post-sorting with the reference vector. To do this, we can use a distribution distance metric D. First, define signed permutation space as
Sa = = PD
| P is permutation matrix, is diagonal matrix with Da E (-1,1))
Then, solve the optimization problem:
min D , SESa
where D can be any distribution distance, such as Kullback-Leibler divergence. In Assumption A, we assume pair-wise different noise component, thus the optimization questions have minimums value if and only if each noise component of z(1) and Sz(k) have the same distribution, thus help align the noise component order. We solve this optimization problem for each environment 2 2, thus obtaining p(k), All following steps in our algorithm remain the same when using this alternative approach.
One small gap remains: even though all post-sorting noise vectors have a consistent order with e(), g(1) is not the ground truth order of e This ambiguity cannot be eliminated, consistent with the nature of the CRL method, and is the same with other CRL methods, such as [40, 28]. Fortunately, the ground truth order is not so important in practice. What people mainly care about the semantic label for each latent node. Some CRL generative models, such as [54], may be helpful for performing fake interventions and manually assigning semantic labels. However, this is beyond the scope of this paper, and will not discuss it further.
Even though the distribution matching optimization method offers greater flexibility, it is computa- tionally expensive. First, note that the cardinality |Sal = d! x 2 which represents vast search space when d large. Furthermore, calculating (+::-) generally computationally intensive. For example, the KL method requires density estimation, and the Maximum Mcan Discrepancy (MMD) method necessitates the computation of pairwise distances among samples. These challenges render this alternative difficult to implement. Consequently, we opt use the v function to facilitate efficient sorting, but it may need carefully design.
D Discussion on Sample Complexity
The sample complexity of our method must be considered from two perspectives: one involves using ICA to estimate g(4) and (*) and the other pertains to utilizing ((0) and a test function to sort the rows of (K)), Since the sorting step depends on the choice of test function, we assume for simplicity that ( is already sorted by the ground truth order. Thus, we only focus on the asymptotic behavior of M (k), which closely relates to the properties of the ICA estimator.
There various algorithms for solving ICA [18, 17, 41]; each algorithm exhibits different asymptotic statistical properties. If we apply the findings in Auddy and Yuan [2), we assume that the estimated ICA unmixing function has the following statistical accuracy:
Theorem 4. If the sample size n. Z g(d, 8), then with probability at least 1 - h(n,, d, o, e), we have:
((T (x()) S C.Cpp.ppppp. ((().
where (( represents the i-th row of the estimated unmixing function MK), C is a constant, and f, 9, and h are known functions. For instance, in Auddy and Yuan [2], p(d,n) n)) vd/n and
f(5) Mog(118), Here, denotes the loss function, and the Loz norm can serve as an option.
Under this theorem, for two environments is and k', if node i does not shift, we have:
|| Ma ll2 < |M Mill + +IMM Milk S 2 C. p(d,n)f(()
with probability of at least 2n( (1, d, 6, Thus, by setting the threshold a as Cpp(d, n)) (()), we can control the false discovery rate to be at most 2h(n, d, e,)). A similar sample complexity theorem can be extended to cases involving more than two environments, as long as the statistical properties of the ICA solution are known.
14

---

E
Detailed Proofs
E.1 Proof of Proposition
Lemma 1. Under problem setting, for any E Rdx1, the equation x" H = y² H holds if and only if = y-
Proof. Given that G possesses full column rank, it follows that H = GJ has full row rank. Conse-
quently, the null space of H" is {0). Therefore, if xF H = y" H, implies H² (x y) = 0. This leads to the conclusion that r y = 0, which in turn implies
y.
Proof of Proposition 2. Recall that B(k) = (12)) 2Ta - Ak) Since A is weighted adjacency matrix, its diagonal entries are zero. Thus,
( bj = (2(
A if it
((x) As = (2
if i=
Under Definition 1, if node Z, is shifted, it implies either 1) 22 + (K())
2) (k)
+ (k²), or 3)
both conditions hold. In scenarios 1) and 3), B * B resulting in B(()) * B In scenario
2), while ( = there exists j E [d] such that A!h) * A(K) leading to B( + B If
node Z4 is not shifted, then A 1(4) A(4)) and 2 s2 implying B,a) B B((²) Therefore,
7 is shifted if and only if B7(() + B(K)) According to Lemma 1, B # B (k) if and only if
B H + B, H. which is equivalent to M! # (K)
In conclusion, Z shifted if and only if MM + M!!))
E,2 Proof of Theorem 3
Lemma 2. Under problem setting, it is not possible for an intervention on the latent node Zi to result
in M!h) -M, (k)
(k), would imply (²x)) (B(k), However, we know (b)) = (0(()))1(
Proof. We prove this by contradiction. Suppose that (a(h) - -,,
According to Lemma 1, this (k) where A(k) is the
weight matrix for a DAG. Since A! 0, it follows that ((() (1) Therefore, both B(())
and ((K) are positive. It impossible for B(*)) be equal to - B B(()) Consequently, the scenario where M. (k) (k) cannot occur.
Proof of Theorem 3. Recall from the data generation process that (A) (k) (A)
When input (k) to ICA, we have (4) p()) ((*) (k), and E(h) = p(() (k)) Without loss of generality, we assume that e(*) is ordered increasingly with respect to v. Thus, post sorting with respect vi, we eliminate the ambiguity of p(k), and we get (1) (1) (K)) and ((k) (kk)
We now ready to prove that Z; is not shifted if and only if (R) (M)) This immediately implies that Z, is not shifted, then M(
(R) = (M()) thus satisfying (k) M
If (R) (K'), there two cases: M₁ (b) M, M -M("k) We prove in Lemma 2 that the scenario M((e) = - -M(k') cannot exist. The only surviving situation is M.() M,)))
which indicates that Z, is not shifted.
15

---

Experiments on Synthetic Data Compared with DCI
As described Section 5.1, instead of generating the mixing function from Oni5 0.25], set G I, such that X Z and Z can be directly observed. In this setup, finding general interventions in linear causal representations reduces to identifying general interventions in linear SEM, setting for which the existing method DCI [52] is designed. Table 2 presents the performance comparison between our method and DCI under these conditions, demonstrating that our method outperforms DCI in most cases.
Graph Type
d
Method
Precision
Recall
F1
DCI
0.60
0.60
0.60
5
Ours
0.80
0.80
0.80
DCI
0.87
1.00
0.92
ER 2
10
Ours
1.00
1.00
1.00
DCI
0.74
1.00
0.84
15
Ours
0.66
1.00
0.78
DCI
0.83
1.00
0.89
10
Ours
1.00
1.00
1.00
ER 4
DCT
0.71
1.00
0.81
15
Ours
0.62
0.93
0.73
DCI
0.70
0.80
0.73
5
Ours
1.00
1.00
1.00
DCI
0.67
1.00
0.79
2
10
Ours
1.00
1.00
1.00
DCI
0.65
1.00
0.78
15
Ours
0.70
0.93
0.78
DCI
0.60
0.60
0.60
5
Ours
0.80
0.80
0.80
DCI
0.77
1.00
0.85
SF 4
10
Ours
1.00
1.00
1.00
DCI 0.56
0.93
0.68
15
Ours
0.67
1.00
0.79
Table 2: Comparison of Precision, Recall, and Fl scores for different graph types, d values, and methods between our method and DCI.
G Additional Information on Real Data
This section provides detailed information on the procedures employed in analyzing the real dataset.
Preprocessing The initial dataset comprised 19,719 observations, which can be downloaded from attps://wwid kaggle com/ http///tttttttt//aaaaeeeeaeaaeee httas//eeaaaaaaaael...... responsseeerreeeeeeeeeeeeee /data. In the preprocess-
ocean- tiran-five--e five ing phase, any observation with missing value in any column excluded, leaving a total of 19,710 observations for further analysis. Subsequently, we applied max-min value normalization to the scores of each question, ensuring that all scores were normalized fall within the range [0, 1]. This normalization step is crucial for achieving uniformity in the data scale, thereby facilitating accurate analysis and comparison across the dataset.
Labeling the Noise To derive mcaningful psychological insights, it is crucial to assign semantic labels to all latent nodes. Given that the noise components are pairwise distinct and unique to the latent node Z,, consider intervening each noise component, then remixing and observing the changes in the observational space. This approach enables us to assign semantic labels to both the noise components and their corresponding latent nodes. We utilize observations from the male dataset as the reference context, which comprises 7,603 observations. Following the initial step of our method, obtain the sorted Mamale and maale The mixing function G is derived from (M
16

---

To identify the semantic label for the first component of é, we set its corresponding noise vector component 0, effectively nullifying the first component Emate, This intervention yields an estimated noise matrix samples from emelle denoted as Ciny The intervened reconstruction, X = G(é and the original score distribution, X Xmale = G(Emalle)", allow us to compare question scores pre- and post-interveniie instinttervenion. Figure plots these distributions, revealing significant shifts for questions pertaining to the Agreeableness dimension, with minimal impact on other scores, thereby identifying the first noise component as Agreeableness. This process replicated for the second through fifth columns of em with results illustrated in Figures 9, 8, 5, and 6. Each plot demonstrates that interventions result in significant distribution changes for questions related to single personality dimension, with negligible effects on others. Consequently, we label these noise components as Openness, Conscientiousness, Extraversion, and Neuroticism, respectively. These labels will be used for all the following analysis.
Shifted Nodes Detection We then applied our method to data from male and female contexts.
The calculated L male.female values are (0.074, 0.0497, 0.078, 0.638, 0.633). Based on these results, identify shifts in the last personality dimensions, specifically labeled Extraversion and Neuroticism. Additionally, we conducted a comparative analysis of personality dimensions between the US and UK, which have 8,753 and 1,531 observations, respectively. The computed L U values are {0.302, 0.258, 0.109, 0.189, 0.088). indicating that no latent node is considered having undergone shifts between these two countries.
Extroversoo
Nou Tourotiiooen
Carsclle Sas
Openeess
Figure 5: Intervention on the fourth component of the noise vector and subsequent re-mixing generate new observed space new score distribution. Notably, only Extraversion exhibits significant changes after intervention, leading us to label the fourth component of the noise vector (after sorting) Extraversion.
17

---

Extrverron
Neuroffillem
Correc Coreeilerioo Sasrtii PRPV
Opernesss
Figure 6: Intervention on the fifth component of the noise vector and subsequent re-mixing generate a new observed space - a new score distribution. Notably, only Neuroticism exhibits significant changes after intervention, leading us label the fifth component the noise vector (after sorting) as Neuroticism.
Extravesion
Niu Nauroflccu
Openness
Figure Intervention on the first component the noise vector and subsequent re-mixing generate new observed space new score distribution. Notably, only Agreeableness exhibits significant changes after intervention, leading us to label the first component of the noise vector (after sorting) as Agreeableness.
18

---

Extroverat
Neurros couuroticiom
Contccientoo Seteepeeee
Oper
Figure Intervention on the third component of the noise vector and subsequent re-min re-mixing generate new observed space new score distribution. Notably, only Conscientiousness exhibits significant changes after intervention, leading us to label the third component of the noise vector (after sorting) Conscientiousness.
Estravesson
Nouroticlem
Opennodd
Figure Intervention on the second component the noise vector and subsequent re-mixing generate new observed space score distribution. Notably, only Openness exhibits significant changes after intervention, leading us to label the second component of the noise vector (after sorting) as Openness.
19

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: has been discussed in Appendix A.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model sell-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
20

---

Justification:
Theorems and proposition has been completely proven in Appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
The whole algorithm has been discussed in Algorithm 1.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
21

---

Answer: [Yes]
Justification:
The code is published iLCS.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https:/ /tts:///nsppppp cc/
des publ c/ gui CodeSulom ss onPol cyy for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run reproduce the results. See the NeurIPS code and data submission guidelines (ht tnttpp:: / /nips. cc/
"nips.cc/public.. httpppublllcggl/eee /wcc/uuisssssse Codee Submi ss sssonnoooo conPoliee olicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The algorithm is detailed in Algorithm 1, and the implementation details are discussed in Section 5.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We do not report statistical significance of the experiments as it follows the traditions of CRL literature, such as [40] and [4].
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by emror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
22

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [No]
Justification:
We do not report the computer resources used as it is not the primary concern this paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported didn't It make it into the paper).
paper (e.g., preliminary or failed experiments that NeurlPS Code of Ethics https: /ttp:///////ii..eeeee cc ct//:uullc//tttticccleeeelllll
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the
Answer: [Yes]
Justification: The research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: is discussed Section
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
23

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: The paper has no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: One dataset generated by simulation, and the source of the real data has been explicitly mentioned.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use the package should be provided. For popular datasets, paperswi thoode com/ dataaaaaaa asetss has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The primary asset of the paper is the code, and the code is published in https: //tgttgttbt/tt///////////t com Ti http//aaaanyooinnngeee iLCS.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing research with human subjects
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
25


Reviews:
[{"review_id": "tuXbpLuxtk", "reviewer": "Reviewer_tbD8", "summary": "This work studies the problem of detecting the mechanism shifts in a novel way by considering the latent nodes. The authors prove identifiability results based on assumptions softer from prior identifiability results for causal representation learning. Their method is based on ICA and is evaluated empirically on synthetic data and a real-world psychometric dataset.", "questions": "Line 56: Do you think that the problem you solve is relevant to identifying the locations of the root causes of a linear SEM (as in [1])?\nFigure 1 caption, 7th line: Typo, should be $Z_4\\to Z_1$ \nLine 143: What are the implications of this assumption? Is it also present in prior identifiability results?\nLine 231: What is the effect of the observed space dimension $p$ being larger or smaller with respect to the problem you solve (does it become easier or harder)?\nFigure 3: the font in the legend and xticks must be larger.\nLine 293-294: Here I got a bit confused. This is different from what you show in the example of Fig. 1, right? Because in Fig. 1 you show interventions (shifts) across different countries.\nLine 297: Can you briefly explain how your methodology can be generalized to a nonlinear data-generating process?\nLine 303: What would distribution shift imply for image data? \n\n[1] Misiakos, P., Wendler, C., & Püschel, M. (2024). Learning DAGs from data with few root causes. Advances in Neural Information Processing Systems, 36.", "limitations": "The authors have discussed some limitations in the appendix. However, a significant limitation of not being applicable to a large number of nodes is not included.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The paper is very well written and the scope and contribution are clearly stated and illustrated with examples. \n\n**Related work** The related work part is detailed and correctly positions the paper at the midpoint between causal representation learning and causal mechanism shift detection. \n\n**Novelty** The paper is novel as it proposes a methodology for a known problem (detection of causal mechanism shifts), in a new setting that considers latent variables, which is the case in the causal representation learning field.\n\n**Theory** The authors propose and theoretically prove a softer identifiability result that allows for fewer than $d$ and unrestricted interventions (soft/hard and possibly applied to multiple nodes).\n\n**Experiments** The method is evaluated on synthetic experiments and an interesting real-world experiment on a psychometric dataset, which provides evidence for its applicability in practice.", "weaknesses": "This work has some possible weaknesses\n\n**Significance of contribution** The proposed work solves a simpler problem than causal representation learning, which doesn't require learning the whole mixing matrix $B$, but rather only the distribution shifts. I am concerned that this simplification of the problem might make it easier solvable and I wonder whether other methods of CRL could be transformed easily so they could perform well on this task. \n\n**Experiments** Following my previous concern, I wonder why you did not compare against prior CRL methods or methods for causal mechanism shifts. Such a comparison would enhance the experimental results. Can you adapt the CRL methods or the causal mechanism shift techniques for observable variables to your setting?\n\n**Sample complexity** From Figure 2 it seems that indeed for a large (towards infinity) number of samples, your method can accurately detect the mechanism shifts. I am concerned, however, that the sample complexity is high which makes nodes with over hundreds of nodes out of reach. This can also be seen from Table 1 where your performance drops quite early, even from graphs with 60 nodes. From a theoretical perspective, it would be interesting to compute how many samples are required regarding the number of nodes (sample complexity result - either theoretical or experimental study). Such a theorem would be appreciated by the community.\n\n**Real experiment: size of changes** The results are very interesting and agree with psychological findings. However, the number of nodes and changes are very small. A larger number of nodes (up to 100) would be more sufficient to show that your algorithm is valid in practice. \n\n**Limitations** You should explain either why the application to a large number of nodes is not needed in practice or add it to the limitations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the reviewer's recognition of our paper's novelty and contribution. We now address the reviewer's concerns.\n\n### From Weaknesses\n\n> Significance of contributions...\n\nPlease refer to the global response for this concern.\n\n> Experiments...\n\nThe most recent CRL method with official code released is [2]. In their simulation, their official code is based on all Gaussian noise and samples data precision matrices from the Wishart distribution. However, in our case, our noise setting is non-Gaussian, requiring the calculation of the data precision matrix from samples. Since the estimation of the precision matrix involves the inverse of a not full-rank matrix, their code becomes numerically unstable. Thus, it is hard to accommodate their code in our setting.\n\nInstead, as you suggested, it is a good idea to compare the existing methods of directly finding causal mechanism shifts in a fully observable setting. We compared with methods DCI [3], as they are the most recent methods with official codes publicly available. The results for the simulation are shown in the table of pdf. Our method achieves higher performance in most settings.\n\n[2] Squires, C., et al. Linear causal disentanglement via interventions.\n\n[3] Wang, Y., et al. Direct estimation of differences in causal graphs\n\n\n> Sample complexity....\n\nPlease refer to our gloabl response.\n\n> Real world experiment...\n\n1. We believe that our experimental setting is relatively high-dimensional and involves larger graph sizes compared with existing CRL methods. For example, [2] considers settings where $d = 5$ latent nodes, [3] considers settings with $d = 5, 10$, and [4] considers settings where $d = 5, 8, 10$. In contrast, our experiments consider $d$ ranging from $5$ to $40$, which represents a relatively high-dimensional graph compared to existing baseline methods.\n\n    [4] Jin, J., et al. \"Learning causal representations from general environments: Identifiability and intrinsic ambiguity.\" \n\n2. The main contribution of our paper is to provide an identifiability result to detect the shifted nodes under fewer assumptions. Thus, we believe that the experimental part is sufficient to validate our method and proof.\n\n3. It is challenging to find CRL datasets, under which the detected shifted nodes can also be validated by scientific papers. \n\n> Limitations\n\nOur method can be applied to any number of latent nodes, and our identifiability results holds. As we explained in previous responses, datasets with large number of latent nodes are hard to obtain, and we have already conducted synthetic experiments on relatively large latent graph sizes compared with other CRL methods. We will add more discussion about this in the limitations section of the revision.\n\n### From Questions \n\n> Line 56...\n\nThank you for your suggestion,we will add more discussion about it in our revision. We note that there are differences between our problem setting and the one in the mentioned paper, even in the direct observation setting. Using notations in [1], their goal is to find the SEM $A$ from the equation $X = C(I + \\bar{A})$, where $(I - A)(I - \\bar{A})=I$. They assume that $C$, the root cause, is sparse.\n\nIn contrast, our objective is to identify the difference in $A$ across different environments, where the difference in $A$ can be sparse. The sparsity in the difference of $A$ between different environments does not necessarily imply a sparse root cause $C$. To elaborate further, suppose we have datasets from two environments, $X^1 = A^1 X^1 + N$ and $X^2 = A^2 X^2 + N$. When we take the difference, we can consider $X^1 - X^2$ as arising from the process $X = (A^1 - A^2)X + N'$. Even though $B = (A^1 - A^2)$ may be sparse, $(I + \\bar{B})^{-1}$ is not necessarily sparse.\n\nTherefore, while the question addressed in that paper is relevant, it is not exactly the same as ours. We will add the citation and include a  brief discussion about the differences between our work and that paper in the revision.\n\n\n> Fig1 caption...\n\nThanks for point this out. We will correct them in the revision.\n\n> L143...\n\nThis assumption is introduced to ensure a consistent order of latent noise components after applying ICA. When ICA is applied to each individual environment, the order of noise components may differ. Therefore, this assumption helps us eliminate permutation ambiguity. However, this assumption is not necessary. We discuss how to relax this assumption in Appendix C.\n\n> L231...\n\nFor any given $d$, a higher $p$ can be regarded as more data being provided since it offers additional auxiliary information. On the other hand, higher dimensional data makes optimization in ICA more challenging. Lower $p$ values provide less auxiliary information, but they are easier to optimize. Generally, we believe that higher $p$ will offer more benefits than disadvantages. For instance, in the third plot of Figure 2, we observe that higher $p$ values consistently yield higher F1 scores.\n\n> L293..\n\nYes. Sorry for the confusion. Figure 1 and Figure 3 are two different example, and line 293-294 is based on the example of Figure 3. We will make it more clear in the revision.\n\n> L297..\n\nPlease refer to the global response.\n\n> Line 303..\n\nA good example of image distribution shifts is the WILDS dataset [1]. For instance, it provides a dataset where tissue slide images  shift across patients' health conditions and hospitals. Identifying the mechanism shifts in this context involves using the tissue image dataset to determine, for example, whether some particular coloring changes across healthy and diseased patients. Since there is no evidence that the (mixing) mapping from latent to observation is linear, we did not include such an image example in our paper. \n\n[1] Koh, P., et al. \"Wilds: A benchmark of in-the-wild distribution shifts.\" ICML 2021.\n\n---\nWe hope our answers will make you feel more positive about our work. Please let us know any follow-up questions!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "P6nnwqFyPt", "reviewer": "Reviewer_Eg5F", "summary": "This paper considers the setting of linear causal representation learning (CRL) with possibly multi-node interventions. Instead of focusing on the task of identifying the causal structure, which is recently shown to be impossible, the authors instead focus on the task of identifying mechanism shifts i.e. which nodes are intervened in each environment. The authors show that this identification task is actually possible, and design an identification algorithm to achieve it. Lastly, the authors empirically demonstrate the effectiveness of their approach.", "questions": "In the paper [1], the authors consider linear CRL (the same setting as this paper) and design an identification algorithm that works for general environments (i.e. multi-node interventions). Their algorithm can fully recover the causal graph, as well as recover the mixing matrix up to a surrounding node ambiguity (SNA). They show that SNA is am intrinsic barrier in this setting.\n\nWhat I'm wondering is that, given their identification algorithm, if it is the case that the task of \"identifying mechanism shift\" can be straightforwardly resolved. Because if you can recover the mixing matrix, then you can also recover the noise-to-latent matrix (i.e. $B^{(k)}$) in the current paper). Then you can identify the mechanism shifts simply by comparing the entries of different $B^{(k)}$'s. Of course, the mixing matrix is actually recovered with some ambuguities. However, given that such ambiguities are inevitable as shown in [1], I suspect that such ambiguities do not affect the task of identifying mechanism shift.\n\n*I am happy to raise my score if the above concern is appropriately addressed.*", "limitations": "N/A", "rating": 3, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper is well-written and easy to follow. Most mathematical definitions and statements are supported with sufficient explanations.\n\n2. The task that the paper focuses on i.e. identifying mechanism shift is quite interesting, and can possibly be considered in other cases where full identification is hard or even impossible.", "weaknesses": "1. It seems that step 3 in Sec. 4.2 is stated without any proof of why it works. Is it just a heuristic or that it can provably lead to identification?\n\n2. It seems to me that the main results of this paper is very closely related to [1], but the authors do not discuss in detail about this issue. (Please see the \"Question part\" for more details)\n\n[1] Jin, Jikai, and Vasilis Syrgkanis. \"Learning causal representations from general environments: Identifiability and intrinsic ambiguity.\" arXiv preprint arXiv:2311.12267 (2023).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your efforts in evaluating our paper. Below, we address your concerns.\n\n### From weaknesses\n\n1. In the population setting, step 3 will lead to identifiability provably, and it is proven in Theorem 3 that $L_i^{k,k'} = 0$ if and only if $Z_i$ is not a shifted node between environments $k$ and $k'$. However, in the finite sample setting, since we have no precise measure of the accuracy of the ICA estimation, the choice of $\\alpha$ is heuristic.\n\n### From questions\n\n* Yes, if we can estimate $B^{(k)}$ by the method in [1], SNA ambiguity does not affect the identifiability of our task. However, estimating $B^{(k)}$ up to SNA **requires** two assumptions: the number of environments $K$ should be at least equal to the number of latent nodes $d$, and there should be at least $\\Theta(d^2)$ interventions. **Our algorithm does not rely on these two assumptions**. For example, for *any* number of latent nodes, our method can localize mechanism shifts even when given only $K=2$ environments. \n\n* Even if we were able to estimate $B^{(k)}$, if the objective is to identify shifted nodes, estimating less can be more efficient. Our paper shows that estimating $B^{(k)}$ is not necessary at all for achieving that objective.\n\nWe will add these comments in the revision to emphasize the difference to [1].\n\n---\nWe hope our answers will make you feel more positive about our work. Please let us know any follow-up questions!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1ASfxAaIs0", "reviewer": "Reviewer_33zq", "summary": "This work studies the nontrivial problem of causal representation learning from the perspective of mechanism shifts within the latent SCM. Specifically, the authors relax existing restrictive assumptions in interventional causal representation learning, such as data generated from single-node perfect interventions and the number of environments necessary to facilitate identifiability and show that it is possible to identify the latent nodes that shift between environments/distributions from more general soft/hard and add/reverse interventions given access to fewer environments than the number of causal variables. Furthermore, the authors develop a practical algorithm to recover the latent sources attributable to the distribution shift and evaluate their method on synthetic data and a psychometric dataset. The main contribution in this work seems to be the use of a test function to score the noise factors learned from ICA to construct a sorted permutation matrix, which is used to construct the new unmixing matrix for scrambling the independent SCM noise variables.", "questions": "- Proposition 2 and Theorem 3 are contradicting. What are the criteria for a node to be identified as a shifted node? Do you mean that if the ith row of the unmixing matrix is **different between environments**, then node i is a shifted node? This seems to be the case when looking at the proof of Proposition 2 in the appendix. I would appreciate it if the authors could provide some clarification on this.\n- Do the authors have any intuition about the setting where noise factors are correlated and the iid assumption of noise across environments is violated? In this scenario, we would no longer have the standard linear ICA result to build off of.", "limitations": "Limitations are discussed in the appendix.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The theoretical result and intuition of identifying latent sources of distribution shift is interesting and is a step toward more feasible CRL for real-world application. Most work focuses on a supervised discriminative setting for studying distribution shifts. This work stands out in being one of the first to identify latent sources of distribution shifts.\n- The empirical evaluation is extensive and considers real-world datasets to evaluate the proposed CRL algorithm. The results from the Psychometrics dataset suggest that the algorithm proposed is capable of identifying the latent shifts from the data distribution to a great degree and in line with human interpretation. The test statistic proposed to quantify the degree of distribution shift between nodes w.r.t unmixing matrix is practical.\n- Generalizing the class of interventions for interventional CRL to identify only shifted nodes is a useful result for real-world CRL, where interventions may be multi-node and more complex.", "weaknesses": "- This work considers linearity in both the mixing and the SCM, which can be somewhat of a restricting assumption in practice.\n- Proposition 2 and Theorem 3 seem to contradict each other. By the current logic, the two statements imply that both shifted and non-shifted nodes require the same condition of the row corresponding to the variable index in the unmixing matrix to be invariant across environments (w/ sign flip). This should also be made clear in Section 4.2 in Step 3.\n- Minor points\n    - In the caption of Figure 1, for the UK environment, it should be the edge Z_4 → Z_1 is removed instead of Z_5 → Z_1 removed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer's valuable suggestions and for recognizing the novelty of our work. We next address the reviewer's concerns.\n\n### From weaknesses\n\n* Please refer to our global response regarding linear models.\n\n* Thanks for pointing this out. We apologize for the typo in Proposition 2, line 187. It should state that \"$Z_i$ is identified as a **not** shifted latent node between $k$ and $k'$ if and only if $M_i^{(k)} = M_i^{(k')}$\", and Theorem 3 is correct. We now believe Proposition 2 and Theorem 3 are consistent after correction. \n\n    Additionally, in Section 4.3, Step 3, line 213, it should read \"$Z_i$ is a **non-shift** node between $k$ and $k'$ if and only if $\\widetilde{M_i}^{(k)} = \\pm \\widetilde{M_i}^{(k')}$\", and in line 215, it should state \"there is **no shift** in node $Z_i$ if and only if $\\widetilde{M_i}^{(k)}=\\pm \\widetilde{M_i}^{(k')}$\". We will correct these in the revision. \nThank you again for your careful review.\n\n* Correct. Thanks for pointing this out. We will correct it in the revision.\n\n### From questions\n\n* As we pointed above, there are some a couple of typos in Proposition 2 and Section 4.2, Step 3. After correction, as you suggested, the illustration of our method should be consistent. A node $Z_i$ is a shifted node if and only if $M_i^{(k)} \\neq M_i^{(k')}$, which means that the $i$-th row of $M^{(k)}$ and $M^{(k')}$ are different, or $\\widetilde{M_i}^{(k)} \\neq \\pm \\widetilde{M_i}^{(k')}$.\n\n* Great question! This is a non-trivial question that makes for an exciting future direction. It is important to note that the correlated noise assumption impacts the identifiability of $B^{(k)}$. Its effect on the ICA solution would come secondary. For our setting and for CRL in general, it is crucial that $B^{(k)}$ is unique to avoid identifiability issues. If we assume the noise component is correlated, it implies that $\\Omega$ is not a diagonal matrix, and it is also possible that $\\Omega$ is not full rank. Given that $\\Omega^{1/2}B = I - A$, if $\\Omega$ is not full rank, then $B$ is not unique. \n\n---\nWe hope our answers will make you feel more positive about our work. Please let us know any follow-up questions!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IUwMPV22Jl", "reviewer": "Reviewer_JGz2", "summary": "Recently, causal representation learning has drawn a lot of attention in the representation learning area, where it considers a causal relation among the latent generative factors. This work is under the setting of linear causal representation learning. Data $X$ comes from a linear mixing $X=GZ$ of the unknown latent factors $Z$, and latent factors $Z$ follow a linear SCM. They are trying to estimate the causal mechanism's shifting (causally changed) node, under the more general/relax intervention. They claim that ‘it is possible to identify the latent sources of distribution shifts while bypassing the estimation of the mixing function and the SCM (causal relation matrix). The experiments were provided to justify their setting.", "questions": "Besides the three questions I mentioned in the cons section, I do have another question here, but it does not lower the marks of this work, just discussion.\n1. This question is not an issue for this work since it seems many reference works use interventional settings. But it would be interesting if the author could discuss this. Could the author clarify why we must use interventional data? Is there any alternative way since the interventional data is kind of a semi-supervised strong assumption? I know there is a paper about the 'Challenging Common Assumptions in unsupervised CRL', but is there any other way we can avoid using interventional data?", "limitations": "N/A", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 4, "contribution": 2, "strengths": "Pros:\n1. This paper is very well-written. It's pretty clear and easy to understand. The reviewer enjoys reading it.\n2. They are considered a more relaxed setting of intervention. It's an interesting idea where you directly find the shifted node.", "weaknesses": "Cons:\n1. The first issue is the main motivation of this work. Their idea is to bypass the estimation of the mixing function $G$ and the SCM $B$ (causal relation matrix). However, the reviewer believes this is the main motivation for causal representation learning. Why would we want a shift node, if we can't even identify the causal relationship in CRL? Could the author please clarify the point here?\n\n2. The reviewer believes the strong assumption limits the significance of this work. The `access to a test function (Assumption B)' is a really strong assumption that seems not realistic. Since you got the permutation ambiguity fixed. And I believe you also fixed the sign ambiguity in your assumption. Combined with the bounded variance you basically got an ICA with only scaling ambiguity. What is the difference between this work and the classical linear ICA result?\n\n3. The reviewer would like to see the clarification of the motivation and connection between linear ICA and this work. For linear ICA, we can get mixing function $M$ and the independent sources $\\epsilon$ in an unsupervised way, with an identifiability guarantee. But for causal representation cases, the $M$ decomposes into $G$ and $B$. The natural/intuitive idea is to identify those factors with certain assumptions. Could the author provide any intuition for this?\n\nThe reviewer would love to raise the score if these questions were resolved in the discussion session.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your efforts on reviewing our paper! Below, we address your concerns.\n\n### From weakness\n\n1. We provided some motivation from Line 34 to Line 38 in the paper, and also included a toy example in Figure 1. Additionally, we provided a Psychometrics data application (Section 5.2) to reflect the motivation and practical significance of our method. In addition, we would like to provide more potential applications here to illustrate in which scenarios one might be interested in localizing shifted nodes.\n\n    For example, examining the mechanism changes in the gene regulatory network structure between healthy individuals and those with cancer may provide insights into the genetic factors contributing to specific cancers. Within biological pathways, genes could regulate various target gene groups depending on the cellular environment or the presence of particular disease conditions [1,2]. In the analysis of EEG signals [3], it is of interest to detect neurons or different brain regions that interact differently when the subject is performing different activities. All these questions can be formulated as lozalizing shifted nodes.\n\n    The complementary question of finding shifted nodes is to find the shared non-shifted nodes, which can also be used for social science analysis. For instance, we may have data from different countries and different ethnicities but aim to identify the potential causes that influence the years of education pupils receive [4]. Based on the applications in these fields, I believe that estimating shifted nodes in CRL is an interesting and valuable topic.\n\n    [1] Hudson, N. J., Reverter, A., \\& Dalrymple, B. P. (2009). A differential wiring analysis of expression data correctly identifies the gene containing the causal mutation. PLoS computational biology.\n\n    [2] Pimanda, J. E., Ottersbach, K., Knezevic, K., Kinston, S., Chan, W. Y., Wilson, N. K., ... \\& Göttgens, B. (2007). Gata2, Fli1, and Scl form a recursively wired gene-regulatory circuit during early hematopoietic development. Proceedings of the National Academy of Sciences.\n\n    [3] Sanei, S., \\& Chambers, J. A. (2013). EEG signal processing. John Wiley \\& Sons.\n\n    [4] Ghassami, A., Salehkaleybar, S., Kiyavash, N., \\& Zhang, K. (2017). Learning causal structures using regression invariance. Advances in Neural Information Processing Systems, 30.\n\n\n2. Assumption B is included for better presentation and understanding of our method's workflow, but it is not necessary for our  identifiability results. As mentioned in line 148 and Appendix C, there are alternatives to this assumption. With estimated noise samples, we can perform distribution matching to achieve a consistent order of noise components. Therefore, this assumption is not necessary.\n\n    Regarding sign and scaling ambiguity, no, we do not fix sign ambiguity in our assumptions, as ICA inherently has sign ambiguity. In Theorem 3, we prove that the unmixing matrix estimated from the ICA method, even with sign ambiguity, will not affect our estimation of shifted nodes. Additionally, in Theorem 1, it is proven that under certain assumptions, the ICA solution only has permutation and sign ambiguity, so there should be no scaling ambiguity.\n\n\n    ICA is an important component of our method. The main contribution of our paper is to prove that the ICA solution can help us detect the shifted latent nodes. We also provide the identifiability result for the algorithm designed based on ICA. We believe this is not trivial since we are solving a new problem with theoretically backed methods.\n\n3. Thank you for the question. We refer you to the global response for a detailed answer to this question. In short, in the \"inadequate\" intervention data setting (small number of intervention environments, inadequate number of interventions, and general intervention setting), estimating $G$ and $B$ from $M$ is impossible. Our method bypasses the step of estimating $B$ but can still estimate shifted nodes with fewer assumption constraints.\n\n\n### From questions\n\nThank you for the question. Assuming interventional distributions is a modeling choice, which may or may not hold in practice. Our setting is concerned with identifying the sources of distribution changes (shifted nodes) across two or more populations. In this case, one formal way to characterize distribution changes is through modeling interventions in a causal graph. We do not think one *must* assume this but we believe it can be a reasonable assumption in multiple scenarios.\n\n\n---\nWe hope our answers will make you feel more positive about our work. Please let us know any follow-up questions!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We appreciate the time and efforts all reviewers have invested in evaluating our manuscript. We are grateful for the constructive feedback and insightful comments. It has come to our attention that there are some common questions regarding our paper, particularly concerning:

1. Can existing CRL methods be applied to our question? Is it a more difficult or simplified question than CRL?

2. Linear mixing and causal structure assumption. Can the method be generalized to nonlinear mixing functions?

3. What is the sample complexity of our method?

We next address these concerns and hope our answers will make the reviewers feel more positive about our work.

### Question 1

In some aspects, our setting does indeed simplify the CRL problem as we do not aim to identify the entire mixing matrix and latent causal structure. However, in other aspects, our setting faces other challenging scenarios such as having access to fewer than $d$ environments (even $d=2$), and considering more general interventions, such as edge weight changes, edge additions/removals, and edge reversals.

Most existing CRL methods [1,2,3] rely on each latent node having a perfect intervention *in at least* one environment. Recent developments such as [4] are capable of recovering $B^{(k)}$ under more general interventions (**none to our knowledge allow edge reversals**) but they still require *at least* $d$ environments and $\Theta(d)$ interventions. To our knowledge, no existing CRL method can address the problem of identifying latent shifted nodes in scenarios with fewer environments, fewer interventions, and under general types of interventions.

[1] Seigal, A., Squires, C. and Uhler, C. [2022], ‘Linear causal disentanglement via interventions’.

[2] Buchholz, S., Rajendran, G., Rosenfeld, E., Aragam, B., Schölkopf, B., \& Ravikumar, P. (2023). Learning linear causal representations from interventions under general nonlinear mixing. Advances in Neural Information Processing Systems, 36.

[3] Ahuja, K., Mahajan, D., Wang, Y., \& Bengio, Y. (2023). Interventional causal representation learning. In International conference on machine learning.

[4] Jin, J., \& Syrgkanis, V. (2023). Learning causal representations from general environments: Identifiability and intrinsic ambiguity. arXiv preprint arXiv:2311.12267.


### Question 2

We believe linear models are good starting points for underexplored settings such as ours, moreover, sometimes linear methods can also serve as rough approximations for nonlinear models. Currently, our method cannot be directly generalized to nonlinear mixing functions. To speculate, one way to address this could be using nonlinear ICA methods; however, a formal proof requires further research and this presents an exciting future direction.

### Question 3

The sample complexity of our algorithm is tied to the sample complexity of ICA since our algorithm uses ICA. Since there are different algorithms for solving ICA, we will assume that the estimated ICA unmixing function has the following statistical accuracy:

If $n \ge g(d,\delta)$, then with probability at least $1 - h(n,d,\delta,\epsilon)$ we have:
$$
l(\hat{M}_i - M_i) \le C \cdot p(d,n)f(\delta),
$$
where $\hat{M}_i$ is the $i$-th row of the estimated unmixing matrix $\hat{M}$. Here, $C$ is a constant, and $p$, $f$, $g$, and $h$ are known functions. For instance, in [5], $p(d,n) = \sqrt{\frac{d}{n}}$ and $f(\delta) = \sqrt{\log(1/\delta)}$. The loss function $l$ can be chosen to be the $L_2$ norm. Therefore, for two environments $k$ and $k'$, if node $i$ is not shifted:

$$
||\hat{M}_i^k - \hat{M}_i^{k'}||_2 \le ||\hat{M}_i^k - M_i||_2 + ||\hat{M}_i^{k'} - M_i||_2 \le 2 \cdot C \cdot p(d,n)f(\delta)
$$

with at least probability $1 - 2h(n,d,\delta,\epsilon)$. Thus, if we set the threshold $\alpha$ in our algorithm to $2 \cdot C \cdot p(d,n)f(\delta)$, we can control the false discovery rate to be at most $2h(n,d,\delta,\epsilon)$. A similar sample complexity theorem can be extended to more than $2$ environments with a similar technique as long as we know the sample complexity of an ICA algorithm. We will add a discussion on this in the revision.

[5] Auddy, A., \& Yuan, M. (2023). Large dimensional independent component analysis: Statistical optimality and computational tractability. arXiv preprint arXiv:2303.18156.

Meta-review: This work considers linear causal representation learning and causal mechanism shifts. Reviewers praised the novelty of the work, and generally viewed it as a significant and innovative contribution to the field. I therefore recommend acceptance.
Decision: Accept (poster)