Original: Conditional Independence Testing with Heteroskedastic Data and Applications to Causal Discovery
Revised: Conditional Independence Testing with Heteroskedastic Data and Applications to Causal Discovery

Original: Abstract
Revised: Abstract

Original: Conditional independence (CI) testing is frequently used in data analysis and machine learning for various scientific fields and it forms the basis of constraint-based causal discovery.
Revised: Conditional independence (CI) testing is frequently used in data analysis and machine learning for various scientific fields and it forms the basis of constraint-based causal discovery.

Original: Oftentimes, CI testing relies on strong, rather unrealistic assumptions.
Revised: Oftentimes, CI testing relies on strong, rather unrealistic assumptions.

Original: One of these assumptions is homoskedasticity, in other words, a constant conditional variance is assumed.
Revised: One of these assumptions is homoskedasticity, in other words, a constant conditional variance is assumed.

Original: We frame heteroskedasticity in a structural causal model framework and present an adaptation of the partial correlation CI test that works well in the presence of heteroskedastic noise, given that expert knowledge about the heteroskedastic relationships is available.
Revised: We frame heteroskedasticity in a structural causal model framework and present an adaptation of the partial correlation CI test that works well in the presence of heteroskedastic noise, given that expert knowledge about the heteroskedastic relationships is available.

Original: Further, we provide theoretical consistency results for the proposed CI test which carry over to causal discovery under certain assumptions.
Revised: Further, we provide theoretical consistency results for the proposed CI test which carry over to causal discovery under certain assumptions.

Original: Numerical causal discovery experiments demonstrate that the adapted partial correlation CI test outperforms the standard test in the presence of heteroskedasticity and is on par for the homoskedastic case.
Revised: Numerical causal discovery experiments demonstrate that the adapted partial correlation CI test outperforms the standard test in the presence of heteroskedasticity and is on par for the homoskedastic case.

Original: Finally, we discuss the general challenges and limits as to how expert knowledge about heteroskedasticity can be accounted for in causal discovery.
Revised: Finally, we discuss the general challenges and limits as to how expert knowledge about heteroskedasticity can be accounted for in causal discovery.

Original:  Introduction
Revised:  Introduction

Original: Conditional independence (CI) testing is a frequently used step across a wide range of machine learning tasks for various scientific fields.
Revised: Conditional independence (CI) testing is a frequently used step across a wide range of machine learning tasks for various scientific fields.

Original: It is also very challenging.
Revised: It is also very challenging.

Original: Discovering causal relationships from purely observational data is an even more challenging task and an important topic in sciences where real experiments are infeasible, e.g. in climate research [Ebert-Uphoff and Deng, 2012, Runge et al., 2019a].
Revised: Discovering causal relationships from purely observational data is an even more challenging task and an important topic in sciences where real experiments are infeasible, e.g. in climate research [Ebert-Uphoff and Deng, 2012, Runge et al., 2019a].

Original: One can distinguish several frameworks that address this problem: score-based approaches [Chickering, 2002], restricted structural causal models [Peters et al., 2017], and constraintbased methods [Spirtes et al., 2000] that rely on CI testing.
Revised: One can distinguish several frameworks that address this problem: score-based approaches [Chickering, 2002], restricted structural causal models [Peters et al., 2017], and constraintbased methods [Spirtes et al., 2000] that rely on CI testing.

Original: A typical representative is the PC algorithm
Revised: A typical representative is the PC algorithm

Original: [Spirtes and Glymour, 1991] which can be combined with any conditional independence test and is thus adaptable to a wide range of data distributions.
Revised: [Spirtes and Glymour, 1991] which can be combined with any conditional independence test and is thus adaptable to a wide range of data distributions.

Original: It utilizes the Faithfulness assumption to conclude that no causal link can exist between two variables X and Y if a CI test suggests that they are independent given a set Z .
Revised: It utilizes the Faithfulness assumption to conclude that no causal link can exist between two variables X and Y if a CI test suggests that they are independent given a set Z .

Original: When a linear additive noise model may be assumed, a popular CI test is the partial correlation test [Lawrance, 1976] which has the advantages of fast computation time and that the null distribution is known analytically.
Revised: When a linear additive noise model may be assumed, a popular CI test is the partial correlation test [Lawrance, 1976] which has the advantages of fast computation time and that the null distribution is known analytically.

Original: As a disadvantage rather strict and often unrealistic assumptions have to be satisfied.
Revised: As a disadvantage rather strict and often unrealistic assumptions have to be satisfied.

Original: One of these assumptions is homoskedasticity, meaning that the variance of the error term is constant.
Revised: One of these assumptions is homoskedasticity, meaning that the variance of the error term is constant.

Original: When this assumption is violated, i.e., in the case of heteroskedasticity, the variance can, for instance, depend on the sampling index or the value of one or multiple influencing variables.
Revised: When this assumption is violated, i.e., in the case of heteroskedasticity, the variance can, for instance, depend on the sampling index or the value of one or multiple influencing variables.

Original: In regression analysis one distinguishes between impure and pure heteroskedasticity.
Revised: In regression analysis one distinguishes between impure and pure heteroskedasticity.

Original: Impure heteroskedasticity stems from an insufficient model specification, e.g. if unobserved variables or confounders are present.
Revised: Impure heteroskedasticity stems from an insufficient model specification, e.g. if unobserved variables or confounders are present.

Original: Another reason might be that the model fails to capture the full relationships of the variables, e.g. the underlying model might be linear with multiplicative noise rather than additive.
Revised: Another reason might be that the model fails to capture the full relationships of the variables, e.g. the underlying model might be linear with multiplicative noise rather than additive.

Original: On the other hand, pure heteroskedasticity is non-constant noise variance that is present despite a correct model.
Revised: On the other hand, pure heteroskedasticity is non-constant noise variance that is present despite a correct model.

Original: The sources for heteroskedasticity in real data are manifold.
Revised: The sources for heteroskedasticity in real data are manifold.

Original: For example, in environmental sciences precipitation in different areas might exhibit different variances that are unaccounted for by other variables in the system, i.e. location-scaled noise.
Revised: For example, in environmental sciences precipitation in different areas might exhibit different variances that are unaccounted for by other variables in the system, i.e. location-scaled noise.

Original: Such a problem could be introduced by aggregating data of different catchments and not adding a variable that is well enough correlated with catchment location [Merz et al., 2021].
Revised: Such a problem could be introduced by aggregating data of different catchments and not adding a variable that is well enough correlated with catchment location [Merz et al., 2021].

Original: An example for sampling index-dependent heteroskedasticity in the time series case are seasonal effects that are present in many climate variables [Proietti, 2004].
Revised: An example for sampling index-dependent heteroskedasticity in the time series case are seasonal effects that are present in many climate variables [Proietti, 2004].

Original: Finally, an example where the noise variance of a variable is dependent on an observed cause is the distance to sea influencing the variability of temperature.
Revised: Finally, an example where the noise variance of a variable is dependent on an observed cause is the distance to sea influencing the variability of temperature.

Original: This is a special case of state-dependent noise.
Revised: This is a special case of state-dependent noise.

Original: One assumption of the PC algorithm is causal sufficiency, meaning that there are no unobserved confounders, formally excluding impure heteroskedasticity.
Revised: One assumption of the PC algorithm is causal sufficiency, meaning that there are no unobserved confounders, formally excluding impure heteroskedasticity.

Original: However, in practice this assumption is often violated which can lead to heteroskedastic noise.
Revised: However, in practice this assumption is often violated which can lead to heteroskedastic noise.

Original: If unaccounted heteroskedasticity is present in the data, the estimator of the ordinary least squares (OLS) regression slope parameter is still unbiased.
Revised: If unaccounted heteroskedasticity is present in the data, the estimator of the ordinary least squares (OLS) regression slope parameter is still unbiased.

Original: However, the estimator of the covariance matrix of the parameter estimates can be biased and inconsistent under heteroskedasticity [Long and Ervin, 2000].
Revised: However, the estimator of the covariance matrix of the parameter estimates can be biased and inconsistent under heteroskedasticity [Long and Ervin, 2000].

Original: This can skew subsequent partial correlation significance tests and affect the link detection rate of a causal discovery method.
Revised: This can skew subsequent partial correlation significance tests and affect the link detection rate of a causal discovery method.

Original: Furthermore, heteroskedasticity might even lead to the detection of wrong links (false positives).
Revised: Furthermore, heteroskedasticity might even lead to the detection of wrong links (false positives).

Original: Moreover, the Gauss-Markov theorem assumes homoskedasticity and, hence, with heteroskedastic data the OLS slope estimator is no longer guaranteed to be the most efficient linear unbiased estimator, which can further harm power of the CI test.
Revised: Moreover, the Gauss-Markov theorem assumes homoskedasticity and, hence, with heteroskedastic data the OLS slope estimator is no longer guaranteed to be the most efficient linear unbiased estimator, which can further harm power of the CI test.

Original: There are multiple ways to treat heteroskedasticity, either already during the modeling step by addressing possible model misspecification, by pre-processing the data, e.g. by applying a logtransform, or post-hoc using robust statistics for CI testing.
Revised: There are multiple ways to treat heteroskedasticity, either already during the modeling step by addressing possible model misspecification, by pre-processing the data, e.g. by applying a logtransform, or post-hoc using robust statistics for CI testing.

Original: Adapting the model, for example, by using CI tests allowing for multiplicative dependencies [Runge, 2018] might not always be feasible for limited sample sizes.
Revised: Adapting the model, for example, by using CI tests allowing for multiplicative dependencies [Runge, 2018] might not always be feasible for limited sample sizes.

Original: Pre-processing in real-world problems also comes with drawbacks: The transformed variables are difficult to interpret and can introduce dependencies or spurious links.
Revised: Pre-processing in real-world problems also comes with drawbacks: The transformed variables are difficult to interpret and can introduce dependencies or spurious links.

Original: In this work we propose an adapted weighted least-squares (WLS) partial correlation variant as a
Revised: In this work we propose an adapted weighted least-squares (WLS) partial correlation variant as a

Original: CI test for the PC algorithm that is able to deal with particular forms of heteroskedasticity.
Revised: CI test for the PC algorithm that is able to deal with particular forms of heteroskedasticity.

Original: Our contributions are theoretical consistency results as well as numerical experiments demonstrating that this approach yields well-calibrated CI tests leading to controlled false positive rates and also improves upon detection power as compared to the standard partial correlation CI test.
Revised: Our contributions are theoretical consistency results as well as numerical experiments demonstrating that this approach yields well-calibrated CI tests leading to controlled false positive rates and also improves upon detection power as compared to the standard partial correlation CI test.

Original: Our approach requires expert knowledge in that it needs to be known which of the variables the heteroskedasticity depends on, or if it depends on the sampling index.
Revised: Our approach requires expert knowledge in that it needs to be known which of the variables the heteroskedasticity depends on, or if it depends on the sampling index.

Original:  Related Work
Revised:  Related Work

Original: The effect of heteroskedasticity on the standard Pearson correlation test has been investigated, for example, by Wilcox and Muska [2001].
Revised: The effect of heteroskedasticity on the standard Pearson correlation test has been investigated, for example, by Wilcox and Muska [2001].

Original: Remedies for heteroskedasticity also have been extensively studied.
Revised: Remedies for heteroskedasticity also have been extensively studied.

Original: Hayes [2007] discuss and evaluate heteroskedasticity-consistent standard errors.
Revised: Hayes [2007] discuss and evaluate heteroskedasticity-consistent standard errors.

Original: Practical approaches to choose weights for WLS, consistency and asymptotic results have been obtained, e.g. by Neumann [1994], Fan and Yao [1998], Carroll [1982], Robinson [1987], Brown and Levine [2007].
Revised: Practical approaches to choose weights for WLS, consistency and asymptotic results have been obtained, e.g. by Neumann [1994], Fan and Yao [1998], Carroll [1982], Robinson [1987], Brown and Levine [2007].

Original: Romano and Wolf [2017] propose a method that combines WLS with heteroskedasticity-consistent standard errors.
Revised: Romano and Wolf [2017] propose a method that combines WLS with heteroskedasticity-consistent standard errors.

Original: Recently, within the causal discovery framework of restricted structural causal models [Peters et al., 2017] several authors [Xu et al., 2022, Tagasovska et al., 2020] have relaxed the assumption of homoskedasticity.
Revised: Recently, within the causal discovery framework of restricted structural causal models [Peters et al., 2017] several authors [Xu et al., 2022, Tagasovska et al., 2020] have relaxed the assumption of homoskedasticity.

Original: In these works the authors focus on identifying cause from effect only in the bivariate setting.
Revised: In these works the authors focus on identifying cause from effect only in the bivariate setting.

Original: Specifically, Xu et al.
Revised: Specifically, Xu et al.

Original: [2022] base their inference score on the log-likelihood of regression residuals and apply a binning-scheme on regions where variance is approximately constant.
Revised: [2022] base their inference score on the log-likelihood of regression residuals and apply a binning-scheme on regions where variance is approximately constant.

Original: More closely related to our work is the robust-PC method of Kalisch and Bühlmann [2008] who use an estimator in a recursive partial correlation formula to robustify the PC algorithm against outliers and otherwise contaminated data.
Revised: More closely related to our work is the robust-PC method of Kalisch and Bühlmann [2008] who use an estimator in a recursive partial correlation formula to robustify the PC algorithm against outliers and otherwise contaminated data.

Original: Non-constant conditional variance can also be regarded as a specific kind of distributional shift or context change.
Revised: Non-constant conditional variance can also be regarded as a specific kind of distributional shift or context change.

Original: The effects of distributional shifts on causal discovery have been investigated, for instance, in Huang et al.
Revised: The effects of distributional shifts on causal discovery have been investigated, for instance, in Huang et al.

Original: [2020], Mooij et al.
Revised: [2020], Mooij et al.

Original: [2020] where the authors propose a framework that includes the environment into the structural causal model formulation using context variables.
Revised: [2020] where the authors propose a framework that includes the environment into the structural causal model formulation using context variables.

Original:  Problem setting 3.1 Heteroskedasticity in causal models
Revised:  Problem setting 3.1 Heteroskedasticity in causal models

Original: In this work, we consider discovering causal relationships in linear models in the presence of nonconstant error variance which is potentially dependent on the parents or on the sampling index.
Revised: In this work, we consider discovering causal relationships in linear models in the presence of nonconstant error variance which is potentially dependent on the parents or on the sampling index.

Original: To translate this into a structural causal model (SCM), we represent heteroskedasticity as a scaling function of the noise variable.
Revised: To translate this into a structural causal model (SCM), we represent heteroskedasticity as a scaling function of the noise variable.

Original: In this way, it can also be viewed as state-dependent or multiplicative noise.
Revised: In this way, it can also be viewed as state-dependent or multiplicative noise.

Original: Consider finitely many random variables V =
Revised: Consider finitely many random variables V =

Original: ( X 1 , . . .
Revised: ( X 1 , . . .

Original: , X d ) with joint distribution P X over a domain X =
Revised: , X d ) with joint distribution P X over a domain X =

Original: X 1 × . . .
Revised: X 1 × . . .

Original: × X d .
Revised: × X d .

Original: Then we are interested in n samples from the following SCM with assignments
Revised: Then we are interested in n samples from the following SCM with assignments

Original: [Equation]
Revised: [Equation]

Original: where f i are linear functions, t ∈ T stands for the sample index, and we have the heteroskedasticity functions h
Revised: where f i are linear functions, t ∈ T stands for the sample index, and we have the heteroskedasticity functions h

Original: [Equation]
Revised: [Equation]

Original: R ≥ 0 .
Revised: ≥ 0 .

Original: The noise variables N i are assumed independent standard Gaussian distributions.
Revised: The noise variables N i are assumed independent standard Gaussian distributions.

Original: The parent set of the variable X it is denoted by Pa ( X it ) , and H ( X it ) ⊂
Revised: The parent set of the variable X it is denoted by Pa ( X it ) , and H ( X it ) ⊂

Original: Pa ( X it ) ∪ { t } which can also be the empty set.
Revised: Pa ( X it ) ∪ { t } which can also be the empty set.

Original: Furthermore, we make the restriction that the causal relationships are stable over time, i.e. the parent sets Pa ( X it ) as well as the functions
Revised: Furthermore, we make the restriction that the causal relationships are stable over time, i.e. the parent sets Pa ( X it ) as well as the functions

Original: f
Revised: f

Original: i are not time-dependent.
Revised: i are not time-dependent.

Original: For h i ≡ const.
Revised: For h i ≡ const.

Original: the homoskedastic version of this SCM is obtained, which simply is a linear SCM with Gaussian noise.
Revised: the homoskedastic version of this SCM is obtained, which simply is a linear SCM with Gaussian noise.

Original: To learn such a causal model from data, a well-known and easy to implement method is to use a constraint-based causal discovery method with CI tests based on partial correlation.
Revised: To learn such a causal model from data, a well-known and easy to implement method is to use a constraint-based causal discovery method with CI tests based on partial correlation.

Original: To recap the partial correlation test and to illustrate our adaptations, consider the following simple example model for a time-series, or otherwise indexed, SCM (1).
Revised: To recap the partial correlation test and to illustrate our adaptations, consider the following simple example model for a time-series, or otherwise indexed, SCM (1).

Original: The general multivariate case is discussed in section 4.2.
Revised: The general multivariate case is discussed in section 4.2.

Original: [Equation]
Revised: [Equation]

Original: for some constants a , b and c , and standard normal independent noise terms N X , N Y , N Z , and N E .
Revised: for some constants a , b and c , and standard normal independent noise terms N X , N Y , N Z , and N E .

Original: The constant c is set to zero for (conditionally) independent X and Y .
Revised: The constant c is set to zero for (conditionally) independent X and Y .

Original: Note that here we are only interested in discovering the relationship between X and Y and not in learning the whole causal graph.
Revised: Note that here we are only interested in discovering the relationship between X and Y and not in learning the whole causal graph.

Original: An intuitive way to define and understand partial correlation is in terms of the correlation between residuals.
Revised: An intuitive way to define and understand partial correlation is in terms of the correlation between residuals.

Original: The partial correlation between X and Y given a controlling variable Z , or a set thereof, is the correlation between the residuals r X and r Y resulting from the linear regression of X on Z and of
Revised: The partial correlation between X and Y given a controlling variable Z , or a set thereof, is the correlation between the residuals r X and r Y resulting from the linear regression of X on Z and of

Original: Y on Z , respectively.
Revised: Y on Z , respectively.

Original: The linear regression that is used in the standard variant of the partial correlation test is ordinary least squares (OLS) regression, thus we refer to this CI test as ParCorr-OLS .
Revised: The linear regression that is used in the standard variant of the partial correlation test is ordinary least squares (OLS) regression, thus we refer to this CI test as ParCorr-OLS .

Original: The
Revised: The

Original: Pearson correlation coefficient ρ is then estimated by(cid:98) ρ ( X, Y | Z ) =
Revised: Pearson correlation coefficient ρ is then estimated by

Original: [Equation]
Revised: [Equation]

Original: (cid:92) V ar ( r X ) (cid:92) V ar ( r Y ) .
Revised: \ V ar ( r Y ) .

Original: We use the hat operator to indicate estimators.
Revised: We use the hat operator to indicate estimators.

Original: For testing the null hypothesis H 0 : ρ = 0 versus the alternative 
Revised: For testing the null hypothesis H 0 : ρ = 0 versus the alternative H : ρ ̸ = 0 , we ˆ ρ

Original: [Equation] [Equation]
Revised: = ˆ [Equation]

Original: This  k is the number of variables we are conditioning 
Revised: This statistic is t ( n − 2 − k -distributed, where k is the number of variables we are conditioning on, and n is the sample size.

Original: However, for the partial correlation test to be correct, the assumptions of OLS, in particular homoskedasticity, have to be fulfilled.
Revised: However, for the partial correlation test to be correct, the assumptions of OLS, in particular homoskedasticity, have to be fulfilled.

Original: In the next section, we investigate what happens if this assumption is violated.
Revised: In the next section, we investigate what happens if this assumption is violated.

Original: 3.2 Effects of heteroskedasticity on partial correlation
Revised: 3.2 Effects of heteroskedasticity on partial correlation

Original: Under heteroskedasticity, the estimator of slope regression parameters in OLS regression is still unbiased.
Revised: Under heteroskedasticity, the estimator of slope regression parameters in OLS regression is still unbiased.

Original: However, the estimator of the covariance matrix of the parameter estimates can be biased and inconsistent under heteroskedasticity [Wooldridge, 2009, p.268].
Revised: However, the estimator of the covariance matrix of the parameter estimates can be biased and inconsistent under heteroskedasticity [Wooldridge, 2009, p.268].

Original: This also affects the subsequent residual-based correlation test in ways that we summarize below.
Revised: This also affects the subsequent residual-based correlation test in ways that we summarize below.

Original: Proofs for and further discussions of the following statements are provided in the Supplement A.1.
Revised: Proofs for and further discussions of the following statements are provided in the Supplement A.1.

Original: Effect 1.
Revised: Effect 1.

Original: Under the null hypothesis, the studentized Pearson correlation coefficient is t ( n − 2 − k ) distributed if X is independent of the variables inducing heteroskedasticity in Y .
Revised: Under the null hypothesis, the studentized Pearson correlation coefficient is t ( n − 2 − k ) distributed if X is independent of the variables inducing heteroskedasticity in Y .

Original: This means that if only one node is compromised by heteroskedasticity, or, more generally, the heteroskedasticity in X is independent of that in Y , the type-I error rate of the t-test will not be affected.
Revised: This means that if only one node is compromised by heteroskedasticity, or, more generally, the heteroskedasticity in X is independent of that in Y , the type-I error rate of the t-test will not be affected.

Original: Please refer to the middle plot in figure 1 for a visualization of the associated null distribution in comparison to the analytical one used by the partial correlation test.
Revised: Please refer to the middle plot in figure 1 for a visualization of the associated null distribution in comparison to the analytical one used by the partial correlation test.

Original: On the other hand, in the left plot the case where both X and Y are affected by the same kind of heteroskedasticity is shown, leading to a different null distribution.
Revised: On the other hand, in the left plot the case where both X and Y are affected by the same kind of heteroskedasticity is shown, leading to a different null distribution.

Original: Effect 2.
Revised: Effect 2.

Original: If X and Y are dependent and at least one is affected by heteroskedasticity, the detection power of the t-test might be degraded.
Revised: If X and Y are dependent and at least one is affected by heteroskedasticity, the detection power of the t-test might be degraded.

Original: In particular, if X and Y are dependent, X is affected by linear heteroskedasticity, the mean of the t -distribution is closer to zero compared to the distribution of weighted least squares based studentized partial correlation coefficient, which is introduced in the next section and essentially transforms the data to be homoskedastic, for fixed sample size.
Revised: In particular, if X and Y are dependent, X is affected by linear heteroskedasticity, the mean of the t -distribution is closer to zero compared to the distribution of weighted least squares based studentized partial correlation coefficient, which is introduced in the next section and essentially transforms the data to be homoskedastic, for fixed sample size.

Original: See the right plot in figure 1.
Revised: See the right plot in figure 1.

Original: This effect is an immediate consequence of the reduced efficiency of the OLS slope estimate.
Revised: This effect is an immediate consequence of the reduced efficiency of the OLS slope estimate.

Original: Intuitively, heteroskedasticity masks the relationship between X and Y .
Revised: Intuitively, heteroskedasticity masks the relationship between X and Y .

Original:  Weighted least squares partial correlation test and causal discovery
Revised:  Weighted least squares partial correlation test and causal discovery

Original: We have seen that the standard partial correlation test is sensitive to heteroskedastic noise since it is based on an OLS regression step.
Revised: We have seen that the standard partial correlation test is sensitive to heteroskedastic noise since it is based on an OLS regression step.

Original: Therefore, we propose to replace the OLS regression by the weighted least squares (WLS) approach which is known to be able to handle non constant error variance.
Revised: Therefore, we propose to replace the OLS regression by the weighted least squares (WLS) approach which is known to be able to handle non constant error variance.

Original: We will refer to the resulting CI test as ParCorr-WLS .
Revised: We will refer to the resulting CI test as ParCorr-WLS .

Original: 4.1 Weighted least squares partial correlation test
Revised: 4.1 Weighted least squares partial correlation test

Original: The idea of WLS is to perform a re-weighting of each data point depending on how far it is from the true regression line.
Revised: The idea of WLS is to perform a re-weighting of each data point depending on how far it is from the true regression line.

Original: It is reasonable to assume data points where the error has low variance to be more informative than those with high error variance.
Revised: It is reasonable to assume data points where the error has low variance to be more informative than those with high error variance.

Original: Therefore, ideally the weights are chosen as the inverse variance of the associated error.
Revised: Therefore, ideally the weights are chosen as the inverse variance of the associated error.

Original: To formalize this idea, consider the linear model y =
Revised: To formalize this idea, consider the linear model y =

Original: Xβ + ε with E[ ε | X ] = 0 , Cov[ ε | X
Revised: Xβ + ε with E[ ε | X ] = 0 , Cov[ ε | X

Original: ] = diag( σ 2 i ) i
Revised: ] = diag( σ 2 i ) i

Original: =1 ,...,n for n observations.
Revised: =1 ,...,n for n observations.

Original: The variance of the error term ε i is denoted by σ 2 i .
Revised: The variance of the error term ε i is denoted by σ 2 i .

Original: Note, as opposed to the assumptions of OLS, the entries of the conditional variance matrix are allowed to differ from each other.
Revised: Note, as opposed to the assumptions of OLS, the entries of the conditional variance matrix are allowed to differ from each other.

Original: Denote the weight matrix by W :=
Revised: Denote the weight matrix by W :=

Original: diag( 1 σ 2 i ) i =1 ,...,n .
Revised: diag( 1 σ 2 i ) i =1 ,...,n .

Original: The WLS method estimates β by solving the adjusted optimization problem
Revised: The WLS method estimates β by solving the adjusted optimization problem

Original: [Equation]
Revised: [Equation]

Original: This objective is quadratic, thus we can write down the solution in closed form
Revised: This objective is quadratic, thus we can write down the solution in closed form

Original: [Equation]
Revised: [Equation]

Original: If the true weights are known, WLS is equivalent to applying OLS to a linearly transformed, homoskedastic version of the data as the weights have the effect of standardizing the scale of the errors.
Revised: If the true weights are known, WLS is equivalent to applying OLS to a linearly transformed, homoskedastic version of the data as the weights have the effect of standardizing the scale of the errors.

Original: Thus, the following lemma holds.
Revised: Thus, the following lemma holds.

Original: Proofs for these statement can be found, for instance, in Greene [2003].
Revised: Proofs for these statement can be found, for instance, in Greene [2003].

Original: Lemma 1.
Revised: Lemma 1.

Original: If the weights are chosen as the reciprocal of the error variance per sample, the WLS estimator is consistent, efficient, and asymptotically normal, as well as BLUE.
Revised: If the weights are chosen as the reciprocal of the error variance per sample, the WLS estimator is consistent, efficient, and asymptotically normal, as well as BLUE.

Original: Moreover, note that the weighted residuals W ( y − X ˆ β ) are homoskedastic.
Revised: Moreover, note that the weighted residuals W ( y − X ˆ β ) are homoskedastic.

Original: The goal now is to approximate the conditional variance function of both X and Y in SCM (2).
Revised: The goal now is to approximate the conditional variance function of both X and Y in SCM (2).

Original: Since it works analogously for both cases, we illustrate the approach for X t =
Revised: Since it works analogously for both cases, we illustrate the approach for X t =

Original: aZ + s ( Z, t ) · N X , i.e. we approximate σ 2 ( z, t ) =
Revised: aZ + s ( Z, t ) · N X , i.e. we approximate σ 2 ( z, t ) =

Original: V ar ( X t | Z
Revised: V ar ( X t | Z

Original: = z ) .
Revised: = z ) .

Original: For that, we use a residual-based non-parametric estimator for the conditional variance, similar to the approach of Robinson [1987].
Revised: For that, we use a residual-based non-parametric estimator for the conditional variance, similar to the approach of Robinson [1987].

Original: Motivated by the identity V ar ( X | Z ) =
Revised: Motivated by the identity V ar ( X | Z ) =

Original: E [( X − E
Revised: E [( X − E

Original: [ X | Z ]) 2 | Z ] , and noting that this is the regression of ( X − aZ ) 2 on Z , the first step is using OLS regression to obtain the squared residuals ( X − ˆ aZ ) 2 .
Revised: [ X | Z ]) 2 | Z ] , and noting that this is the regression of ( X − aZ ) 2 on Z , the first step is using OLS regression to obtain the squared residuals ( X − ˆ aZ ) 2 .

Original: Afterwards, we use a non-parametric regression method to regress these residuals on Z and thereby predict the conditional mean by using a linear combination of the k residuals closest in Z value.
Revised: Afterwards, we use a non-parametric regression method to regress these residuals on Z and thereby predict the conditional mean by using a linear combination of the k residuals closest in Z value.

Original: For sampling index-dependent heteroskedasticity this turns into a windowing approach, which essentially smoothes the squared residuals.
Revised: For sampling index-dependent heteroskedasticity this turns into a windowing approach, which essentially smoothes the squared residuals.

Original: Algorithm 1 details the proposed partial correlation CI test based on the feasible WLS approach that employs our weight approximation method.
Revised: Algorithm 1 details the proposed partial correlation CI test based on the feasible WLS approach that employs our weight approximation method.

Original: For the test to perform well, it is crucial to know the type of heteroskedasticity, more precisely, which of the predictors the variance depends on.
Revised: For the test to perform well, it is crucial to know the type of heteroskedasticity, more precisely, which of the predictors the variance depends on.

Original: In practice, this kind of expert knowledge could be obtained by performing a test for heteroskedasticity, e.g. as suggested in Wooldridge [2009, p.277] or by investigating plots of the residuals.
Revised: In practice, this kind of expert knowledge could be obtained by performing a test for heteroskedasticity, e.g. as suggested in Wooldridge [2009, p.277] or by investigating plots of the residuals.

Original: Here we make the limiting assumption that the heteroskedasticity only depends on one of the predictors or on the sampling index.
Revised: Here we make the limiting assumption that the heteroskedasticity only depends on one of the predictors or on the sampling index.

Original: Further extensions are considered in the section 6.
Revised: Further extensions are considered in the section 6.

Original: Now, we formulate our main assumption under which it is possible to obtain a consistency result for the WLS method that uses our weight approximation method.
Revised: Now, we formulate our main assumption under which it is possible to obtain a consistency result for the WLS method that uses our weight approximation method.

Original: Assumption 1 (Heteroskedastic relationships) .
Revised: Assumption 1 (Heteroskedastic relationships) .

Original: For each node X i , i = 1 , . . .
Revised: For each node X i , i = 1 , . . .

Original: , d in SCM (1) , the skedasticity or noise scaling function h i only depends on one of the predictors H or on the sampling index t , i.e. its domain is one dimensional, or it is constant.
Revised: , d in SCM (1) , the skedasticity or noise scaling function h i only depends on one of the predictors H or on the sampling index t , i.e. its domain is one dimensional, or it is constant.

Original: Furthermore, it is known what it depends on.
Revised: Furthermore, it is known what it depends on.

Original: We also have to impose a rather technical assumption on the functions
Revised: We also have to impose a rather technical assumption on the functions

Original: h i .
Revised: h i .

Original: Assumption 2 (Weight approximation) .
Revised: Assumption 2 (Weight approximation) .

Original: Assumptions (3.3) - (3.5) from Robinson [1987].
Revised: Assumptions (3.3) - (3.5) from Robinson [1987].

Original: The proof of the following lemma can be found in Robinson [1987].
Revised: The proof of the following lemma can be found in Robinson [1987].

Original: Lemma 2.
Revised: Lemma 2.

Original: Under assumptions 1 and 2, the WLS estimator that uses the reciprocal of the approximated variance as weights is consistent and attains the correct covariance matrix asymptotically.
Revised: Under assumptions 1 and 2, the WLS estimator that uses the reciprocal of the approximated variance as weights is consistent and attains the correct covariance matrix asymptotically.

Original: Assumption 3 (Technical assumptions) .
Revised: Assumption 3 (Technical assumptions) .

Original: See Supplement A.2.
Revised: See Supplement A.2.

Original: We now state the first main theorem.
Revised: We now state the first main theorem.

Original: Theorem 1.
Revised: Theorem 1.

Original: Under the assumptions 1 - 3 ParCorr-WLS CI test with estimated weights (Algorithm 1) is consistent for testing the conditional independence between two potentially heteroskedastic variables
Revised: Under the assumptions 1 - 3 ParCorr-WLS CI test with estimated weights (Algorithm 1) is consistent for testing the conditional independence between two potentially heteroskedastic variables

Original: X and Y conditioned on a set of variables Z .
Revised: X and Y conditioned on a set of variables Z .

Original: Algorithm
Revised: Algorithm

Original: Data: Expert knowledge E as map with keys X and Y and values in {false, ‘sampling index’, ‘heteroskedastic parent H ’}, where H ∈ V , observational data with sample size n for nodes X , Y , Z 1 , . . .
Revised: Data: Expert knowledge E as map with keys X and Y and values in {false, ‘sampling index’, ‘heteroskedastic parent H ’}, where H ∈ V , observational data with sample size n for nodes X , Y , Z 1 , . . .

Original: , Z k , H , window length λ , significance level α
Revised: , Z k , H , window length λ , significance level α

Original: Result: boolean indicating whether there is a (conditional) dependence between X and Y given
Revised: Result: boolean indicating whether there is a (conditional) dependence between X and Y given

Original: Z 1 , . .
Revised: Z 1 , . .

Original: , Z k resid :=
Revised: , Z k resid :=

Original: [] ; for node in { X , Y } do if k == 

Revised: [] ; for node in { X , Y } do if k ==

Original: then ˜ r = node else obtain residuals ˜ r
Revised: then ˜ r = node else obtain residuals ˜ r

Original: =
Revised: =

Original: (˜ r 1 , . .
Revised: (˜ r 1 , . .

Original: ˜ r n ) by regressing node on Z 1 , . . .
Revised: ˜ r n ) by regressing node on Z 1 , . . .

Original: , Z k using OLS; endif E [ node ] =
Revised: , Z k using OLS; endif E [ node ] =

Original: = ‘false’ then append ˜ r to resid ; else if E [ node ]
Revised: = ‘false’ then append ˜ r to resid ; else if E [ node ]

Original: ==
Revised: ==

Original: ‘sampling index’ then compute weights w i =
Revised: ‘sampling index’ then compute weights w i =

Original: ( 1 λ (cid:80) min( n,i + λ 2 ) j =max(1 ,i − λ 2 )
Revised: [Equation]

Original: ˜ r 2 j ) − 1 , for i = 1 , . . . , n ; obtain residuals r by regressing node on Z 1 , . . 
Revised: ; obtain residuals r by regressing node on Z 1 , . . .

Original: , Z k using WLS with the weights w ; append w · r to resid ; else sort ˜ r such that their corresponding values of H increase; compute weights w i
Revised: , Z k using WLS with the weights w ; append w · r to resid ; else sort ˜ r such that their corresponding values of H increase; compute weights w i

Original: =
Revised: =

Original: ( 1 λ (cid:80) min( n,i + λ 2 )
Revised: ( 1 λ [Equation]

Original: j =max(1 ,i − λ 2 ) ˜ r 2 j ) − 1 , for i = 1 , . . . , n ; revert the sorting in the indices of w ; obtain residuals r by regressing node on Z 1 , . ..
Revised: , for i = 1 , . . . , n ; revert the sorting in the indices of w ; obtain residuals r by regressing node on Z 1 , . .

Original: , Z k using WLS with the weights w ; append w · r to resid ; end calculate studentized Pearson correlation 
Revised: , Z k using WLS with the weights w ; append w · r to resid ; end calculate studentized Pearson correlation t between r

Original: t between r X = resid [0] and r Y = resid [1] ; perform t -test with (two-sided) significance level α , i.e. reject if | t | > t (1 − α 2 , n − 2 − k )
Revised: X = resid [0] and r Y = resid [1] ; perform t -test with (two-sided) significance level α , i.e. reject if | t | > t (1 − α 2 , n − 2 − k )

Original: 4.2 Extension to the PC algorithm
Revised: 4.2 Extension to the PC algorithm

Original: A well-known and widely used algorithm for discovering causal relationships in terms of the completed partially directed acyclic graph (CPDAG) from observational data is the PC algorithm as introduced in Spirtes and Glymour [1991].
Revised: A well-known and widely used algorithm for discovering causal relationships in terms of the completed partially directed acyclic graph (CPDAG) from observational data is the PC algorithm as introduced in Spirtes and Glymour [1991].

Original: It consists of two phases: The first one is concerned with learning the skeleton of adjacencies based on iterative CI testing.
Revised: It consists of two phases: The first one is concerned with learning the skeleton of adjacencies based on iterative CI testing.

Original: Subsequently, a set of rules is applied to determine the orientation of the found links.
Revised: Subsequently, a set of rules is applied to determine the orientation of the found links.

Original: To ensure consistency of this method, the following assumptions have to be fulfilled.
Revised: To ensure consistency of this method, the following assumptions have to be fulfilled.

Original: Details can be found in Spirtes et al.
Revised: Details can be found in Spirtes et al.

Original: Let G =
Revised: Let G =

Original: ( V, E ) be a graph consisting of a set of vertices V and a set of edges E ⊂ V × V .
Revised: ( V, E ) be a graph consisting of a set of vertices V and a set of edges E ⊂ V × V .

Original: Let P denote the probability distribution of V .
Revised: Let P denote the probability distribution of V .

Original: Assumption 4 (PC algorithm) .
Revised: Assumption 4 (PC algorithm) .

Original: The Causal Markov condition, Causal Faithfulness, and Causal
Revised: The Causal Markov condition, Causal Faithfulness, and Causal

Original: Sufficiency are fulfilled for the SCM (1) with graph G .
Revised: Sufficiency are fulfilled for the SCM (1) with graph G .

Original: Under these assumptions and if the utilized conditional independence test is consistent, it can be shown that the PC algorithm converges in probability to the correct causal structure, i.e.
Revised: Under these assumptions and if the utilized conditional independence test is consistent, it can be shown that the PC algorithm converges in probability to the correct causal structure, i.e.

Original: [Equation]
Revised: [Equation]

Original: where G denotes the ground truth CPDAG and ˆ G n is the finite sample output of the PC algorithm.
Revised: where G denotes the ground truth CPDAG and ˆ G n is the finite sample output of the PC algorithm.

Original: See Kalisch and Bühlmann [2007] for a proof.
Revised: See Kalisch and Bühlmann [2007] for a proof.

Original: In the following, we need to further restrict Assumption 1 to prove consistency of the PC algorithm under heteroskedasticity.
Revised: In the following, we need to further restrict Assumption 1 to prove consistency of the PC algorithm under heteroskedasticity.

Original: Assumption 5 (Heteroskedastic relationships regarding PC algorithm) .
Revised: Assumption 5 (Heteroskedastic relationships regarding PC algorithm) .

Original: Assumption 1 is further limited to the case where there is only sampling index dependent heteroskedasticity or homoskedasticity.
Revised: Assumption 1 is further limited to the case where there is only sampling index dependent heteroskedasticity or homoskedasticity.

Original: Given Assumption 5, we can apply our proposed method ParCorr-WLS in every CI test of the PC algorithm.
Revised: Given Assumption 5, we can apply our proposed method ParCorr-WLS in every CI test of the PC algorithm.

Original: Using Lemma 1, we thus can establish consistency of the PC algorithm with ParCorr-WLS if the true weights are known.
Revised: Using Lemma 1, we thus can establish consistency of the PC algorithm with ParCorr-WLS if the true weights are known.

Original: Lemma 2 yields the same result for ParCorr-WLS with estimated weights.
Revised: Lemma 2 yields the same result for ParCorr-WLS with estimated weights.

Original: Theorem 2.
Revised: Theorem 2.

Original: Under Assumptions 2,3,4,5 the output of the PC algorithm, with ParCorr-WLS as a CI test, converges in probability to the correct causal graph.
Revised: Under Assumptions 2,3,4,5 the output of the PC algorithm, with ParCorr-WLS as a CI test, converges in probability to the correct causal graph.

Original: Note that we prove consistency of the PC algorithm with ParCorr-WLS only in the case of sampling index dependent heteroskedasticity.
Revised: Note that we prove consistency of the PC algorithm with ParCorr-WLS only in the case of sampling index dependent heteroskedasticity.

Original: We discuss the challenges to extending this to more general types of heteroskedasticity in section 6.
Revised: We discuss the challenges to extending this to more general types of heteroskedasticity in section 6.

Original:  Experiments
Revised:  Experiments

Original: In the following, we conduct experiments evaluating our proposed CI test separately and in conjunction with the PC algorithm.
Revised: In the following, we conduct experiments evaluating our proposed CI test separately and in conjunction with the PC algorithm.

Original: Throughout the experiments, heteroskedasticity strength refers to the parameter s in the scaling functions h of linear and periodic type given by
Revised: Throughout the experiments, heteroskedasticity strength refers to the parameter s in the scaling functions h of linear and periodic type given by

Original: [Equation]
Revised: [Equation]

Original: In other words, in case of linear heteroskedasticity strength is the slope of the variance function, and for periodic heteroskedasticity it refers to the amplitude of the variance function.
Revised: In other words, in case of linear heteroskedasticity strength is the slope of the variance function, and for periodic heteroskedasticity it refers to the amplitude of the variance function.

Original: 5.1 Conditional independence testing
Revised: 5.1 Conditional independence testing

Original: We generate the data from the SCM (2) where we consider various types of heteroskedasticity, i.e. functions h , namely linear and periodic as given in Eq.
Revised: We generate the data from the SCM (2) where we consider various types of heteroskedasticity, i.e. functions h , namely linear and periodic as given in Eq.

Original: Plots of the simulated data are provided in the Supplement.
Revised: Plots of the simulated data are provided in the Supplement.

Original: Each of the types can either be Z - or sampling index-dependent.
Revised: Each of the types can either be Z - or sampling index-dependent.

Original: A visualization of the considered heteroskedasticity-types can be found in the Supplement A.5.
Revised: A visualization of the considered heteroskedasticity-types can be found in the Supplement A.5.

Original: We use the Kolmogorov-Smirnov (KS) statistic to quantify how uniform the distribution of p-values is, and therefore as a metric for type-I errors, as in Runge [2018].
Revised: We use the Kolmogorov-Smirnov (KS) statistic to quantify how uniform the distribution of p-values is, and therefore as a metric for type-I errors, as in Runge [2018].

Original: Type-II errors are measured by the area under the power curve (AUPC).
Revised: Type-II errors are measured by the area under the power curve (AUPC).

Original: The metrics were evaluated with a sample size of 500 fromrealizations of theSCM.
Revised: The metrics were evaluated with a sample size of 500 fromrealizations of the

Original: Error bars indicate the bootstrapped standard errors. Figure 2 shows that ParCorr-WLS is well calibrated in the presence of heteroskedasticity, regardless if it affects only one or both of the variables X and Y .
Revised: Figure 2 shows that ParCorr-WLS is well calibrated in the presence of heteroskedasticity, regardless if it affects only one or both of the variables X and Y .

Original: On the other hand, the ParCorr-OLS test becomes ill-calibrated in the case of heteroskedastic X and Y as expected by Effect 1.
Revised: On the other hand, the ParCorr-OLS test becomes ill-calibrated in the case of heteroskedastic X and Y as expected by Effect 1.

Original: In particular, this means that if we can choose the weights reasonably well, we are able to overcome multiplicative confounding with our proposed CI test.
Revised: In particular, this means that if we can choose the weights reasonably well, we are able to overcome multiplicative confounding with our proposed CI test.

Original: Regarding power as measured by AUPC, we observe a rather rapid decrease for ParCorr-OLS as the heteroskedasticity increases (compare to Effect 2).
Revised: Regarding power as measured by AUPC, we observe a rather rapid decrease for ParCorr-OLS as the heteroskedasticity increases (compare to Effect 2).

Original: Our proposed method ParCorr-WLS has higher power in the heteroskedastic scenario, and even for homoskedastic noise (heteroskedasticity strength equal zero) the power is comparable to that of ParCorr-OLS.
Revised: Our proposed method ParCorr-WLS has higher power in the heteroskedastic scenario, and even for homoskedastic noise (heteroskedasticity strength equal zero) the power is comparable to that of ParCorr-OLS.

Original: The drop in power for ParCorr-WLS can be explained by an overall increase in noisiness of the data as heteroskedasticity strength is growing.
Revised: The drop in power for ParCorr-WLS can be explained by an overall increase in noisiness of the data as heteroskedasticity strength is growing.

Original: Refer to the Supplement A.5 for a plot of AUPC of ParCorr-OLS and ParCorr-WLS on data with homoskedastic but increasing noise.
Revised: Refer to the Supplement A.5 for a plot of AUPC of ParCorr-OLS and ParCorr-WLS on data with homoskedastic but increasing noise.

Original: Similar effects are present for all considered types of heteroskedasticity (see Supplement).
Revised: Similar effects are present for all considered types of heteroskedasticity (see Supplement).

Original: The results remain very similar if we do not use the ground truth weights but estimate them using the window approach with a reasonable window length as detailed in Section 4.1.
Revised: The results remain very similar if we do not use the ground truth weights but estimate them using the window approach with a reasonable window length as detailed in Section 4.1.

Original: 5.2 Causal discovery
Revised: 5.2 Causal discovery

Original: To test our proposed CI test in a more realistic setting, we apply it within the PC algorithm to recover the causal graph from simulated observational data.
Revised: To test our proposed CI test in a more realistic setting, we apply it within the PC algorithm to recover the causal graph from simulated observational data.

Original: We build upon the PC-stable algorithm implementation within the Tigramite software package [Runge et al., 2019b] which is published under the GNU General Public License.
Revised: We build upon the PC-stable algorithm implementation within the Tigramite software package [Runge et al., 2019b] which is published under the GNU General Public License.

Original: The data for these experiments was generated with SCM (1) in the following way.
Revised: The data for these experiments was generated with SCM (1) in the following way.

Original: Given a random ground truth graph, we fix a percentage of heteroskedasticity-affected nodes which are then selected uniformly at random.
Revised: Given a random ground truth graph, we fix a percentage of heteroskedasticity-affected nodes which are then selected uniformly at random.

Original: Throughout the experiments this percentage is set to 0 .
Revised: Throughout the experiments this percentage is set to 0 .

Original: 3 to reduce the chance of parent and child being affected by the same kind of heteroskedasticity.
Revised: 3 to reduce the chance of parent and child being affected by the same kind of heteroskedasticity.

Original: For these affected nodes, we choose as a heteroskedasticity type linear or periodic with equal probability (Eq.
Revised: For these affected nodes, we choose as a heteroskedasticity type linear or periodic with equal probability (Eq.

Original: and let the noise variance h either depend on one randomly selected parent or the sampling index.
Revised: and let the noise variance h either depend on one randomly selected parent or the sampling index.

Original: We also set a fixed strength s per experiment and investigated the effect of increasing the strength on the performance of our method compared to the PC algorithm with ParCorr-OLS.
Revised: We also set a fixed strength s per experiment and investigated the effect of increasing the strength on the performance of our method compared to the PC algorithm with ParCorr-OLS.

Original: All linear dependencies have a coefficient c = 0 .
Revised: All linear dependencies have a coefficient c = 0 .

Original: We tested for various small to medium sized causal graphs (see also Supplement).
Revised: We tested for various small to medium sized causal graphs (see also Supplement).

Original: In these experiments, we estimate the weights based on the expert knowledge as required by Assumption 1, namely which node is affected by heteroskedasticity, and whether the noise variance depends on the sampling index or another node.
Revised: In these experiments, we estimate the weights based on the expert knowledge as required by Assumption 1, namely which node is affected by heteroskedasticity, and whether the noise variance depends on the sampling index or another node.

Original: In the case of the heteroskedasticity depending on another node, we also need to know which node it is.
Revised: In the case of the heteroskedasticity depending on another node, we also need to know which node it is.

Original: We also compare with the PC algorithm which uses ParCorr-WLS based on the ground truth weights for all direct heteroskedastic relations.
Revised: We also compare with the PC algorithm which uses ParCorr-WLS based on the ground truth weights for all direct heteroskedastic relations.

Original: Note, however, that this does not take care of indirect heteroskedasticity due to heteroskedastic parents that are not part of the conditioning set.
Revised: Note, however, that this does not take care of indirect heteroskedasticity due to heteroskedastic parents that are not part of the conditioning set.

Original: Note that this challenging setting is outside of the stricter
Revised: Note that this challenging setting is outside of the stricter

Original: Assumption 5 for which the PC algorithm is consistent.
Revised: Assumption 5 for which the PC algorithm is consistent.

Original: Figure 3 shows, similar to the experiments of the previous section, that the PC algorithm with ParCorrWLS continues to have a rather small false positive rate (FPR) even though the heteroskedasticity strength increases.
Revised: Figure 3 shows, similar to the experiments of the previous section, that the PC algorithm with ParCorrWLS continues to have a rather small false positive rate (FPR) even though the heteroskedasticity strength increases.

Original: In contrast, ParCorr-OLS shows an increase in FPR as the heteroskedasticity strength increases.
Revised: In contrast, ParCorr-OLS shows an increase in FPR as the heteroskedasticity strength increases.

Original: Additionally, even in this rather complicated setting, we see that the true positive rate (TPR) can be improved by using our method.
Revised: Additionally, even in this rather complicated setting, we see that the true positive rate (TPR) can be improved by using our method.

Original: The PC algorithm with ParCorr-WLS has an average runtime of 0 .
Revised: The PC algorithm with ParCorr-WLS has an average runtime of 0 .

Original: 29 seconds on homoskedastic data compared to 0 .
Revised: 29 seconds on homoskedastic data compared to 0 .

Original: 14 seconds for ParCorr-OLS evaluated on AMD 7763.
Revised: 14 seconds for ParCorr-OLS evaluated on AMD 7763.

Original:  Discussion and Outlook
Revised:  Discussion and Outlook

Original: In this work we relaxed the common assumption of a constant variance by explicitly allowing for heteroskedasticity as a multiplicative scaling of noise in an otherwise linear SCM.
Revised: In this work we relaxed the common assumption of a constant variance by explicitly allowing for heteroskedasticity as a multiplicative scaling of noise in an otherwise linear SCM.

Original: Our proposed partial correlation test based on weighted least squares regression is a linear, computationally fast and easy to implement method and constitutes a useful standalone method for various data science tasks, but our focus here is on its use in causal discovery.
Revised: Our proposed partial correlation test based on weighted least squares regression is a linear, computationally fast and easy to implement method and constitutes a useful standalone method for various data science tasks, but our focus here is on its use in causal discovery.

Original: The main strengths of our approach are that it is able to produce more reliable results both as a CI test and in causal discovery in the presence of heteroskedasticity than the standard partial correlation variant.
Revised: The main strengths of our approach are that it is able to produce more reliable results both as a CI test and in causal discovery in the presence of heteroskedasticity than the standard partial correlation variant.

Original: More reliable results here refers to controlled false positives as well as higher detection power.
Revised: More reliable results here refers to controlled false positives as well as higher detection power.

Original: Furthermore, the suggested adaptations do not compromise calibratedness and power of the CI test on homoskedastic data.
Revised: Furthermore, the suggested adaptations do not compromise calibratedness and power of the CI test on homoskedastic data.

Original: The main weakness of our method is that the CI test requires substantial expert knowledge in Assumption 1 and for the consistency of the PC algorithm the even stricter Assumption 5.
Revised: The main weakness of our method is that the CI test requires substantial expert knowledge in Assumption 1 and for the consistency of the PC algorithm the even stricter Assumption 5.

Original: Assumptionrequires that the heteroskedasticity only depends on one of the predictors or on the sampling index, i.e. its domain is one dimensional.
Revised: Assumptionrequires that the heteroskedasticity only depends on one of the predictors or on the sampling index, i.e. its domain is one dimensional.

Original: Furthermore, one needs to know which of these types is the case.
Revised: Furthermore, one needs to know which of these types is the case.

Original: Assumption 5 only allows for sampling-index heteroskedasticity.
Revised: Assumption 5 only allows for sampling-index heteroskedasticity.

Original: The reason for this is that there can be indirect heteroskedasticity at the node X or Y that is not induced by a parent of X or Y , but by some other ancestor.
Revised: The reason for this is that there can be indirect heteroskedasticity at the node X or Y that is not induced by a parent of X or Y , but by some other ancestor.

Original: This heteroskedasticity then propagates through the causal graph and essentially makes X or Y heteroskedastic whenever this path is not blocked by the conditioning set.
Revised: This heteroskedasticity then propagates through the causal graph and essentially makes X or Y heteroskedastic whenever this path is not blocked by the conditioning set.

Original: In this case, the expert knowledge does not tell us about this heteroskedasticity and we would not be able to apply
Revised: In this case, the expert knowledge does not tell us about this heteroskedasticity and we would not be able to apply

Original: ParCorr-WLS to remove it.
Revised: ParCorr-WLS to remove it.

Original: See also Section A.4 in the Supplement for a detailed treatment of cases in which the CI tests within the PC algorithm are consistent under more general heteroskedasticity forms.
Revised: See also Section A.4 in the Supplement for a detailed treatment of cases in which the CI tests within the PC algorithm are consistent under more general heteroskedasticity forms.

Original: In the following, we discuss alternatives and further avenues of research.
Revised: In the following, we discuss alternatives and further avenues of research.

Original: In future work one may alter the required expert knowledge and weight approximation scheme to overcome the issues induced by indirect heteroskedasticity.
Revised: In future work one may alter the required expert knowledge and weight approximation scheme to overcome the issues induced by indirect heteroskedasticity.

Original: Here, a possible remedy could be an iterative approach using information about causal relationships from earlier steps.
Revised: Here, a possible remedy could be an iterative approach using information about causal relationships from earlier steps.

Original: An alternative would be to use the CMIknn CI test [Runge, 2018] that is able to fully treat heteroskedastic multiplicative confounding.
Revised: An alternative would be to use the CMIknn CI test [Runge, 2018] that is able to fully treat heteroskedastic multiplicative confounding.

Original: However, this increased generality comes at the price of reduced detection power, also see Figure 10 in the Supplement.
Revised: However, this increased generality comes at the price of reduced detection power, also see Figure 10 in the Supplement.

Original: Another open question is how to further improve the weight approximation method.
Revised: Another open question is how to further improve the weight approximation method.

Original: For instance, by iteratively repeating the regression and smoothing steps.
Revised: For instance, by iteratively repeating the regression and smoothing steps.

Original: Another important consideration is the choice of the window length.
Revised: Another important consideration is the choice of the window length.

Original: Linear properties of the variance function allow us to use a larger window length.
Revised: Linear properties of the variance function allow us to use a larger window length.

Original: However, if the variance function shows a high variability, crucial information might be lost if the chosen window length is too large.
Revised: However, if the variance function shows a high variability, crucial information might be lost if the chosen window length is too large.

Original: Model selection criteria might be employed to alleviate this problem.
Revised: Model selection criteria might be employed to alleviate this problem.

Original: Furthermore, one can consider extensions of ParCorr-WLS to multiple heteroskedastic influencing factors, e.g., Spokoiny [2002] extend the residual-based conditional variance function estimation to dimensions larger than one.
Revised: Furthermore, one can consider extensions of ParCorr-WLS to multiple heteroskedastic influencing factors, e.g., Spokoiny [2002] extend the residual-based conditional variance function estimation to dimensions larger than one.

Original: Another multidimensional method based on differences is discussed in
Revised: Another multidimensional method based on differences is discussed in

Original: Cai et al.
Revised: Cai et al.

Original: It would also be interesting to explore using generalized least squares to be able to account for additional correlation of the residuals.
Revised: It would also be interesting to explore using generalized least squares to be able to account for additional correlation of the residuals.

Original: Potentially, the weighted least squares partial correlation coefficient could also be combined with a permutation-based test to extend the method to problems with non-Gaussian noise.
Revised: Potentially, the weighted least squares partial correlation coefficient could also be combined with a permutation-based test to extend the method to problems with non-Gaussian noise.

Original: Concluding , our proposed ParCorr-WLS CI test makes constraint-based causal discovery methods better applicable to real world problems where the assumption of homoskedasticity is violated, such as in climate research or neuroscience.
Revised: Concluding , our proposed ParCorr-WLS CI test makes constraint-based causal discovery methods better applicable to real world problems where the assumption of homoskedasticity is violated, such as in climate research or neuroscience.

Original: Ethically, we believe that our rather fundamental work has a low potential for misuse.
Revised: Ethically, we believe that our rather fundamental work has a low potential for misuse.

Original: Acknowledgments and Disclosure of Funding
Revised: Acknowledgments and Disclosure of Funding

Original: WG was supported by the Helmholtz AI project CausalFlood .
Revised: WG was supported by the Helmholtz AI project CausalFlood .

Original: UN and JW were supported by grant no. 948112 Causal Earth of the European Research Council (ERC).
Revised: UN and JW were supported by grant no. 948112 Causal Earth of the European Research Council (ERC).

Original: This work used resources of the
Revised: This work used resources of the

Original: Deutsches Klimarechenzentrum (DKRZ) granted by its Scientific Steering Committee (WLA) under project ID bd1083.
Revised: Deutsches Klimarechenzentrum (DKRZ) granted by its Scientific Steering Committee (WLA) under project ID bd1083.

Original: We thank the anonymous reviewers for their helpful comments
Revised: We thank the anonymous reviewers for their helpful comments

Original: Checklist
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] See Sections 1 and 6, as well as the assumptions, specifically Assumption 1.
Revised: [Yes] See Sections 1 and 6, as well as the assumptions, specifically Assumption 1.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] See Section 6.
Revised: [Yes] See Section 6.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 2.
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] See Assumptions 1-5.
Revised: [Yes] See Assumptions 1-5.

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes] We provide references and intuitive justifications for the results in the main text and complete proofs in the Supplement. 3.
Revised: [Yes] We provide references and intuitive justifications for the results in the main text and complete proofs in the Supplement. 3.

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] The code and instructions on how to execute it are provided in the supplemental material.
Revised: [Yes] The code and instructions on how to execute it are provided in the supplemental material.

Original: The code includes a method for generating the synthetic data.
Revised: The code includes a method for generating the synthetic data.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] All hyperparameters are listed in Section 5.
Revised: [Yes] All hyperparameters are listed in Section 5.

Original: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] See Figures 2 and 3.
Revised: [Yes] See Figures 2 and 3.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] See Section 5. 4.
Revised: [Yes] See Section 5. 4.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] See Section 5.
Revised: [Yes] See Section 5.

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [Yes] See Section 5.
Revised: [Yes] See Section 5.

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes] The code is included in the supplemental material.
Revised: [Yes] The code is included in the supplemental material.

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [N/A] 5.
Revised: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?[N/A] A Supplement
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: Here we provide proofs of the statements made in the main text as well as further figures of numerical experiments and a more detailed discussion of heteroskedasticity effects regarding causal discovery.

Original: A.1 Proof of Effects 1 andA.1.1 Proof of Effect 1:

Original: Proof.

Original: Let ( X i , Y i ) i

Original: =1 ,...,n be an independent sample with Pearson correlation coefficient ρ , and we assume the linear model Y i

Original: =

Original: X i β

Original: + h

Original: ( Z i )

Original: ϵ i , where Z i and ϵ

Original: i are independent and standard normal, and h is the noise scaling function.

Original: Note that w.l.o.g.

Original: we assume only Y to be heteroskedastic w.r.t.

Original: Z .

Original: Testing whether the Pearson correlation between X and Y is zero is equivalent to testing whether the slope parameter β is equal to zero.

Original: Under the null hypothesis, calculating Var( h ( Z i )

Original: ϵ i |

Original: X i ) gives

Original: [Equation]

Original: using the independence of Z i and X i , the fact that ϵ i is standard normal, and that the Z i are identically distributed.

Original: Therefore, this is a homoskedastic problem.

Original: A.1.2 Discussion of Effect 2:

Original: We start by discussing the homoskedastic case to see where non-constant variance of noise leads to problems within the t-test.

Original: Let ( X i , Y i ) i

Original: =1 ,...,n be an independent sample from a bivariate normal distribution with Pearson correlation coefficient ρ , and we assume the linear model Y i

Original: =

Original: [-X i β 
-]
Revised: [-X i β 
-]

Original: + ϵ i , where ϵ i is standard normal, then

Original: [Equation]

Original: This also is true for the finite sample estimators of these entities, since

Original: [Equation]

Original: and

Original: [Equation]

Original: Furthermore, the following relationship holds for the estimator of the studentized correlation coefficient and the estimator of the slope parameter β

Original: [Equation]

Original: For homoskedastic noise the second factor is an estimator of the standard error of ˆ β , which we derive by using the mean of the squared residual as an estimator for the error variance.

Original: We know that

Original: [Equation]

Original: and thus

Original: Var

Original: [Equation]

Original: If we assume the design to be fixed and Var( ϵ i )

Original: =

Original: σ 2 to be constant, this simplifies to

Original: [Equation]

Original: Therefore we use the following estimator

Original: [Equation]

Original: We now assume that Y is heteroskedastic, i.e. the model takes the form

Original: [Equation]

Original: We need to show

Original: [Equation]

Original: where ˆ β OLS is the OLS estimator for the slope parameter and ˆ β WLS is its WLS version.

Original: Since OLS and WLS generally return similar values for the slope estimator (especially for large n ) and both are unbiased estimators for β 0 , we focus on the denominator.

Original: We get E

Original: [Equation]

Original: from the reduced efficiency of the OLS regression on heteroskedastic data, i.e. from

Original: [Equation]

Original: if we assume that the estimators (6) for Var(ˆ β OLS ) and Var(ˆ β WLS ) are unbiased.

Original: The unbiasedness is clear for the WLS-case, as well as for the case where the heteroskedasticity in X is independent from the one in Y , see Effect 1.

Original: In the case of heteroskedasticity in Y that is dependent on X , the estimator might be biased.

Original: If it is over-estimating Var( β ) , the power is also reduced.

Original: The situation of under-estimating the variance can, in fact, quasi-increase power but comes with the major drawback of increased probability of type I error.

Original: A.2 Proof of TheoremA.2.1 Assumptions

Original: Consider random variables X

Original: =

Original: ( X 1 , . . .

Original: , X n ) T , Y =

Original: ( Y 1 , . . .

Original: , Y n ) T and Z k =

Original: ( Z k 1 , . . .

Original: , Z kn ) T for k = 1 , . . .

Original: , d , where n is the number of samples.

Original: Denote the design matrix of the regressor variable by Z =

Original: ( Z 1 , . . .

Original: , Z d ) ∈ R n × d .

Original: We assume that all variables are centered Gaussians and generated according to the model

Original: [Equation]

Original: with error variables ε X

Original: =

Original: ( ε X 1 , . .

Original: . , ε Xn ) T , ε Y =

Original: ( ε Y 1 , . . .

Original: , ε Yn ) T .

Original: We make the following assumptions:

Original: (A1)

Original: The error vectors ε X , ε Y have zero mean and diagonal covariance matrices Σ X = diag( σ 2 X, 1 , . . .

Original: , σ 2 X,n ) , Σ Y = diag( σ 2 Y, 1 , . . .

Original: , σ 2 Y,n ) .

Original: Similarly, for any k , the regressor variables Z k =

Original: ( Z k 1 , . . .

Original: , Z kn ) T have diagonal covariance matrices.

Original: (A2)

Original: The matrix Z T Z is almost surely positive definite, of full rank d , and we have that lim n →∞

Original: [Equation]

Original: T

Original: [Equation]

Original: converges entrywise in probability to a matrix Q .

Original: Note that the k, ℓ -entry of Z T Z is (cid:80) ni

Original: =1 Z ki Z ℓi .

Original: (A3)

Original: We have well defined asymptotic average variances

Original: [Equation]

Original: and the limits

Original: [Equation]

Original: for

Original: k =

Original: [Equation]

Original: d exist in probability.

Original: (A4)

Original: The partial correlations

Original: [Equation]

Original: are constant, i.e. independent of the sampling index i .

Original: Moreover, conditional covariances between different sample indices are zero.

Original: (A5)

Original: The variances of each variable are bounded by a constant independent of the sampling index i .

Original: (A6)

Original: For parent-dependent heteroskedasticity, let H be the heteroskedasticity-inducing parent.

Original: Denote by S i the set of λ nearest neighbours of X i (or Y i respectively) in H -value.

Original: In the case of sampling index dependent heteroskedasticity, S i denotes the set of nearest neighbours in sampling index value.

Original: We assume

Original: [Equation]

Original: Furthermore, the limit of the variances averaged over the nearest neighbours exists and converges to the right value, i.e.

Original: [Equation]

Original: A.2.2 Proof

Original: We will now sketch the proof of consistency of the estimator for the partial correlation ρ X,Y | Z in the case where we regress on only one variable, i.e. d = 1 .

Original: The result can however easily be generalized to higher dimensions although the computations become slightly more involved.

Original: Recall that in the first step of ParCorr-WLS, we compute the residuals

Original: [Equation]

Original: where ˆ α, ˆ β are OLS-estimates of α, β .

Original: More precisely, recall that ˆ α is given by the formula

Original: [Equation]
Revised: [N/A]

Original: Hence, using Slutsky’s theorem, we compute that

Original: [Equation]

Original: which converges in probability as 1 n ( C · Z 2 i + σ 2 X,i )

Original: + σ 2 X,i where C is a constant.

Original: Recall the definition of the set of nearest neighbours S i from assumption (A6).

Original: Following the results in Hall and Carroll [1989], we establish the convergence of ˆ σ 2 X,i = 1 λ (cid:80) r

Original: j ∈S i r 2 j .

Original: Furthermore, we see that the smoothed variances of the residuals behave asymptotically as

Original: [Equation]

Original: After estimation of the variances, we determine the residuals of the WLS regression with weight matrix ˆ W X which is equivalent to the OLS regression after scaling all variables by ˆ W X .

Original: In other words, we define

Original: [Equation]

Original: where ˜ α is the OLS-estimator w.r.t.

Original: the regressor ˆ W X Z and the error variable ˆ W X ˜ ε .

Original: Define ˆ W Y and

Original: R Yi analogously.

Original: The following asymptotic moment formulas can be computed from the definition of R X i , R Y i .

Original: Lemma 3.

Original: Assume (A1)-(A6).

Original: Then E [ R Xi | Z ] = 0 and for large n

Original: [Equation]

Original: where p is a polynomial of degree 4 .

Original: [-In particular, by (A2),(A5) and the fact that for Gaussians higher moments are functions of the variance, Var( (cid:0) R Xi (cid:1) 2 ) ≤ K for some constant K > 
-]
Revised: [-In particular, by (A2),(A5) and the fact that for Gaussians higher moments are functions of the variance, Var( (cid:0) R Xi (cid:1) 2 ) ≤ K for some constant K > 
-]

Original: independent of i .

Original: The same statements hold for the residuals R Yi .

Original: Lemma 4.

Original: Assume (A1)-(A6).

Original: Then, the estimator

Original: [Equation]

Original: consistently estimates ρ X,Y | Z .

Original: Proof.

Original: By Slutsky’s theorem, it suffices to prove that the three sums in the definition of ˆ ρ X,

Original: Y

Original: | Z converge in probability and to compute the limits.

Original: Convergence follows from the law of large numbers and the variance bounds Var( (cid:0) R Xi (cid:1) 2 ) ≤ K, Var( (cid:0) R Yi (cid:1) 2 ) ≤ K ′ of Lemma 3.

Original: The limit of the numerator is equal to lim n →∞ 1 n (cid:80) ni

Original: =1 E [ R Xi R Yi ] , so we compute this quantity first conditioned on Z .

Original: This yields for large n

Original: [Equation]

Original: Hence lim n →∞ 1 n (cid:80) ni =1 E

Original: [ R Xi R Yi ]

Original: =

Original: ρ X,Y | Z .

Original: To compute lim n →∞ 1 n (cid:80) ni

Original: =1 E

Original: [( R Xi ) 2 ] we use the first formula of Lemma 3 to see that

Original: [Equation]

Original: so that lim n →∞ 1 n (cid:80) ni =1 E [

Original: [Equation]

Original: R Xi

Original: [Equation]

Original: 2 ] = 1 .

Original: A similar computation yields lim n →∞ 1 n (cid:80) ni

Original: =1 E

Original: [( R Yi ) 2 ] = 1 , so

Original: that indeed ˆ ρ X,Y | Z → ρ X,Y | Z as n → ∞ in probability.

Original: A.3 Proof of Theorem

Original: Proof.

Original: Assumption 5 allows us to apply ParCorr-WLS in every conditional independence test within the PC algorithm.

Original: Using Theorem 1, we get the result.

Original: A.4 Extension of Theorem 2 to different heteroskedasticity forms

Original: As discussed in section 4.2, the PC algorithm with ParCorr-OLS is not consistent.

Original: If we use ParCorrWLS with a consistent estimate for the variance of the residual in every conditional independence test, we can make the PC algorithm consistent.

Original: The scope of the current work is a situation where heteroskedastic noise can be dependent on one of the parents of the node or on the sampling index.

Original: However, only for the stricter set of assumptions for

Original: Theorem 2 we can prove consistency, namely only for sample-index dependent heteroskedasticity.

Original: In the following, we will discuss different situations of heteroskedasticity affecting variables involved in the CI tests within the PC algorithm.

Original: In some of these situations, we are able to apply

Original: our proposed

Original: ParCorr-WLS CI test, thus obtaining consistency.

Original: However, in other situations we are not able to use background knowledge about heteroskedasticity and therefore suffer the same problems as in the standard version of the PC algorithm with ParCorr-OLS.

Original: Let us look at the following cases.

Original: Case 1: The heteroskedasticity in X is independent of that in Y .

Original: In case of direct heteroskedasticity, we apply ParCorr-WLS.

Original: However, if we encounter a situation where the heteroskedasticity in either X or Y is caused indirectly through a parent, compare figure 4, we lack the background knowledge about it, and therefore apply ParCorr-OLS.

Original: Due to Effect 1, we get type I error control for both ParCorr-OLS and ParCorr-WLS.

Original: With regard to type II errors, by Effect 2 it is clear that the power of ParCorr-OLS is reduced.

Original: In a setting where the sample size is large enough to get a reasonable weight approximation for WLS, we can only gain detection power by using ParCorr-WLS within the PC algorithm.

Original: Case 2: There is dependence between the heteroskedasticity in X and that in Y .

Original: If X and Y are uncorrelated, i.e. ρ X,Y | Z = 0 , and the heteroskedasticity of at least one of them is directly induced by a parent or by the sampling index, then we apply WLS whereby we remove the heteroskedasticity and are in Case 1 or in the homoskedastic case again.

Original: If the heteroskedasticity is indirect, meaning one of the parents of X and/or Y , that is not included in the conditioning set, is heteroskedastic, our expert knowledge doesn’t include this information and we perform ParCorr-OLS.

Original: By Effect 1, we know that this suffers from inflated false positives.

Original: However, if the link is not removed in this step of the PC algorithm, the conditioning set will be increased in subsequent steps.

Original: This means at some point it will include the parent that caused the indirect heteroskedasticity, if the link did not get removed before.

Original: When this happens we again will be in Case 1 or in the homoskedastic case.

Original: The main challenge, or in other words the reason why we cannot prove consistency for this more general form of heteroskedasticity, arises if X and Y are correlated.

Original: If at least one of them is affected by direct heteroskedasticity, then we again apply WLS whereby we remove the heteroskedasticity and are in Case 1 or in the homoskedastic case as above.

Original: But, if the heteroskedasticity is indirect in both X and Y , then we apply ParCorr-OLS.

Original: Since the power for this test is very low in this situation, and we do not obtain a consistent estimator for the standard deviation of the regression parameter, we potentially remove the link between X and Y .

Original: A.5 Additional Plots