Original: Conditional Independence Testing with Heteroskedastic Data and Applications to Causal Discovery
Revised: Conditional Independence Testing with Heteroskedastic Data and Applications to Causal Discovery

Original: Anonymous Author(s)

Original: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: Conditional independence (CI) testing is frequently used in data analysis and machine learning for various scientiﬁc ﬁelds and it forms the basis of constraint-basedcausal discovery.
Revised: Conditional independence (CI) testing is frequently used in data analysis and machine learning for various scientific fields and it forms the basis of constraint-basedcausal discovery.

Original: Oftentimes, CI testing relies on strong, rather unrealistic assumptions.
Revised: Oftentimes, CI testing relies on strong, rather unrealistic assumptions.

Original: One of these assumptions is homoskedasticity, in other words, a constantconditional variance is assumed.
Revised: One of these assumptions is homoskedasticity, in other words, a constantconditional variance is assumed.

Original: We frame heteroskedasticity in a structural causalmodel framework and present an adaptation of the partial correlation CI test thatworks well in the presence of heteroskedastic noise, given that expert knowledgeabout the heteroskedastic relationships is available.
Revised: We frame heteroskedasticity in a structural causalmodel framework and present an adaptation of the partial correlation CI test thatworks well in the presence of heteroskedastic noise, given that expert knowledgeabout the heteroskedastic relationships is available.

Original: Further, we provide theoreticalconsistency results for the proposed CI test which carry over to causal discoveryunder certain assumptions.
Revised: Further, we provide theoreticalconsistency results for the proposed CI test which carry over to causal discoveryunder certain assumptions.

Original: Numerical causal discovery experiments demonstratethat the adapted partial correlation CI test outperforms the standard test in thepresence of heteroskedasticity and is on par for the homoskedastic case.
Revised: Numerical causal discovery experiments demonstratethat the adapted partial correlation CI test outperforms the standard test in thepresence of heteroskedasticity and is on par for the homoskedastic case.

Original: Finally,we discuss the general challenges and limits as to how expert knowledge aboutheteroskedasticity can be accounted for in causal discovery.
Revised: Finally,we discuss the general challenges and limits as to how expert knowledge aboutheteroskedasticity can be accounted for in causal discovery.

Original:  Introduction
Revised:  Introduction

Original: Conditional independence (CI) testing is a frequently used step across a wide range of machinelearning tasks for various scientiﬁc ﬁelds.
Revised: Conditional independence (CI) testing is a frequently used step across a wide range of machinelearning tasks for various scientific fields.

Original: It is also very challenging.
Revised: It is also very challenging.

Original: Discovering causal relationshipsfrom purely observational data is an even more challenging task and an important topic in scienceswhere real experiments are infeasible, e.g. in climate research [Ebert-Uphoff and Deng, 2012, Rungeet al., 2019a].
Revised: Discovering causal relationshipsfrom purely observational data is an even more challenging task and an important topic in scienceswhere real experiments are infeasible, e.g. in climate research [Ebert-Uphoff and Deng, 2012, Rungeet al., 2019a].

Original: One can distinguish several frameworks that address this problem: score-basedapproaches [Chickering, 2002], restricted structural causal models [Peters et al., 2017] and constraintbased methods [Spirtes et al., 2000] that rely on CI testing.
Revised: One can distinguish several frameworks that address this problem: score-basedapproaches [Chickering, 2002], restricted structural causal models [Peters et al., 2017], and constraintbased methods [Spirtes et al., 2000] that rely on CI testing.

Original: A typical representative is the PCalgorithm [Spirtes and Glymour, 1991] which can be combined with any conditional independencetest and is thus adaptable to a wide range of data distributions.
Revised: A typical representative is the PCalgorithm [Spirtes and Glymour, 1991] which can be combined with any conditional independencetest and is thus adaptable to a wide range of data distributions.

Original: It utilizes the Faithfulness assumptionto conclude that no causal link can exist between two variables X and Y if a CI test suggests thatthey are independent given a set Z .
Revised: It utilizes the Faithfulness assumptionto conclude that no causal link can exist between two variables X and Y if a CI test suggests thatthey are independent given a set Z .

Original: When a linear additive noise model may be assumed, a popular CI test is the partial correlation test[Lawrance, 1976] which has the advantages of fast computation time and that the null distributionis known analytically.
Revised: When a linear additive noise model may be assumed, a popular CI test is the partial correlation test[Lawrance, 1976] which has the advantages of fast computation time and that the null distributionis known analytically.

Original: As a disadvantage rather strict and often unrealistic assumptions have to besatisﬁed.
Revised: As a disadvantage rather strict and often unrealistic assumptions have to besatisfied.

Original: One of these assumptions is homoskedasticity, meaning that the variance of the error termis constant.
Revised: One of these assumptions is homoskedasticity, meaning that the variance of the error termis constant.

Original: When this assumption is violated, i.e., in the case of heteroskedasticity, the variance can,for instance, depend on the sampling index or the value of one or multiple inﬂuencing variables.
Revised: When this assumption is violated, i.e., in the case of heteroskedasticity, the variance can,for instance, depend on the sampling index or the value of one or multiple influencing variables.

Original: In regression analysis one distinguishes between impure and pure heteroskedasticity.
Revised: In regression analysis one distinguishes between impure and pure heteroskedasticity.

Original: Impure heteroskedasticity stems from an insufﬁcient model speciﬁcation, e.g. if unobserved variables or con- founders are present.
Revised: Impure heteroskedasticity stems from an insufficient model specification, e.g. if unobserved variables or con- founders are present.

Original: Another reason might be that the model fails to capture the full relationships ofthe variables, e.g. the underlying model might be linear with multiplicative noise rather than additive.
Revised: Another reason might be that the model fails to capture the full relationships ofthe variables, e.g. the underlying model might be linear with multiplicative noise rather than additive.

Original: On the other hand, pure heteroskedasticity is non-constant noise variance that is present despite acorrect model.
Revised: On the other hand, pure heteroskedasticity is non-constant noise variance that is present despite acorrect model.

Original: The sources for heteroskedasticity in real data are manifold.
Revised: The sources for heteroskedasticity in real data are manifold.

Original: For example, in environmental sciencesprecipitation in different areas might exhibit different variances that are unaccounted for by othervariables in the system, i.e. location-scaled noise.
Revised: For example, in environmental sciencesprecipitation in different areas might exhibit different variances that are unaccounted for by othervariables in the system, i.e. location-scaled noise.

Original: Such a problem could be introduced by aggregatingdata of different catchments and not adding a variable that is well enough correlated with catchmentlocation [Merz et al., 2021].
Revised: Such a problem could be introduced by aggregatingdata of different catchments and not adding a variable that is well enough correlated with catchmentlocation [Merz et al., 2021].

Original: An example for sampling index-dependent heteroskedasticity in the timeseries case are seasonal effects that are present in many climate variables [Proietti, 2004].
Revised: An example for sampling index-dependent heteroskedasticity in the timeseries case are seasonal effects that are present in many climate variables [Proietti, 2004].

Original: Finally, anexample where the noise variance of a variable is dependent on an observed cause is the distance tosea inﬂuencing the variability of temperature.
Revised: Finally, anexample where the noise variance of a variable is dependent on an observed cause is the distance tosea influencing the variability of temperature.

Original: This is a special case of state-dependent noise.
Revised: This is a special case of state-dependent noise.

Original: One assumption of the PC algorithm is causal sufﬁciency, meaning that there are no unobservedconfounders, formally excluding impure heteroskedasticity.
Revised: One assumption of the PC algorithm is causal sufficiency, meaning that there are no unobservedconfounders, formally excluding impure heteroskedasticity.

Original: However, in practice this assumption isoften violated which can lead to heteroskedastic noise.
Revised: However, in practice this assumption isoften violated which can lead to heteroskedastic noise.

Original: If unaccounted heteroskedasticity is present in the data, the estimator of the ordinary least squares(OLS) regression slope parameter is still unbiased.
Revised: If unaccounted heteroskedasticity is present in the data, the estimator of the ordinary least squares(OLS) regression slope parameter is still unbiased.

Original: However, the estimator of the covariance matrixof the parameter estimates can be biased and inconsistent under heteroskedasticity [Long and Ervin,2000].
Revised: However, the estimator of the covariance matrixof the parameter estimates can be biased and inconsistent under heteroskedasticity [Long and Ervin,2000].

Original: This can skew subsequent partial correlation signiﬁcance tests and affect the link detectionrate of a causal discovery method.
Revised: This can skew subsequent partial correlation significance tests and affect the link detectionrate of a causal discovery method.

Original: Furthermore, heteroskedasticity might even lead to the detectionof wrong links (false positives).
Revised: Furthermore, heteroskedasticity might even lead to the detectionof wrong links (false positives).

Original: Moreover, the Gauss-Markov theorem assumes homoskedasticityand, hence, with heteroskedastic data the OLS slope estimator is no longer guaranteed to be the mostefﬁcient linear unbiased estimator, which can further harm power of the CI test.
Revised: Moreover, the Gauss-Markov theorem assumes homoskedasticityand, hence, with heteroskedastic data the OLS slope estimator is no longer guaranteed to be the mostefficient linear unbiased estimator, which can further harm power of the CI test.

Original: There are multiple ways to treat heteroskedasticity, either already during the modeling step byaddressing possible model misspeciﬁcation, by pre-processing the data, e.g. by applying a logtransform, or post-hoc using robust statistics for CI testing.
Revised: There are multiple ways to treat heteroskedasticity, either already during the modeling step byaddressing possible model misspecification, by pre-processing the data, e.g. by applying a logtransform, or post-hoc using robust statistics for CI testing.

Original: Adapting the model, for example, byusing CI tests allowing for multiplicative dependencies [Runge, 2018] might not always be feasiblefor limited sample sizes.
Revised: Adapting the model, for example, byusing CI tests allowing for multiplicative dependencies [Runge, 2018] might not always be feasiblefor limited sample sizes.

Original: Pre-processing in real-world problems also comes with drawbacks:
Revised: Pre-processing in real-world problems also comes with drawbacks:

Original: Thetransformed variables are difﬁcult to interpret and can introduce dependencies or spurious links.
Revised: Thetransformed variables are difficult to interpret and can introduce dependencies or spurious links.

Original: In this work we propose an adapted weighted least-squares (WLS) partial correlation variant as a
Revised: In this work we propose an adapted weighted least-squares (WLS) partial correlation variant as a

Original: CI test for the PC algorithm that is able to deal with particular forms of heteroskedasticity.
Revised: CI test for the PC algorithm that is able to deal with particular forms of heteroskedasticity.

Original: Ourcontributions are theoretical consistency results as well as numerical experiments demonstratingthat this approach yields well-calibrated CI tests leading to controlled false positive rates and alsoimproves upon detection power as compared to the standard partial correlation CI test.
Revised: Ourcontributions are theoretical consistency results as well as numerical experiments demonstratingthat this approach yields well-calibrated CI tests leading to controlled false positive rates and alsoimproves upon detection power as compared to the standard partial correlation CI test.

Original: Our approachrequires expert knowledge in that it needs to be known which of the variables the heteroskedasticitydepends on, or if it depends on the sampling index.
Revised: Our approachrequires expert knowledge in that it needs to be known which of the variables the heteroskedasticitydepends on, or if it depends on the sampling index.

Original:  Related Work
Revised:  Related Work

Original: The effect of heteroskedasticity on the standard Pearson correlation test has been investigated, forexample, by Wilcox and Muska [2001].
Revised: The effect of heteroskedasticity on the standard Pearson correlation test has been investigated, forexample, by Wilcox and Muska [2001].

Original: Remedies for heteroskedasticity also have been extensivelystudied.
Revised: Remedies for heteroskedasticity also have been extensivelystudied.

Original: Hayes [2007] discuss and evaluate heteroskedasticity-consistent standard errors.
Revised: Hayes [2007] discuss and evaluate heteroskedasticity-consistent standard errors.

Original: Practicalapproaches to choose weights for WLS, consistency and asymptotic results have been obtained, e.g.by Neumann [1994], Fan and Yao [1998], Carroll [1982], Robinson [1987], Brown and Levine [2007].
Revised: Practicalapproaches to choose weights for WLS, consistency and asymptotic results have been obtained, e.g.by Neumann [1994], Fan and Yao [1998], Carroll [1982], Robinson [1987], Brown and Levine [2007].

Original: Romano and Wolf [2017] propose a method that combines WLS with heteroskedasticity-consistentstandard errors.
Revised: Romano and Wolf [2017] propose a method that combines WLS with heteroskedasticity-consistentstandard errors.

Original: Recently, within the causal discovery framework of restricted structural causal models [Peters et al.
Revised: Recently, within the causal discovery framework of restricted structural causal models [Peters et al.

Original: ,2017] several authors [Xu et al., 2022, Tagasovska et al., 2020] have relaxed the assumption ofhomoskedasticity.
Revised: ,2017] several authors [Xu et al., 2022, Tagasovska et al., 2020] have relaxed the assumption ofhomoskedasticity.

Original: In these works the authors focus on identifying cause from effect only in thebivariate setting
Revised: In these works the authors focus on identifying cause from effect only in thebivariate setting.

Original:  Xu et al.
Revised: Specifically, Xu et al.

Original: [ basetheir inference score on the log-likelihood of regression residuals and apply a binning-scheme on regions where variance is approximately constant
Revised: [2022] base

Original: Non-constant conditional variance can also be regarded as a speciﬁc kind of distributional shift orcontext change.
Revised: Non-constant conditional variance can also be regarded as a specific kind of distributional shift orcontext change.

Original: The effects of distributional shifts on causal discovery have been investigated, forinstance, in Huang et al.
Revised: The effects of distributional shifts on causal discovery have been investigated, forinstance, in Huang et al.

Original: [2020], Mooij et al.
Revised: [2020], Mooij et al.

Original: [2020] where the authors propose a framework thatincludes the environment into the structural causal model formulation using context variables.
Revised: [2020] where the authors propose a framework thatincludes the environment into the structural causal model formulation using context variables.

Original:  Problem setting
Revised:  Problem setting

Original: 3.1 Heteroskedasticity in causal models
Revised: 3.1 Heteroskedasticity in causal models

Original: In this work, we consider discovering causal relationships in linear models in the presence of nonconstant error variance which is potentially dependent on the parents or on the sampling index.
Revised: In this work, we consider discovering causal relationships in linear models in the presence of nonconstant error variance which is potentially dependent on the parents or on the sampling index.

Original: Totranslate this into a structural causal model (SCM), we represent heteroskedasticity as a scalingfunction of the noise variable.
Revised: Totranslate this into a structural causal model (SCM), we represent heteroskedasticity as a scalingfunction of the noise variable.

Original: In this way, it can also be viewed as state-dependent or multiplicativenoise.
Revised: In this way, it can also be viewed as state-dependent or multiplicativenoise.

Original: Consider ﬁnitely many random variables V =
Revised: Consider finitely many random variables V =

Original: ( X 1 , . . .
Revised: ( X 1 , . . .

Original: , X d ) with joint distribution P X over adomain X 
Revised: , X d ) with joint distribution P X over adomain X =

Original: = X 1 ⇥ . . .
Revised: X 1 × . . .

Original: ⇥ X d .
Revised: × X d .

Original: Then we are interested in n samples from the following SCM with 99 assignments
Revised: Then we are interested in n samples from the following SCM with 99 assignments

Original: [Equation]
Revised: [Equation]

Original: where f i are linear functions, t 2 T stands for the sample index, and we have the heteroskedasticityfunctions
Revised: where f i are linear functions, t ∈ [0 , ∞ ) stands for the sample index, and we have the heteroskedasticity functions

Original: h i : X ⇥ T ! R  0 .
Revised: h i : X × [0 , ∞ ) −→ R ≥ 0 .

Original: The noise variables N i are assumed independent standard Gaussiandistributions.
Revised: The noise variables N i are assumed independentstandard Gaussian distributions.

Original: The parent set of the variable X it is denoted by Pa ( X it ) , and H ( X it ) 
Revised: The parent set of the variable X it is denoted by Pa ( X it ) , and H ( X it ) ⊂

Original: ⇢ Pa ( X it ) [ { t }which can also be the empty set.
Revised: Pa ( X it ) ∪ { t } which can also be the empty set.

Original: Furthermore, we make the restriction that  the causal relationshipsare stable over time, i.e. the parent sets Pa ( X it ) as well as the functions f i are not time-dependent.
Revised: Furthermore, we make the restriction that 104 the causal relationships are stable over time, i.e. the parent sets Pa ( X it ) as well as the functions f iare not time-dependent.

Original: For h i ⌘ const.
Revised: For h i ≡ const.

Original: the homoskedastic version of this SCM is obtained, which simply is a linear SCMwith Gaussian noise.
Revised: the homoskedastic version of this SCM is obtained, which simply is a linear SCMwith Gaussian noise.

Original: To learn such a causal model from data, a well-known and easy to implementmethod is to use a constraint-based causal discovery method with CI tests based on partial correlation.
Revised: To learn such a causal model from data, a well-known and easy to implementmethod is to use a constraint-based causal discovery method with CI tests based on partial correlation

Original: To recap the partial correlation test and to illustrate our adaptations, consider the following simpleexample model for a time-series, or otherwise indexed, SCM (1).
Revised: To recap the partial correlation test and to illustrate our adaptations, consider the following simpleexample model for a time-series, or otherwise indexed, SCM (1).

Original: The general multivariate case isdiscussed in section 4.2.
Revised: The general multivariate case isdiscussed in section 4.2.

Original: [Equation]
Revised: [Equation]

Original: for some constants a , b and c , and standard normal independent noise terms N X , N Y , N Z , and N E .
Revised: for some constants a , b and c , and standard normal independent noise terms N X , N Y , N Z , and N E .

Original: The constant c is set to zero for (conditionally) independent X and Y .
Revised: The constant c is set to zero for (conditionally) independent X and Y .

Original: Note that here we are onlyinterested in discovering the relationship between X and Y and not in learning the whole causalgraph.
Revised: Note that here we are onlyinterested in discovering the relationship between X and Y and not in learning the whole causalgraph.

Original: An intuitive way to deﬁne and understand partial correlation is in terms of the correlation betweenresiduals.
Revised: An intuitive way to define and understand partial correlation is in terms of the correlation betweenresiduals.

Original: The partial correlation between X and Y given a controlling variable Z , or a set thereof, isthe correlation between the residuals r X and r Y resulting from the linear regression of X on Z and of
Revised: The partial correlation between X and Y given a controlling variable Z , or a set thereof, isthe correlation between the residuals r X and r Y resulting from the linear regression of X on Z and of

Original: Y on Z , respectively.
Revised: Y on Z , respectively.

Original: The linear regression that is used in the standard variant of the partial correlationtest is ordinary least squares (OLS) regression, thus we refer to this CI test as ParCorr-OLS .
Revised: The linear regression that is used in the standard variant of the partial correlationtest is ordinary least squares (OLS) regression, thus we refer to this CI test as ParCorr-OLS .

Original: The
Revised: The

Original: Pearson correlation coefﬁcient ⇢ is then estimated by
Revised: Pearson correlation coefficient ρ is then estimated by

Original: [Equation]
Revised: [Equation]

Original: We use thehat operator to indicate estimators.
Revised: We use hatoperator to indicate estimators.

Original: For testing the null hypothesis H 0 : ⇢ = 0 versus the alternative
Revised: For testing the null hypothesis H 0 : ρ = 0 versus the alternative

Original: H 1 : ⇢ 6 = 0 , we use the studentized version of the partial correlation as a test statistic T (ˆ ⇢ ) =
Revised: H 1 : ρ ̸ = 0 , we use the studentized version of the partial correlation as a test statistic T (ˆ ρ ) =

Original: ˆ ⇢ p n  2 p ˆ⇢ 2 .
Revised: ˆ ρ √ n − 2 √− ˆ

Revised: [Equation]

Original: This statistic is t ( n  2  k ) -distributed, where k is the number of variables we are conditioning on,and n is the sample size.
Revised: This statistic is t ( n − 2 − k ) -distributed, where k is the number of variables we are conditioning on,and n is the sample size.

Original: However, for the partial correlation test to be correct, the assumptions of OLS, in particular homoskedasticity, have to be fulﬁlled.
Revised: However, for the partial correlation test to be correct, the assumptions of OLS, in particular homoskedasticity, have to be fulfilled.

Original: In the next section, we investigate what happens if this assumptionis violated.
Revised: In the next section, we investigate what happens if this assumptionis violated.

Original: 3.2 Effects of heteroskedasticity on partial correlation
Revised: 3.2 Effects of heteroskedasticity on partial correlation

Original: Under heteroskedasticity, the estimator of slope regression parameters in OLS regression is stillunbiased.
Revised: Under heteroskedasticity, the estimator of slope regression parameters in OLS regression is stillunbiased.

Original: However, the estimator of the covariance matrix of the parameter estimates can be biased and inconsistent under heteroskedasticity [Wooldridge, 2009, p.268].
Revised: However, the estimator of the covariance matrix of the parameter estimates can be biasedand inconsistent under heteroskedasticity [Wooldridge, 2009, p.268].

Original: This also affects the subsequentresidual-based correlation test in ways that we summarize below.
Revised: This also affects the subsequentresidual-based correlation test in ways that we summarize below.

Original: Proofs for the following statementsare provided in the Supplement A.1.
Revised: Proofs for the following statementsare provided in the Supplement A.1.

Original: Effect 1.
Revised: Effect 1.

Original: Under the null hypothesis, the studentized Pearson correlation coefﬁcient is t ( n  2  k )distributed if the heteroskedasticity in X is independent of the one in Y .
Revised: Under the null hypothesis, the studentized Pearson correlation coefficient is t ( n − 2 − k ) 136 distributed if the heteroskedasticity in X is independent of the one in Y .

Original: This means that if only one node is compromised by heteroskedasticity, or, more generally, theheteroskedasticity in X is independent of that in Y , the type-I error rate of the t-test will not beaffected.
Revised: This means that if only one node is compromised by heteroskedasticity, or, more generally, theheteroskedasticity in X is independent of that in Y , the type-I error rate of the t-test will not beaffected.

Original: Please refer to the middle plot in ﬁgure 1 for a visualization of the associated null distributionin comparison to the analytical one used by the partial correlation test.
Revised: Please refer to the middle plot in figure 1 for a visualization of the associated null distributionin comparison to the analytical one used by the partial correlation test.

Original: On the other hand, in theleft plot the case where both X and Y are affected by the same kind of heteroskedasticity is shown,leading to a different null distribution.
Revised: On the other hand, in theleft plot the case where both X and Y are affected by the same kind of heteroskedasticity is shown,leading to a different null distribution.

Original: Effect 2.
Revised: Effect 2.

Original: If X and Y are dependent and at least one is affected by heteroskedasticity, the detectionpower of the t-test is degraded.
Revised: If X and Y are dependent and at least one is affected by heteroskedasticity, the detectionpower of the t-test is degraded.

Original: In particular, if the heteroskedasticity terms in X and Y are dependent, the mean of the t -distributionis closer to zero compared to the distribution of weighted least squares based studentized partialcorrelation coefﬁcient, which is introduced in the next section and essentially transforms the data tobe homoskedastic, for ﬁxed sample size.
Revised: In particular, if the heteroskedasticity terms in X and Y are dependent, the mean of the t -distributionis closer to zero compared to the distribution of weighted least squares based studentized partialcorrelation coefficient, which is introduced in the next section and essentially transforms the data tobe homoskedastic, for fixed sample size.

Original: See the right plot in ﬁgure 1.
Revised: See the right plot in figure 1.

Original: This effect is an immediateconsequence of the reduced efﬁciency of the OLS slope estimate.
Revised: This effect is an immediateconsequence of the reduced efficiency of the OLS slope estimate.

Original: Intuitively, heteroskedasticitymasks the relationship between X and Y .
Revised: Intuitively, heteroskedasticitymasks the relationship between X and Y .

Original:  Weighted least squares partial correlation test and causal discovery
Revised:  Weighted least squares partial correlation test and causal discovery

Original: We have seen that the standard partial correlation test is sensitive to heteroskedastic noise since itis based on an OLS regression step.
Revised: We have seen that the standard partial correlation test is sensitive to heteroskedastic noise since itis based on an OLS regression step.

Original: Therefore, we propose to replace the OLS regression by theweighted least squares (WLS) approach which is known to be able to handle non constant errorvariance.
Revised: Therefore, we propose to replace the OLS regression by theweighted least squares (WLS) approach which is known to be able to handle non constant errorvariance.

Original: We will refer to the resulting CI test as ParCorr-WLS .
Revised: We will refer to the resulting CI test as ParCorr-WLS .

Original: 4.1 Weighted least squares partial correlation test
Revised: 4.1 Weighted least squares partial correlation test

Original: The idea of WLS is to perform a re-weighting of each data point depending on how far it is from thetrue regression line.
Revised: The idea of WLS is to perform a re-weighting of each data point depending on how far it is from thetrue regression line.

Original: It is reasonable to assume data points where the error has low variance to bemore informative than those with high error variance.
Revised: It is reasonable to assume data points where the error has low variance to bemore informative than those with high error variance.

Original: Therefore, ideally the weights are chosen asthe inverse variance of the associated error.
Revised: Therefore, ideally the weights are chosen asthe inverse variance of the associated error.

Original: To formalize this idea, consider the linear model y 
Revised: To formalize this idea, consider the linear model y =

Original: = X  + " with E[ " | X ] = 0 , Cov[ " | X ] =diag(  2 i ) i =1 ,...,n for n observations.
Revised: Xβ + ε with E[ ε | X ] = 0 , Cov[ ε | X ] =diag( σ 2 i ) i =1 ,...,n for n observations.

Original: The variance of the error term " i is denoted by  2 i .
Revised: The variance of the error term ε i is denoted by σ 2 i .

Original: Note, asopposed to the assumptions of OLS, the entries of the conditional variance matrix are allowed todiffer from each other.
Revised: Note, asopposed to the assumptions of OLS, the entries of the conditional variance matrix are allowed to differ from each other.

Original: Denote the weight matrix by W :=
Revised: Denote the weight matrix by W :=

Original: diag( 1  2 i ) i =1 ,...,n .
Revised: diag( 1 σ 2 i ) i =1 ,...,n .

Original: The WLS methodestimates  by solving the adjusted optimization problem
Revised: The WLS methodestimates β by solving the adjusted optimization problem

Original: [Equation]
Revised: [Equation]

Original: This objective is quadratic, thus we can write down the solution in closed form
Revised: This objective is quadratic, thus we can write down the solution in closed form

Original: [Equation]
Revised: [Equation]

Original: If the true weights are known, WLS is equivalent to applying OLS to a linearly transformed, homoskedastic version of the data as the weights have the effect of standardizing the scale of the errors.
Revised: If the true weights are known, WLS is equivalent to applying OLS to a linearly transformed, homoskedastic version of the data as the weights have the effect of standardizing the scale of the errors.

Original: Thus, the following lemma holds.
Revised: Thus, the following lemma holds.

Original: Proofs for these statement can be found, for instance, in Greene[2003].
Revised: Proofs for these statement can be found, for instance, in Greene[2003].

Original: Lemma 1.
Revised: Lemma 1.

Original: If the weights are chosen as the reciprocal of the error variance per sample, the WLSestimator is consistent, efﬁcient, and asymptotically normal, as well as BLUE.
Revised: If the weights are chosen as the reciprocal of the error variance per sample, the WLSestimator is consistent, efficient, and asymptotically normal, as well as BLUE.

Original: Moreover, note that the weighted residuals W ( y  X ˆ  ) are homoskedastic.
Revised: Moreover, note that the weighted residuals W ( y − X ˆ β ) are homoskedastic.

Original: The goal now is to approximate the conditional variance function of both X and Y in SCM (2).
Revised: The goal now is to approximate the conditional variance function of both X and Y in SCM (2).

Original: Sinceit works analogously for both cases, we illustrate the approach for X t =
Revised: Since it works analogously for both cases, we illustrate the approach for X t =

Original: aZ + s ( Z, t ) · N X , i.e. weapproximate  2 ( z, t ) =
Revised: aZ + s ( Z, t ) · N X ,i.e. we approximate σ 2 ( z, t ) =

Original: V ar ( X t | Z
Revised: V ar ( X t | Z

Original: = z ) .
Revised: = z ) .

Original: For that, we use a residual-based non-parametric estimator for the conditional variance, similar to theapproach of Robinson [1987].
Revised: For that, we use a residual-based non-parametricestimator for the conditional variance, similar to the approach of Robinson [1987].

Original: Motivated by the identity V ar ( X| Z ) =
Revised: Motivated by theidentity V ar ( X ) =

Original: E [( X  E
Revised: E [( X − E

Original: [ X| Z ]) 2| Z ] ,andnoting that this is the regression of ( X  aZ ) 2 on Z , the ﬁrst step is using OLS regression to obtainthe squared residuals ( X  ˆ aZ ) 2 .
Revised: [ X ]) 2 ] , the first step is using OLS regression to obtain the squaredresiduals ( X − ˆ aZ ) 2 .

Original: Afterwards, weuse a non-parametric regression method to regressthese residuals on Z and thereby predict the conditional meanby using a linear combination of thek residuals closest in Z value. For time-dependent heteroskedasticity this turns into a windowingapproach, which essentially smoothes the squared residuals.
Revised: Afterwards, we predict the conditional mean using a windowing approach,which essentially smoothes the squared residuals.

Original: Algorithm 1 details the proposed partial correlation CI test based on the feasible WLS approachthat employs our weight approximation method.
Revised: Algorithm 1 details the proposed partial correlation CI test based on the feasible WLS approachthat employs our weight approximation method.

Original: For the test to perform well, it is crucial to knowthe type of heteroskedasticity, more precisely, which of the predictors the variance depends on.
Revised: For the test to perform well, it is crucial to knowthe type of heteroskedasticity, more precisely, which of the predictors the variance depends on.

Original: Inpractice, this kind of expert knowledge could be obtained by performing a test for heteroskedasticity,e.g. as suggested in Wooldridge [2009, p.277] or by investigating plots of the residuals.
Revised: Inpractice, this kind of expert knowledge could be obtained by performing a test for heteroskedasticity,e.g. as suggested in Wooldridge [2009, p.277] or by investigating plots of the residuals.

Original: Here wemake the limiting assumption that the heteroskedasticity only depends on one of the predictors or onthe sampling index.
Revised: Here wemake the limiting assumption that the heteroskedasticity only depends on one of the predictors or onthe sampling index.

Original: Further extensions are considered in the section 6.
Revised: Further extensions are considered in the section 6.

Original: Now, we formulate our main assumption under which it is possible to obtain a consistency result forthe WLS method that uses our weight approximation method.
Revised: Now, we formulate our main assumption under which it is possible to obtain a consistency result forthe WLS method that uses our weight approximation method.

Original: Assumption 1 (Heteroskedastic relationships) .
Revised: Assumption 1 (Heteroskedastic relationships) .

Original: For each node X i , i = 1 , . . .
Revised: For each node X i , i = 1 , . . .

Original: , d in SCM (1) , theskedasticity or noise scaling function h i only depends on one of the predictors H or on the samplingindex t , i.e. its domain is one dimensional, or it is constant.
Revised: , d in SCM (1) , theskedasticity or noise scaling function h i only depends on one of the predictors H or on the samplingindex t , i.e. its domain is one dimensional, or it is constant.

Original: Furthermore, it is known what it dependson.
Revised: Furthermore, it is known what it dependson.

Original: We also have to impose a rather technical assumption on the functions
Revised: We also have to impose a rather technical assumption on the functions

Original: h i .
Revised: h i .

Original: Assumption 2 (Weight approximation) .
Revised: Assumption 2 (Weight approximation) .

Original: Assumptions (3.3) - (3.5) from Robinson [1987].
Revised: Assumptions (3.3) - (3.5) from Robinson [1987].

Original: The proof of the following lemma can be found in Robinson [1987].
Revised: The proof of the following lemma can be found in Robinson [1987].

Original: Lemma 2.
Revised: Lemma 2.

Original: Under assumptions 1 and 2, the WLS estimator that uses the reciprocal of the approximated variance as weights is consistent and attains the correct covariance matrix asymptotically.
Revised: Under assumptions 1 and 2, the WLS estimator that uses the reciprocal of the approximated variance as weights is consistent and attains the correct covariance matrix asymptotically.

Original: Assumption 3 (Technical assumptions) .
Revised: Assumption 3 (Technical assumptions) .

Original: See Supplement A.2.
Revised: See Supplement A.2.

Original: We now state the ﬁrst main theorem.
Revised: We now state the first main theorem.

Original: Theorem 1.
Revised: Theorem 1.

Original: Under the assumptions 1 - 3 ParCorr-WLS CI test with estimated weights (Algorithm 1) isconsistent for testing the conditional independence between two potentially heteroskedastic variables
Revised: Under the assumptions 1 - 3 ParCorr-WLS CI test with estimated weights (Algorithm 1) isconsistent for testing the conditional independence between two potentially heteroskedastic variables

Original: X and Y conditioned on a set of variables Z .
Revised: X and Y conditioned on a set of variables Z .

Original: 4.2 Extension to the PC algorithm
Revised: 2 Extension to the PC algorithm

Original: A well-known and widely used algorithm for discovering causal relationships in terms of the completed partially directed acyclic graph (CPDAG) from observational data is the PC algorithm asintroduced in Spirtes and Glymour [1991].
Revised: A well-known and widely used algorithm for discovering causal relationships in terms of the completed partially directed acyclic graph (CPDAG) from observational data is the PC algorithm asintroduced in Spirtes and Glymour [1991].

Original: It consists of two phases: The ﬁrst one is concernedwith learning the skeleton of adjacencies based on iterative CI testing.

Original: Subsequently, a set of rules isapplied to determine the orientation of the found links.

Original: Algorithm

Original: Data: Expert knowledge E as map with keys X and Y and values in {false, ‘sampling index’, ‘heteroskedastic parent H ’}, where H 2 V , observational data with sample size n for nodes X , Y , Z 1 , . . .

Original: , Z k , H , window length  , signiﬁcance level ↵

Original: Result: boolean indicating whether there is a (conditional) dependence between X and Y given Z 1 , . . , Z k resid := [] ; for node in { X , Y } do if k == 

Revised: It consists of two phases: The first one is concerned for node in { X , Y } do if k ==

Original: then ˜ r = node else obtain residuals ˜ r
Revised: then ˜ r = node else obtain residuals ˜ r

Original: =
Revised: =

Original: (˜ r 1 , . .
Revised: (˜ r 1 , . .

Original: ˜ r n ) by regressing node on Z 1 , . . .
Revised: ˜ r n ) by regressing node on Z 1 , . . .

Original: , Z k using OLS; endif
Revised: , Z k using OLS; endif

Original: E [ node ] =
Revised: E [ node ] =

Original: = ‘false’ then append ˜ r to resid ; else if E [ node ]
Revised: = ‘false’ then append ˜ r to resid ; else if E [ node ]

Original: ==
Revised: ==

Original: ‘sampling index’ then compute weights w i
Revised: ‘sampling index’ then compute weights w i

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: fori = 1 ,
Revised: for

Original: [Equation]
Revised: [Equation]

Original: . . , n ; obtain residuals r by regressing node on Z 1 , . . 
Revised: ; obtain residuals r by regressing node on Z 1 , . . .

Original: , Z k using WLS with the weights w ; append w · r to resid ; else sort ˜ r such that their corresponding values of H increase; compute weights w i
Revised: , Z k using WLS with the weights w ; append w · r to resid ; else sort ˜ r such that their corresponding values of H increase; compute weights w i

Original: =
Revised: =

Original: ( 1 
Revised: ( 1 λ

Original: P
Revised: P

Original: [Equation]
Revised: [Equation]

Original: for [Equation] ; revert the sorting in the indices of w ; obtain residuals r by regressing node on Z 1 , . . .
Revised: ; revert the sorting in the indices of w ; obtain residuals r by regressing node on Z 1 , . . .

Original: , Z k using WLS with the weights w ; append w · r to resid ; end calculate studentized Pearson correlation t between r X = resid [0] and r Y = resid [1] ; perform t -test with (two-sided) signiﬁcance level ↵ , i.e. reject if | t | > t (1  ↵ 2 , n  2  k ) 
Revised: , Z k using WLS with the weights w ; append w · r to resid ; end calculate studentized Pearson correlation t between r X = resid [0] and r Y = resid [1] ; perform t -test with (two-sided) significance level α , i.e. reject if | t | > t (1 − α 2 , n − 2 − k ) with learning the skeleton of adjacencies based on iterative CI testing.

Revised: Subsequently, a set of rules isapplied to determine the orientation of the found links.

Original: To ensure consistency of this method, the following assumptions have to be fulﬁlled.
Revised: To ensure consistency of this method, the following assumptions have to be fulfilled.

Original: Details can befound in Spirtes et al.
Revised: Details can befound in Spirtes et al.

Original: Let G =
Revised: Let G =

Original: ( V, E ) be a graph consisting of a set of vertices V and a setof edges E ⇢ V ⇥ V .
Revised: ( V, E ) be a graph consisting of a set of vertices V and a setof edges E ⊂ V × V .

Original: Let P denote the probability distribution of V .
Revised: Let P denote the probability distribution of V .

Original: Assumption 4 (PC algorithm) .
Revised: Assumption 4 (PC algorithm) .

Original: The Causal Markov condition, Causal Faithfulness, and Causal
Revised: The Causal Markov condition, Causal Faithfulness, and Causal

Original: Sufﬁciency are fulﬁlled for the SCM (1) with graph G .
Revised: Sufficiency are fulfilled for the SCM (1) with graph G .

Original: Under these assumptions and if the utilized conditional independence test is consistent, it can beshown that the PC algorithm converges in probability to the correct causal structure, i.e.
Revised: Under these assumptions and if the utilized conditional independence test is consistent, it can beshown that the PC algorithm converges in probability to the correct causal structure, i.e.

Original: [Equation]
Revised: [Equation]

Original: where G denotes the ground truth CPDAG and ˆ G n is the ﬁnite sample output of the PC algorithm.
Revised: where G denotes the ground truth CPDAG and ˆ G n is the finite sample output of the PC algorithm.

Original: See Kalisch and Bühlmann [2007] for a proof.
Revised: See Kalisch and Bühlmann [2007] for a proof.

Original: In the following, we need to further restrict Assumption 1 to prove consistency of the PC algorithmunder heteroskedasticity.
Revised: In the following, we need to further restrict Assumption 1 to prove consistency of the PC algorithmunder heteroskedasticity.

Original: Assumption 5 (Heteroskedastic relationships regarding PC algorithm) .
Revised: Assumption 5 (Heteroskedastic relationships regarding PC algorithm) .

Original: Assumption 1 is further limited to the case where there is only sampling index dependent heteroskedasticity or homoskedasticity.
Revised: Assumption 1 is further limited to the case where there is only sampling index dependent heteroskedasticity or homoskedasticity.

Original: Given Assumption 5, we can apply our proposed method ParCorr-WLS in every CI test of the PCalgorithm.
Revised: Given Assumption 5, we can apply our proposed method ParCorr-WLS in every CI test of the PCalgorithm.

Original: Using Lemma 1, we thus can establish consistency of the PC algorithm with ParCorr-WLSif the true weights are known.
Revised: Using Lemma 1, we thus can establish consistency of the PC algorithm with ParCorr-WLSif the true weights are known.

Original: Lemma 2 yields the same result for ParCorr-WLS with estimatedweights.
Revised: Lemma 2 yields the same result for ParCorr-WLS with estimatedweights.

Original: Theorem 2.
Revised: Theorem 2.

Original: Under Assumptions 2,3,4,5 the output of the PC algorithm, with ParCorr-WLS as a CItest, converges in probability to the correct causal graph.
Revised: Under Assumptions 2,3,4,5 the output of the PC algorithm, with ParCorr-WLS as a CItest, converges in probability to the correct causal graph.

Original: Note that we prove consistency of the PC algorithm with ParCorr-WLS only in the case of samplingindex dependent heteroskedasticity.
Revised: Note that we prove consistency of the PC algorithm with ParCorr-WLS only in the case of samplingindex dependent heteroskedasticity.

Original: We discuss the challenges to extending this to more general typesof heteroskedasticity in section 6.
Revised: We discuss the challenges to extending this to more general typesof heteroskedasticity in section 6.

Original:  Experiments
Revised:  Experiments

Original: In the following, we conduct experiments evaluating our proposed CI test separately and in conjunction with the PC algorithm.
Revised: In the following, we conduct experiments evaluating our proposed CI test separately and in conjunction with the PC algorithm.

Original: Throughout the experiments, heteroskedasticity strength refers to theparameter s in the scaling functions h of linear and periodic type given by
Revised: Throughout the experiments, heteroskedasticity strength refers to theparameter s in the scaling functions h of linear and periodic type given by

Original: [Equation]
Revised: [Equation]

Original: In other words, in case of linear heteroskedasticity strength is the slope of the variance function, andfor periodic heteroskedasticity it refers to the amplitude of the variance function.
Revised: In other words, in case of linear heteroskedasticity strength is the slope of the variance function, andfor periodic heteroskedasticity it refers to the amplitude of the variance function.

Original: 5.1 Conditional independence testing
Revised: 5.1 Conditional independence testing

Original: We generate the data from the SCM (2) where we consider various types of heteroskedasticity, i.e.functions h , namely linear and periodic as given in Eq.
Revised: We generate the data from the SCM (2) where we consider various types of heteroskedasticity, i.e.functions h , namely linear and periodic as given in Eq.

Original: Plots of the simulated data are providedin the Supplement.
Revised: Plots of the simulated data are providedin the Supplement.

Original: Each of the types can either be Z - or sampling index-dependent.
Revised: Each of the types can either be Z - or sampling index-dependent.

Original: A visualizationof the considered heteroskedasticity-types can be found in the Supplement A.5.
Revised: A visualizationof the considered heteroskedasticity-types can be found in the Supplement A.5.

Original: We use the Kolmogorov-Smirnov (KS) statistic to quantify how uniform the distribution of p-valuesis, and therefore as a metric for type-I errors, as in Runge [2018].
Revised: We use the Kolmogorov-Smirnov (KS) statistic to quantify how uniform the distribution of p-valuesis, and therefore as a metric for type-I errors, as in Runge [2018].

Original: Type-II errors are measured by thearea under the power curve (AUPC).
Revised: Type-II errors are measured by thearea under the power curve (AUPC).

Original: The metrics were evaluated with a sample size of 500 from 100realizations of the SCM.
Revised: The metrics were evaluated with a sample size of 500 from 100realizations of the SCM.

Original: Error bars indicate the bootstrapped standard errors.
Revised: Error bars indicate the bootstrapped standard errors.

Original: Figure 2 shows that ParCorr-WLS is well calibrated in the presence of heteroskedasticity, regardlessif it affects only one or both of the variables X and Y .
Revised: Figure 2 shows that ParCorr-WLS is well calibrated in the presence of heteroskedasticity, regardlessif it affects only one or both of the variables X and Y .

Original: On the other hand, the ParCorr-OLS testbecomes ill-calibrated in the case of heteroskedastic X and Y as expected by Effect 1.
Revised: On the other hand, the ParCorr-OLS testbecomes ill-calibrated in the case of heteroskedastic X and Y as expected by Effect 1.

Original: In particular,this means that if we can choose the weights reasonably well, we are able to overcome multiplicativeconfounding with our proposed CI test.
Revised: In particular,this means that if we can choose the weights reasonably well, we are able to overcome multiplicativeconfounding with our proposed CI test.

Original: Regarding power as measured by AUPC, we observe a rather rapid decrease for ParCorr-OLS as theheteroskedasticity increases (compare to Effect 2).
Revised: Regarding power as measured by AUPC, we observe a rather rapid decrease for ParCorr-OLS as theheteroskedasticity increases (compare to Effect 2).

Original: Our proposed method ParCorr-WLS has higher power in the heteroskedastic scenario, and even for homoskedastic noise (heteroskedasticity strengthequal zero) the power is comparable to that of ParCorr-OLS.
Revised: Our proposed method ParCorr-WLS has higher power in the heteroskedastic scenario, and even for homoskedastic noise (heteroskedasticity strengthequal zero) the power is comparable to that of ParCorr-OLS.

Original: The drop in power for ParCorr-WLS canbe explained by an overall increase in noisiness of the data as heteroskedasticity strength is growing.
Revised: The drop in power for ParCorr-WLS canbe explained by an overall increase in noisiness of the data as heteroskedasticity strength is growing.

Original: Refer to the Supplement A.5 for a plot of AUPC of ParCorr-OLS and ParCorr-WLS on data withhomoskedastic but increasing noise.
Revised: Refer to the Supplement A.5 for a plot of AUPC of ParCorr-OLS and ParCorr-WLS on data withhomoskedastic but increasing noise.

Original: Similar effects are present for all considered types of heteroskedasticity (see Supplement).
Revised: Similar effects are present for all considered types of heteroskedasticity (see Supplement).

Original: The resultsremain very similar if we do not use the ground truth weights but estimate them using the windowapproach with a reasonable window length as detailed in Section 4.1.
Revised: The resultsremain very similar if we do not use the ground truth weights but estimate them using the windowapproach with a reasonable window length as detailed in Section 4.1.

Original: 5.2 Causal discovery
Revised: 5.2 Causal discovery

Original: To test our proposed CI test in a more realistic setting, we apply it within the PC algorithm torecover the causal graph from simulated observational data.
Revised: To test our proposed CI test in a more realistic setting, we apply it within the PC algorithm torecover the causal graph from simulated observational data.

Original: We build upon the PC-stable algorithmimplementation within the Tigramite software package [Runge et al., 2019b] which is publishedunder the GNU General Public License.
Revised: We build upon the PC-stable algorithmimplementation within the Tigramite software package [Runge et al., 2019b] which is publishedunder the GNU General Public License.

Original: The data for these experiments was generated with SCM (1) in the following way.
Revised: The data for these experiments was generated with SCM (1) in the following way.

Original: Given a randomground truth graph, we ﬁx a percentage of heteroskedasticity-affected nodes which are then selecteduniformly at random.
Revised: Given a randomground truth graph, we fix a percentage of heteroskedasticity-affected nodes which are then selecteduniformly at random.

Original: Throughout the experiments this percentage is set to 0 .
Revised: Throughout the experiments this percentage is set to 0 .

Original: 3 to reduce the chance ofparent and child being affected by the same kind of heteroskedasticity.
Revised: 3 to reduce the chance ofparent and child being affected by the same kind of heteroskedasticity.

Original: For these affected nodes, wechoose as a heteroskedasticity type linear or periodic with equal probability (Eq.
Revised: For these affected nodes, wechoose as a heteroskedasticity type linear or periodic with equal probability (Eq.

Original: and let the noisevariance h either depend on one randomly selected parent or the sampling index.
Revised: and let the noisevariance h either depend on one randomly selected parent or the sampling index.

Original: We also set a ﬁxedstrength s per experiment and investigated the effect of increasing the strength on the performanceof our method compared to the PC algorithm with ParCorr-OLS.
Revised: We also set a fixedstrength s per experiment and investigated the effect of increasing the strength on the performanceof our method compared to the PC algorithm with ParCorr-OLS.

Original: All linear dependencies have acoefﬁcient c = 0 .
Revised: All linear dependencies have acoefficient c = 0 .

Original: We tested for various small to medium sized causal graphs (see also Supplement).
Revised: We tested for various small to medium sized causal graphs (see also Supplement).

Original: In these experiments, we estimate the weights based on the expert knowledge as required by Assumption 1, namely which node is affected by heteroskedasticity, and whether the noise variancedepends on the sampling index or another node.
Revised: In these experiments, we estimate the weights based on the expert knowledge as required by Assumption 1, namely which node is affected by heteroskedasticity, and whether the noise variancedepends on the sampling index or another node.

Original: In the case of the heteroskedasticity depending onanother node, we also need to know which node it is.
Revised: In the case of the heteroskedasticity depending onanother node, we also need to know which node it is.

Original: We also compare with the PC algorithm whichuses ParCorr-WLS based on the ground truth weights for all direct heteroskedastic relations.
Revised: We also compare with the PC algorithm whichuses ParCorr-WLS based on the ground truth weights for all direct heteroskedastic relations.

Original: Note,however, that this does not take care of indirect heteroskedasticity due to heteroskedastic parentsthat are not part of the conditioning set.
Revised: Note,however, that this does not take care of indirect heteroskedasticity due to heteroskedastic parentsthat are not part of the conditioning set.

Original: Note that this challenging setting is outside of the stricter
Revised: Note that this challenging setting is outside of the stricter

Original: Assumption 5 for which the PC algorithm is consistent.
Revised: Assumption 5 for which the PC algorithm is consistent.

Original: Figure 3 shows, similar to the experiments of the previous section, that the PC algorithm with ParCorrWLS continues to have a rather small false positive rate (FPR) even though the heteroskedasticitystrength increases.
Revised: Figure 3 shows, similar to the experiments of the previous section, that the PC algorithm with ParCorrWLS continues to have a rather small false positive rate (FPR) even though the heteroskedasticitystrength increases.

Original: ParCorr-OLS shows a small increase from s = 0 to s = 1 .
Revised: ParCorr-OLS shows a small increase from s = 0 to s = 1 .

Original: Additionally, even inthis rather complicated setting, we see that the true positive rate (TPR) can be improved by usingour method.
Revised: Additionally, even inthis rather complicated setting, we see that the true positive rate (TPR) can be improved by usingour method.

Original: The PC algorithm with ParCorr-WLS has an average runtime of 0 .
Revised: The PC algorithm with ParCorr-WLS has an average runtime of 0 .

Original: 29 seconds onhomoskedastic data compared to 0 .
Revised: 29 seconds onhomoskedastic data compared to 0 .

Original: 14 seconds for ParCorr-OLS evaluated on AMD 7763.
Revised: 14 seconds for ParCorr-OLS evaluated on AMD 7763.

Original:  Discussion and Outlook
Revised:  Discussion and Outlook

Original: In this work we relaxed the common assumption of a constant variance by explicitly allowing forheteroskedasticity as a multiplicative scaling of noise in an otherwise linear SCM.
Revised: In this work we relaxed the common assumption of a constant variance by explicitly allowing forheteroskedasticity as a multiplicative scaling of noise in an otherwise linear SCM.

Original: Our proposedpartial correlation test based on weighted least squares regression is a linear, computationally fast andeasy to implement method and constitutes a useful standalone method for various data science tasks,but our focus here is on its use in causal discovery.
Revised: Our proposedpartial correlation test based on weighted least squares regression is a linear, computationally fast andeasy to implement method and constitutes a useful standalone method for various data science tasks,but our focus here is on its use in causal discovery.

Original: The main strengths of our approach are that itis able to produce more reliable results both as a CI test and in causal discovery in the presence ofheteroskedasticity than the standard partial correlation variant.
Revised: The main strengths of our approach are that itis able to produce more reliable results both as a CI test and in causal discovery in the presence ofheteroskedasticity than the standard partial correlation variant.

Original: More reliable results here refers tocontrolled false positives as well as higher detection power.
Revised: More reliable results here refers tocontrolled false positives as well as higher detection power.

Original: Furthermore, the suggested adaptationsdo not compromise calibratedness and power of the CI test on homoskedastic data.
Revised: Furthermore, the suggested adaptationsdo not compromise calibratedness and power of the CI test on homoskedastic data.

Original: The main weakness of our method is that the CI test requires substantial expert knowledge in Assumption 1 and for the consistency of the PC algorithm the even stricter Assumption 5.
Revised: The main weakness of our method is that the CI test requires substantial expert knowledge in Assumption 1 and for the consistency of the PC algorithm the even stricter Assumption 5.

Original: Assumption 1requires that the heteroskedasticity only depends on one of the predictors or on the sampling index,i.e. its domain is one dimensional.
Revised: Assumption 1requires that the heteroskedasticity only depends on one of the predictors or on the sampling index,i.e. its domain is one dimensional.

Original: Furthermore, one needs to know which of these types is the case.
Revised: Furthermore, one needs to know which of these types is the case.

Original: Assumption 5 only allows for sampling-index heteroskedasticity.
Revised: Assumption 5 only allows for sampling-index heteroskedasticity.

Original: The reason for this is that there canbe indirect heteroskedasticity at the node X or Y that is not induced by a parent of X or Y , but by some other ancestor.
Revised: The reason for this is that there canbe indirect heteroskedasticity at the node X or Y that is not induced by a parent of X or Y , but by some other ancestor.

Original: This heteroskedasticity then propagates through the causal graph and essentiallymakes X or Y heteroskedastic whenever this path is not blocked by the conditioning set.
Revised: This heteroskedasticity then propagates through the causal graph and essentiallymakes X or Y heteroskedastic whenever this path is not blocked by the conditioning set.

Original: In this case,the expert knowledge does not tell us about this heteroskedasticity and we would not be able to apply
Revised: In this case,the expert knowledge does not tell us about this heteroskedasticity and we would not be able to apply

Original: ParCorr-WLS to remove it.
Revised: ParCorr-WLS to remove it.

Original: See also Section A.4 in the Supplement for a detailed treatment of casesin which the CI tests within the PC algorithm are consistent under more general heteroskedasticityforms.
Revised: See also Section A.4 in the Supplement for a detailed treatment of casesin which the CI tests within the PC algorithm are consistent under more general heteroskedasticityforms.

Original: In the following, we discuss alternatives and further avenues of research.
Revised: In the following, we discuss alternatives and further avenues of research.

Original: In future work one may alter the required expert knowledge and weight approximation scheme toovercome the issues induced by indirect heteroskedasticity.
Revised: In future work one may alter the required expert knowledge and weight approximation scheme toovercome the issues induced by indirect heteroskedasticity.

Original: Here, a possible remedy could be an iterative approach using information about causal relationships from earlier steps.
Revised: Here, a possible remedy could be an iterative approach using information about causal relationships from earlier steps.

Original: An alternative wouldbe to use the CMIknn CI test [Runge, 2018] that is able to fully treat heteroskedastic multiplicativeconfounding.
Revised: An alternative wouldbe to use the CMIknn CI test [Runge, 2018] that is able to fully treat heteroskedastic multiplicativeconfounding.

Original: However, this increased generality comes at the price of  reduced detection power, alsosee Figure 10 in the Supplement.
Revised: However, this increased generality comes at the price of much reduced detection power.

Original: Another open question is how to further improve the weight approximation method.
Revised: Another open question is how to further improve the weight approximation method.

Original: For instance,by iteratively repeating the regression and smoothing steps.
Revised: For instance,by iteratively repeating the regression and smoothing steps.

Original: Another important consideration is thechoice of the window length.
Revised: Another important consideration is thechoice of the window length.

Original: Linear properties of the variance function allow us to use a largerwindow length.
Revised: Linear properties of the variance function allow us to use a largerwindow length.

Original: However, if the variance function shows a high variability, crucial information mightbe lost if the chosen window length is too large.
Revised: However, if the variance function shows a high variability, crucial information mightbe lost if the chosen window length is too large.

Original: Model selection criteria might be employed toalleviate this problem.
Revised: Model selection criteria might be employed toalleviate this problem.

Original: Furthermore, one can consider extensions of ParCorr-WLS to multiple heteroskedastic inﬂuencingfactors, e.g., Spokoiny [2002] extend the residual-based conditional variance function estimation todimensions larger than one.
Revised: Furthermore, one can consider extensions of ParCorr-WLS to multiple heteroskedastic influencingfactors, e.g., Spokoiny [2002] extend the residual-based conditional variance function estimation todimensions larger than one.

Original: Another multidimensional method based on differences is discussed in
Revised: Another multidimensional method based on differences is discussed in

Original: Cai et al.
Revised: Cai et al.

Original: It would also be interesting to explore using generalized least squares to be able to account foradditional correlation of the residuals.
Revised: It would also be interesting to explore using generalized least squares to be able to account foradditional correlation of the residuals.

Original: Potentially, the weighted least squares partial correlationcoefﬁcient could also be combined with a permutation-based test to extend the method to problemswith non-Gaussian noise.
Revised: Potentially, the weighted least squares partial correlationcoefficient could also be combined with a permutation-based test to extend the method to problemswith non-Gaussian noise.

Original: Concluding , our proposed ParCorr-WLS CI test makes constraint-based causal discovery methodsbetter applicable to real world problems where the assumption of homoskedasticity is violated, suchas in climate research or neuroscience.
Revised: Concluding , our proposed ParCorr-WLS CI test makes constraint-based causal discovery methodsbetter applicable to real world problems where the assumption of homoskedasticity is violated, suchas in climate research or neuroscience.

Original: Ethically, we believe that our rather fundamental work has alow potential for misuse.
Revised: Ethically, we believe that our rather fundamental work has alow potential for misuse.

Original: Checklist
Revised: Checklist

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’scontributions and scope?
Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] See Sections 1 and 6, as well asthe assumptions, speciﬁcally Assumption 1.
Revised: [Yes] See Sections 1 and 6, as well asthe assumptions, specifically Assumption 1.(c)

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: Did you discuss any potential negative societal impacts of your work?

Original: [Yes] SeeSection 6.(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: [Yes] SeeSection 6.(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] See Assumptions 1-5.(b)
Revised: [Yes] See Assump- tions 1-5.

Original:  Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes] We provide references and intuitive justiﬁcations for the results in the main text and complete proofs in the Supplement.3.
Revised: [Yes] We provide referencesand intuitive justifications for the results in the main text and complete proofs in theSupplement.3.

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?
Revised: mental results (either in the supplemental material or as a URL)?

Original: [Yes] The code andinstructions on how to execute it are provided in the supplemental material.
Revised: [Yes] The code and instructions on how to execute it are provided in the supplemental material.

Original: The codeincludes a method for generating the synthetic data.
Revised: The code includes a method for generating the synthetic data.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?

Original: [Yes] All hyperparameters are listed in Section 5.
Revised: [Yes] All hyperparameters are listed in Section 5.

Original: (c) Did you report error bars (e.g., with respect to the random seed after running experi-
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experi-

Original: ments multiple times)?
Revised: ments multiple times)?

Original: [Yes] See Figures 2 and 3.
Revised: [Yes] See Figures 2 and 3.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type

Original: of GPUs, internal cluster, or cloud provider)?
Revised: of GPUs, internal cluster, or cloud provider)?

Original: [Yes] See Section 5.
Revised: [Yes] See Section 5.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] See Section 5.
Revised: [Yes] See Section 5.

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [Yes] See Section 5.
Revised: [Yes] See Section 5.(c)

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes]The code is included in the supplemental material.
Revised: [Yes]The code is included in the supplemental material.

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [N/A]
Revised: [N/A]

Original: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: information or offensive content?
Revised: information or offensive content?

Original: [N/A]
Revised: [N/A]

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable?
Revised: applicable?

Original: [N/A]
Revised: [N/A]

Original: [Table] 
Revised: [Table] [Table] [Table] [Table]

Original: [Table] 
Revised: [Table] [Table] [Table] [Table]