Original: Causal Discovery for Linear Mixed Data
Revised: Causal Discovery for Linear Mixed Data

Original: : Bernhard Sch¨olkopf, Caroline Uhler and Kun Zhang
Revised: : Bernhard Sch¨olkopf, Caroline Uhler and Kun Zhang

Original: Abstract
Revised: Abstract

Original: Discovery of causal relationships from observational data, especially from mixed data that consist of both continuous and discrete variables, is a fundamental yet challenging problem.
Revised: Discovery of causal relationships from observational data, especially from mixed data that consist of both continuous and discrete variables, is a fundamental yet challenging problem.

Original: Traditional methods focus on polishing the data type processing policy, which may lose data information.
Revised: Traditional methods focus on polishing the data type processing policy, which may lose data information.

Original: Compared with such methods, the constraint-based and score-based methods for mixed data derive certain conditional independence tests or score functions from the data’s characteristics.
Revised: Compared with such methods, the constraint-based and score-based methods for mixed data derive certain conditional independence tests or score functions from the data’s characteristics.

Original: However, they may return the Markov equivalence class due to the lack of identiﬁability guarantees, which may limit their applicability or hinder their interpretability of causal graphs.
Revised: However, they may return the Markov equivalence class due to the lack of identiﬁability guarantees, which may limit their applicability or hinder their interpretability of causal graphs.

Original: Thus, in this paper, based on the structural causal models of continuous and discrete variables, we provide sufﬁcient identiﬁability conditions in bivariate as well as multivariate cases.
Revised: Thus, in this paper, based on the structural causal models of continuous and discrete variables, we provide sufﬁcient identiﬁability conditions in bivariate as well as multivariate cases.

Original: We show that if the data follow our proposed restricted Linear Mixed causal model (LiM), such a model is identiﬁable.
Revised: We show that if the data follow our proposed restricted Linear Mixed causal model (LiM), such a model is identiﬁable.

Original: In addition, we proposed a two-step hybrid method to discover the causal structure for mixed data.
Revised: In addition, we proposed a two-step hybrid method to discover the causal structure for mixed data.

Original: Experiments on both synthetic and real-world data empirically demonstrate the identiﬁability and efﬁcacy of our proposed LiM model.
Revised: Experiments on both synthetic and real-world data empirically demonstrate the identiﬁability and efﬁcacy of our proposed LiM model.

Original: 1. Introduction
Revised: 1. Introduction

Original: Identifying the causal structure from purely observational data, termed as causal discovery, has been rapidly developed for the past decades with growing interest and has been widely applied in many domains (Pearl, 2000; Spirtes et al., 1993; Shimizu, 2014; Zhang and Hyv¨arinen, 2016).
Revised: Identifying the causal structure from purely observational data, termed as causal discovery, has been rapidly developed for the past decades with growing interest and has been widely applied in many domains (Pearl, 2000; Spirtes et al., 1993; Shimizu, 2014; Zhang and Hyv¨arinen, 2016).

Original: The traditional approaches to causal discovery roughly fall into two categories, namely constraint-based methods (Spirtes and Glymour, 1991; Spirtes et al., 1995), and score-based ones (Chickering, 2002).
Revised: The traditional approaches to causal discovery roughly fall into two categories, namely constraint-based methods (Spirtes and Glymour, 1991; Spirtes et al., 1995), and score-based ones (Chickering, 2002).

Original: Since they may output Markov equivalence classes, i.e., a set of causal structures entailing the same conditional independences, they do not offer complete causal information.
Revised: Since they may output Markov equivalence classes, i.e., a set of causal structures entailing the same conditional independences, they do not offer complete causal information.

Original: To distinguish different causal structures in the Markov equivalence class, several scholars derive additional assumptions on the data distribution and propose causal methods based on Structural Causal Models (SCM).
Revised: To distinguish different causal structures in the Markov equivalence class, several scholars derive additional assumptions on the data distribution and propose causal methods based on Structural Causal Models (SCM).

Original: These methods, including Linear Non-Gaussian Acyclic Models (LiNGAM) (Shimizu et al., 2006),
Revised: These methods, including Linear Non-Gaussian Acyclic Models (LiNGAM) (Shimizu et al., 2006),

Original: Additive Nonlinear Models (ANM) (Hoyer et al., 2009), and Post Nonlinear (PNL) (Zhang and
Revised: Additive Nonlinear Models (ANM) (Hoyer et al., 2009), and Post Nonlinear (PNL) (Zhang and

Original: Hyv¨arinen, 2009), achieve the unique identiﬁability of the causal structure.
Revised: Hyv¨arinen, 2009), achieve the unique identiﬁability of the causal structure.

Original: Most existing causal discovery methods focus on cases where the involved variables are either continuous or discrete only.
Revised: Most existing causal discovery methods focus on cases where the involved variables are either continuous or discrete only.

Original: However, in many real-world scenarios such as economics (Wei et al., 2018), bioinformatics (Sedgewick et al., 2019), etc., the collected data often are a mixture of both continuous and discrete variables.
Revised: However, in many real-world scenarios such as economics (Wei et al., 2018), bioinformatics (Sedgewick et al., 2019), etc., the collected data often are a mixture of both continuous and discrete variables.

Original: When encountering such mixed data, one may ignore the discrete variables and apply the methods for continuous variables to estimate the partial causal network; or utilize a discretization policy to discretize the continuous variables, so that they can use those methods for discrete causal networks (Monti and Cooper, 1998; Chen et al., 2017).
Revised: When encountering such mixed data, one may ignore the discrete variables and apply the methods for continuous variables to estimate the partial causal network; or utilize a discretization policy to discretize the continuous variables, so that they can use those methods for discrete causal networks (Monti and Cooper, 1998; Chen et al., 2017).

Original: Both methods attempt to convert mixed data types into the same type, which is naive and can make it easy to lose data information, inducing non-negligible estimation errors.
Revised: Both methods attempt to convert mixed data types into the same type, which is naive and can make it easy to lose data information, inducing non-negligible estimation errors.

Original: Yamayoshi et al.
Revised: Yamayoshi et al.

Original: (2020) proposed a Latent-LiNGAM algorithm, which assumed that observed categorical variables are the result of discretizing latent continuous variables via link functions.
Revised: (2020) proposed a Latent-LiNGAM algorithm, which assumed that observed categorical variables are the result of discretizing latent continuous variables via link functions.

Original: But they did not concentrate on the model’s theoretical analysis with mixed data.
Revised: But they did not concentrate on the model’s theoretical analysis with mixed data.

Original: Apart from these methods, by and large, causal discovery algorithms for mixed data can be categorized into two classes: constraint-based and score-based ones.
Revised: Apart from these methods, by and large, causal discovery algorithms for mixed data can be categorized into two classes: constraint-based and score-based ones.

Original: Constraintbased algorithms are those variants of PC (Pearl, 2000; Spirtes and Glymour, 1991), including Cui et al.
Revised: Constraintbased algorithms are those variants of PC (Pearl, 2000; Spirtes and Glymour, 1991), including Cui et al.

Original: ; Sedgewick et al.
Revised: ; Sedgewick et al.

Original: ; Tsagris et al.
Revised: ; Tsagris et al.

Original: (2018), which cannot guarantee identiﬁability and are sensitive to samples.
Revised: (2018), which cannot guarantee identiﬁability and are sensitive to samples.

Original: Unlike the PC variants, score-based algorithms for mixed data do not use (un)conditional independence tests, but instead, optimize a likelihood score derived from mixed data’s characteristics with the commonly-used greedy equivalence search framework.
Revised: Unlike the PC variants, score-based algorithms for mixed data do not use (un)conditional independence tests, but instead, optimize a likelihood score derived from mixed data’s characteristics with the commonly-used greedy equivalence search framework.

Original: Such efforts include Li and
Revised: Such efforts include Li and

Original: Shimizu (2018); Huang et al.
Revised: Shimizu (2018); Huang et al.

Original: ; Andrews et al.
Revised: ; Andrews et al.

Original: ; Wei et al.
Revised: ; Wei et al.

Original: (2018), etc.
Revised: (2018), etc.

Original: In particular, the ﬁrst three efforts employ different score functions, i.e., with LiNGAM and the logistic regression model, regression model in RKHS, degenerate distributions, and respectively, whereas they do not provide the model’s identiﬁability results and may return Markov equivalence classes.
Revised: In particular, the ﬁrst three efforts employ different score functions, i.e., with LiNGAM and the logistic regression model, regression model in RKHS, degenerate distributions, and respectively, whereas they do not provide the model’s identiﬁability results and may return Markov equivalence classes.

Original: Wei et al.
Revised: Wei et al.

Original: (2018) developed a mixed causal model and proved its identiﬁability in the bivariate cases.
Revised: (2018) developed a mixed causal model and proved its identiﬁability in the bivariate cases.

Original: However, in their bivariate identiﬁability, they assumed the noises of continuous variables followed the
Revised: However, in their bivariate identiﬁability, they assumed the noises of continuous variables followed the

Original: Laplace distributions and the intercepts for discrete variables are zeros.
Revised: Laplace distributions and the intercepts for discrete variables are zeros.

Original: Further, the bivariate identiﬁability is not qualiﬁed enough to handle multivariate cases whereas the multivariate data ordinarily exist in many applications.
Revised: Further, the bivariate identiﬁability is not qualiﬁed enough to handle multivariate cases whereas the multivariate data ordinarily exist in many applications.

Original: Thus, in this paper, we propose a structural causal model that consists of both continuous and discrete variables, and give its sufﬁcient identiﬁability conditions in bivariate as well as multivariate cases.
Revised: Thus, in this paper, we propose a structural causal model that consists of both continuous and discrete variables, and give its sufﬁcient identiﬁability conditions in bivariate as well as multivariate cases.

Original: Compared with the mixed model developed by Wei et al.
Revised: Compared with the mixed model developed by Wei et al.

Original: (2018), we allow more non-Gaussian distributions to be followed by the noises of continuous variables and the intercepts for discrete variables are not restricted to be zeros.
Revised: (2018), we allow more non-Gaussian distributions to be followed by the noises of continuous variables and the intercepts for discrete variables are not restricted to be zeros.

Original: Further, we derive a two-step hybrid method to uniquely estimate the causal structure without discretization.
Revised: Further, we derive a two-step hybrid method to uniquely estimate the causal structure without discretization.

Original: In the ﬁrst phase, we develop a log-likelihood score function to characterize the joint distribution for mixed data.
Revised: In the ﬁrst phase, we develop a log-likelihood score function to characterize the joint distribution for mixed data.

Original: It is optimized globally accompanied by the acyclicity and sparsity constraints.
Revised: It is optimized globally accompanied by the acyclicity and sparsity constraints.

Original: This global search however may stuck in local optima, which share the same skeleton with the ground truth structure 1 .
Revised: This global search however may stuck in local optima, which share the same skeleton with the ground truth structure 1 .

Original: To mitigate this issue, in the second phase, we search structures over the skeleton spaces and ﬁnd the graph with the best score.
Revised: To mitigate this issue, in the second phase, we search structures over the skeleton spaces and ﬁnd the graph with the best score.

Original: Our contributions mainly are detailed in two-fold:
Revised: Our contributions mainly are detailed in two-fold:

Original: (i) For the mixed causal models that contain both continuous and discrete data, we prove the identiﬁability conditions in bivariate as well as multivariate cases.
Revised: (i) For the mixed causal models that contain both continuous and discrete data, we prove the identiﬁability conditions in bivariate as well as multivariate cases.

Original: with which we enrich the identiﬁability space for causal discovery with mixed data.
Revised: with which we enrich the identiﬁability space for causal discovery with mixed data.

Original: (ii)
Revised: (ii)

Original: We propose a score-based optimization method to infer the causal structure between mixed data.
Revised: We propose a score-based optimization method to infer the causal structure between mixed data.

Original: Experiments on synthetic and real-world data demonstrate our proposed method’s efﬁcacy, compared with other methods.
Revised: Experiments on synthetic and real-world data demonstrate our proposed method’s efﬁcacy, compared with other methods.

Original: 2. Preliminary and Model Deﬁnition
Revised: 2. Preliminary and Model Deﬁnition

Original: We consider linear mixed causal models.
Revised: We consider linear mixed causal models.

Original: Speaking concretely, suppose we are given p observed random variables, including discrete and continuous ones,
Revised: Speaking concretely, suppose we are given p observed random variables, including discrete and continuous ones,

Original: i.e., X = { x 1 , ..., x p } .
Revised: i.e., X = { x 1 , ..., x p } .

Original: Since a categorical variable with T categories can be regarded as ( T − 1) binary variables, we assume that each discrete variable is binary (Wei et al., 2018).
Revised: Since a categorical variable with T categories can be regarded as ( T − 1) binary variables, we assume that each discrete variable is binary (Wei et al., 2018).

Original: Further, we use the following assumptions from Li and
Revised: Further, we use the following assumptions from Li and

Original: Shimizu (2018):
Revised: Shimizu (2018):

Original: A1.
Revised: A1.

Original: Observed variables x i ( i = 1 , . . .
Revised: Observed variables x i ( i = 1 , . . .

Original: , p ) form a Directed Acyclic Graph (DAG).
Revised: , p ) form a Directed Acyclic Graph (DAG).

Original: A2.
Revised: A2.

Original: The value assigned to each continuous variable x
Revised: The value assigned to each continuous variable x

Original: i is a linear function of its parent variables denoted by x pa( i ) plus a non-Gaussian error term e i , that is,
Revised: i is a linear function of its parent variables denoted by x pa( i ) plus a non-Gaussian error term e i , that is,

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: the error terms e i are continuous random variables with non-Gaussian densities
Revised: the error terms e i are continuous random variables with non-Gaussian densities

Original: , and the error
Revised: , and the error

Original: variables e i
Revised: variables e i

Original: are independent of each other.
Revised: are independent of each other.

Original: The coefﬁcients
Revised: The coefﬁcients

Original: b
Revised: b

Original: ij and intercepts
Revised: ij and intercepts

Original: c i
Revised: c i

Original: are
Revised: are

Original: constants
Revised: constants

Original: A3.
Revised: A3.

Original: For each discrete variable x i , its value equals 1 if the linear function of its parent variables x pa( i ) plus a Logistic error term e i is larger than 0, otherwise, its value equals 0.
Revised: For each discrete variable x i , its value equals 1 if the linear function of its parent variables x pa( i ) plus a Logistic error term e i is larger than 0, otherwise, its value equals 0.

Original: That is,
Revised: That is,

Original: x i
Revised: x i

Original: [Equation]
Revised: [Equation]

Original: otherwise
Revised: otherwise

Original: [Equation]
Revised: [Equation]

Original: where the error terms e i are identical to those in Eq.(1), but follow the Logistic distribution.
Revised: where the error terms e i are identical to those in Eq.(1), but follow the Logistic distribution.

Original: Deﬁnition 1 (Linear Mixed causal model, LiM)
Revised: Deﬁnition 1 (Linear Mixed causal model, LiM)

Original: If a causal model for mixed data satisﬁes assumptions A1-A3, then this SCM is called a Linear Mixed causal model, abbreviated as LiM.
Revised: If a causal model for mixed data satisﬁes assumptions A1-A3, then this SCM is called a Linear Mixed causal model, abbreviated as LiM.

Original: Let F = { f con , f dis | f con ( x, e ) =
Revised: Let F = { f con , f dis | f con ( x, e ) =

Original: bx + e + c, f dis ( x, e ) =
Revised: bx + e + c, f dis ( x, e ) =

Original: (cid:40) 1 , bx + e + c >0 , otherwise } be a set of two functions which work on continuous and discrete variables, respectively, where x, e, b and c are a random variable of X , error term, coefﬁcient and intercept, respectively.
Revised: (cid:40) 1 , bx + e + c >0 , otherwise } be a set of two functions which work on continuous and discrete variables, respectively, where x, e, b and c are a random variable of X , error term, coefﬁcient and intercept, respectively.

Original: P =
Revised: P =

Original: { P con , P dis } denotes the set of probabilistic distributions for continuous and discrete variables.
Revised: { P con , P dis } denotes the set of probabilistic distributions for continuous and discrete variables.

Original: Using these notations, our model can be rewritten as:
Revised: Using these notations, our model can be rewritten as:

Original: [Equation]
Revised: [Equation]

Original: where f i ∈ F , and P ( e i ) ∈ P .
Revised: where f i ∈ F , and P ( e i ) ∈ P .

Original: 3. Identiﬁability Conditions of the LiM
Revised: 3. Identiﬁability Conditions of the LiM

Original: Here we provide a sufﬁcient identiﬁability condition for the LiM model, inspired by ideas of Wei et al.
Revised: Here we provide a sufﬁcient identiﬁability condition for the LiM model, inspired by ideas of Wei et al.

Original: (2018) and Peters et al.
Revised: (2018) and Peters et al.

Original: 3.1. Bivariate cases
Revised: 3.1. Bivariate cases

Original: The LiM model of Section 2 is equivalent to the model of Wei et al.
Revised: The LiM model of Section 2 is equivalent to the model of Wei et al.

Original: (2018) if the intercepts c i are taken to be zeros and the error terms e i follow the Laplace distributions L (0 , α i ) .
Revised: (2018) if the intercepts c i are taken to be zeros and the error terms e i follow the Laplace distributions L (0 , α i ) .

Original: Laplace distributions are commonly used in non-Gaussian models including independent component analysis and are known to be robust against the misspeciﬁcation of the distributions if the right distribution is super-Gaussian (Hyv¨arinen et al., 2001).
Revised: Laplace distributions are commonly used in non-Gaussian models including independent component analysis and are known to be robust against the misspeciﬁcation of the distributions if the right distribution is super-Gaussian (Hyv¨arinen et al., 2001).

Original: Wei et al.
Revised: Wei et al.

Original: (2018) provided a sufﬁcient identiﬁability condition for two-variable cases of their model, i.e., the two variables do not have the same marginal distributions if they are binary, all the probabilities and densities are positive, and the error variables are of non-zero variances.
Revised: (2018) provided a sufﬁcient identiﬁability condition for two-variable cases of their model, i.e., the two variables do not have the same marginal distributions if they are binary, all the probabilities and densities are positive, and the error variables are of non-zero variances.

Original: We show the identiﬁability of our model in a similar manner to Wei et al.
Revised: We show the identiﬁability of our model in a similar manner to Wei et al.

Original: The major difference lies in the fact that we allow more non-Gaussian distributions to be followed by the continuous error terms, rather than only the Laplace distributions.
Revised: The major difference lies in the fact that we allow more non-Gaussian distributions to be followed by the continuous error terms, rather than only the Laplace distributions.

Original: Now we characterize the condition that the non-Gaussian distributions of our model need to satisfy.
Revised: Now we characterize the condition that the non-Gaussian distributions of our model need to satisfy.

Original: Condition 1 The limit of non-Gaussian density ratio λ , deﬁned as λ :=
Revised: Condition 1 The limit of non-Gaussian density ratio λ , deﬁned as λ :=

Original: lim x →±∞ P con ( x ) P con ( x − b ) , satisﬁes:
Revised: lim x →±∞ P con ( x ) P con ( x − b ) , satisﬁes:

Original: [Equation]
Revised: [Equation]

Original: where C is a non-zero ﬁnite constant.
Revised: where C is a non-zero ﬁnite constant.

Original: In other words, λ is neither equal to zero nor inﬁnity ( λ (cid:54) = 0 , ∞ ).
Revised: In other words, λ is neither equal to zero nor inﬁnity ( λ (cid:54) = 0 , ∞ ).

Original: It’s noteworthy that basic non-Gaussian distributions satisfy Condition 1, including the Laplace distribution, Uniform distribution, Exponential distribution, and Gamma distribution, etc.
Revised: It’s noteworthy that basic non-Gaussian distributions satisfy Condition 1, including the Laplace distribution, Uniform distribution, Exponential distribution, and Gamma distribution, etc.

Original: With
Revised: With

Original: Condition 1 in the LiM model, we obtain our identiﬁability result in the bivariate case.
Revised: Condition 1 in the LiM model, we obtain our identiﬁability result in the bivariate case.

Original: Theorem 2 Let the data X
Revised: Theorem 2 Let the data X

Original: = { x i , x j } be generated by the LiM model in Eqs.(1)-(2) with Condition 1.
Revised: = { x i , x j } be generated by the LiM model in Eqs.(1)-(2) with Condition 1.

Original: Under the conditions that x i and x j do not share the same marginal distributions and
Revised: Under the conditions that x i and x j do not share the same marginal distributions and

Original: P ( x i = 1 | x j = 0) =
Revised: P ( x i = 1 | x j = 0) =

Original: P ( x j = 1 | x i = 0) holds 3 if they are both discrete, and all the probabilities are positive, the model is identiﬁable.
Revised: P ( x j = 1 | x i = 0) holds 3 if they are both discrete, and all the probabilities are positive, the model is identiﬁable.

Original: Proof We prove the identiﬁability for the bivariate case from three aspects: i) both variables are continuous; ii) both variables are discrete; iii) one is continuous and the other is discrete.
Revised: Proof We prove the identiﬁability for the bivariate case from three aspects: i) both variables are continuous; ii) both variables are discrete; iii) one is continuous and the other is discrete.

Original: i), if both variables are continuous, the model of Section 2 is a LiNGAM model (Shimizu et al., 2006).
Revised: i), if both variables are continuous, the model of Section 2 is a LiNGAM model (Shimizu et al., 2006).

Original: Therefore, the model is identiﬁable.
Revised: Therefore, the model is identiﬁable.

Original: ii), suppose that two variables { x i , x j } are binary.
Revised: ii), suppose that two variables { x i , x j } are binary.

Original: Assume that all the probabilities are positive and their marginal distributions are different.
Revised: Assume that all the probabilities are positive and their marginal distributions are different.

Original: Then, we compare the following two models x j → x i and x i → x j .
Revised: Then, we compare the following two models x j → x i and x i → x j .

Original: The conditional probability P ( x i | x j ) of the ﬁrst model x j → x i is written as:
Revised: The conditional probability P ( x i | x j ) of the ﬁrst model x j → x i is written as:

Original: [Equation]
Revised: [Equation]

Original: The conditional probability P ( x j | x i ) of the second model x i → x j is written as:
Revised: The conditional probability P ( x j | x i ) of the second model x i → x j is written as:

Original: [Equation]
Revised: [Equation]

Original: Assume that two models give the same joint distribution of observed variables x i and x j .
Revised: Assume that two models give the same joint distribution of observed variables x i and x j .

Original: Denote
Revised: Denote

Original: P ( x i = 1) by k i and P ( x j = 1) by k j .
Revised: P ( x i = 1) by k i and P ( x j = 1) by k j .

Original: Then,
Revised: Then,

Original: [Equation]
Revised: [Equation]

Original: If x i = x j = 0 , we get (1 − k i )(1 − 1+ e −
Revised: If x i = x j = 0 , we get (1 − k i )(1 − 1+ e −

Original: ( cj ) )
Revised: ( cj ) )

Original: = (1 − k j )(1 −
Revised: = (1 − k j )(1 −

Original: [Equation]
Revised: [Equation]

Original: 1+ e − ( ci ) ) .
Revised: 1+ e − ( ci ) ) .

Original: From P ( x i = 1 | x j = 0)
Revised: From P ( x i = 1 | x j = 0)

Original: =
Revised: =

Original: P ( x j = 1 | x i = 0) , we have
Revised: P ( x j = 1 | x i = 0) , we have

Original: [Equation]
Revised: [Equation]

Original: 1+ e − ( ci ) , which induces k i = k j .
Revised: 1+ e − ( ci ) , which induces k i = k j .

Original: It contradicts with the assumption that the marginal distributions of x i and x j are different.
Revised: It contradicts with the assumption that the marginal distributions of x i and x j are different.

Original: iii), suppose that one is continuous and the other is binary.
Revised: iii), suppose that one is continuous and the other is binary.

Original: Without loss of generality, assume that x i is continuous and x j is binary.
Revised: Without loss of generality, assume that x i is continuous and x j is binary.

Original: We adopt the contradictory method, i.e., assuming two models in the following give the same joint distribution, then some conclusion would be drawn to contradict our model’s conditions.
Revised: We adopt the contradictory method, i.e., assuming two models in the following give the same joint distribution, then some conclusion would be drawn to contradict our model’s conditions.

Original: Hence, we ﬁrst compare such two models.
Revised: Hence, we ﬁrst compare such two models.

Original: The ﬁrst model x i → x j is written as:
Revised: The ﬁrst model x i → x j is written as:

Original: [Equation]
Revised: [Equation]

Original: where x i =
Revised: where x i =

Original: e i ∼ Non − Gaussian ( · ) .
Revised: e i ∼ Non − Gaussian ( · ) .

Original: The second model x j → x i is written as:
Revised: The second model x j → x i is written as:

Original: [Equation]
Revised: [Equation]

Original: 12) where x j = e j ∼ Logistic (0 , 1) .
Revised: 12) where x j = e j ∼ Logistic (0 , 1) .

Original: Assume that the two models give the same distribution of observed variables.
Revised: Assume that the two models give the same distribution of observed variables.

Original: For the ﬁrst model, the conditional probability of x j = 1 given x i is given by
Revised: For the ﬁrst model, the conditional probability of x j = 1 given x i is given by

Original: [Equation]
Revised: [Equation]

Original: Then,
Revised: Then,

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: [Equation]
Revised: [Equation]

Original: For the second model, the conditional probability of x j = 1 given x i is given by
Revised: For the second model, the conditional probability of x j = 1 given x i is given by

Original: [Equation]
Revised: [Equation]

Original: Due to Condition 1, we obtain the limit of the density ratio as
Revised: Due to Condition 1, we obtain the limit of the density ratio as

Original: [Equation]
Revised: [Equation]

Original: where λ (cid:54) = 0 and λ (cid:54) = ∞ .
Revised: where λ (cid:54) = 0 and λ (cid:54) = ∞ .

Original: Note that the limits lim x
Revised: Note that the limits lim x

Original: i →±∞ P ( x j = 1 | x i ) under the second model are greater than 0 and smaller than 1 due to the assumption P ( x j = 1) > 0 .
Revised: i →±∞ P ( x j = 1 | x i ) under the second model are greater than 0 and smaller than 1 due to the assumption P ( x j = 1) > 0 .

Original: This means that the limits lim
Revised: This means that the limits lim

Original: x i →±∞ P ( x j = 1 | x i ) under the second model are different from those of the ﬁrst model, which contradicts the assumption that the two models give the same distribution of observed variables.
Revised: x i →±∞ P ( x j = 1 | x i ) under the second model are different from those of the ﬁrst model, which contradicts the assumption that the two models give the same distribution of observed variables.

Original: Thus, the model is bivariate identiﬁable if two variables x i , x j do not have the same marginal distributions and P ( x i = 1 | x j = 0)
Revised: Thus, the model is bivariate identiﬁable if two variables x i , x j do not have the same marginal distributions and P ( x i = 1 | x j = 0)

Original: =
Revised: =

Original: P ( x j = 1 | x i = 0) holds in case that they are binary, all the probabilities and densities are positive, and the error variables are of non-zero variances.
Revised: P ( x j = 1 | x i = 0) holds in case that they are binary, all the probabilities and densities are positive, and the error variables are of non-zero variances.

Original: Deﬁnition 3 (Bivariate Identiﬁable Set) Let F = { f con , f dis | f con ( x, e ) = bx + e + c, f dis ( x, e )
Revised: Deﬁnition 3 (Bivariate Identiﬁable Set) Let F = { f con , f dis | f con ( x, e ) = bx + e + c, f dis ( x, e )

Original: =
Revised: =

Original: (cid:40) 1 , bx + e + c >0 , otherwise } be a set of two functions which work on continuous and discrete variables, respectively.
Revised: (cid:40) 1 , bx + e + c >0 , otherwise } be a set of two functions which work on continuous and discrete variables, respectively.

Original: P =
Revised: P =

Original: { P con , P dis } denotes the set of probabilistic distributions for continuous and discrete variables.
Revised: { P con , P dis } denotes the set of probabilistic distributions for continuous and discrete variables.

Original: Consider a mixed causal model with two variables x i and x j , i.e., x j = e j and x i =
Revised: Consider a mixed causal model with two variables x i and x j , i.e., x j = e j and x i =

Original: f i ( x j , e i ) with x j ⊥ e i , where ⊥ denotes the independence relation.
Revised: f i ( x j , e i ) with x j ⊥ e i , where ⊥ denotes the independence relation.

Original: We call a set
Revised: We call a set

Original: B ⊆ F × P × P as a bivariate identiﬁability set if the triple ( f i , P ( x j ) , P ( e i )) where f ∈ F , and
Revised: B ⊆ F × P × P as a bivariate identiﬁability set if the triple ( f i , P ( x j ) , P ( e i )) where f ∈ F , and

Original: P ( x j ) , P ( e i ) ∈
Revised: P ( x j ) , P ( e i ) ∈

Original: P hold, follows our LiM model’s assumptions.
Revised: P hold, follows our LiM model’s assumptions.

Original: Using the deﬁnition of the bivariate identiﬁable set B , if the triple ( f i , P ( x j ) , P ( e i )) follows our
Revised: Using the deﬁnition of the bivariate identiﬁable set B , if the triple ( f i , P ( x j ) , P ( e i )) follows our

Original: LiM model, we have ( f i , P ( x j ) , P ( e i ))
Revised: LiM model, we have ( f i , P ( x j ) , P ( e i ))

Original: ∈ B , which means that the bivariate mixed causal model is identiﬁable.
Revised: ∈ B , which means that the bivariate mixed causal model is identiﬁable.

Original: 3.2. From bivariate to multivariate cases
Revised: 3.2. From bivariate to multivariate cases

Original: Here we ﬁrst review brieﬂy the multivariate identiﬁablity of additive noise models for continuous random variables (Hoyer et al., 2009; Peters et al., 2014) and thereafter give our multivariate identiﬁablity of the LiM model for mixed data.
Revised: Here we ﬁrst review brieﬂy the multivariate identiﬁablity of additive noise models for continuous random variables (Hoyer et al., 2009; Peters et al., 2014) and thereafter give our multivariate identiﬁablity of the LiM model for mixed data.

Original: Intuitively, Peters et al.
Revised: Intuitively, Peters et al.

Original: (2014) showed that under the assumption of causal minimality and that of positive densities, if two different additive noise graphs are assumed to give the same distribution of observed variables, it results in contradiction to the bivariate identiﬁability.
Revised: (2014) showed that under the assumption of causal minimality and that of positive densities, if two different additive noise graphs are assumed to give the same distribution of observed variables, it results in contradiction to the bivariate identiﬁability.

Original: They also pointed out that whenever they have a restriction that ensures identiﬁability in the bivariate case, the multivariate version remains valid.
Revised: They also pointed out that whenever they have a restriction that ensures identiﬁability in the bivariate case, the multivariate version remains valid.

Original: In fact, most parts of their proof other than the bivariate identiﬁability condition use only the general properties of non-parametric structural causal models with no hidden common causes and no cycles, and do not depend on the assumptions of additive noise models.
Revised: In fact, most parts of their proof other than the bivariate identiﬁability condition use only the general properties of non-parametric structural causal models with no hidden common causes and no cycles, and do not depend on the assumptions of additive noise models.

Original: Therefore, our LiM model also is identiﬁable for cases with more than two variable using the idea of Peters et al.
Revised: Therefore, our LiM model also is identiﬁable for cases with more than two variable using the idea of Peters et al.

Original: based on the bivariate identiﬁability.
Revised: based on the bivariate identiﬁability.

Original: We then deﬁne the restricted mixed causal models to constrain conditional distributions and give identiﬁability analysis, in a similar manner to Peters et al.
Revised: We then deﬁne the restricted mixed causal models to constrain conditional distributions and give identiﬁability analysis, in a similar manner to Peters et al.

Original: Deﬁnition 4 (Restricted LiM model) Consider a LiM model with p variables.
Revised: Deﬁnition 4 (Restricted LiM model) Consider a LiM model with p variables.

Original: We call this SCM a restricted LiM model if for all i ∈ { 1 , ..., p } , j ∈ pa( i ) , and all sets S ⊆ { 1 , ..., p } with pa( i ) \{ j } ⊆
Revised: We call this SCM a restricted LiM model if for all i ∈ { 1 , ..., p } , j ∈ pa( i ) , and all sets S ⊆ { 1 , ..., p } with pa( i ) \{ j } ⊆

Original: S ⊆ nd( i ) \{ i, j } , there exists an x ∗ S with P S ( x ∗ S ) > 0 , s.t.
Revised: S ⊆ nd( i ) \{ i, j } , there exists an x ∗ S with P S ( x ∗ S ) > 0 , s.t.

Original: [Equation]
Revised: [Equation]

Original: satisﬁes the assumptions and Condition 1 of the LiM’s model in Section 2, i.e.,
Revised: satisﬁes the assumptions and Condition 1 of the LiM’s model in Section 2, i.e.,

Original: [Equation]
Revised: [Equation]

Original: ) where the underbrace with x j represents the input component of f i for the variable x j , and f i ∈ F .
Revised: ) where the underbrace with x j represents the input component of f i for the variable x j , and f i ∈ F .

Original: pa( i ) and nd( i ) are index sets of x i ’s parents and non-descendants.
Revised: pa( i ) and nd( i ) are index sets of x i ’s parents and non-descendants.

Original: Further, we require that the noise variables to have non-vanishing densities.
Revised: Further, we require that the noise variables to have non-vanishing densities.

Original: Theorem 5 Let the data X
Revised: Theorem 5 Let the data X

Original: = { x 1 , ..., x p } be generated by a restricted LiM model.
Revised: = { x 1 , ..., x p } be generated by a restricted LiM model.

Original: Under the conditions that any two discrete variables do not share the same marginal distributions, all the probabilities are positive, and P ( X ) satisﬁes the Markov and faithfulness conditions, the model is identiﬁable.
Revised: Under the conditions that any two discrete variables do not share the same marginal distributions, all the probabilities are positive, and P ( X ) satisﬁes the Markov and faithfulness conditions, the model is identiﬁable.

Original: Proof The theorem is proved by contradiction.
Revised: Proof The theorem is proved by contradiction.

Original: We assume that our restricted mixed causal model is not identiﬁable, i.e., there exist two restricted mixed causal models G 1 and G 2 , which induce the identical joint distribution P ( X ) .
Revised: We assume that our restricted mixed causal model is not identiﬁable, i.e., there exist two restricted mixed causal models G 1 and G 2 , which induce the identical joint distribution P ( X ) .

Original: In such a case, we will show that G 1 =
Revised: In such a case, we will show that G 1 =

Original: G 2 to induce the identiﬁability.
Revised: G 2 to induce the identiﬁability.

Original: Consider two variables x i and x j in X where for the sets Q := pa( i ) G 1 \{ j } , R :=
Revised: Consider two variables x i and x j in X where for the sets Q := pa( i ) G 1 \{ j } , R :=

Original: pa( j ) G 2 \{ i } , and S :=
Revised: pa( j ) G 2 \{ i } , and S :=

Original: Q ∪ R , they satisfy i)
Revised: Q ∪ R , they satisfy i)

Original: x j → x i in G 1 and x i → x j in G 2 ; and ii) S ⊆ nd( i ) G 1 \{ j } and S ⊆ nd( j ) G 2 \{ i } .
Revised: x j → x i in G 1 and x i → x j in G 2 ; and ii) S ⊆ nd( i ) G 1 \{ j } and S ⊆ nd( j ) G 2 \{ i } .

Original: Such two variables do exist (Peters et al., 2014).
Revised: Such two variables do exist (Peters et al., 2014).

Original: Firstly, due to ii) we get e i ⊥ ( x j , x S ) and e j ⊥ ( x i , x S ) .
Revised: Firstly, due to ii) we get e i ⊥ ( x j , x S ) and e j ⊥ ( x i , x S ) .

Original: Let x ∗ S
Revised: Let x ∗ S

Original: = { x q , x r } .
Revised: = { x q , x r } .

Original: For the graph G 1 , we get ( f i ( x q , · ) , P ( x j | x S = x ∗ S ) , P ( e i ))
Revised: For the graph G 1 , we get ( f i ( x q , · ) , P ( x j | x S = x ∗ S ) , P ( e i ))

Original: ∈ B , which satisﬁes the assumptions of our bivariate mixed causal model.
Revised: ∈ B , which satisﬁes the assumptions of our bivariate mixed causal model.

Original: It induces
Revised: It induces

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: where x ∗ j := x j | x S = x S ∗ and f i ∈ F .
Revised: where x ∗ j := x j | x S = x S ∗ and f i ∈ F .

Original: For the graph G 2 , we get ( f j ( x r , · ) , P ( x i | x S = x ∗ S ) , P ( e j )) ∈
Revised: For the graph G 2 , we get ( f j ( x r , · ) , P ( x i | x S = x ∗ S ) , P ( e j )) ∈

Original: B , which satisﬁes the assumptions of our bivariate LiM model.
Revised: B , which satisﬁes the assumptions of our bivariate LiM model.

Original: It induces
Revised: It induces

Original: [Equation]
Revised: [Equation]

Original: where x ∗ i :
Revised: where x ∗ i :

Original: = x i | x S = x S ∗ and f j ∈ F .
Revised: = x i | x S = x S ∗ and f j ∈ F .

Original: The above analysis contradicts the bivariate identiﬁability result in Theorem 2, hence we have G 1 =
Revised: The above analysis contradicts the bivariate identiﬁability result in Theorem 2, hence we have G 1 =

Original: G 2 .
Revised: G 2 .

Original: 4. Optimization Method
Revised: 4. Optimization Method

Original: To uncover the causal structure for mixed data that consist of both continuous and discrete variables, we propose an integrated hybrid score-based learning method.
Revised: To uncover the causal structure for mixed data that consist of both continuous and discrete variables, we propose an integrated hybrid score-based learning method.

Original: The objective function is based on the negative log-likelihood of the data.
Revised: The objective function is based on the negative log-likelihood of the data.

Original: By instantiating the negative log-likelihood with the joint probability distribution of the mixed data, we get
Revised: By instantiating the negative log-likelihood with the joint probability distribution of the mixed data, we get

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where B is the adjacency matrix, and x i,t is the t th sample of the
Revised: where B is the adjacency matrix, and x i,t is the t th sample of the

Original: i th variable x i .
Revised: i th variable x i .

Original: n is the sample size.
Revised: n is the sample size.

Original: P d and P c denote the probability distribution of discrete and continuous variables, respectively.
Revised: P d and P c denote the probability distribution of discrete and continuous variables, respectively.

Original: z i is an indicator variable, where z i = 

Revised: z i is an indicator variable, where z i =

Original: if x i is discrete while z i = 0 otherwise.
Revised: if x i is discrete while z i = 0 otherwise.

Original: σ ( B
Revised: σ ( B

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: i ) bilxl,t ) , while p i is the density function of the non-Gaussian error terms e i of continuous variables.
Revised: i ) bilxl,t ) , while p i is the density function of the non-Gaussian error terms e i of continuous variables.

Original: In our method, we speciﬁed p i to be the density function of Laplace distributions.
Revised: In our method, we speciﬁed p i to be the density function of Laplace distributions.

Original: But, we can use other density functions as well.
Revised: But, we can use other density functions as well.

Original: Thereafter, we seek to solve the following continuous optimization problem:
Revised: Thereafter, we seek to solve the following continuous optimization problem:

Original: [Equation]
Revised: [Equation]

Original: where h ( B ) is an acyclicity constraint which ensures that B is a DAG (Zheng et al., 2018), Λ is a regularization parameter, and || · || 1 is an l 1 sparsity regularization.
Revised: where h ( B ) is an acyclicity constraint which ensures that B is a DAG (Zheng et al., 2018), Λ is a regularization parameter, and || · || 1 is an l 1 sparsity regularization.

Original: Following the optimization procedure in (Zeng et al., 2021b), we leverage the Quadratic Penalty Method (QPM) to estimate B , converting Eq.(32) into an unconstrained function
Revised: Following the optimization procedure in (Zeng et al., 2021b), we leverage the Quadratic Penalty Method (QPM) to estimate B , converting Eq.(32) into an unconstrained function

Original: [Equation]
Revised: [Equation]

Original: where S = L ( B ) + Λ || B
Revised: where S = L ( B ) + Λ || B

Original: || 1 + ρ 2 h ( B ) 2 is the quadratic penalty function, and ρ is a regularization parameter.
Revised: || 1 + ρ 2 h ( B ) 2 is the quadratic penalty function, and ρ is a regularization parameter.

Original: Then we utilize the L-BFGS-B (Byrd et al., 1995) to solve Eq.(33).
Revised: Then we utilize the L-BFGS-B (Byrd et al., 1995) to solve Eq.(33).

Original: Due to the machine precision, it is well-known that it is hard for the estimated b ij to receive absolute zeros if such pairs have no edges (Zheng et al., 2018).
Revised: Due to the machine precision, it is well-known that it is hard for the estimated b ij to receive absolute zeros if such pairs have no edges (Zheng et al., 2018).

Original: Hence we give a small ﬁxed threshold (cid:15) to rule out those whose estimated effects are lower than (cid:15) .
Revised: Hence we give a small ﬁxed threshold (cid:15) to rule out those whose estimated effects are lower than (cid:15) .

Original: Require: Data X ; indicator vector Z ; threshold (cid:15) ; tolerance parameter ω .
Revised: Require: Data X ; indicator vector Z ; threshold (cid:15) ; tolerance parameter ω .

Original: Ensure: Connection strengths matrix B ∗ .
Revised: Ensure: Connection strengths matrix B ∗ .

Original: search 1: Optimize Eq.(32) to obtain ˆ B using QPM with the tolerance parameter ω , 2: Rule out edges whose connection strengths are below (cid:15) : ˆ B = ˆ B ◦ 0 (ˆ b ij < (cid:15) ) .
Revised: search 1: Optimize Eq.(32) to obtain ˆ B using QPM with the tolerance parameter ω , 2: Rule out edges whose connection strengths are below (cid:15) : ˆ B = ˆ B ◦ 0 (ˆ b ij < (cid:15) ) .

Original: Phase II: Local search 3: Initiate a temporary minimum log-likelihood as L tmp ( B ∗ ) =
Revised: Phase II: Local search 3: Initiate a temporary minimum log-likelihood as L tmp ( B ∗ ) =

Original: L (ˆ B ) .
Revised: L (ˆ B ) .

Original: 4: while B ∈ Ske(ˆ B ) and h ( B ) <
Revised: 4: while B ∈ Ske(ˆ B ) and h ( B ) <

Original: ω do 5:
Revised: ω do 5:

Original: Compute the negative log-likelihood L ( B ) for B .
Revised: Compute the negative log-likelihood L ( B ) for B .

Original: 6: if L ( B ) <
Revised: 6: if L ( B ) <

Original: L tmp ( B ∗ ) then 7:
Revised: L tmp ( B ∗ ) then 7:

Original: L tmp ( B ∗ ) =
Revised: L tmp ( B ∗ ) =

Original: L ( B ) .
Revised: L ( B ) .

Original: 8: end if 9: end while 10: return B ∗ with the minimal log-likelihood L tmp ( B ∗ ) .
Revised: 8: end if 9: end while 10: return B ∗ with the minimal log-likelihood L tmp ( B ∗ ) .

Original: However, such an optimization method may easily stuck in local optima, which means the estimated skeleton is consistent with the ground-truth skeleton, while the estimated causal structure may not be consistent with the ground-truth one.
Revised: However, such an optimization method may easily stuck in local optima, which means the estimated skeleton is consistent with the ground-truth skeleton, while the estimated causal structure may not be consistent with the ground-truth one.

Original: To mitigate this issue, after obtaining the estimated adjacency matrix ˆ B , we tackle further the following combinatorial optimization problem:
Revised: To mitigate this issue, after obtaining the estimated adjacency matrix ˆ B , we tackle further the following combinatorial optimization problem:

Original: [Equation]
Revised: [Equation]

Original: where Ske(ˆ B ) represents a set where the containing DAGs entail the same skeleton as ˆ B and ω is a tolerance parameter.
Revised: where Ske(ˆ B ) represents a set where the containing DAGs entail the same skeleton as ˆ B and ω is a tolerance parameter.

Original: Compared with the traditional approaches that search over the discrete space of DAGs with the full graph, we perform our structure search over the narrowed space of DAGs within the estimated skeleton, which possesses an advantage in computational efﬁciency.
Revised: Compared with the traditional approaches that search over the discrete space of DAGs with the full graph, we perform our structure search over the narrowed space of DAGs within the estimated skeleton, which possesses an advantage in computational efﬁciency.

Original: The full algorithm is outlined in Algorithm 1.
Revised: The full algorithm is outlined in Algorithm 1.

Original: As demonstrated in Algorithm 1, our LiM approach ﬁrstly performs global updates, estimating the connection strengths matrix ˆ B in one step with continuous optimization techniques.
Revised: As demonstrated in Algorithm 1, our LiM approach ﬁrstly performs global updates, estimating the connection strengths matrix ˆ B in one step with continuous optimization techniques.

Original: Then, it performs a local update to search over the skeleton space, estimating one candidate DAG with one changing edge at each iteration in a combinatorial optimization manner.
Revised: Then, it performs a local update to search over the skeleton space, estimating one candidate DAG with one changing edge at each iteration in a combinatorial optimization manner.

Original: To conclude, the LiM approach is a two-step hybrid method, which takes advantages of both global and local search to avoid falling into local optima and to be more computationally efﬁcient.
Revised: To conclude, the LiM approach is a two-step hybrid method, which takes advantages of both global and local search to avoid falling into local optima and to be more computationally efﬁcient.

Original: 5. Experiments
Revised: 5. Experiments

Original: In this section, we performed simulation experiments and employed our method to real-world application data to learn the causal graph with mixed data, evaluating the efﬁcacy of our proposed method.
Revised: In this section, we performed simulation experiments and employed our method to real-world application data to learn the causal graph with mixed data, evaluating the efﬁcacy of our proposed method.

Original: 5.1. Synthetic data
Revised: 5.1. Synthetic data

Original: To generate the data in simulations, we ﬁrstly established a randomly unweighted DAG according to the ER models (Zheng et al., 2018), where the number of edges was randomly selected.
Revised: To generate the data in simulations, we ﬁrstly established a randomly unweighted DAG according to the ER models (Zheng et al., 2018), where the number of edges was randomly selected.

Original: Given the
Revised: Given the

Original: DAG, we assigned uniformly the edge weights from [ − 2 , − 1] (cid:83) [1 , 2] to get an adjacency matrix B .
Revised: DAG, we assigned uniformly the edge weights from [ − 2 , − 1] (cid:83) [1 , 2] to get an adjacency matrix B .

Original: Without loss of generality, the number of discrete variables was selected randomly from [1 , ( p − 1)] , and thereafter we randomly assigned the discrete and continuous variables.
Revised: Without loss of generality, the number of discrete variables was selected randomly from [1 , ( p − 1)] , and thereafter we randomly assigned the discrete and continuous variables.

Original: Finally, the data were generated according to our LiM model in Eqs.(1)-(2).
Revised: Finally, the data were generated according to our LiM model in Eqs.(1)-(2).

Original: We compared our method with a constraint-based method, a variant of PC algorithm (PC) (Spirtes and Glymour, 1991) as a representative.
Revised: We compared our method with a constraint-based method, a variant of PC algorithm (PC) (Spirtes and Glymour, 1991) as a representative.

Original: It discretized all continuous variables into discrete ones, following Li and Shimizu (2018).
Revised: It discretized all continuous variables into discrete ones, following Li and Shimizu (2018).

Original: Besides, since PC may return a DAG pattern, PDAG, instead of a unique DAG, we took its instance for evaluation. 4 We compared with score-based methods, including the scores of Notears (Zheng et al., 2018) with the Logistic or Laplace distributions.
Revised: Besides, since PC may return a DAG pattern, PDAG, instead of a unique DAG, we took all possible instance for evaluation We compared with score-based methods, including the scores of Notears (Zheng et al., 2018) with the Logistic or Laplace distributions.

Original: We also compared with a commonly-used functional-based method, the LiNGAM method (Shimizu et al., 2006).
Revised: We also compared with a commonly-used functional-based method, the LiNGAM method (Shimizu et al., 2006).

Original: To emphasize the necessity of our local search phase, we took our LiM method without the second phase as a comparison as well (mixed).
Revised: To emphasize the necessity of our local search phase, we took our LiM method without the second phase as a comparison as well (mixed).

Original: In these experiments, we evaluated the performance of all methods in terms of precision, recall and F1 score with both edges and causal directions, where the F1 score is deﬁned as F1 = 

Revised: In these experiments, we evaluated the performance of all methods in terms of precision, recall and F1 score with both edges and causal directions, where the F1 score is deﬁned as F1 =

Original: × Precision × Recall
Revised: × Precision × Recall

Original: Precision+Recall .
Revised: Precision+Recall .

Original: For those continuous optimization methods, we chose the threshold (cid:15) = 0 .
Revised: For those continuous optimization methods, we chose the threshold (cid:15) = 0 .

Original: 1 , the tolerance ω
Revised: 1 , the tolerance ω

Original: = 1 e − 8 , and the regularization parameter Λ = 0 .
Revised: = 1 e − 8 , and the regularization parameter Λ = 0 .

Original: 1 , while for those which exploit conditional independence tests, we ﬁxed the signiﬁcance level to be 0.01.
Revised: 1 , while for those which exploit conditional independence tests, we ﬁxed the signiﬁcance level to be 0.01.

Original: For other parameters, we adopted their default settings.
Revised: For other parameters, we adopted their default settings.

Original: We performed the simulations using i) different sample sizes, i.e., n = 50, 100, 500, 1000, 2000, 5000, with bivariate and 5 mixed variables in turn.
Revised: We performed the simulations using i) different sample sizes, i.e., n = 50, 100, 500, 1000, 2000, 5000, with bivariate and 5 mixed variables in turn.

Original: In addition, we generated the data with ii) different numbers of discrete variables ranging from 1 to ( p − 1) where each graph hassamples, to test the robustness of our proposed LiM method.
Revised: In addition, we generated the data with ii) different numbers of discrete variables ranging from 1 to ( p − 1) where each graph hassamples, to test the robustness of our proposed LiM method.

Original: For each setting, we experimented with 50 realizations and reported the average results.
Revised: For each setting, we experimented with 50 realizations and reported the average results.

Original: Sensitivity to different sample sizes.
Revised: Sensitivity to different sample sizes.

Original: Figure 1 gives the results of the recovered causal graph with 2 or 5 mixed variables, compared with PC, logistic, laplace, LiNGAM, and mixed methods.
Revised: Figure 1 gives the results of the recovered causal graph with 2 or 5 mixed variables, compared with PC, logistic, laplace, LiNGAM, and mixed methods.

Original: The x-axis shows the sample sizes, while the y-axis is the recall, precision, or F1 score.
Revised: The x-axis shows the sample sizes, while the y-axis is the recall, precision, or F1 score.

Original: Overall, our LiM method gives the best accuracy in both settings, which veriﬁed the identiﬁability results, especially in bivariate cases.
Revised: Overall, our LiM method gives the best accuracy in both settings, which veriﬁed the identiﬁability results, especially in bivariate cases.

Original: More speciﬁcally, the LiM, and PC methods’ accuracies increase remarkably along with the sample sizes up to 1000 in multi-variate causal networks.
Revised: More speciﬁcally, the LiM, and PC methods’ accuracies increase remarkably along with the sample sizes up to 1000 in multi-variate causal networks.

Original: When the sample size reaches 1000, it decreases slightly with increasing sample sizes.
Revised: When the sample sizes reach 1000, it decreases slightly with increasing sample sizes.

Original: This phenomenon might be originated from the overﬁtting problem of optimization.
Revised: This phenomenon might be originated from the overﬁtting problem of optimization.

Original: Though our method is sensitive to sample sizes for learning causal networks, it does perform better than other comparisons.
Revised: Though our method is sensitive to sample sizes for learning causal networks, it does perform better than other comparisons.

Original: On the contrary, PC’s unsatisfactory performance is basically due to the usage of conditional independence tests and the incapability of handling mixed data.
Revised: On the contrary, PC’s unsatisfactory performance is basically due to the usage of conditional independence tests and the incapability of handling mixed data.

Original: Score-based methods are more robust to sample sizes compared with the constraint-based ones.
Revised: Score-based methods are more robust to sample sizes compared with the constraint-based ones.

Original: However, since their score functions may not ﬁt the mixed data or they may be trapped in local optima problems, their performances are not comparable despite stability.
Revised: However, since their score functions may not ﬁt the mixed data or they may be trapped in local optima problems, their performances are not comparable despite stability.

Original: Sensitivity to different discrete variables.
Revised: Sensitivity to different discrete variables.

Original: Figure 2 reports the results of the recovered causal graph with different numbers of discrete variables ranging from 1 to p − 1 , where there are a total of 4 or 5 observed variables.
Revised: Figure 2 reports the results of the recovered causal graph with different numbers of discrete variables ranging from 1 to p − 1 , where there are a total of 4 or 5 observed variables.

Original: The x-axis shows the number of discrete variables, while the y-axis is the recall, precision or F1 score.
Revised: The x-axis shows the number of discrete variables, while the y-axis is the recall, precision or F1 score.

Original: As shown, we can see that overall, our method and mixed method’s performances tend to decrease distinctly as the number of discrete variables increases.
Revised: As shown, we can see that overall, our method and mixed method’s performances tend to decrease distinctly as the number of discrete variables increases.

Original: The reason might be the violation of the assumption P ( x i = 1 | x j = 0)
Revised: The reason might be the violation of the assumption P ( x i = 1 | x j = 0)

Original: =
Revised: =

Original: P ( x j = 1 | x i = 0) and with the 
Revised: P ( x j = 1 | x i = 0) and with the increasing number of discrete variables, such an assumption is more likely to be violated. But our

Original: [Figure]
Revised: [Figure]

Original: increasing number of discrete variables, such an assumption is more likely to be violated.

Original: But our method performs overall better than other comparisons, indicating the capability of handling mixed data 
Revised: method performs overall better than other comparisons, indicating the capability of handling mixed data [Figure]

Original: 5.2. Real-world data
Revised: 5.2. Real-world data

Original: Boston housing data set.
Revised: Boston housing data set.

Original: We then applied our LiM method to a real-world Boston housing data set, which was collected at the UCI Repository (Dua and Graff, 2017).
Revised: We then applied our LiM method to a real-world Boston housing data set, which was collected at the UCI Repository (Dua and Graff, 2017).

Original: Such a data set contains 506 data points and we chose 11 variables for the experiments, where the chosen continuous variables are identical to Zhang et al.
Revised: Such a data set contains 506 data points and we chose 11 variables for the experiments, where the chosen continuous variables are identical to Zhang et al.

Original: (2011) and the only binary variable is included.
Revised: (2011) and the only binary variable is included.

Original: They are CRI (continuous, per capita crime rate by town), IND (proportion of non-retail business acres per town), CHA (binary, if tract bounds river or not), NOX (continuous, nitric oxides concentration), RM (continuous, average number of rooms per dwelling), AGE (continuous, proportion of owner-occupied units built prior to 1940), DIS (continuous, weighted distances to ﬁve Boston employment centres), TAX (continuous, full-value property-tax rate per 10,000 dollars), B (continuous, the proportion of blacks by town),
Revised: They are CRI (continuous, per capita crime rate by town), IND (proportion of non-retail business acres per town), CHA (binary, if tract bounds river or not), NOX (continuous, nitric oxides concentration), RM (continuous, average number of rooms per dwelling), AGE (continuous, proportion of owner-occupied units built prior to 1940), DIS (continuous, weighted distances to ﬁve Boston employment centres), TAX (continuous, full-value property-tax rate per 10,000 dollars), B (continuous, the proportion of blacks by town),

Original: LST (continuous, percentage lower status of the population), and MED (continuous, median value of owner-occupied homes).
Revised: LST (continuous, percentage lower status of the population), and MED (continuous, median value of owner-occupied homes).

Original: We used the same settings as in the simulation experiments, i.e., we set the ruled-out threshold (cid:15) = 0 .
Revised: We used the same settings as in the simulation experiments, i.e., we set the ruled-out threshold (cid:15) = 0 .

Original: 1 and tolerance parameter ω
Revised: 1 and tolerance parameter ω

Original: = 1 e − 8 .
Revised: = 1 e − 8 .

Original: To provide reliable performance, due to the different scales of the large number of variables, we standardized the continuous variables before employing our method.
Revised: To provide reliable performance, due to the different scales of the large number of variables, we standardized the continuous variables before employing our method.

Original: The resulting causal graphs are demonstrated in Figure 3.
Revised: The resulting causal graphs are demonstrated in Figure 3.

Original: Since the logistic method estimated more than 40 edges, and LiNGAM estimated nearly 20 edges, which have much more spurious edges than others, we only showed the comparison results of laplace and[Figure]
Revised: Since the logistic method estimated more than 40 edges, and LiNGAM estimated nearly 20 edges, which have much more spurious edges than others, we only showed the comparison results of laplace and

Original: PC methods.
Revised: PC methods.

Revised: As shown in Figure 3(a), on the one hand, though it is arguable

Original: As shown in Figure 3(a), on the one hand, though it is arguable that RM may not  be an effect variable, we still found some interesting conclusions which were accordance with our common understandings.
Revised: that RM may not [Figure] be an effect variable, we still found some interesting conclusions which were accordance with our common understandings.

Original: For example, MED is inﬂuenced by LST , which is determined by some house-related indicators,
Revised: For example, MED is inﬂuenced by LST , which is determined by some house-related indicators,

Original: i
Revised: i

Original: [Figure]
Revised: [Figure]

Original: e., IND , AGE and TAX ; there is no direct link between NOX and MED but they are dependent through intermediate causal relationships (Margaritis, 2005); it is reasonable that TAX , which reﬂects the government’s housing policy, inﬂuences IND , LST , and CRI (Kenyon et al., 2012).
Revised: e., IND , AGE and TAX ; there is no direct link between NOX and MED but they are dependent through intermediate causal relationships (Margaritis, 2005); it is reasonable that TAX , which reﬂects the government’s housing policy, inﬂuences IND , LST [Figure] and CRI (Kenyon et al., 2012).

Original: On the other hand, laplace and PC methods estimated the TAX as an effect variable, which was not consistent with our common understanding (Kenyon et al., 2012).
Revised: On the other hand, laplace and PC methods estimated the TAX as an effect variable, which was not consistent with our common understanding (Kenyon et al., 2012).

Original: Overall, the results illustrated the effectiveness of our proposed LiM method in inferring causal graphs from mixed data.
Revised: Overall, the results illustrated the effectiveness of our proposed LiM method in inferring causal graphs from mixed data.

Original: 6. Conclusions
Revised: 6. Conclusions

Original: In this paper, we provided complete identiﬁability conditions for causal discovery with linear mixed data that consist of continuous and discrete variables, both in bivariate and multivariate cases.
Revised: In this paper, we provided complete identiﬁability conditions for causal discovery with linear mixed data that consist of continuous and discrete variables, both in bivariate and multivariate cases.

Original: Further, we proposed a two-step hybrid approach to uniquely identify the causal structure.
Revised: Further, we proposed a two-step hybrid approach to uniquely identify the causal structure.

Original: Experiments on synthetic as well as real-world data demonstrate that our LiM method outperformed the comparisons.
Revised: Experiments on synthetic as well as real-world data demonstrate that our LiM method outperformed the comparisons.

Original: There are several questions that we aim to answer in the future research.
Revised: There are several questions that we aim to answer in the future research.

Original: First, in this paper, we give mathematical identiﬁability for mixed data under the causal sufﬁciency.
Revised: First, in this paper, we give mathematical identiﬁability for mixed data under the causal sufﬁciency.

Original: But in many realistic applications, causal sufﬁciency may not hold and there may exist latent variables or confounders in the underlying causal graphs.
Revised: But in many realistic applications, causal sufﬁciency may not hold and there may exist latent variables or confounders in the underlying causal graphs.

Original: Thus, it is signiﬁcant to develop a general identiﬁability condition[Figure] without the causal sufﬁciency for mixed data.
Revised: Thus, it is signiﬁcant to develop a general identiﬁability condition without the causal sufﬁciency for mixed data.

Original: Second, the linearity assumption may decrease the  learning effect and limit the applicability.
Revised: Second, the linearity assumption may decrease the [Figure] learning effect and limit the applicability.

Original: It is essential to generalize the identiﬁability to cases where there are nonlinear relationships, not only for bivarate but also multi-variate causal networks.
Revised: It is essential to generalize the identiﬁability to cases where there are nonlinear relationships, not only for bivarate but also multi-variate causal networks.

Original: As for the nonlinear causal modeling, there are some aspects that we could consider deeper research.
Revised: As for the nonlinear causal modeling, there are some aspects that we could consider deeper research.

Original: For instance, we could focus on the cases where the mixed data are generated by the PNL (Zhang and Hyv¨arinen, 2009), or where they are generated by a nonlinear deterministic relation without noise (Janzing et al., 2012; Zeng et al., 2021a), etc.
Revised: For instance, we could focus on the cases where the mixed data are generated by the PNL (Zhang and Hyv¨arinen, 2009), or where they are generated by a nonlinear deterministic relation without noise (Janzing et al., 2012; Zeng et al., 2021a), etc.

Original: Third, in practice, there may also exist other types of discrete variables, e.g., multi-categorical variables or ordinal variables.
Revised: Third, in practice, there may also exist other types of discrete variables, e.g., multi-categorical variables or ordinal variables.

Original: Ignoring ordinal information may have adverse effects on model estimation (von Eye and Wiedermann, 2018).
Revised: Ignoring ordinal information may have adverse effects on model estimation (von Eye and Wiedermann, 2018).

Original: Thus, it is desirable to extend our method to cover general data types.
Revised: Thus, it is desirable to extend our method to cover general data types.

Original: Acknowledgments
Revised: Acknowledgments

Original: We are grateful to the anonymous reviewers whose careful comments and suggestions helped improve this manuscript.
Revised: We are grateful to the anonymous reviewers whose careful comments and suggestions helped improve this manuscript.

Original: This work was supported by the Grants ONR N00014-20-1-2501 and KAKENHI 20K11708.
Revised: This work was supported by the Grants ONR N00014-20-1-2501 and KAKENHI 20K11708.