Original: E STIMATING T REATMENT E FFECTS VIA O RTHOGONAL R EGULARIZATION
Revised: E STIMATING T REATMENT E FFECTS VIA O RTHOGONAL R EGULARIZATION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Decision-making often requires accurate estimation of causal effects from observational data.
Revised: Decision-making often requires accurate estimation of causal effects from observational data.

Original: This is challenging as outcomes of alternative decisions are not observed and have to be estimated.
Revised: This is challenging as outcomes of alternative decisions are not observed and have to be estimated.

Original: Previous methods estimate outcomes based on unconfoundedness but neglect any constraints that unconfoundedness imposes on the outcomes.
Revised: Previous methods estimate outcomes based on unconfoundedness but neglect any constraints that unconfoundedness imposes on the outcomes.

Original: In this paper, we propose a novel regularization framework in which we formalize unconfoundedness as an orthogonality constraint.
Revised: In this paper, we propose a novel regularization framework in which we formalize unconfoundedness as an orthogonality constraint.

Original: We provide theoretical guarantees that this yields an asymptotically normal estimator for the average causal effect.
Revised: We provide theoretical guarantees that this yields an asymptotically normal estimator for the average causal effect.

Original: Compared to other estimators, its asymptotic variance is strictly smaller.
Revised: Compared to other estimators, its asymptotic variance is strictly smaller.

Original: Based on our regularization framework, we develop d eep o rthogonal n etworks for u nconfounded t reatments (DONUT) which learn outcomes that are orthogonal to the treatment assignment.
Revised: Based on our regularization framework, we develop d eep o rthogonal n etworks for u nconfounded t reatments (DONUT) which learn outcomes that are orthogonal to the treatment assignment.

Original: Using a variety of benchmark datasets for causal inference, we demonstrate that DONUT outperforms the state-of-the-art substantially.
Revised: Using a variety of benchmark datasets for causal inference, we demonstrate that DONUT outperforms the state-of-the-art substantially.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Estimating the causal effect of an intervention (i. e., treatment effect) is an important problem in many domains such as marketing (Brodersen et al., 2015), economics (Heckman et al., 1997), and epidemiology (Robins et al., 2000).
Revised: Estimating the causal effect of an intervention (i. e., treatment effect) is integral for individual decision making in many domains such as marketing (Brodersen et al., 2015; Hatt & Feuerriegel, 2020), economics (Heckman et al., 1997), and epidemiology (Robins et al., 2000).

Original: For instance, in order to control an epidemic, it is relevant for public decision-makers to estimate the causal effect of school-closures (intervention) on the infection rate (outcome).
Revised: For instance, in order to control an epidemic, it is relevant for public decision-makers to estimate the causal effect of school-closures (intervention) on the infection rate (outcome).

Original: The causal effect of an intervention can be estimated in two ways: randomized control trials (RCTs) and observational studies .
Revised: The causal effect of an intervention can be estimated in two ways: randomized control trials (RCTs) and observational studies .

Original: RCTs are widely recognized as the gold standard for estimating causal effects, yet conducting RCTs is often infeasible (Robins et al., 2000).
Revised: RCTs are widely recognized as the gold standard for estimating causal effects, yet conducting RCTs is often infeasible (Robins et al., 2000).

Original: For instance, randomly allocating different policy interventions during an epidemic might be unethical and impractical.
Revised: For instance, randomly allocating different policy interventions during an epidemic might be unethical and impractical.

Original: Unlike RCTs, observational studies adopt observed data to infer causal effects.
Revised: Unlike RCTs, observational studies adopt observed data to infer causal effects.

Original: For this, covariates must be collected that contain all confounders (i. e., variables that affect both treatment and outcome).
Revised: For this, covariates must be collected that contain all confounders (i. e., variables that affect both treatment and outcome).

Original: This is becoming increasingly common due to ease of access to rich data.
Revised: This is becoming increasingly common due to ease of access to rich data.

Original: In this paper, we estimate the average causal effect of a treatment from observational data.
Revised: In this paper, we estimate the average causal effect of a treatment from observational data.

Original: In order to estimate the causal effect of a treatment, the outcome of an alternative treatment has to be estimated.
Revised: In order to estimate the causal effect of a treatment, the outcome of an alternative treatment has to be estimated.

Original: However, this is challenging, since we do not know what the outcome would have been if another treatment had been applied.
Revised: However, this is challenging, since we do not know what the outcome would have been if another treatment had been applied.

Original: Existing methods for estimating treatment effects use the treatment assignment as a feature and train regression models to estimate the outcomes (Funk et al., 2011; Kallus, 2017b).
Revised: Existing methods for estimating treatment effects use the treatment assignment as a feature and train regression models to estimate the outcomes (Funk et al., 2011; Kallus, 2017b).

Original: Methods based on nearest neighbors and matching are adopted to ﬁnd similar subjects (Ho et al., 2007; Crump et al., 2008; Kallus, 2017a; 2020).
Revised: Methods based on nearest neighbors and matching are adopted to ﬁnd similar subjects (Ho et al., 2007; Crump et al., 2008; Kallus, 2017a; 2020).

Original: Tree and forest-based methods (Wager & Athey, 2018) estimate the treatment effect at the leaf node and train many weak learners to build expressive ensemble models.
Revised: Tree and forest-based methods (Wager & Athey, 2018) estimate the treatment effect at the leaf node and train many weak learners to build expressive ensemble models.

Original: Gaussian process-based methods provide uncertainty quantiﬁcation (Alaa & van der Schaar, 2017; Ray & Szabo, 2019).
Revised: Gaussian process-based methods provide uncertainty quantiﬁcation (Alaa & van der Schaar, 2017; Ray & Szabo, 2019).

Original: Weighting-based approaches re-weight the outcomes using weights based on covariate and treatment data (Kallus, 2018).
Revised: Weighting-based approaches re-weight the outcomes using weights based on covariate and treatment data (Kallus, 2018).

Original: For instance, Fong et al.
Revised: For instance, Fong et al.

Original: ; Yiu & Su (2018) seek weights such that the treatment assignment is unassociated with the covariates.
Revised: ; Yiu & Su (2018) seek weights such that the treatment assignment is unassociated with the covariates.

Original: However, they do not require the treatment assignment to be unassociated with the potential outcomes.
Revised: However, they do not require the treatment assignment to be unassociated with the potential outcomes.

Original: Doubly robust methods combine a model for the outcomes and a model for the treatment propensity in a manner that is robust to misspeciﬁcation (Funk et al., 2011; Benkeser et al., 2017; Chernozhukov et al., 2018).
Revised: Doubly robust methods combine a model for the outcomes and a model for the treatment propensity in a manner that is robust to misspeciﬁcation (Funk et al., 2011; Benkeser et al., 2017; Chernozhukov et al., 2018).

Original: Recently, deep learning has been successful for this task due to its strong predictive performance and ability to learn representations of the data
Revised: Recently, deep learning has been successful for this task due to its strong predictive performance and ability to learn representations of the data

Original: (e. g., Johansson et al., 2016; Louizos et al., 2017; Shalit et al., 2017; Yao et al., 2018;
Revised: (e. g., Johansson et al., 2016; Louizos et al., 2017; Shalit et al., 2017; Yao et al., 2018;

Original: Yoon et al., 2018; Shi et al., 2019).
Revised: Yoon et al., 2018; Shi et al., 2019).

Original: To ensure identiﬁability of the causal effect, state-of-the-art methods for estimating treatment effects are based on unconfoundedness (i. e., all confounders are measured, and thus included in the covariates).
Revised: To ensure identiﬁability of the causal effect, state-of-the-art methods for estimating treatment effects are based on unconfoundedness (i. e., all confounders are measured, and thus included in the covariates).

Original: Hence, unconfoundedness is assumed for identiﬁability, yet, during estimation of the model parameters, any implications on the unobserved outcomes that arise from unconfoundedness have been neglected.
Revised: Hence, unconfoundedness is assumed for identiﬁability, yet, during estimation of the model parameters, any implications on the unobserved outcomes that arise from unconfoundedness have been neglected.

Original: Contribution.
Revised: Contribution.

Original: In this paper, (i) we introduce a regularization framework that exploits unconfoundedness .
Revised: In this paper, (i) we introduce a regularization framework that exploits unconfoundedness .

Original: To this end, we formalize unconfoundedness as an orthogonality constraint.
Revised: To this end, we formalize unconfoundedness as an orthogonality constraint.

Original: This constraint is used during estimation of the model parameters to ensure that the outcomes are orthogonal to the treatment assignment.
Revised: This constraint is used during estimation of the model parameters to ensure that the outcomes are orthogonal to the treatment assignment.

Original: We prove sufﬁcient conditions under which this yields an asymptotically normal estimator for the average causal effect.
Revised: We prove sufﬁcient conditions under which this yields an asymptotically normal estimator for the average causal effect.

Original: Compared to other estimators, its asymptotic variance is strictly smaller.
Revised: Compared to other estimators, its asymptotic variance is strictly smaller.

Original: (ii)
Revised: (ii)

Original: Based on our regularization framework, we develop d eep o rthogonal n etworks for u nconfounded t reatments ( DONUT ) for estimating average causal effects.
Revised: Based on our regularization framework, we develop d eep o rthogonal n etworks for u nconfounded t reatments ( DONUT ) for estimating average causal effects.

Original: DONUT leverages the predictive capabilities of neural networks to learn outcomes that are orthogonal to the treatment assignment.
Revised: DONUT leverages the predictive capabilities of neural networks to learn outcomes that are orthogonal to the treatment assignment.

Original: Using a variety of benchmark datasets for causal inference, we demonstrate that DONUT outperforms the state-of-the-art substantially.
Revised: Using a variety of benchmark datasets for causal inference, we demonstrate that DONUT outperforms the state-of-the-art substantially.

Original:  P ROBLEM S ETUP
Revised:  P ROBLEM S ETUP

Original: Our objective is to estimate the average treatment effect (ATE) of a binary treatment from observational data.
Revised: Our objective is to estimate the average treatment effect (ATE) of a binary treatment from observational data.

Original: For this, we build upon the Neyman-Rubin potential outcomes framework (Rubin, 2005).
Revised: For this, we build upon the Neyman-Rubin potential outcomes framework (Rubin, 2005).

Original: Consider a population where every subject i is described by the d -dimensional covariates
Revised: Consider a population where every subject i is described by the d -dimensional covariates

Original: X i ∈ R d .
Revised: X i ∈ R d .

Original: Each subject is assigned a treatment T i ∈ { 0 , 1 } .
Revised: Each subject is assigned a treatment T i ∈ { 0 , 1 } .

Original: The random variable Y i (1) corresponds to the outcome under treatment, i. e., T i = 1 , whereas Y i (0) corresponds to the outcome under no treatment, i. e., T i = 0 .
Revised: The random variable Y i (1) corresponds to the outcome under treatment, i. e., T i = 1 , whereas Y i (0) corresponds to the outcome under no treatment, i. e., T i = 0 .

Original: These two random variables, Y i (1) , Y i (0)
Revised: These two random variables, Y i (1) , Y i (0)

Original: ∈ R , are known as the potential outcomes.
Revised: ∈ R , are known as the potential outcomes.

Original: Due to the fundamental problem of causal inference, only one of the potential outcomes is observed, but never both.
Revised: Due to the fundamental problem of causal inference, only one of the potential outcomes is observed, but never both.

Original: The observed outcome is denoted by Y i .
Revised: The observed outcome is denoted by Y i .

Original: Our aim is to estimate the average treatment effect
Revised: Our aim is to estimate the average treatment effect

Original: [Equation]
Revised: [Equation]

Original: The following standard assumptions are sufﬁcient for identiﬁability of the causal effect (Imbens &
Revised: The following standard assumptions are sufﬁcient for identiﬁability of the causal effect (Imbens &

Original: Rubin, 2015): consistency (i. e., ∀ t ∈ { 0 , 1 } : Y
Revised: Rubin, 2015): consistency (i. e., ∀ t ∈ { 0 , 1 } : Y

Original: = Y ( t ) , if T = t ); positivity (i. e., ∀ x ∈ R d : 0 < P ( T = 1 | X = x ) < 1 ); and unconfoundedness .
Revised: = Y ( t ) , if T = t ); positivity (i. e., ∀ x ∈ R d : 0 < P ( T = 1 | X = x ) < 1 ); and unconfoundedness .

Original: Unconfoundedness assumes that all confounders are measured and, hence, conditioning on them blocks all backdoor paths.
Revised: Unconfoundedness assumes that all confounders are measured and, hence, conditioning on them blocks all backdoor paths.

Original: This is equivalent to assuming that the potential outcomes Y (1) and Y (0) are independent of the assigned treatment T given the covariates X , i. e.,
Revised: This is equivalent to assuming that the potential outcomes Y (1) and Y (0) are independent of the assigned treatment T given the covariates X , i. e.,

Original: [Equation]
Revised: [Equation]

Original: Based on this, the ATE is equal to
Revised: Based on this, the ATE is equal to

Original: [Equation]
Revised: [Equation]

Original: Our task is to estimate the function f
Revised: Our task is to estimate the function f

Original: ( x, t ) =
Revised: ( x, t ) =

Original: E [ Y | X
Revised: E [ Y | X

Original: = x, T = t ] for all x ∈ R d and t ∈ { 0 , 1 } based on observational data D = { ( X i , T i , Y i ) } ni
Revised: = x, T = t ] for all x ∈ R d and t ∈ { 0 , 1 } based on observational data D = { ( X i , T i , Y i ) } ni

Original: =1 .
Revised: =1 .

Original:  O RTHOGONAL R EGULARIZATION FOR E STIMATING T REATMENT E FFECTS
Revised:  O RTHOGONAL R EGULARIZATION FOR E STIMATING T REATMENT E FFECTS

Original: The key idea of our regularization framework is to exploit the implications on the outcomes, that result from unconfoundedness.
Revised: The key idea of our regularization framework is to exploit the implications on the outcomes, that result from unconfoundedness.

Original: For this, we formalize unconfoundedness as an orthogonality constraint.
Revised: For this, we formalize unconfoundedness as an orthogonality constraint.

Original: This orthogonality constraint ensures that the outcomes are orthogonal to the treatment assignment.
Revised: This orthogonality constraint ensures that the outcomes are orthogonal to the treatment assignment.

Original: We later introduce a speciﬁc variant of our regularization framework based on neural networks, which yields DONUT
Revised: We later introduce a speciﬁc variant of our regularization framework based on neural networks, which yields DONUT

Original: 3. U NCONFOUNDEDNESS AS O RTHOGONALITY C ONSTRAINT
Revised: 3. U NCONFOUNDEDNESS AS O RTHOGONALITY C ONSTRAINT

Original: Under unconfoundedness, the outcomes are independent of the assigned treatment given the covariates, i. e.,
Revised: Under unconfoundedness, the outcomes are independent of the assigned treatment given the covariates, i. e.,

Original: [Equation]
Revised: [Equation]

Original: Using the inner product (cid:104) V, W (cid:105)
Revised: Using the inner product (cid:104) V, W (cid:105)

Original: = 1 n (cid:80) ni
Revised: = 1 n (cid:80) ni

Original: =1 V i
Revised: =1 V i

Original: W
Revised: W

Original: i , we formalize the following orthogonality constraint :
Revised: i , we formalize the following orthogonality constraint :

Original: [Equation]
Revised: [Equation]

Original: for t ∈ { 0 , 1 } , and where f ( x, t )
Revised: for t ∈ { 0 , 1 } , and where f ( x, t )

Original: =
Revised: =

Original: E [ Y | X
Revised: E [ Y | X

Original: = x, T = t ] and π ( x ) =
Revised: = x, T = t ] and π ( x ) =

Original: E [ T | X = x ] .
Revised: E [ T | X = x ] .

Original: The function π is the propensity score .
Revised: The function π is the propensity score .

Original: This is a necessary condition for unconfoundedness, since unconfoundedness requires the covariance between Y ( t ) and T given X to be zero due to the independence of the outcomes and the treatment assignment.
Revised: This is a necessary condition for unconfoundedness, since unconfoundedness requires the covariance between Y ( t ) and T given X to be zero due to the independence of the outcomes and the treatment assignment.

Original: Hence, since the inner product in (5) is the empirical covariance between Y ( t ) and T given X , unconfoundedness requires the (centered) outcomes to be orthogonal to the (centered) treatment assignment with respect to the above inner product.
Revised: Hence, since the inner product in (5) is the empirical covariance between Y ( t ) and T given X , unconfoundedness requires the (centered) outcomes to be orthogonal to the (centered) treatment assignment with respect to the above inner product.

Original: An appropriate method for estimating outcomes based on unconfoundedness should ensure that the orthogonality constraint holds.
Revised: An appropriate method for estimating outcomes based on unconfoundedness should ensure that the orthogonality constraint holds.

Original: Although existing methods are based on unconfoundedness, the orthogonality of the outcomes to the treatment assignment is ignored during estimation of the model parameters.
Revised: Although existing methods are based on unconfoundedness, the orthogonality of the outcomes to the treatment assignment is ignored during estimation of the model parameters.

Original: As a remedy, we propose a regularization framework that accommodates the orthogonality constraint in the estimation procedure.
Revised: As a remedy, we propose a regularization framework that accommodates the orthogonality constraint in the estimation procedure.

Original: 3. P ROPOSED R EGULARIZATION F RAMEWORK
Revised: 3. P ROPOSED R EGULARIZATION F RAMEWORK

Original: In our regularization framework, the orthogonality constraint in (5) is included in the estimation procedure as follows.
Revised: In our regularization framework, the orthogonality constraint in (5) is included in the estimation procedure as follows.

Original: Let H f ⊆
Revised: Let H f ⊆

Original: [ R d × { 0 , 1 } → R ] and H π ⊆
Revised: [ R d × { 0 , 1 } → R ] and H π ⊆

Original: [ R d → [0 , 1]] be function classes for f and π , and
Revised: [ R d → [0 , 1]] be function classes for f and π , and

Original: (cid:15) ∈ R a model parameter.
Revised: (cid:15) ∈ R a model parameter.

Original: Then, the objective is to ﬁnd the solution to the optimization problem
Revised: Then, the objective is to ﬁnd the solution to the optimization problem

Original: [Equation]
Revised: [Equation]

Original: where L FL is the so-called factual loss (see Section 3.2.1) between the estimated models and observed (factual) data.
Revised: where L FL is the so-called factual loss (see Section 3.2.1) between the estimated models and observed (factual) data.

Original: The term Ω OR is our new orthogonal regularization that enforces the orthogonality constraint using the model parameter (cid:15) (see Section 3.2.2).
Revised: The term Ω OR is our new orthogonal regularization that enforces the orthogonality constraint using the model parameter (cid:15) (see Section 3.2.2).

Original: The variable λ ∈ R
Revised: The variable λ ∈ R

Original: + is a hyperparameter controlling the strength of regularization.
Revised: + is a hyperparameter controlling the strength of regularization.

Original: We describe both components of (6), i. e., factual loss and orthogonal regularization, in the following.
Revised: We describe both components of (6), i. e., factual loss and orthogonal regularization, in the following.

Original: 3.2. F ACTUAL L OSS L FL
Revised: 3.2. F ACTUAL L OSS L FL

Original: Similar to previous work (e. g., Shi et al., 2019), the outcome model f and propensity score model π are estimated using the observed data and the factual loss given by
Revised: Similar to previous work (e. g., Shi et al., 2019), the outcome model f and propensity score model π are estimated using the observed data and the factual loss given by

Original: [Equation]
Revised: [Equation]

Original: where α ∈ R + is a hyperparameter weighting the terms of the factual loss.
Revised: where α ∈ R + is a hyperparameter weighting the terms of the factual loss.

Original: We note the following about the estimation of the outcome model f in (7).
Revised: We note the following about the estimation of the outcome model f in (7).

Original: In the ﬁrst term of the factual loss, f is ﬁtted to the observed outcomes and, thus, no information about the unobserved outcomes is used.
Revised: In the ﬁrst term of the factual loss, f is ﬁtted to the observed outcomes and, thus, no information about the unobserved outcomes is used.

Original: As a consequence, the model learns about the observed outcomes, but not about the unobserved outcomes.
Revised: As a consequence, the model learns about the observed outcomes, but not about the unobserved outcomes.

Original: State-of-the-art methods exclusively rely on the factual loss to estimate the outcomes (e. g. Shalit et al., 2017).
Revised: State-of-the-art methods exclusively rely on the factual loss to estimate the outcomes (e. g. Shalit et al., 2017).

Original: This is based on the assumptions that similar subjects have similar outcomes (Yao et al., 2018) and that for every subject in the data, there is a similar subject in the data with the opposite treatment.
Revised: This is based on the assumptions that similar subjects have similar outcomes (Yao et al., 2018) and that for every subject in the data, there is a similar subject in the data with the opposite treatment.

Original: However, this is unlikely in presence of selection bias, i e., when treatment and control groups differ systemically.
Revised: However, this is unlikely in presence of selection bias, i e., when treatment and control groups differ systemically.

Original: This pitfall is addressed by our orthogonal regularization in the following.
Revised: This pitfall is addressed by our orthogonal regularization in the following.

Original: 3.2. O RTHOGONAL R EGULARIZATION Ω OR
Revised: 3.2. O RTHOGONAL R EGULARIZATION Ω OR

Original: We now specify the orthogonal regularization term Ω OR that ensures that the orthogonality constraint in (5) is satisﬁed.
Revised: We now specify the orthogonal regularization term Ω OR that ensures that the orthogonality constraint in (5) is satisﬁed.

Original: We ﬁrst state Ω
Revised: We ﬁrst state Ω

Original: OR and then explain each of its parts.
Revised: OR and then explain each of its parts.

Original: The orthogonal regularization term is given by
Revised: The orthogonal regularization term is given by

Original: [Equation]
Revised: [Equation]

Original: with
Revised: with

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: where the pseudo outcome Y ∗ i (0) and the perturbation function f (cid:15) are required to learn outcomes that are orthogonal to the treatment assignment, as described in the following.
Revised: where the pseudo outcome Y ∗ i (0) and the perturbation function f (cid:15) are required to learn outcomes that are orthogonal to the treatment assignment, as described in the following.

Original: Pseudo outcome.
Revised: Pseudo outcome.

Original: The untreated outcome of a subject can be expressed as Y i (0) =
Revised: The untreated outcome of a subject can be expressed as Y i (0) =

Original: Y i − ψ ( X i )
Revised: Y i − ψ ( X i )

Original: T i , where ψ ( X ) =
Revised: T i , where ψ ( X ) =

Original: f ( X, 1) − f
Revised: f ( X, 1) − f

Original: ( X, 0) is the true treatment effect at X .
Revised: ( X, 0) is the true treatment effect at X .

Original: Hence, if we had access to the treatment effect ψ ( X ) , we would also have access to the untreated outcome Y i (0) even if we did not observe it.
Revised: Hence, if we had access to the treatment effect ψ ( X ) , we would also have access to the untreated outcome Y i (0) even if we did not observe it.

Original: We use the average treatment effect of the current model ﬁt, i. e., ψ ∗ in (9), as proxy for ψ ( X i ) .
Revised: We use the average treatment effect of the current model ﬁt, i. e., ψ ∗ in (9), as proxy for ψ ( X i ) .

Original: This creates a pseudo outcome, Y ∗ i (0)
Revised: This creates a pseudo outcome, Y ∗ i (0)

Original: =
Revised: =

Original: Y i − ψ ∗ T i .
Revised: Y i − ψ ∗ T i .

Original: Under sufﬁcient conditions, this converges to the true outcome (see Section 4).
Revised: Under sufﬁcient conditions, this converges to the true outcome (see Section 4).

Original: Perturbation function.
Revised: Perturbation function.

Original: In order to ensure that the orthogonality constraint is satisﬁed, i. e., (cid:104) Y ( t ) − f ( X, t ) , T − π ( X ) (cid:105) = 0 , we use a perturbation function f (cid:15) .
Revised: In order to ensure that the orthogonality constraint is satisﬁed, i. e., (cid:104) Y ( t ) − f ( X, t ) , T − π ( X ) (cid:105) = 0 , we use a perturbation function f (cid:15) .

Original: Simply adding the inner product as a regularization term is not sufﬁcient, since this does not guarantee that the orthogonality constraint holds at a solution of (6).
Revised: Simply adding the inner product as a regularization term is not sufﬁcient, since this does not guarantee that the orthogonality constraint holds at a solution of (6).

Original: A similar approach is used in targeted minimum loss estimation (e. g., van der Laan & Rose, 2011) and, more general, dual problems in optimization (Zalinescu, 2002).
Revised: A similar approach is used in targeted minimum loss estimation (e. g., van der Laan & Rose, 2011) and, more general, dual problems in optimization (Zalinescu, 2002).

Original: We extend the function f to a perturbation function f (cid:15) using the model parameter (cid:15) as in (10).
Revised: We extend the function f to a perturbation function f (cid:15) using the model parameter (cid:15) as in (10).

Original: As a result, solving the optimization problem in (6) forces the outcome estimates f (cid:15) to satisfy the orthogonality constraint.
Revised: As a result, solving the optimization problem in (6) forces the outcome estimates f (cid:15) to satisfy the orthogonality constraint.

Original: Mathematically, this can be seen by taking the partial derivative of (6) and setting it to zero, i. e.,
Revised: Mathematically, this can be seen by taking the partial derivative of (6) and setting it to zero, i. e.,

Original: [Equation]
Revised: [Equation]

Original: and, hence, (cid:104) Y ∗ (0) − f (cid:15) ( X, 0) , T − π ( X ) (cid:105) = 0 .
Revised: and, hence, (cid:104) Y ∗ (0) − f (cid:15) ( X, 0) , T − π ( X ) (cid:105) = 0 .

Original: It is analytically sufﬁcient to regularize only the untreated outcome.
Revised: It is analytically sufﬁcient to regularize only the untreated outcome.

Original: Whenever one of the outcomes is orthogonal to the treatment assignment, the other outcome is also orthogonal.
Revised: Whenever one of the outcomes is orthogonal to the treatment assignment, the other outcome is also orthogonal.

Original: A proof of this is provided in Appendix A.
Revised: A proof of this is provided in Appendix A.

Original: Then, the outcomes are estimated using f (cid:15) ( x, t ) and, therefore, the estimate for the ATE is ˆ ψ =
Revised: Then, the outcomes are estimated using f (cid:15) ( x, t ) and, therefore, the estimate for the ATE is ˆ ψ =

Original: X i
Revised: X i

Original: ) , which coincides with ψ ∗ , since the perturbation terms in (10) cancel.
Revised: ) , which coincides with ψ ∗ , since the perturbation terms in (10) cancel.

Original: 3. D EEP O
Revised: 3. D EEP O

Original: [Equation]
Revised: [Equation]

Original: N ETWORKS FOR U NCONFOUNDED T REATMENTS
Revised: N ETWORKS FOR U NCONFOUNDED T REATMENTS

Original: Our regularization framework works with any outcome model f and propensity score model π that is estimated through a loss function.
Revised: Our regularization framework works with any outcome model f and propensity score model π that is estimated through a loss function.

Original: The regularization framework ensures that the outcomes are estimated such that they are orthogonal to the treatment assignment.
Revised: The regularization framework ensures that the outcomes are estimated such that they are orthogonal to the treatment assignment.

Original: We introduce a speciﬁc variant of our regularization framework based on feedforward neural networks, which yields d eep o rthogonal n etworks for u nconfounded t reatments ( DONUT ).
Revised: We introduce a speciﬁc variant of our regularization framework based on feedforward neural networks, which yields d eep o rthogonal n etworks for u nconfounded t reatments ( DONUT ).

Original: Neural networks present a suitable model class due to their strong predictive performance.
Revised: Neural networks present a suitable model class due to their strong predictive performance.

Original: The architecture of DONUT is set as follows.
Revised: The architecture of DONUT is set as follows.

Original: For the outcome model f , we use the basic architecture of TARNet (Shalit et al., 2017).
Revised: For the outcome model f , we use the basic architecture of TARNet (Shalit et al., 2017).

Original: TARNet uses a deep feedforward neural network to produce a representation layer, followed by two 2-layer neural networks to predict each of the potential outcomes from the shared representation.
Revised: TARNet uses a deep feedforward neural network to produce a representation layer, followed by two 2-layer neural networks to predict each of the potential outcomes from the shared representation.

Original: For the propensity score model π , we use a logistic regression (Rosenbaum & Rubin, 1983).
Revised: For the propensity score model π , we use a logistic regression (Rosenbaum & Rubin, 1983).

Original: The choice of feedforward neural networks is particularly strengthened by the theoretical discussion in Section 4.
Revised: The choice of feedforward neural networks is particularly strengthened by the theoretical discussion in Section 4.

Original:  T HEORETICAL G UARANTEES
Revised:  T HEORETICAL G UARANTEES

Original: We prove sufﬁcient conditions under which our regularization framework yields an asymptotically normal 6 estimator ˆ ψ for the true ATE ψ .
Revised: We prove sufﬁcient conditions under which our regularization framework yields an asymptotically normal 6 estimator ˆ ψ for the true ATE ψ .

Original: Asymptotic normality is particularly favorable for an estimator, as such estimators converge to the true ATE at a rate 1 / √ n .
Revised: Asymptotic normality is particularly favorable for an estimator, as such estimators converge to the true ATE at a rate 1 / √ n .

Original: We further show that, compared to other estimators, our estimator has strictly smaller asymptotic variance.
Revised: We further show that, compared to other estimators, our estimator has strictly smaller asymptotic variance.

Original: Theorem 1.
Revised: Theorem 1.

Original: (Asymptotic Normality.)
Revised: (Asymptotic Normality.)

Original: Suppose
Revised: Suppose

Original: The estimator ˆ η = ( ˆ f, ˆ π ) for the outcome and propensity score model converges to some ¯ η = ( ¯ f, ¯ π ) in the sense that (cid:107) ˆ η − ¯ η (cid:107) =
Revised: The estimator ˆ η = ( ˆ f, ˆ π ) for the outcome and propensity score model converges to some ¯ η = ( ¯ f, ¯ π ) in the sense that (cid:107) ˆ η − ¯ η (cid:107) =

Original: o p (1) , where either ¯ f = f or ¯ π = π (or both) corresponds to the true function.
Revised: o p (1) , where either ¯ f = f or ¯ π = π (or both) corresponds to the true function.

Original: The treatment effect is homogeneous,
Revised: The treatment effect is homogeneous,

Original: i
Revised: i

Original: e., ∀ x ∈ R d : ψ ( x ) = ψ . 3.
Revised: e., ∀ x ∈ R d : ψ ( x ) = ψ . 3.

Original: The estimators ˆ f
Revised: The estimators ˆ f

Original: and ˆ π take values in P -Donsker classes, i. e., H f , H π
Revised: and ˆ π take values in P -Donsker classes, i. e., H f , H π

Original: [Equation]
Revised: [Equation]

Original: CLT ( P ) .
Revised: CLT ( P ) .

Original: Then ˆ ψ is asymptotically normal, i. e.,
Revised: Then ˆ ψ is asymptotically normal, i. e.,

Original: [Equation]
Revised: [Equation]

Original: where σ 2 is the variance of the outcome.
Revised: where σ 2 is the variance of the outcome.

Original: The proof is provided in Appendix C. We make the following remarks about the result and its conditions.
Revised: The proof is provided in Appendix C. We make the following remarks about the result and its conditions.

Original: Condition 1 requires that either the model for the outcomes f or the model for the propensity score π (or both) is correctly speciﬁed, but not necessarily both.
Revised: Condition 1 requires that either the model for the outcomes f or the model for the propensity score π (or both) is correctly speciﬁed, but not necessarily both.

Original: This means that our estimator is doubly robust , since it is consistent under correct speciﬁcation of either f or π .
Revised: This means that our estimator is doubly robust , since it is consistent under correct speciﬁcation of either f or π .

Original: In particular, neural networks converge at a fast enough rate to invoke the doubly robustness condition (Farrell et al., 2018).
Revised: In particular, neural networks converge at a fast enough rate to invoke the doubly robustness condition (Farrell et al., 2018).

Original: Condition 2 can be easily relaxed to any speciﬁcation of ψ as long as it has ﬁnitely many parameters and given that the appropriate identiﬁcation criteria hold.
Revised: Condition 2 can be easily relaxed to any speciﬁcation of ψ as long as it has ﬁnitely many parameters and given that the appropriate identiﬁcation criteria hold.

Original: This changes the orthogonal regularization in (8) accordingly; see Appendix D for details.
Revised: This changes the orthogonal regularization in (8) accordingly; see Appendix D for details.

Original: However, the advantage of Condition 2 is that it explains why the asymptotic variance of ˆ ψ is smaller compared to other estimators.
Revised: However, the advantage of Condition 2 is that it explains why the asymptotic variance of ˆ ψ is smaller compared to other estimators.

Original: In particular, the difference in asymptotic variance to some estimators (e. g., the inverse probability weighted estimator) can be sizable if the propensity score is close to zero or one.
Revised: In particular, the difference in asymptotic variance to some estimators (e. g., the inverse probability weighted estimator) can be sizable if the propensity score is close to zero or one.

Original: Often, this difference is not offset by weaker restrictions imposed by heterogeneous treatment effects (e. g.,
Revised: Often, this difference is not offset by weaker restrictions imposed by heterogeneous treatment effects (e. g.,

Original: Vansteelandt & Joffe, 2014).
Revised: Vansteelandt & Joffe, 2014).

Original: Condition 3 captures a large class of functions of estimators for f and π from which we can choose.
Revised: Condition 3 captures a large class of functions of estimators for f and π from which we can choose.

Original: An overview of P -Donsker classes is given in Mikosch et al.
Revised: An overview of P -Donsker classes is given in Mikosch et al.

Original: In particular, the class of feedforward neural networks is a P -Donsker class, since Lipschitz parametric functions are P -Donsker functions, and any Lipschitz transformation of a P -Donsker function is again a P -Donsker function.
Revised: In particular, the class of feedforward neural networks is a P -Donsker class, since Lipschitz parametric functions are P -Donsker functions, and any Lipschitz transformation of a P -Donsker function is again a P -Donsker function.

Original: For mathematical details, see Appendix E. Together with the convergence rate for doubly robustness, this provides theoretical justiﬁcation for the use of neural networks in our regularization framework, and therefore in DONUT.
Revised: For mathematical details, see Appendix E. Together with the convergence rate for doubly robustness, this provides theoretical justiﬁcation for the use of neural networks in our regularization framework, and therefore in DONUT.

Original: Similarity to partially linear regression.
Revised: Similarity to partially linear regression.

Original: We ﬁnd an interesting similarity between our estimator ˆ ψ and the estimator obtained by partially linear regression (PLR) (see (1.5) in Chernozhukov et al.
Revised: We ﬁnd an interesting similarity between our estimator ˆ ψ and the estimator obtained by partially linear regression (PLR) (see (1.5) in Chernozhukov et al.

Original: The analytical expression of ˆ ψ yield by our regularization framework in (8) is given in (31) in Appendix C.
Revised: The analytical expression of ˆ ψ yield by our regularization framework in (8) is given in (31) in Appendix C.

Original: This is similar to the estimator obtained by PLR after partialling the effect of
Revised: This is similar to the estimator obtained by PLR after partialling the effect of

Original: X out from T (see (1.5) in Chernozhukov et al.
Revised: X out from T (see (1.5) in Chernozhukov et al.

Original: However, the PLR separately estimates the nuisance functions ˆ f and ˆ π and then plugs them into the estimator.
Revised: However, the PLR separately estimates the nuisance functions ˆ f and ˆ π and then plugs them into the estimator.

Original: Our approach does not use the analytical expression as plug-in estimator, but the estimator arises directly from solving the
Revised: Our approach does not use the analytical expression as plug-in estimator, but the estimator arises directly from solving the

Original: A class F of measurable functions on a probability space (Ω , A , P ) is called a P -Donsker class if, for G P n = √ n ( P n − P ) , the empirical process { G P n f : f ∈ F} n ≥ 1 converges weakly to a P -Brownian bridge G P .
Revised: A class F of measurable functions on a probability space (Ω , A , P ) is called a P -Donsker class if, for G P n = √ n ( P n − P ) , the empirical process { G P n f : f ∈ F} n ≥ 1 converges weakly to a P -Brownian bridge G P .

Original: include details.
Revised: include details.

Original: optimization problem in (6) via (8).
Revised: optimization problem in (6) via (8).

Original: The difference becomes apparent in the experiments (Section 5), where we include the PLR as a baseline and ﬁnd that the performance of DONUT is superior.
Revised: The difference becomes apparent in the experiments (Section 5), where we include the PLR as a baseline and ﬁnd that the performance of DONUT is superior.

Original: Comparison to other estimators.
Revised: Comparison to other estimators.

Original: We compare the asymptotic behavior of our estimator ˆ ψ to other regular estimators (e. g., Funk et al.
Revised: We compare the asymptotic behavior of our estimator ˆ ψ to other regular estimators (e. g., Funk et al.

Original: (2011); Nie & Wager (2017)).
Revised: (2011); Nie & Wager (2017)).

Original: We ﬁnd that, under the conditions of Theorem 1, there does not exist a regular estimator that achieves strictly smaller asymptotic variance than our estimator.
Revised: We ﬁnd that, under the conditions of Theorem 1, there does not exist a regular estimator that achieves strictly smaller asymptotic variance than our estimator.

Original: The reason for this is as follows.
Revised: The reason for this is as follows.

Original: We proof in Appendix C that our regularization framework yields an asymptotically normal estimator for the ATE.
Revised: We proof in Appendix C that our regularization framework yields an asymptotically normal estimator for the ATE.

Original: In particular, we proof that our estimator is efﬁcient (see Appendix C for the efﬁcient inﬂuence function), and therefore it achieves the efﬁciency bound.
Revised: In particular, we proof that our estimator is efﬁcient (see Appendix C for the efﬁcient inﬂuence function), and therefore it achieves the efﬁciency bound.

Original: As a consequence, among all regular estimators, our estimator achieves the smallest asymptotic variance.
Revised: As a consequence, among all regular estimators, our estimator achieves the smallest asymptotic variance.

Original: In Appendix G, we make an example in which we compare the inverse probability weighted estimator to our estimator.
Revised: In Appendix G, we make an example in which we compare the inverse probability weighted estimator to our estimator.

Original: We show that the difference in asymptotic variance becomes particularly pronounced in presence of selection bias (i. e., when treatment and control groups differ systemically).
Revised: We show that the difference in asymptotic variance becomes particularly pronounced in presence of selection bias (i. e., when treatment and control groups differ systemically).

Original: To summarize, we prove sufﬁcient conditions
Revised: To summarize, we prove sufﬁcient conditions

Original: under which our estimator ˆ ψ is asymptotically normal.
Revised: under which our estimator ˆ ψ is asymptotically normal.

Original: Hence, our proposed estimator converges to the true ATE ψ at a fast rate.
Revised: Hence, our proposed estimator converges to the true ATE ψ at a fast rate.

Original: Compared to other regular estimators, its asymptotic variance is strictly smaller, since we prove that our estimator is efﬁcient.
Revised: Compared to other regular estimators, its asymptotic variance is strictly smaller, since we prove that our estimator is efﬁcient.

Original: Since feedforward neural networks converge at a fast enough rate to invoke doubly robustness and belong to a P -Donsker class, these results hold true when using feedforward neural networks in our regularization framework, which yields DONUT.
Revised: Since feedforward neural networks converge at a fast enough rate to invoke doubly robustness and belong to a P -Donsker class, these results hold true when using feedforward neural networks in our regularization framework, which yields DONUT.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: Our proposed DONUT is evaluated against state-of-the-art baselines, where we ﬁnd that its ATE estimation is superior (Section 5.2).
Revised: Our proposed DONUT is evaluated against state-of-the-art baselines, where we ﬁnd that its ATE estimation is superior (Section 5.2).

Original: Its beneﬁts appear especially for problems subject to selection bias, which is conﬁrmed as part of a simulation study (Section 5.3).
Revised: Its beneﬁts appear especially for problems subject to selection bias, which is conﬁrmed as part of a simulation study (Section 5.3).

Original: In the latter, the theoretical guarantees from Section 4 are conﬁrmed.
Revised: In the latter, the theoretical guarantees from Section 4 are conﬁrmed.

Original: 5. S ETUP
Revised: 5. S ETUP

Original: Evaluating methods for estimating causal effects is challenging as we rarely have access to ground truth causal effects.
Revised: Evaluating methods for estimating causal effects is challenging as we rarely have access to ground truth causal effects.

Original: Established procedures for evaluation of such methods rely on semi-synthetic data, which reﬂect the real world.
Revised: Established procedures for evaluation of such methods rely on semi-synthetic data, which reﬂect the real world.

Original: Our experimental setup follows established procedure regarding datasets, baselines, and performance metrics (e. g., Johansson et al., 2016; Shalit et al., 2017).
Revised: Our experimental setup follows established procedure regarding datasets, baselines, and performance metrics (e. g., Johansson et al., 2016; Shalit et al., 2017).

Original: Datasets.
Revised: Datasets.

Original: We evaluate all methods across four benchmark datasets for causal inference: IHDP (e. g.,
Revised: We evaluate all methods across four benchmark datasets for causal inference: IHDP (e. g.,

Original: Johansson et al., 2016), Twins (e. g., Yoon et al., 2018), ACIC 2018 (e. g., Shi et al., 2019), and Jobs (e. g., Shalit et al., 2017).
Revised: Johansson et al., 2016), Twins (e. g., Yoon et al., 2018), ACIC 2018 (e. g., Shi et al., 2019), and Jobs (e. g., Shalit et al., 2017).

Original: The ﬁrst three datasets are semi-synthetic, while the last originated from a
Revised: The ﬁrst three datasets are semi-synthetic, while the last originated from a

Original: RCT.
Revised: RCT.

Original: Details on IHDP, Twins, ACIC 2018, and Jobs are provided in Appendix H.
Revised: Details on IHDP, Twins, ACIC 2018, and Jobs are provided in Appendix H.

Original: Training details.
Revised: Training details.

Original: DONUT is trained using the regularization framework in (6), where both the outcome model and the propensity score model are trained jointly using stochastic gradient descent with momentum.
Revised: DONUT is trained using the regularization framework in (6), where both the outcome model and the propensity score model are trained jointly using stochastic gradient descent with momentum.

Original: The hidden layer size is 200 for the representation layers and 100 for the outcome layers similar to Shalit et al.
Revised: The hidden layer size is 200 for the representation layers and 100 for the outcome layers similar to Shalit et al.

Original: ; Shi et al.
Revised: ; Shi et al.

Original: The hyperparameter α in the factual loss (7) is set to 1 and λ in the orthogonal regularization (8) is determined by hyperparameter optimization over { 10 k } 2 k = − 2 .
Revised: The hyperparameter α in the factual loss (7) is set to 1 and λ in the orthogonal regularization (8) is determined by hyperparameter optimization over { 10 k } 2 k = − 2 .

Original: For IHDP, we follow established practice (e. g., Shalit et al., 2017) and average over 1,000 realizations of the outcomes with 63/27/10 train/validation/test splits.
Revised: For IHDP, we follow established practice (e. g., Shalit et al., 2017) and average over 1,000 realizations of the outcomes with 63/27/10 train/validation/test splits.

Original: Following Shalit et al.
Revised: Following Shalit et al.

Original: (2017); Yoon et al.
Revised: (2017); Yoon et al.

Original: ; Shi et al.
Revised: ; Shi et al.

Original: (2019), we average over 100 different train/validation/test splits for Twins and Jobs, and over 10 splits for each dataset for ACIC, all with ratios 56/24/20.
Revised: (2019), we average over 100 different train/validation/test splits for Twins and Jobs, and over 10 splits for each dataset for ACIC, all with ratios 56/24/20.

Original: Baselines.
Revised: Baselines.

Original: We compare DONUT against 17 state-of-the-art methods for estimating treatment effects, organized in the following groups: (i) Regression methods: Linear regression with treatment as covariate ( OLS/LR-1 ), separate linear regressors for each treatment ( OLS/LR-2 ), and balancing linear regression ( BLR )
Revised: We compare DONUT against 17 state-of-the-art methods for estimating treatment effects, organized in the following groups: (i) Regression methods: Linear regression with treatment as covariate ( OLS/LR-1 ), separate linear regressors for each treatment ( OLS/LR-2 ), and balancing linear regression ( BLR )

Original: (Johansson et al., 2016); (ii) Matching methods: k -nearest neighbor ( k -NN )
Revised: (Johansson et al., 2016); (ii) Matching methods: k -nearest neighbor ( k -NN )

Original: (Crump et al., 2008); (iii) Tree methods: Bayesian additive regression trees ( BART ) (Chipman et al., 2012), random forest ( R-Forest ) (Breiman, 2001), and causal forest ( C-Forest ) (Wager &
Revised: (Crump et al., 2008); (iii) Tree methods: Bayesian additive regression trees ( BART ) (Chipman et al., 2012), random forest ( R-Forest ) (Breiman, 2001), and causal forest ( C-Forest ) (Wager &

Original: Athey, 2018); (iv) Gaussian process methods: Causal multi-task Gaussian process ( CMGP ) (Alaa & van der Schaar, 2017) and debiased Gaussian process ( D-GP ) (Ray & Szabo, 2019); (v) Neural network methods: Balancing neural network ( BNN )
Revised: Athey, 2018); (iv) Gaussian process methods: Causal multi-task Gaussian process ( CMGP ) (Alaa & van der Schaar, 2017) and debiased Gaussian process ( D-GP ) (Ray & Szabo, 2019); (v) Neural network methods: Balancing neural network ( BNN )

Original: (Johansson et al., 2016), treatment-agnostic
Revised: (Johansson et al., 2016), treatment-agnostic

Original: [Table]
Revised: [Table]

Original: representation network ( TARNet ) (Shalit et al., 2017), counterfactual regression with Wasserstein distance ( CFR-WASS ) (Shalit et al., 2017), generative adversarial networks ( GANITE )
Revised: representation network ( TARNet ) (Shalit et al., 2017), counterfactual regression with Wasserstein distance ( CFR-WASS ) (Shalit et al., 2017), generative adversarial networks ( GANITE )

Original: (Yoon et al., 2018), and Dragonnet (Shi et al., 2019); (vi) Plug-in estimators: Augmented inverse probability weighted estimator ( AIPWE ) (Cao et al., 2009), targeted maximum likelihood estimator ( TMLE ) (van der Laan & Rubin, 2006), and partially linear regression ( PLR )
Revised: (Yoon et al., 2018), and Dragonnet (Shi et al., 2019); (vi) Plug-in estimators: Augmented inverse probability weighted estimator ( AIPWE ) (Cao et al., 2009), targeted maximum likelihood estimator ( TMLE ) (van der Laan & Rubin, 2006), and partially linear regression ( PLR )

Original: (Chernozhukov et al., 2018)
Revised: (Chernozhukov et al., 2018)

Original: all of them using the outcome and propensity score model of Dragonnet as nuisance functions.
Revised: all of them using the outcome and propensity score model of Dragonnet as nuisance functions.

Original: Performance metrics.
Revised: Performance metrics.

Original: Following established procedure, we report the following metrics for each dataset.
Revised: Following established procedure, we report the following metrics for each dataset.

Original: For IHDP and ACIC 2018, we use the absolute error in average treatment effect (Johansson et al., 2016): (cid:15) ATE = | 1 n (cid:80) ni
Revised: For IHDP and ACIC 2018, we use the absolute error in average treatment effect (Johansson et al., 2016): (cid:15) ATE = | 1 n (cid:80) ni

Original: =

Revised: =

Original: ( f ( x i , 1) − f ( x i , 0)) − 1 n (cid:80) ni
Revised: ( f ( x i , 1) − f ( x i , 0)) − 1 n (cid:80) ni

Original: =

Revised: =

Original: ( ˆ f ( x i , 1) − ˆ f
Revised: ( ˆ f ( x i , 1) − ˆ f

Original: ( x i , 0))
Revised: ( x i , 0))

Original: | .
Revised: | .

Original: For Twins, we use the absolute error in observed average treatment effect (Yoon et al., 2018): (cid:15) ATE = | 1 n (cid:80) ni
Revised: For Twins, we use the absolute error in observed average treatment effect (Yoon et al., 2018): (cid:15) ATE = | 1 n (cid:80) ni

Original: =

Revised: =

Original: ( y i (1) − y i (0)) − 1 n (cid:80) ni
Revised: ( y i (1) − y i (0)) − 1 n (cid:80) ni

Original: =

Revised: =

Original: (ˆ y i (1) − ˆ y i (0))
Revised: (ˆ y i (1) − ˆ y i (0))

Original: | .
Revised: | .

Original: For Jobs, all treated subjects T were part of the original randomized sample E , and hence, the true average treatment effect can be computed on the treated by ATT = |T
Revised: For Jobs, all treated subjects T were part of the original randomized sample E , and hence, the true average treatment effect can be computed on the treated by ATT = |T

Original: | − 1 (cid:80) i ∈T y
Revised: | − 1 (cid:80) i ∈T y

Original: i − |C ∩ E| − 1 (cid:80)
Revised: i − |C ∩ E| − 1 (cid:80)

Original: i
Revised: i

Original: ∈C∩E y i , where C is the control group.
Revised: ∈C∩E y i , where C is the control group.

Original: Similar to (Shalit et al., 2017), we then use the error:
Revised: Similar to (Shalit et al., 2017), we then use the error:

Original: (cid:15) ATT = | ATT − |T | − 1 (cid:80)
Revised: (cid:15) ATT = | ATT − |T | − 1 (cid:80)

Original: i ∈T ( ˆ f ( x i , 1) − ˆ
Revised: i ∈T ( ˆ f ( x i , 1) − ˆ

Original: f
Revised: f

Original: ( x i , 0))
Revised: ( x i , 0))

Original: | .
Revised: | .

Original: 5. R ESULTS
Revised: 5. R ESULTS

Original: IHDP, Twins, and Jobs have been used to evaluate many methods for estimating treatment effects.
Revised: IHDP, Twins, and Jobs have been used to evaluate many methods for estimating treatment effects.

Original: In
Revised: In

Original: Table 1, the results of the experiments on IHDP, Twins, and Jobs are presented.
Revised: Table 1, the results of the experiments on IHDP, Twins, and Jobs are presented.

Original: Overall, DONUT achieves competitive performance across all datasets.
Revised: Overall, DONUT achieves competitive performance across all datasets.

Original: On IHDP, the CMGP baseline achieves slightly lower estimation error (mean: .11 (CMGP) vs. .13 (DONUT)), but at the drawback of an inferior standard deviation (std.:
Revised: On IHDP, the CMGP baseline achieves slightly lower estimation error (mean: .11 (CMGP) vs. .13 (DONUT)), but at the drawback of an inferior standard deviation (std.:

Original: .10 (CMGP) vs. .01 (DONUT)).
Revised: .10 (CMGP) vs. .01 (DONUT)).

Original: However, as shown by previous work, the small sample size and the limited simulation settings of IHDP make it difﬁcult to draw conclusions about methods (e. g., Yoon et al., 2018; Shi et al., 2019).
Revised: However, as shown by previous work, the small sample size and the limited simulation settings of IHDP make it difﬁcult to draw conclusions about methods (e. g., Yoon et al., 2018; Shi et al., 2019).

Original: In contrast, DONUT achieves superior performance on both Twins and Jobs, where the number of samples is much larger (Twins: n = 11,400;
Revised: In contrast, DONUT achieves superior performance on both Twins and Jobs, where the number of samples is much larger (Twins: n = 11,400;

Original: Jobs: n = 3,212).
Revised: Jobs: n = 3,212).

Original: On these datasets, DONUT is state-of-the-art.
Revised: On these datasets, DONUT is state-of-the-art.

Original: In particular, we point out the difference to PLR which uses the analyitcal expression of our estimator similar to Chernozhukov et al.
Revised: In particular, we point out the difference to PLR which uses the analyitcal expression of our estimator similar to Chernozhukov et al.

Original: (2018) as discussed in Section 4, but as a plug-in estimator.
Revised: (2018) as discussed in Section 4, but as a plug-in estimator.

Original: For a reasonable comparison, we use the outcome models of Dragonnet as nuisance functions.
Revised: For a reasonable comparison, we use the outcome models of Dragonnet as nuisance functions.

Original: We ﬁnd that DONUT achieves superior performance.
Revised: We ﬁnd that DONUT achieves superior performance.

Original: Among neural network-based methods (i. e., BNN, TARNet, CFR-WASS, GANITE, Dragonnet, and DONUT), DONUT performs superior across all datasets.
Revised: Among neural network-based methods (i. e., BNN, TARNet, CFR-WASS, GANITE, Dragonnet, and DONUT), DONUT performs superior across all datasets.

Original: In comparison to TARNet, which shares the basic architecture of DONUT (but without orthogonal regularization), we achieve substantial reduction in ATE estimation error across all datasets (out-of-sample error reduction: 32.1 % on IHDP, 78.1 % on Twins, and 45.5 % on Jobs).
Revised: In comparison to TARNet, which shares the basic architecture of DONUT (but without orthogonal regularization), we achieve substantial reduction in ATE estimation error across all datasets (out-of-sample error reduction: 32.1 % on IHDP, 78.1 % on Twins, and 45.5 % on Jobs).

Original: This demonstrates the effectiveness of our regularization framework.
Revised: This demonstrates the effectiveness of our regularization framework.

Original: [Table]
Revised: [Table]

Original: We further evaluate DONUT using ACIC 2018.
Revised: We further evaluate DONUT using ACIC 2018.

Original: This collection of datasets was introduced for evaluating neural networks for estimating average treatment effects (Shi et al., 2019).
Revised: This collection of datasets was introduced for evaluating neural networks for estimating average treatment effects (Shi et al., 2019).

Original: We compare
Revised: We compare

Original: DONUT against the most competitive, and current state-of-the-art method, i. e., Dragonnet.
Revised: DONUT against the most competitive, and current state-of-the-art method, i. e., Dragonnet.

Original: In addition, we compare against TARNet (i. e., DONUT, but without orthogonal regularization).
Revised: In addition, we compare against TARNet (i. e., DONUT, but without orthogonal regularization).

Original: Table 2 presents the results of the experiments on ACIC 2018.
Revised: Table 2 presents the results of the experiments on ACIC 2018.

Original: The main observation is that DONUT improves estimation relative to TARNet (DONUT without orthogonal regularization) and relative to Dragonnet across a large collection of datasets.
Revised: The main observation is that DONUT improves estimation relative to TARNet (DONUT without orthogonal regularization) and relative to Dragonnet across a large collection of datasets.

Original: These results further conﬁrm the beneﬁt of including orthogonal regularization in the estimation procedure.
Revised: These results further conﬁrm the beneﬁt of including orthogonal regularization in the estimation procedure.

Original: 5. S IMULATION S TUDY ON S ELECTION B IAS
Revised: 5. S IMULATION S TUDY ON S ELECTION B IAS

Original: To evaluate the robustness of DONUT with regards to selection bias (i. e., when treatment and control groups differ substantially), we generate synthetic data with varying selection bias according to a similar protocol as in
Revised: To evaluate the robustness of DONUT with regards to selection bias (i. e., when treatment and control groups differ substantially), we generate synthetic data with varying selection bias according to a similar protocol as in

Original: Yao et al.
Revised: Yao et al.

Original: (2018); Yoon et al.
Revised: (2018); Yoon et al.

Original: Details on the protocol are provided in Appendix I.
Revised: Details on the protocol are provided in Appendix I.

Original:  
Revised:  

Original: 
Revised:

Original:    .XOOEDFN/HLEOHUGLYHUJHQFH       
Revised:    .XOOEDFN/HLEOHUGLYHUJHQFH       

Original: A T E '2187'UDJRQQHW
Revised: A T E '2187'UDJRQQHW

Original: We compare DONUT against Dragonnet, which makes use of the AIPWE to estimate the average treatment effect.
Revised: We compare DONUT against Dragonnet, which makes use of the AIPWE to estimate the average treatment effect.

Original: This is particular interesting for the comparison with DONUT due to the results in Section 4.
Revised: This is particular interesting for the comparison with DONUT due to the results in Section 4.

Original: We showed that, compared to any other estimator, our estimator obtains strictly smaller asymptotic variance, especially in presence of selection bias.
Revised: We showed that, compared to any other estimator, our estimator obtains strictly smaller asymptotic variance, especially in presence of selection bias.

Original: Figure 1 presents the mean and standard deviation of (cid:15) ATE for varying selection bias.
Revised: Figure 1 presents the mean and standard deviation of (cid:15) ATE for varying selection bias.

Original: We report two major insights: (i)
Revised: We report two major insights: (i)

Original: Dragonnet is outperformed by DONUT across different levels of selection bias.
Revised: Dragonnet is outperformed by DONUT across different levels of selection bias.

Original: As selection bias increases, the estimation error of DONUT remains stable.
Revised: As selection bias increases, the estimation error of DONUT remains stable.

Original: (ii)
Revised: (ii)

Original: The standard deviation of both DONUT and Dragonnet increases as selection bias increases.
Revised: The standard deviation of both DONUT and Dragonnet increases as selection bias increases.

Original: However, in line with our ﬁndings that DONUT has the smallest asymptotic variance among all regular estimators, the standard deviation of DONUT remains consistently smaller than the standard deviation of Dragonnet.
Revised: However, in line with our ﬁndings that DONUT has the smallest asymptotic variance among all regular estimators, the standard deviation of DONUT remains consistently smaller than the standard deviation of Dragonnet.

Original: In addition, this difference becomes more pronounced the larger the selection bias.
Revised: In addition, this difference becomes more pronounced the larger the selection bias.

Original: Hence, we observe empirical properties of our estimator that coincide with the theory derived in Section 4.
Revised: Hence, we observe empirical properties of our estimator that coincide with the theory derived in Section 4.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: Understanding causal effects is crucial for reliable decision-making.
Revised: Understanding causal effects is crucial for reliable decision-making.

Original: In this paper, we present a regularization framework for estimating average causal effects.
Revised: In this paper, we present a regularization framework for estimating average causal effects.

Original: We formalize unconfoundedness as an orthogonality constraint that is used to learn outcomes that are orthogonal to the treatment assignment.
Revised: We formalize unconfoundedness as an orthogonality constraint that is used to learn outcomes that are orthogonal to the treatment assignment.

Original: We prove theoretical guarantees that our regularization framework yields an asymptotically normal estimator.
Revised: We prove theoretical guarantees that our regularization framework yields an asymptotically normal estimator.

Original: Based on this, we develop DONUT, which leverages the predictive capabilities of neural networks to estimate average causal effects.
Revised: Based on this, we develop DONUT, which leverages the predictive capabilities of neural networks to estimate average causal effects.

Original: Experiments on datasets show that, in most cases,
Revised: Experiments on datasets show that, in most cases,

Original: DONUT outperforms the state-of-the-art substantially.
Revised: DONUT outperforms the state-of-the-art substantially.

Original: This work provides an interesting avenue for future research on causal inference.
Revised: This work provides an interesting avenue for future research on causal inference.

Original: We hypothesize that most existing models can be improved by incorporating orthogonal regularization.
Revised: We hypothesize that most existing models can be improved by incorporating orthogonal regularization.

Original: We leave the derivation of a unifying theory for future work.
Revised: We leave the derivation of a unifying theory for future work.

Revised: A revised version of this work can be found in Hatt & Feuerriegel (2021).

Original: A S UFFICIENCY OF R EGULARIZING THE U NTREATED O UTCOME
Revised: A S UFFICIENCY OF R EGULARIZING THE U NTREATED O UTCOME

Original: We show that it is sufﬁcient to regularize Y (0) , since orthogonality of Y (0) implies orthogonality of
Revised: We show that it is sufﬁcient to regularize Y (0) , since orthogonality of Y (0) implies orthogonality of

Original: Y (1) to T given X .
Revised: Y (1) to T given X .

Original: Suppose Y (0) satisﬁes the orthogonality constraint,
Revised: Suppose Y (0) satisﬁes the orthogonality constraint,

Original: i
Revised: i

Original: [Equation]
Revised: [Equation]

Original: e.,
Revised: e.,

Original: [Equation]
Revised: [Equation]

Original: Then,
Revised: Then,

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: ) using that Y (0) =
Revised: ) using that Y (0) =

Original: Y − ψ ( X ) T and Y (1) =
Revised: Y − ψ ( X ) T and Y (1) =

Original: Y + ψ ( X ) (1 − T ) in (18) and (20).
Revised: Y + ψ ( X ) (1 − T ) in (18) and (20).

Original: Hence, (cid:104) Y (1) − f
Revised: Hence, (cid:104) Y (1) − f

Original: ( X, 1) , T − π ( X ) (cid:105) is zero and, therefore, Y (1) is orthogonal to T given X .
Revised: ( X, 1) , T − π ( X ) (cid:105) is zero and, therefore, Y (1) is orthogonal to T given X .

Original: Nevertheless, it is straightforward to adapt the orthogonal regularizer in (8) to accommodate both outcomes.
Revised: Nevertheless, it is straightforward to adapt the orthogonal regularizer in (8) to accommodate both outcomes.

Original: Consider the following adapted orthogonal regularizer
Revised: Consider the following adapted orthogonal regularizer

Original: [Equation]
Revised: [Equation]

Original: with
Revised: with

Original: Y
Revised: Y

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: , (27) where (cid:15) is a additional model parameters.
Revised: , (27) where (cid:15) is a additional model parameters.

Original: Similar to the result in Section 3.2.2.
Revised: Similar to the result in Section 3.2.2.

Original: , this yields that (cid:104) Y ∗ (0) − f (cid:15) ( X, 0) , T − π ( X ) (cid:105) and (cid:104)
Revised: , this yields that (cid:104) Y ∗ (0) − f (cid:15) ( X, 0) , T − π ( X ) (cid:105) and (cid:104)

Original: Y ∗ (1) − f (cid:15) ( X, 1) , T − π ( X ) (cid:105) are zero, when the partial derivative of (24) w. r. t. (cid:15) is zero.
Revised: Y ∗ (1) − f (cid:15) ( X, 1) , T − π ( X ) (cid:105) are zero, when the partial derivative of (24) w. r. t. (cid:15) is zero.

Original: B A SYMPTOTICALLY N ORMAL E STIMATORS
Revised: B A SYMPTOTICALLY N ORMAL E STIMATORS

Original: In this section, we give a brief discussion on asymptotically normal estimators.
Revised: In this section, we give a brief discussion on asymptotically normal estimators.

Original: We refer to van der
Revised: We refer to van der

Original: Vaart (1998) for an in-depth discussion.
Revised: Vaart (1998) for an in-depth discussion.

Original: Suppose ˆ ψ is an estimator for the true parameter ψ .
Revised: Suppose ˆ ψ is an estimator for the true parameter ψ .

Original: Then, ˆ ψ is an asymptotically normal estimator if it satisﬁes
Revised: Then, ˆ ψ is an asymptotically normal estimator if it satisﬁes

Original: [Equation]
Revised: [Equation]

Original: where φ ( Z ) is referred to as the inﬂuence function at Z =
Revised: where φ ( Z ) is referred to as the inﬂuence function at Z =

Original: ( X, T, Y )
Revised: ( X, T, Y )

Original: (Kandasamy et al., 2015).
Revised: (Kandasamy et al., 2015).

Original: The inﬂuence function has mean zero and ﬁnite variance (i. e., E [ φ ( Z )]
Revised: The inﬂuence function has mean zero and ﬁnite variance (i. e., E [ φ ( Z )]

Original: = 

Revised: =

Original: and Var ( φ ( Z )) < ∞ ).
Revised: and Var ( φ ( Z )) < ∞ ).

Original: The existence and uniqueness of the inﬂuence function follows by the Riesz representation theorem.
Revised: The existence and uniqueness of the inﬂuence function follows by the Riesz representation theorem.

Original: Asymptotic normality is a favorable property of an estimator, since, by the central limit theorem, √
Revised: Asymptotic normality is a favorable property of an estimator, since, by the central limit theorem, √

Original: [Equation]
Revised: [Equation]

Original: Hence, an asymptotically normal estimator is asymptotically normal distributed and unbiased.
Revised: Hence, an asymptotically normal estimator is asymptotically normal distributed and unbiased.

Original: Moreover, the variance shrinks in proportion to 1 / √ n as the sample size grows and is given by the variance of the inﬂuence function.
Revised: Moreover, the variance shrinks in proportion to 1 / √ n as the sample size grows and is given by the variance of the inﬂuence function.

Original: As a consequence, the distribution of the estimator ˆ ψ converges weakly to a dirac delta function centered around the true ψ .
Revised: As a consequence, the distribution of the estimator ˆ ψ converges weakly to a dirac delta function centered around the true ψ .

Original: C P ROOF OF T HEOREM
Revised: C P ROOF OF T HEOREM

Original: In observational studies, the covariates X are often high-dimensional and limited knowledge about the nuisance functions (i. e., outcome and propensity score functions f and π ) is available.
Revised: In observational studies, the covariates X are often high-dimensional and limited knowledge about the nuisance functions (i. e., outcome and propensity score functions f and π ) is available.

Original: In such a case, it is reasonable to use ﬂexible, data-adaptive methods for estimating the nuisance functions, e. g., neural networks.
Revised: In such a case, it is reasonable to use ﬂexible, data-adaptive methods for estimating the nuisance functions, e. g., neural networks.

Original: However, the complexity of these methods makes asymptotic analysis difﬁcult, because the estimators used to construct ˆ η = ( ˆ f, ˆ π ) are not described by a single ﬁnite-dimensional parameter.
Revised: However, the complexity of these methods makes asymptotic analysis difﬁcult, because the estimators used to construct ˆ η = ( ˆ f, ˆ π ) are not described by a single ﬁnite-dimensional parameter.

Original: Nevertheless, under some conditions, we can learn about the asymptotics of ˆ ψ using tools from empirical process theory.
Revised: Nevertheless, under some conditions, we can learn about the asymptotics of ˆ ψ using tools from empirical process theory.

Original: First, we need to introduce some notation.
Revised: First, we need to introduce some notation.

Original: Throughout the proof, we will use P { f ( Z ) } =
Revised: Throughout the proof, we will use P { f ( Z ) } =

Original: (cid:82) f ( z ) d P to denote expectations of f ( Z ) for a random variable Z (treating the function f as ﬁxed).
Revised: (cid:82) f ( z ) d P to denote expectations of f ( Z ) for a random variable Z (treating the function f as ﬁxed).

Original: Hence,
Revised: Hence,

Original: P { ˆ f ( Z ) } is random if ˆ f is random (e. g., estimated from the sample).
Revised: P { ˆ f ( Z ) } is random if ˆ f is random (e. g., estimated from the sample).

Original: In contrast, E [ ˆ f ( Z )] is a ﬁxed non-random quantity, which averages over randomness in both Z and ˆ f and thus will not equal
Revised: In contrast, E [ ˆ f ( Z )] is a ﬁxed non-random quantity, which averages over randomness in both Z and ˆ f and thus will not equal

Original: P { ˆ f ( Z ) } except when ˆ f = f is ﬁxed and non-random.
Revised: P { ˆ f ( Z ) } except when ˆ f = f is ﬁxed and non-random.

Original: Moreover, we let P n = 1 n (cid:80) ni
Revised: Moreover, we let P n = 1 n (cid:80) ni

Original: =

Revised: =

Original: δ Z i denote the empirical measure such that sample averages can be written as 1 n (cid:80) ni
Revised: δ Z i denote the empirical measure such that sample averages can be written as 1 n (cid:80) ni

Original: =1 f ( Z i ) =
Revised: =1 f ( Z i ) =

Original: (cid:82) f ( z ) d P n = (cid:98) P n ( f ( Z )) .
Revised: (cid:82) f ( z ) d P n = (cid:98) P n ( f ( Z )) .

Original: For clarity, we will denote the true nuisance functions and average treatment effect as η 0 = ( f 0 , π 0 ) and ψ 0 , respectively.
Revised: For clarity, we will denote the true nuisance functions and average treatment effect as η 0 = ( f 0 , π 0 ) and ψ 0 , respectively.

Original: We further denote the triplet ( Y, T, X ) by Z .
Revised: We further denote the triplet ( Y, T, X ) by Z .

Original: For ˆ f = f (cid:15) , ˆ π , and ˆ ψ = 1 n (cid:80) ni
Revised: For ˆ f = f (cid:15) , ˆ π , and ˆ ψ = 1 n (cid:80) ni

Original: =1 f (cid:15) ( X i , 1) − f (cid:15) ( X i , 0) from (8), the regularization framework yields (cid:104)
Revised: =1 f (cid:15) ( X i , 1) − f (cid:15) ( X i , 0) from (8), the regularization framework yields (cid:104)

Original: [Equation]
Revised: [Equation]

Original: (30) by construction.
Revised: (30) by construction.

Original: Hence, we receive an analytical expression of the estimator of the average treatment effect by solving the above for ˆ ψ , i. e.,
Revised: Hence, we receive an analytical expression of the estimator of the average treatment effect by solving the above for ˆ ψ , i. e.,

Original: [Equation]
Revised: [Equation]

Original: Therefore, the estimator is given by ˆ ψ = (cid:98) P n ( m ( Z ; ˆ η )) , where
Revised: Therefore, the estimator is given by ˆ ψ = (cid:98) P n ( m ( Z ; ˆ η )) , where

Original: [Equation]
Revised: [Equation]

Original: and η = ( π, f ) denotes the nuisance functions.
Revised: and η = ( π, f ) denotes the nuisance functions.

Original: We recall the conditions in Theorem 1.
Revised: We recall the conditions in Theorem 1.

Original: Suppose the estimator ˆ η = ( ˆ f, ˆ π ) converges to some ¯ η = ( ¯ f, ¯ π ) in the sense that (cid:107) ˆ η − ¯ η (cid:107)
Revised: Suppose the estimator ˆ η = ( ˆ f, ˆ π ) converges to some ¯ η = ( ¯ f, ¯ π ) in the sense that (cid:107) ˆ η − ¯ η (cid:107)

Original: =
Revised: =

Original: o p (1) 8 , where either ¯ f = f 0 or ¯ π = π 0 (or both) corresponds to the true nuisance function.
Revised: o p (1) 8 , where either ¯ f = f 0 or ¯ π = π 0 (or both) corresponds to the true nuisance function.

Original: Thus at least one nuisance estimator needs to converge to the correct function, but one can be misspeciﬁed.
Revised: Thus at least one nuisance estimator needs to converge to the correct function, but one can be misspeciﬁed.

Original: Then, P { m ( Z ; ¯ η ) } = P { m ( Z ; η 0 ) }
Revised: Then, P { m ( Z ; ¯ η ) } = P { m ( Z ; η 0 ) }

Original: = ψ 0 , from the straightforward to check fact that P { ( m ( Z ; ¯ π, f 0 ) } =
Revised: = ψ 0 , from the straightforward to check fact that P { ( m ( Z ; ¯ π, f 0 ) } =

Original: P { m ( Z ; π 0 , ¯ f ) } for any ¯ π and ¯ f . Consider the decomposition
Revised: P { m ( Z ; π 0 , ¯ f ) } for any ¯ π and ¯ f . Consider the decomposition

Original: [Equation]
Revised: [Equation]

Original: If the estimators for the nuisance functions ˆ η take values in P -Donsker classes, then m ( Z ; ˆ η ) also belongs to a P -Donsker class, since Lipschitz transformations of Donsker functions are again Donsker functions.
Revised: If the estimators for the nuisance functions ˆ η take values in P -Donsker classes, then m ( Z ; ˆ η ) also belongs to a P -Donsker class, since Lipschitz transformations of Donsker functions are again Donsker functions.

Original: The Donsker property together with the continuous mapping theorem yields that ( P n − P ) { m ( Z ; ˆ η ) } is asymptotically equivalent to ( P n − P ) { m ( Z ; ¯ η ) } up to o p (1 / √ n ) error (see Mikosch et al.
Revised: The Donsker property together with the continuous mapping theorem yields that ( P n − P ) { m ( Z ; ˆ η ) } is asymptotically equivalent to ( P n − P ) { m ( Z ; ¯ η ) } up to o p (1 / √ n ) error (see Mikosch et al.

Original: (1997) for more details on P -Donsker classes).
Revised: (1997) for more details on P -Donsker classes).

Original: Therefore,
Revised: Therefore,

Original: [Equation]
Revised: [Equation]

Original: It is left to show that P { m ( Z ; ˆ η ) − m ( Z ; ¯ η ) } is asymptotically negligible.
Revised: It is left to show that P { m ( Z ; ˆ η ) − m ( Z ; ¯ η ) } is asymptotically negligible.

Original: This term equals
Revised: This term equals

Original: [Equation]
Revised: [Equation]

Original: where we use P { m ( Z ; ¯ η ) }
Revised: where we use P { m ( Z ; ¯ η ) }

Original: =
Revised: =

Original: P { m ( Z ; η 0 ) } in (34), iterated expectation in (35), and P { T ( T − ˆ π ( X )) }
Revised: P { m ( Z ; η 0 ) } in (34), iterated expectation in (35), and P { T ( T − ˆ π ( X )) }

Original: =
Revised: =

Original: P { π 0 ( X )(1 − ˆ π ( X )) } (likewise for π 0 instead of ˆ π ) in (37).
Revised: P { π 0 ( X )(1 − ˆ π ( X )) } (likewise for π 0 instead of ˆ π ) in (37).

Original: In (39), and similarly in (341), we use that ψ 0 ( X ) =
Revised: In (39), and similarly in (341), we use that ψ 0 ( X ) =

Original: ψ 0 and, therefore, f 0 ( X, 1) − f 0 ( X, 0)
Revised: ψ 0 and, therefore, f 0 ( X, 1) − f 0 ( X, 0)

Original: =
Revised: =

Original: ψ 0 .
Revised: ψ 0 .

Original: As a result, by simplifying, the above equals
Revised: As a result, by simplifying, the above equals

Original: [Equation]
Revised: [Equation]

Original: Therefore, by the fact that π 0 and ˆ π are bounded away from zero and one, along with the CauchySchwarz inequality (i. e., P { fg } ≤ (cid:107) f (cid:107)(cid:107) g (cid:107) ), we have that (up to a multiplicative constant) | P { m ( Z ; ˆ η ) − m ( Z ; ¯ η ) }| is bounded above by
Revised: Therefore, by the fact that π 0 and ˆ π are bounded away from zero and one, along with the CauchySchwarz inequality (i. e., P { fg } ≤ (cid:107) f (cid:107)(cid:107) g (cid:107) ), we have that (up to a multiplicative constant) | P { m ( Z ; ˆ η ) − m ( Z ; ¯ η ) }| is bounded above by

Original: [Equation]
Revised: [Equation]

Original: Thus, for example if ˆ π is based on a correctly speciﬁed parametric model (e. g., logistic regression), so that (cid:107) ˆ π − π 0 (cid:107)
Revised: Thus, for example if ˆ π is based on a correctly speciﬁed parametric model (e. g., logistic regression), so that (cid:107) ˆ π − π 0 (cid:107)

Original: =
Revised: =

Original: o p (1 / √ n ) , then we only need ˆ f to be consistent, i. e., (cid:107) ˆ f − f 0 (cid:107) =
Revised: o p (1 / √ n ) , then we only need ˆ f to be consistent, i. e., (cid:107) ˆ f − f 0 (cid:107) =

Original: o p (1) , to make the product term P { m ( Z ; ˆ η ) − m ( Z ; ¯ η ) } =
Revised: o p (1) , to make the product term P { m ( Z ; ˆ η ) − m ( Z ; ¯ η ) } =

Original: o p (1 / √ n ) asymptotically negligible.
Revised: o p (1 / √ n ) asymptotically negligible.

Original: Then the doubly robust estimator satisﬁes ˆ ψ − ψ 

Revised: Then the doubly robust estimator satisﬁes ˆ ψ − ψ

Original: = ( P n − P ) { m ( Z ; η 0 ) } + o p (1 / √ n ) and it is efﬁcient with inﬂuence function φ ( Z ; ψ, η ) = m ( Z ; η ) − ψ .
Revised: = ( P n − P ) { m ( Z ; η 0 ) } + o p (1 / √ n ) and it is efﬁcient with inﬂuence function φ ( Z ; ψ, η ) = m ( Z ; η ) − ψ .

Original: Thus, this proves the asymptotic normality of ˆ ψ .
Revised: Thus, this proves the asymptotic normality of ˆ ψ .

Original: It remains to show that the asymptotic variance of ˆ ψ equals
Revised: It remains to show that the asymptotic variance of ˆ ψ equals

Original: [Equation]
Revised: [Equation]

Original: Note that in the paper, E [ f ( Z )] is used similarly to P { f ( Z ) } .
Revised: Note that in the paper, E [ f ( Z )] is used similarly to P { f ( Z ) } .

Original: We introduced the notion P { f ( Z ) } here to make clear that we only integrate over randomness in Z and not the function estimates ˆ f .
Revised: We introduced the notion P { f ( Z ) } here to make clear that we only integrate over randomness in Z and not the function estimates ˆ f .

Original: As seen in Appendix B, the asymptotic variance of an estimator is the variance of its inﬂuence functions.
Revised: As seen in Appendix B, the asymptotic variance of an estimator is the variance of its inﬂuence functions.

Original: From above, we know that the efﬁcient inﬂuence function of ˆ ψ is φ ( Z ; ψ, η ) = m
Revised: From above, we know that the efﬁcient inﬂuence function of ˆ ψ is φ ( Z ; ψ, η ) = m

Original: ( Z ; η ) − ψ .
Revised: ( Z ; η ) − ψ .

Original: Thus, using that P { φ ( Z ; ψ, η ) } = 0 (by deﬁnition),
Revised: Thus, using that P { φ ( Z ; ψ, η ) } = 0 (by deﬁnition),

Original: [Equation]
Revised: [Equation]

Original: This, together with
Revised: This, together with

Original: [Equation]
Revised: [Equation]

Original: where U =
Revised: where U =

Original: Y − f
Revised: Y − f

Original: ( X, T ) ∼ N (0 , σ 2 ) and P { T ( T − π ( X )) }
Revised: ( X, T ) ∼ N (0 , σ 2 ) and P { T ( T − π ( X )) }

Original: =
Revised: =

Original: P { Var ( T | X ) } proves the statement.
Revised: P { Var ( T | X ) } proves the statement.

Original: D O RTHOGONAL R EGULARIZATION UNDER L INEAR S PECIFICATION OF THE T REATMENT E FFECT
Revised: D O RTHOGONAL R EGULARIZATION UNDER L INEAR S PECIFICATION OF THE T REATMENT E FFECT

Original: In Theorem 1, condition 2 ensures the homogeneity of the treatment effect, i. e., ψ ( X ) =
Revised: In Theorem 1, condition 2 ensures the homogeneity of the treatment effect, i. e., ψ ( X ) =

Original: ψ .
Revised: ψ .

Original: This can be easily relaxed to any speciﬁcation of ψ as long as it has ﬁnitely many parameters and given that the appropriate identiﬁcation criteria hold (for linear speciﬁcation, this is the non-singularity of the design matrix).
Revised: This can be easily relaxed to any speciﬁcation of ψ as long as it has ﬁnitely many parameters and given that the appropriate identiﬁcation criteria hold (for linear speciﬁcation, this is the non-singularity of the design matrix).

Original: We explain in this section how the orthogonal regularization in (8) changes when linear speciﬁcation of the treatment effect is considered, i. e., ψ ( x ) = θ (cid:62) x , where θ ∈ R d .
Revised: We explain in this section how the orthogonal regularization in (8) changes when linear speciﬁcation of the treatment effect is considered, i. e., ψ ( x ) = θ (cid:62) x , where θ ∈ R d .

Original: Recall the orthogonal regularizer from (8)
Revised: Recall the orthogonal regularizer from (8)

Original: [Equation]
Revised: [Equation]

Original: where the perturbation function f (cid:15) remains unchanged.
Revised: where the perturbation function f (cid:15) remains unchanged.

Original: The pseudo outcome Y ∗ i (0) is now given as
Revised: The pseudo outcome Y ∗ i (0) is now given as

Original: [Equation]
Revised: [Equation]

Original: where θ is given by the system of linear equations,
Revised: where θ is given by the system of linear equations,

Original: [Equation]
Revised: [Equation]

Original: where v ( t, x ) is an arbitrary function of the dimension of θ , which ensures that the system of linear equations possesses a unique solution.
Revised: where v ( t, x ) is an arbitrary function of the dimension of θ , which ensures that the system of linear equations possesses a unique solution.

Original: For instance, in the case of ψ ( x ) = θ (cid:62) x , the choice v ( t i , x i ) =
Revised: For instance, in the case of ψ ( x ) = θ (cid:62) x , the choice v ( t i , x i ) =

Original: x i t i could be made.
Revised: x i t i could be made.

Original: The procedure goes as follows.
Revised: The procedure goes as follows.

Original: First, the above system of linear equations is solved for θ .
Revised: First, the above system of linear equations is solved for θ .

Original: Second, the pseudo outcome Y ∗ i (0) =
Revised: Second, the pseudo outcome Y ∗ i (0) =

Original: Y i − ( θ (cid:62) X i )
Revised: Y i − ( θ (cid:62) X i )

Original: T i can be computed.
Revised: T i can be computed.

Original: Finally, this is plugged into the regularizer.
Revised: Finally, this is plugged into the regularizer.

Original: The proof for asymptotic normality goes similar to the one under homogeneous treatment effect.
Revised: The proof for asymptotic normality goes similar to the one under homogeneous treatment effect.

Original: E F EEDFORWARD N EURAL N ETWORKS ARE IN P -D ONSKER CLASS
Revised: E F EEDFORWARD N EURAL N ETWORKS ARE IN P -D ONSKER CLASS

Original: We show that feedforward neural networks are in a P -Donsker class.
Revised: We show that feedforward neural networks are in a P -Donsker class.

Original: A feedforward neural network f d is deﬁned by the recursion
Revised: A feedforward neural network f d is deﬁned by the recursion

Original: [Equation]
Revised: [Equation]

Original:  for d ∈ N ≥ 1 , matrices { W i } di =1 of appropriate dimensions, activation function σ , which applies element-wise, and d is the depth of the neural network.
Revised: [Equation] d, (53) for d ∈ N ≥ 1 , matrices { W i } di =1 of appropriate dimensions, activation function σ , which applies element-wise, and d is the depth of the neural network.

Original: f 1 is clearly in a P -Donsker class as it is Lipschitz parametric.
Revised: f 1 is clearly in a P -Donsker class as it is Lipschitz parametric.

Original: The Donsker property is preserved under Lipschitz transformations.
Revised: The Donsker property is preserved under Lipschitz transformations.

Original: Hence, f i ( x ) , for i = 2 , . . .
Revised: Hence, f i ( x ) , for i = 2 , . . .

Original: , d , is in a P -Donsker class, if the activation function is a Lipschitz transformation.
Revised: , d , is in a P -Donsker class, if the activation function is a Lipschitz transformation.

Original: This is the case for most common activation functions.
Revised: This is the case for most common activation functions.

Original: We give two examples in the following.
Revised: We give two examples in the following.

Original: The ReLU activation function, σ ReLU ( x ) = max(0 , x ) , preserves the Donsker property, since a constant function is clearly in a P -Donsker class and taking the maximum of two P -Donsker functions is again a P -Donsker function due to the preservation under Lipschitz transformations.
Revised: The ReLU activation function, σ ReLU ( x ) = max(0 , x ) , preserves the Donsker property, since a constant function is clearly in a P -Donsker class and taking the maximum of two P -Donsker functions is again a P -Donsker function due to the preservation under Lipschitz transformations.

Original: The same follows for the ELU activation function,
Revised: The same follows for the ELU activation function,

Original: [Equation]
Revised: [Equation]

Original: where α ≥ 0 , since it is a Lipschitz transformation.
Revised: where α ≥ 0 , since it is a Lipschitz transformation.

Original: F A SYMPTOTICS OF ˆ ψ UNDER H ETEROGENEOUS T REATMENT E FFECTS
Revised: F A SYMPTOTICS OF ˆ ψ UNDER H ETEROGENEOUS T REATMENT E FFECTS

Original: Our estimator retains a useful interpretation under heterogeneous treatment effects.
Revised: Our estimator retains a useful interpretation under heterogeneous treatment effects.

Original: In this case, ˆ ψ converges to
Revised: In this case, ˆ ψ converges to

Original: [Equation]
Revised: [Equation]

Original: where ψ ( X ) = f ( X, 1) − f
Revised: where ψ ( X ) = f ( X, 1) − f

Original: ( X, 0) is the true treatment effect at X .
Revised: ( X, 0) is the true treatment effect at X .

Original: This can be interpreted as a weighted average of treatment effects ψ ( X ) .
Revised: This can be interpreted as a weighted average of treatment effects ψ ( X ) .

Original: As a consequence, most weight is given to subpopulations with large Var ( T | X ) , i. e., which are most informative about the treatment effect.
Revised: As a consequence, most weight is given to subpopulations with large Var ( T | X ) , i. e., which are most informative about the treatment effect.

Original: We consider the same notation as in the proof in Appendix C and resume it.
Revised: We consider the same notation as in the proof in Appendix C and resume it.

Original: From the proof in
Revised: From the proof in

Original: Appendix C, we know that
Revised: Appendix C, we know that

Original: [Equation]
Revised: [Equation]

Original: since the estimator of the nuisance functions ˆ η = ( ˆ f, ˆ π ) belong to a P -Donsker class, and therefore m ( Z ; ˆ η ) belongs to a P -Donsker class.
Revised: since the estimator of the nuisance functions ˆ η = ( ˆ f, ˆ π ) belong to a P -Donsker class, and therefore m ( Z ; ˆ η ) belongs to a P -Donsker class.

Original: Similar to the proof in Appendix C, it is left to investigate the term P { m ( Z ; ˆ η ) − m
Revised: Similar to the proof in Appendix C, it is left to investigate the term P { m ( Z ; ˆ η ) − m

Original: Z ; ¯ η ) } .
Revised: Z ; ¯ η ) } .

Original: We show that this term is asymptotically not negligible when the treatment effect is heterogeneous.
Revised: We show that this term is asymptotically not negligible when the treatment effect is heterogeneous.

Original: Again, by iterated expectations, this term equals
Revised: Again, by iterated expectations, this term equals

Original: [Equation]
Revised: [Equation]

Original: The ﬁrst term can be bounded from above (up to a multiplicative constant) by the Cauchy-Schwarz inequality and, therefore, is asymptotically negligible if one of the nuisance functions is correctly speciﬁed.
Revised: The ﬁrst term can be bounded from above (up to a multiplicative constant) by the Cauchy-Schwarz inequality and, therefore, is asymptotically negligible if one of the nuisance functions is correctly speciﬁed.

Original: Thus, only the second term remains.
Revised: Thus, only the second term remains.

Original: Using ψ ( X ) = f
Revised: Using ψ ( X ) = f

Original: 0 ( X, 1) − f 

Revised: 0 ( X, 1) − f

Original: ( X, 0) , the second term can be written as
Revised: ( X, 0) , the second term can be written as

Original: [Equation]
Revised: [Equation]

Original: Since π 0 ( X )(1 − π 0 ( X )) =
Revised: Since π 0 ( X )(1 − π 0 ( X )) =

Original: Var ( T | X ) , this concludes the proof.
Revised: Var ( T | X ) , this concludes the proof.

Original: G C OMPARISON TO I NVERSE P ROBABILITY W EIGHTED E STIMATOR
Revised: G C OMPARISON TO I NVERSE P ROBABILITY W EIGHTED E STIMATOR

Original: We compare our estimator ˆ ψ to the inverse probability weighted (IPW) estimator (e. g., Funk et al., 2011).
Revised: We compare our estimator ˆ ψ to the inverse probability weighted (IPW) estimator (e. g., Funk et al., 2011).

Original: The IPW estimator is a popular estimator for treatment effects due to its improvement in efﬁciency and reduction in bias compared to unweighted estimators.
Revised: The IPW estimator is a popular estimator for treatment effects due to its improvement in efﬁciency and reduction in bias compared to unweighted estimators.

Original: Under identical conditions as in Theorem 1, the IPW estimator, denoted as ˆ ψ IPW , is asymptotically normal with asymptotic variance σ 2 E [1 / Var ( T | X )] (e. g., Kennedy, 2016).
Revised: Under identical conditions as in Theorem 1, the IPW estimator, denoted as ˆ ψ IPW , is asymptotically normal with asymptotic variance σ 2 E [1 / Var ( T | X )] (e. g., Kennedy, 2016).

Original: We can compare the asymptotic behavior of our estimator ˆ ψ and the IPW estimator ˆ ψ IPW .
Revised: We can compare the asymptotic behavior of our estimator ˆ ψ and the IPW estimator ˆ ψ IPW .

Original: Both estimators are asymptotically unbiased, but with different asymptotic variance as the following result shows.
Revised: Both estimators are asymptotically unbiased, but with different asymptotic variance as the following result shows.

Original: Corollary 1.
Revised: Corollary 1.

Original: Under the conditions in Theorem 1, the asymptotic variance of ˆ ψ is strictly smaller than the asymptotic variance of ˆ ψ IPW .
Revised: Under the conditions in Theorem 1, the asymptotic variance of ˆ ψ is strictly smaller than the asymptotic variance of ˆ ψ IPW .

Original: Proof.
Revised: Proof.

Original: The statement follows by Jensen’s inequality.
Revised: The statement follows by Jensen’s inequality.

Original: The difference between the asymptotic variances becomes particularly pronounced in presence of selection bias, i. e., when the treatment group is systematically different from the control group.
Revised: The difference between the asymptotic variances becomes particularly pronounced in presence of selection bias, i. e., when the treatment group is systematically different from the control group.

Original: Corollary 2.
Revised: Corollary 2.

Original: If the propensity score π ( x ) is close to 0 or 1 for some x ∈ R d , then the asymptotic variance of ˆ ψ IPW can take arbitrary large values.
Revised: If the propensity score π ( x ) is close to 0 or 1 for some x ∈ R d , then the asymptotic variance of ˆ ψ IPW can take arbitrary large values.

Original: Proof.
Revised: Proof.

Original: For some small (cid:15) > 0 , let π ( X ) ∈
Revised: For some small (cid:15) > 0 , let π ( X ) ∈

Original: [0 , 1] \ ( (cid:15), 1 − (cid:15) ) .
Revised: [0 , 1] \ ( (cid:15), 1 − (cid:15) ) .

Original: Then, by Bhatia-Davis inequality and since T is binary, Var ( T | X ) =
Revised: Then, by Bhatia-Davis inequality and since T is binary, Var ( T | X ) =

Original: π ( X )(1 − π ( X )) .
Revised: π ( X )(1 − π ( X )) .

Original: Hence, Var ( T | X ) ≤ (cid:15) , and, therefore, E [1 / Var ( T | X )] ≥ 1 /(cid:15) , which proves the statement.
Revised: Hence, Var ( T | X ) ≤ (cid:15) , and, therefore, E [1 / Var ( T | X )] ≥ 1 /(cid:15) , which proves the statement.

Original: H D ETAILED D ESCRIPTION OF THE D ATASETS H. IHDP
Revised: H D ETAILED D ESCRIPTION OF THE D ATASETS H. IHDP

Original: Hill (Hill, 2011) introduced a semi-synthetic dataset created from the Infant Healthand Development
Revised: Hill (Hill, 2011) introduced a semi-synthetic dataset created from the Infant Healthand Development

Original: Program (IHDP).
Revised: Program (IHDP).

Original: This dataset is based on a randomized experiment that examines the effect of home visits by specialists on future cognitive scores.
Revised: This dataset is based on a randomized experiment that examines the effect of home visits by specialists on future cognitive scores.

Original: The dataset consists of 747 children ( t = 1 : 139, t = 0 : 608) with 25 covariates.
Revised: The dataset consists of 747 children ( t = 1 : 139, t = 0 : 608) with 25 covariates.

Original: Similar to Shalit et al.
Revised: Similar to Shalit et al.

Original: , we use 1,000 realizations from setting
Revised: , we use 1,000 realizations from setting

Original: A in the NPCI package (Dorie, 2016).
Revised: A in the NPCI package (Dorie, 2016).

Original: H. T WINS
Revised: H. T WINS

Original: This dataset is made up of all births in the USA between 1989 and 1991 (Almond et al., 2005).
Revised: This dataset is made up of all births in the USA between 1989 and 1991 (Almond et al., 2005).

Original: Only twins are considered among these births.
Revised: Only twins are considered among these births.

Original: Treatment (i. e., T = 1 ) is deﬁned as being the heavier twin (and T = 0 as being the lighter twin).
Revised: Treatment (i. e., T = 1 ) is deﬁned as being the heavier twin (and T = 0 as being the lighter twin).

Original: The outcome is deﬁned as 1-year mortality.
Revised: The outcome is deﬁned as 1-year mortality.

Original: There are 30 covariates available for each pair of twins that relate to the parents, pregnancy and birth: marital status; race; residence; number of previous births; pregnancy risk factors; quality of care during pregnancy; and number of gestation weeks prior to birth.
Revised: There are 30 covariates available for each pair of twins that relate to the parents, pregnancy and birth: marital status; race; residence; number of previous births; pregnancy risk factors; quality of care during pregnancy; and number of gestation weeks prior to birth.

Original: Only twins that weigh less than 2 kg and have no missing covariates (list-wise deletion) are taken into account.
Revised: Only twins that weigh less than 2 kg and have no missing covariates (list-wise deletion) are taken into account.

Original: This creates a complete dataset (without missing data).
Revised: This creates a complete dataset (without missing data).

Original: The ﬁnal cohort consists of 11,400 twin pairs, whose mortality rate is 17.7 % for lighter twins and 16.1 % for heavier ones.
Revised: The ﬁnal cohort consists of 11,400 twin pairs, whose mortality rate is 17.7 % for lighter twins and 16.1 % for heavier ones.

Original: In this setting, we observed both T = 

Revised: In this setting, we observed both T =

Original: (lighter twin) and T = 1 (heavier twin) for each pair of twins;
Revised: (lighter twin) and T = 1 (heavier twin) for each pair of twins;

Original: Therefore, the true treatment effect in this dataset is known.
Revised: Therefore, the true treatment effect in this dataset is known.

Original: In order to simulate an observational study, one of the twins is selectively observed based on information using the covariates (which leads to selection bias) as follows: T | x ∼ Bern ( Sigmoid ( w (cid:62) x + n ))
Revised: In order to simulate an observational study, one of the twins is selectively observed based on information using the covariates (which leads to selection bias) as follows: T | x ∼ Bern ( Sigmoid ( w (cid:62) x + n ))

Original: where w (cid:62) ∼ U (( − 0 .
Revised: where w (cid:62) ∼ U (( − 0 .

Original: 1 , 0 . 1) 30 × 1 ) and n ∼ N (0 , 0 . 1) .
Revised: 1 , 0 . 1) 30 × 1 ) and n ∼ N (0 , 0 . 1) .

Original: H. J OBS
Revised: H. J OBS

Original: Jobs studied in LaLonde (1986) consists of randomized data based on the National Supported Work program and non-randomized observational study data.
Revised: Jobs studied in LaLonde (1986) consists of randomized data based on the National Supported Work program and non-randomized observational study data.

Original: A (random) subset of randomized data is used to evaluate the algorithms.
Revised: A (random) subset of randomized data is used to evaluate the algorithms.

Original: The dataset consists of 722 randomized samples ( T = 1 : 297, T = 0 : 425) and 2,490 non-randomized samples ( T = 1 : 0, T = 0 : 2,490), all with 7 covariates.
Revised: The dataset consists of 722 randomized samples ( T = 1 : 297, T = 0 : 425) and 2,490 non-randomized samples ( T = 1 : 0, T = 0 : 2,490), all with 7 covariates.

Original: H. ACIC
Revised: H. ACIC

Original: ACIC 2018 is a collection of semi-synthetic datasets derived from the linked birth and infant death data (LBIDD) (MacDorman & Atkinso, 1998), and was developed for the 22018 Atlantic Causal
Revised: ACIC 2018 is a collection of semi-synthetic datasets derived from the linked birth and infant death data (LBIDD) (MacDorman & Atkinso, 1998), and was developed for the 22018 Atlantic Causal

Original: Inference Conference competition (ACIC)
Revised: Inference Conference competition (ACIC)

Original: (Shimoni et al., 2018).
Revised: (Shimoni et al., 2018).

Original: The simulation includes 63 different data generation processes with a sample size from 1,000 to 50,000.
Revised: The simulation includes 63 different data generation processes with a sample size from 1,000 to 50,000.

Original: Each dataset is a realization from a separate distribution, which itself is randomly drawn in accordance with the settings of the data generation process.
Revised: Each dataset is a realization from a separate distribution, which itself is randomly drawn in accordance with the settings of the data generation process.

Original: Similar to Shi et al.
Revised: Similar to Shi et al.

Original: (2019), we randomly pick 3 datasets 10 of size either 5k or
Revised: (2019), we randomly pick 3 datasets 10 of size either 5k or

Original: 10kfor each generating process settings and excludeall datasets with indication of strong selection bias.
Revised: 10k process settings and exclude with indication of strong selection bias.

Original: This yields a total number of 97 datasets.
Revised: This yields a total number of 97 datasets.

Original: I D ATA G ENERATING P ROCESS FOR S IMULATION S TUDY
Revised: I D ATA G ENERATING P ROCESS FOR S IMULATION S TUDY

Original: For the simulation study, we follow a similar protocol as in Yao et al.
Revised: For the simulation study, we follow a similar protocol as in Yao et al.

Original: ; Yoon et al.
Revised: ; Yoon et al.

Original: We generate 2,500 untreated samples from N ( 0 10 × 1 , 0 .
Revised: We generate 2,500 untreated samples from N ( 0 10 × 1 , 0 .

Original: 5 × ΣΣ (cid:62) ) and 5,000 treated samples from
Revised: 5 × ΣΣ (cid:62) ) and 5,000 treated samples from

Original: N ( µ 1 , 0 .
Revised: N ( µ 1 , 0 .

Original: 5 × ΣΣ (cid:62) ) , where Σ ∼ U ((0 , 1) 10 × 10 ) .
Revised: 5 × ΣΣ (cid:62) ) , where Σ ∼ U ((0 , 1) 10 × 10 ) .

Original: Varying µ 1 yields different levels of selection bias, which is measured by the Kullback-Leibler divergence.
Revised: Varying µ 1 yields different levels of selection bias, which is measured by the Kullback-Leibler divergence.

Original: The larger the Kullback-Leibler divergence, the greater the distributional distance between treatment and control group, and, thus, the larger the selection bias.
Revised: The larger the Kullback-Leibler divergence, the greater the distributional distance between treatment and control group, and, thus, the larger the selection bias.

Original: The outcome is generated as Y | X = x , T = t ∼ ( w (cid:62) x + t + n ) , where w ∼ U (( − 1 , 1) 10 × 1 ) , and n ∼ N (0 , 0 . 1) .
Revised: The outcome is generated as Y | X = x , T = t ∼ ( w (cid:62) x + t + n ) , where w ∼ U (( − 1 , 1) 10 × 1 ) , and n ∼ N (0 , 0 . 1) .