Question: It is unclear to me how the estimator μ(t,Xi) is constructed. Is it the weighted average of something (?), where the weights wi(x) are presumably the frequency with which the ith training example falls into the the same leaf as x (see Athey et al. 2019)?
Evidence: Heterogeneous treatment effect (HTE) estimation has been of growing interest for decision-makers in a wide spectrum of contexts. It uncovers the effect of interventions at sub-group levels, thereby providing highly tailored suggestions rather than a one-size-fits-all policy. When it comes to precision medicine, HTE provides the leveraged information that physicians need to precisely treat different patients with proper dosages of drugs depending on their genes, living habits, and EHR history. Recently, with the emerging big data that exhibits astronomical complexities, HTE has been essential for online marketplaces. For largescale data that presents a huge challenge, algorithms that adapt the booming machine learning (ML) techniques to HTE estimation are proposed. However, these methods largely focus on binary or discrete treatments and thereby being not widely applicable, given the fact that continuous treatments are prevalent in practice. Examples include targeting customers with continuous incentives, customizing the duration of ads to improve user engagement, and design optimal interest rates to maximize revenue while control for default.
For HTE estimation with continuous treatments, it could be feasible to discretize the treatments and utilize the above approaches, but the limitations are two-phase and bring non-trivial challenges to researchers. First, ordinal treatments like different levels of education cannot be treated as categorical otherwise the order information across treatments will be lost, and secondly, discretizing or bucketing has the limitations of model inaccuracy and cannot recognize the right patterns of the outcome across treatments. To elucidate the complex relationship between treatment and outcome, researchers put forward ideas of dose-response function (DRF).
The estimation of DRF in the presence of high dimensional features can be viewed as a natural extension of regression problems with additional complexities. The success of random forest (RF) for regression problems motivates the development of causal forest (CF) and orthogonal random forest and the modification of Bayesian Additive Random Forest. These methods partition the feature space with a splitting criterion fitted on treatments that can provide accurate estimations for HTE. But often they posit linear or partially linear models of DRF which are vulnerable to model misspecifications. Figure is an illustration of model misspecification of random forest and CF. Doubly Robust estimators (DREs) allows for misspecification of either the treatment or outcome regression. Recent work combines the estimation of generalized propensity score with regression models of the outcome that can further reduce the bias and use kernel regression for non-parametric estimation of DRF. However, it may suffer from the curse of dimensionality when the feature space is high-dimensional given large-scale instances. To advance, provide kernel-based DML estimators that can easily deal with high-dimensional data. Nevertheless, the localization and weighting mechanism of CF is not taken into account, which is essential for capturing heterogeneity. It remains unexplored that how to effectively combine ML methods with localized estimations under no model assumptions.
In this paper, we propose generalized causal forest (GCF) by extending CF to a generalized one with a brand new splitting criterion, and herein the generalization is two-fold. First, we generalize the local linear model of CF to a non-parametric one with DRF. Meanwhile, we employ the distance metrics in the functional space of Partial DRF as the splitting criterion and use kernel-based DML estimators as the DRF approximation.
When it comes to implementations, many open source packages integrate recent research with the trending ML methods that facilitate HTE estimation, such as EconML, CausalML and GRF. EconML, CausalML offer a suite of high-performance ML algorithms like DML, while GRF specially supports CF. However, they are only compatible with single-machine systems and may not be efficient when it comes to large-scale instances. Apache Spark is integrated for handling large-scale data with distributed and in-memory computing, while inclusively provides APIs of trending ML algorithms with the built-in MLlib. Therefore, as one of our contributions, we build our algorithm on Spark that enables large-scale data processing and easy use of any ML techniques. To summarize, the main contributions are:
• We are the first to generalize CF's partially linear model to a non-parametric one with DRF and apply kernel-based DML estimators. The proposed algorithm pictures the nonparametric behaviors of how outcome varies with treatments. • We propose a distance-based splitting criterion with various distance metrics in the functional space of Partial DRF that generalizes the differences of CAPE in CF. • Our proposed GCF has been tested on both synthetic and large-scale real-world datasets. It improves on SOTA in terms of multiple evaluation metrics. We implement GCF on Apache Spark MLlib and achieves higher computatinal efficiency by distributed computing.
The rest of the paper is organized as follows. In Section 2, we present related works in the literature.
In Section 3, we introduce some notations and backgrounds. In Section 4, we introduce the Generalized Causal Forest. In Section 5, we examine the empirical performance of GCF by applying it to both synthetic and real-world data sets. We conclude the paper with some discussions in Section 6.
In this paper, we propose generalized causal forest (GCF) by extending CF to a generalized one with a brand new splitting criterion, and herein the generalization is two-fold.
Answer: The construction of the causal estimand is two-fold. First, the in-node estimator is derived from the kernel-based DML estimator as described in sec 4.2. Then we average all estimands across nodes. The weight is determined by the sample in each node. We have added detailed descriptions in the algorithm pseudocode in our newly updated paper.