An empirical comparison of tree-based methods for propensity score estimation.

Evaluation of treatment effectiveness in nonrandomized studies is complicated by exposure group differences on measured and unmeasured characteristics that are independently related to the outcome. The resulting confounding can be controlled using propensity scores to balance observed confounders between treatment groups. The propensity score represents the probability of receiving treatment conditional on a set of confounders (Rosenbaum and Rubin 1983). Individuals with similar propensity scores can be expected to have similar values on measured background characteristics. Given a correctly specified propensity score model, once one conditions on the propensity score, differences in measured characteristics between the treatment groups should be from chance alone, assuming no unmeasured confounders (Rosenbaum and Rubin 1983). The true propensity score, or the predicted probability of treatment, is not known. Therefore, researchers must estimate the propensity score, typically using parametric models (Austin 2011). Logistic regression is one of the most common methods used to estimate propensity scores, but it requires several assumptions. The relation between continuous and ordinal independent variables and the log odds of the dependent variable must be linear. Furthermore, the model assumes additivity. Therefore, researchers must consider the functional form of covariates as well as interaction terms (D'Agostino 1998). Violations can result in misspecification of the propensity score model and a biased effect estimate (Drake 1993). Regression tree-based methods, including bagging and random forest classification (RFC), are nonparametric methods derived from learning-based algorithms that offer alternative strategies for estimating the propensity score. The methods use a series of classification trees to estimate the average probability of membership in a given class. These techniques may improve predictive accuracy compared to classical statistical techniques such as linear and logistic regression (Breiman 2001b). For example, in simulation studies, regardless of nonlinearity or nonadditivity, random forest performed well in terms of covariate balance between treatment groups and may result in further reduction in bias of the effect estimate when compared to traditional logistic regression (Setoguchi et al. 2008; Lee, Lessler, and Stuart 2010). There has been relatively little investigation into the use of tree-based methods to estimate the propensity score (Westreich, Lessler, and Funk 2010). In this article, we illustrate the use of three tree-based methods: bagging, RFC, and a single classification tree. We evaluate these methods in the context of an analysis to understand the effect of physical and occupational therapy services on the motor skills of preschoolers who were born with very low birth weight (VLBW). We consider the propensity scores estimated by tree-based methods in comparison to a logistic regression model. We then compare the distribution of the estimated propensity scores, the balance of covariates, and the change in effect estimates after applying inverse probability of treatment weights (IPTW).