Comments on Fifty Years of Classification and Regression Trees

We wish to congratulate the author for a nice overview of the tree-based methods, and the author clearly highlighted the recursive partitioning technique (Friedman, 1977; Breiman et al., 1984; Zhang and Singer, 2010) behind the tree-based methods. As the author summarized, there are two major types of tree methods: classification trees and regression trees, as precisely reected in the title of the classical book by Breiman et al. (1984). In our own experience, for regression problems, other nonparametric methods, including adaptive splines (Friedman, 1991) that are based on a similar partitioning technique, appear more desirable than regression trees, with the exception of survival analysis (Zhang, 1997, 2004; Zhang and Singer, 2010). With the advent of high-throughput genomic technologies, classification trees have become one of the most common and convenient bioinformatic tools. In what follows, we would like to share some of the recent developments in this area. Genome-wide association studies (GWASs) collect data for hundreds of thousands or millions of single nucleotide polymorphisms (SNPs) to study diseases of complex inheritance patterns, which can be recorded qualitatively (e.g., breast cancer) or in a quantitative scale (e.g., blood pressure). GWASs typically employ the case-control design, and the logistic regression model is generally applied to assess the association between each of the SNPs and the disease response, although more advanced techniques, especially nonparametric regression, have been proposed to incorporate multiple SNPs and interactions. A clear advantage of classification trees is that they make no model assumption and that they can select important variables (or features) and detect interactions among the variables. Zhang et al. (2000) was among the early applications of tree-based methods to genetic association analysis. Since then, interests in tree-based genetic analyses have grown substantially. For examples, Chen et al. (2007) developed a forest-based method on haplotypes instead of SNPs to detect gene-gene interactions, and importantly, they detected both a known variant and an unreported haplotype that were associated with age-related macular degeneration. Wang et al. (2009) further demonstrated the utility of this forest-based approach. Yao et al. (2009) applied GUIDE to the Framingham Heart Study (FHS) and detected combinations of SNPs that affect the disease risk. Garcia-Magarinos et al. (2009) demonstrated that the tree-based methods were effective in detecting interactions with pre-selected variables that were marginally associated with the disease outcome, but were susceptible to the local maximum problem when many noise variables were present. Chen et al. (2011) combined the classification tree and Bayesian search strategy, which improved the power to detect high order gene-gene interactions at the cost of high computation demand. Tree-based methods are extensively used in gene expression analysis to classify tissue types. Here the setting is very different from the GWAS applications. In GWAS applications, we deal with a very large number of discrete risk factors (e.g., the number of copies of a particular allele). In expression analysis, the number of variables is large but not so large, usually in the order of tens of thousands, and the variables tend to be continuous. For example, Zhang et al. (2001) demonstrated that classification trees can discriminate distinct colon cancers more accurately than other methods. Huang et al. (2003) found that aggregated gene expression patterns can predict the breast cancer outcomes with about 90% accuracy using tree models. Zhang et al. (2003) introduced deterministic forests for gene expression data in cancer diagnosis which have similar power to random forests but are easier in scientific interpretation. Pang et al. (2006) developed a random forest method incorporating pathway information and demonstrated that it has low prediction error in gene expression analysis. Furthermore, Diaz-Uriarte and De Andres (2006) demonstrated that random forest can be useful in variable selection by using a smaller set of genes and maintaining a comparable prediction accuracy. Of a related note, Wang and Zhang (2009) attempted to address the basic questions: how many trees are really needed in a random forest? They provided empirical evidence that a random forest can be reduced in size so much to allow scientific interpretation. As more and more data are generated from new technologies such as the Next-Generation Sequencing, tree-based methods will be very useful for analyzing such large and complex data after necessary extensions. Closely related to genomic data analysis is the personalized medicine. Zhang et al. (2010) presented a proof of concept that tree-based methods have some unique advantages over parametric methods to identify patient characteristics that may affect their treatment responses. In summary, tree-based methods have thrived in the past several decades, and they will become more useful and the methodological developments will be more challenging than ever, as more information increases in both size and complexity.