# Tree-Structured Regression Model Using a Projection Pursuit Approach
## 1 Introduction  
Classification and regression trees are commonly applied to data, mainly for interpretability. The first regression tree was AID (automatic interaction detection [1]). At each node, AID determines the splitting rules using the sum of squared deviations as an impurity measure. It splits the data recursively and stops when the impurity measure is less than the predefined threshold value. It uses the sample mean of data in the final node as the predicted value. There are several problems with AID: overfitting, selection bias when the variables are highly correlated, and the masking problem. To overcome the overfitting problem, CART (the classification and regression tree [2]) was developed. It uses a pruning method to reduce the size of the tree to prevent overfitting. However, this increases the computational cost. AID and CART use piecewise constant regression approaches. It is difficult to solve the masking problem with these approaches because the predictors can be used in two ways: for splitting and for fitting the model in the final node [3]. To overcome this problem, CTREE (the conditional inference tree [4]) uses a permutation test, and GUIDE (generalized, unbiased, interaction detection and estimation [5]) uses bootstrap calibration. For decades, various algorithms have been proposed to apply complex models instead of constants values to the final nodes of the regression tree. M5 [6] and MSP [7] allow the construction of linear models at final nodes. HTL [8] allows for a nonlinear regressor at final nodes. One advantage of predicting using such complex models at final nodes over the traditional constant values is that they are generally much simpler and more accurate. Instead of making a single prediction from a complex model, ensemble models use summarize the predictions from many simple models. Tree-based ensemble models have been developed, e.g., bagging [9], boosting [10], random forests [11], BART (Bayesian Additive Regression Model [12]), and KTBoost [13]. In general, the ensemble technique is known to reduce overfitting. Accordingly, the performance of the model can be expected to be improved. Furthermore, there have been many efforts to apply tree algorithms to various regression problems like logistic regression and quantile regression [14]. Regression tree algorithms have recently extebd to the autoregressive tree model for time-series analysis [15] and therandom effect tree model for panel data analysis [16]. The flexibility of fitting various types of data well is a powerful feature of regression trees. Therefore, regression tree algorithms have been developed and widely used in regression problems [17; 18; 19; 20]. Another regression approach is projection pursuit regression (PPR [21]). This approach uses a sum of empirically determined univariate functions of the projected predictor values. To find the regression model, PPR finds the projection of the predictors, fits an empirical model with the projected predictor values and calculates the residuals. This procedure is repeated with the residuals until the residual sum of squares is very small. The final projection pursuit regression model is the sum of the empirical models in each iteration. Even though projection pursuit regression predicts the response variable quite well, it is not easy to understand the model itself, and it is quite difficult to explain patterns in the data. The projection pursuit idea is applied to improve the interpretability of the model, and a new tree with a piecewise regression approach--the projection pursuit regression tree--is proposed. With the projection pursuit regression tree, the data analyst can explore the partitioned data as well as the data as a whole. The main difference between the general tree-based regression methods and the projection pursuit regression tree is the way they partition data. The general tree-based regression methods are focused on the space of independent variables and each final node tend to have wider-ranging values of the dependent variable.