# Tree-Structured Regression Model Using a Projection Pursuit Approach
## 3 Projection Pursuit Regression Tree
### Features of the Projection Pursuit Regression Tree  
The projection pursuit regression tree is developed to explore data in each partitioned data space instead of the whole data space by focusing on the value of the dependent variable (\(Y\)). This tree always divides the data by \(Y\) values; the observations with smaller \(Y\) values are assigned to the left node and the observations with larger \(Y\) values are assigned to the right node. After fitting the projection pursuit regression tree, all final nodes are sorted by their estimated values--the node that is furthest to the left has the smallest estimated value, and the rightmost node has the largest estimated value. Therefore, with this tree structure, the most important variables for large or small \(Y\) values can be determined.  
To examine this feature of the projection pursuit regression tree more deeply, 100 observations with one dependent variable and four independent variables are simulated. Let Q1, Q2, and Q3 be the 25%, 50%, and 75% quartiles of Y. The \(Y\) values are divided into four groups--G1, G2, G3 and G4--with these quartiles. \(X1\) is linearly associated with \(Y\) in G1 and does not have any relation with \(Y\) in the other groups. Similarly, X2 shows a linear relationship with \(Y\) only in G2, X3 shows an association with \(Y\) only in G3, and X4 shows an association with \(Y\) only in G4. The structures of the data are presented in Figure 5. The relationships between \(Y\) and the Xs depend on the range of \(Y\) values.  
Figure 6 shows the result of the projection pursuit regression tree with depth 3, and Table 2 shows the coefficients of projection in each node and the overall importance measure of the projection pursuit regression tree as well as the importance measure of the random forest for regression. The length of the projection is stick to one and the value of the projection coefficient depends on the number of variables. For consistent decision, the number of variables in each coefficient is multiplied for comparison. The projection coefficients of projection pursuit in each node show the role of each variable in separating the left and the right nodes. If the absolute value of a coefficient is large (great than 1), the corresponding variable plays an important role in this separation.  
Figure 4: The scatter plot of \(Y\) and the predicted values using CART and the projection pursuit regression model with methods Models 1-5.  
In node 1, all \(Y\) values are divided into two groups--(G1, G2) and (G3, G4). To separate the smaller \(Y\) groups (G1, G2) from the larger \(Y\) groups (G3, G4), \(X2\) and X3 should play an important role, as indicated by the coefficients of node 1. Additionally, \(X1\) and \(X2\) are important for separating G1 and G2. This separation occurs in node 2 and the coefficients of node 2 are large in \(X1\) and \(X2\). In separating G3 and G4 in node 3, \(X3\) and \(X4\) are important. To separate G1 into smaller and larger groups (node 4), \(X1\) should primarily be used. Additionally, \(X2\) should be used for G2 (node 5), \(X3\) should be used for G3 (node 10), and \(X4\) should be used for G4 (node 11). All these features can be found in Table 2 by the corresponding coefficients of each node in the projection pursuit regression tree. Table 2 show the importance measure of the random forest for regression. The order of importance is \(X3\), \(X2\), \(X1\), and \(X4\). It shows the same result of the overall importance of the projection pursuit regression tree. Table 3 shows MSE of multiple regression line (lm), randomForest, rpart and 5 models of the projection pursuit regression tree. According to Table 3, all methods in the projection regression tree show better performance than the other three methods, lm, randomForest, and rpart.