# Tree-Structured Regression Model Using a Projection Pursuit Approach
## 3 Projection Pursuit Regression Tree
### Construction of the Projection Pursuit Regression Tree
In each node, the observations are separated into a group with small \(Y\) values and a group with large \(Y\) values; the group with small \(Y\) values is assigned to the left node, and the group with large \(Y\) values is assigned to the right node. To retain this directional convention, the direction of the optimal projection in step 3 is modified. To explain how the projection pursuit regression tree works, a toy example is used. One-hundred observations with two independent variables, \(X1\) and \(X2\), and one dependent variable, \(Y\), are randomly chosen. In this toy example data, \(X1\) and \(X2\) are highly correlated with \(Y\) (Figure 1a,b). In step 1, the data are divided into two groups: group 1 (\(\bullet\)) and group 2 (\(\blacktriangle\)). In step 2, the LDA index is used to find the optimal projection to separate group 1 and group 2. Figure 1c is the scatter plot of the projected values of \(X1\) and \(X2\) vs. \(Y\). Figure 2 shows the difference between CART and the projection pursuit regression tree in dividing the space of independent variables. The dashed vertical line in the scatter plot of \(X1\) and \(X2\) represents the cut-off point in each method. The observations on the left/lower side of this line are assigned to the left node, and those on the right/upper side are assigned to the right node. CART uses the variable \(X1\) to separate the two groups. Because CART uses one variable at a time, the separating line is parallel to the axes of the variables. In contrast, the projection pursuit regression tree uses a linear combination of  
Figure 1: Toy example: 100 observations; \(X1\) and \(X2\) are highly correlated with Y.  
the independent variables \(a_{1}X1+a_{2}X2\), and this separating line can be in any direction, including the direction parallel to axes. These procedures continue until a node is declared as a final node. To decide whether a node is final, two different ways are provided: one is for exploring data, and the other is for predicting. For the exploratory data analysis, a tree with a predefined depth can be constructed. With this approach, the projection pursuit regression tree divides the \(Y\) values into \(2^{depth}\) groups and explores the properties of each group. Figure 3 is a simple projection pursuit regression tree for the toy example with depth 1. The data with small \(Y\) values have small \(X1\) and \(X2\) values, and the data with large \(Y\) values have large \(X1\) and \(X2\) values. For the prediction purpose, the final node is declared if the number of observations in a node is small or if the between-groups sum of squares is relatively smaller than the within-groups sum of squares. The second criterion is similar to the classical [PERSON]'s LDA idea. After constructing the tree structure with the observed data, the numerical values are assigned to the final nodes. Five different models are provided in our projection pursuit regression tree to assign numerical values in the final nodes. Let \(Y_{i^{*}}\) and \(\mathbf{X}_{i^{*}}=\left[X_{i^{*}1},\cdots,X_{i^{*}p}\right]^{T}\) be the \(i^{*}\)th observations in a specific final node, \(i^{*}=1,\cdots,n^{*}\)  
Figure 3: Simple projection pursuit regression tree for the toy example with depth 1. Figure 2: How to separate the space of independent variables in CART and in the projection pursuit regression tree. where \(n^{*}\) is the number of observations in the node and \(\alpha^{*}\) is the best projection of the previous node. Then, for a new observation \(\mathbf{X}^{*}=\left[X_{1}^{*},\cdots,X_{p}^{*}\right]^{T}\),