# Tree-Structured Regression Model Using a Projection Pursuit Approach
## 3 Projection Pursuit Regression Tree  
### Construction of the Projection Pursuit Regression Tree  
Multiple regression models are widely used, but it is difficult to interpret the final model, especially when there are many independent variables. Additionally, if there are nonlinear patterns between the dependent variable and independent variables, the multiple regression model is more difficult to interpret. Most regression trees have better interpretability than the ordinary multiple regression model. With recursive partitioning of the independent variable space, the regression tree discovers interesting features in each partitioned space of the independent variables. However, one of the main purposes of regression is to explain the dependent variable using the independent variables. In this paper, the dependent variable is focused and a new regression tree--the projection pursuit regression tree--is proposed. The original idea of the projection pursuit regression tree comes from the projection pursuit classification tree [24]. In the projection pursuit regression tree, a modified approach to split nodes and assign values in the final node is used. Let \((Y_{1}^{*},\mathbf{X}_{1}^{*}),\cdots,(Y_{n}^{*},\mathbf{X}_{n}^{*})\) be the set of observations in a specific node and \(\mathbf{X}_{i}^{*}\) be a \(p\)-dimensional vector. Then, in each node,  
* Divide \(Y_{1}^{*},\cdots,Y_{n}^{*}\) into two groups: group 1 with small values and group 2 with large values (eg. Assign \(Y_{i}^{*}\) to group 1 if \(Y_{i}<\text{median}(Y_{1}^{*},\cdots,Y_{n}^{*})\). Otherwise, assign \(Y_{i}^{*}\) to group 2.) * Find the optimal 1-dimensional projection \(\alpha^{*}\) to separate group 1 and group 2 using one of the projection pursuit indices that involves class information--the LDA, PDA, or Lr index. * Project the data in the node onto the optimal projection \(\alpha^{*}\), i.e., \(\alpha^{*T}\mathbf{X}^{*}\). If \(ave(\alpha^{*T}\mathbf{X}_{i}^{*}|Y_{i}^{*}\in\text{group 1})\geq ave(\alpha^{*T} \mathbf{X}_{i}^{*}|Y_{i}^{*}\in\text{group 2})\), use \(\alpha^{*}=-\alpha^{*}\) as the optimal projection. * Find the best cut-off, \(c\), to separate group 1 and group 2. * Group 1 is assigned to the left node, and group 2 is assigned to the right node. In step 2, the projection pursuit index is needed to separate group 1 and group 2. The LDA index, Lr index [25], and PDA index [26] were developed to find the projection from a separating view of groups. The LDA index is based on linear discriminant analysis and is useful for finding the view that maximizes between-group variation relative to within-group variation. If the correlations among variables are high or the number of variables is relatively large compared with the number of observations, the PDA index is useful to escape the data piling problem [27] in the LDA index. The amount of penalty on correlations can be controlled with the tuning parameter \(\lambda\). The PDA index with \(\lambda=0\) is the same as the LDA index. Both the LDA and PDA indexes use all information in the variance-covariance matrix of the independent variables. On the other hand, the Lr index ignores the covariances among variables and uses only the variances of the independent variables \((L_{r}index(\mathbf{A})=\left(\frac{\sum_{l=1}^{d}\sum_{l=1}^{d}\sum_{l=1}^{p _{l}}(g_{ij}-g_{j})^{r}}{\sum_{l=1}^{d}\sum_{l=1}^{p_{l}}\sum_{l=1}^{p_{l}}(g_ {ij}-g_{ij})^{r}}\right)^{1/r}\), where \(y_{ijl}\) is the \(l\)th variable of the projected data in \(i\)th group \(j\)th observation and \(\mathbf{A}\) is the \(p\times q\) projection matrix). \(r\) determines how to calculate the distance between two points in each dimension. With different selections of indices and the parameters associated with indices, the interesting features of our data space can be found. The projection pursuit regression tree is designed to provide an exploratory data analysis tool, and the tree structure is constructed to retain this feature.