## 4 Structure of Ensemble of Clustered and Penalized Random-Forest Model  
In this section, we present a combination of machine-learning algorithms and statistical methods that reduce RF trees and increase prediction accuracy. We called this method ECAPRAF which is an optimization problem to obtain the optimal value of regression coefficients defined on RF trees and reduce them within clusters. The coefficients are obtained by solving the following equation:  
\[\underset{\mathbf{\beta}_{k}}{minimize}\ \sum_{k=1}^{K}\sum_{u=1}^{d_{k}}( \overline{y}_{k}(z_{u})-T_{k}(z_{u})\mathbf{\beta}_{k})^{2}+p_{\lambda}(\beta_{t}) \tag{8}\]  
Figure 3 shows the hybrid structure of the proposed model (ECAPRAF). As shown in the figure, the dataset \(\{(\mathbf{x}_{1},y_{1}),\dots,(\mathbf{x}_{N},y_{N})\}\) is clustered into \(k\) clusters \(C_{1},\dots,C_{k}\) to identify the homogeneous subset of data. Within each cluster, the data are divided into the training set \(D=\{(\mathbf{x}_{1},y_{1}),\dots,(\mathbf{x}_{k},y_{k})\}\), and the test set \(Z=\{z_{1},z_{2},\dots,z_{d}\}\), which are used for training and evaluation, respectively. The \(M_{k}\) trees are trained by a training set in which \(\text{T}_{j_{k}}(\mathbf{z}_{u})\), \(j_{k}=1,\dots,M_{k}\), \(u=1,\dots,d_{k}\) are obtained from RF and then are evaluated by a test set. Finally, the \(T_{k}(\mathbf{z}_{u})=\left[1,T_{1_{k}}(\mathbf{z}_{u}),\dots,\ T_{M_{k}}(\mathbf{z}_{u})\right]\) matrix, which is generated from the output of the RF trees, represents the explanatory variables. The predicted results, obtained from the RF trees, are \(\overline{y}_{k}=\frac{1}{M_{k}}\sum_{n=1}^{M_{k}}T_{n_{k}}(\mathbf{z}_{u}),\ k=1, \dots,K\). This step produces the predicted trees to perform the next stage. The constructed trees are used as the inputs of the lasso, elastic-net, and group-lasso regressions with the response variables \(\overline{y}_{k}\) to automatically incorporate these trees into the model. Suppose that in the \(k\)-th cluster, these trees form the linear regression with the independent variable \(\text{T}_{k}\) and dependent variable \(\overline{Y}_{k}\). The linear-regression model corresponding to the cluster is:  
\[\overline{Y}_{k}=T_{k}(\mathbf{z}_{u})\mathbf{\beta}_{k}+\varepsilon_{k}\ ;\ \varepsilon_{k} \sim N\Big{(}0,\sigma_{k}^{2}\Big{)};\ \ k=1,\dots,K \tag{12}\]  
where \(\mathbf{\beta}_{k}=\left(\mathbf{\beta}_{1_{k}},\mathbf{\beta}_{2_{k}},\dots,\mathbf{\beta}_ {M_{k}}\right)^{\prime}\) is the regression coefficient. The purpose is to optimize and estimate the coefficients of trees so that some of them are inclined toward zero or removed from the model using shrinkage methods. Consequently, some of the coefficients  
Figure 3: Detailed structure of ECAPRAF Model. First, dataset is clustered into K classes to identify the homogeneous subset of data and assign them to similar groups. Inside each cluster, the data are trained to produce RF trees for a more accurate prediction. The trees are reduced by shrinkage methods in order to reduce computational time. Finally, all clusters are ensembled. become zero and are removed from the model. Reducing trees creates a more appropriate and efficient model than the traditional RF. In principle, it reduces error and enhances the model performance. Finally, \(m_{k}\) trees are selected in each cluster. The weak learners are corrected by other strong learners and the error is reduced by averaging so that the sum of the whole trees of clusters becomes m.  
To designate the value of the tuning parameter \(\lambda_{k}\) in Equation (11), 10-fold cross-validation is implemented to gain the best \(\lambda_{k}\) within clusters that gives the minimum error. In the case where no prior information is available for the value of \(k\), the number of groups can be specified using formal statistical tests.