## 1 Introduction  
Ensemble learning is a powerful tool for the classification and prediction of various issues that has been extensively studied in statistics and machine learning. These methods use several algorithms to enhance the performance of the model and increase the prediction accuracy so that several weak algorithms are combined with a special pattern to create a strong learner that performs better than a single learner. The most well-known algorithms of ensemble learning are bagging, boosting, and random forest (RF). Bagging [1] produces regression trees by random selection with replacement. In addition, the generated trees do not appertain to previous trees, and each one is independent from its peers. These deep trees grow without pruning, hence a high variance and low bias exist in each tree, whose average causes low variance. Random forest is a developed form of bagging that is used for both regression and classification. Some methods similar to RF discussed in the literature do not work as well as the RF proposed by [PERSON] [2]. [PERSON] [3] proposed a method in which each node is randomly selected from the k best splits. [PERSON] et al. [4] suggested the first random-selection algorithm based on the best separators in which k trees are generated by the random vector. In the RF, a subset of predictors is selected as a separator in each node. Each node is grown and split by randomly selecting attributes. The number of trees in the RF has always been a concern to researchers. A large number of trees does not always mean good prediction accuracy, but in some cases, causes increases in error, overfitting, and decreases in accuracy. By using RF, [PERSON] et al. [5] proposed a method in which less OOB error determined the most optimal RF trees. To overcome the problem of a large number of trees, we propose a method in which shrinkage methods, which perform variable selection, are used to reduce the number of RF trees. This method not only increases accuracy but also prevents overfitting. In our method, from all the RF trees existing in the prediction, a subset of trees is selected using shrinkage methods. In [6], the post-selection boosting random-forest algorithm was suggested, which utilizes lasso regression to reduce the number of RF trees and improve the performance of the RF algorithm. On the other hand, clustering the correlated data, identifying the homogeneous subsets of predictors, and assigning them to similar clusters can be effective at increasing the prediction accuracy. For this reason, k-means clustering is used in this paper. **Major Contribution:** In present paper, we propose a hybrid approach called ensemble of clustered and penalized random forest (ECPARAF), which is shown in Figure 1. This method has four fundamental parts. These parts include clustering, predicting, shrinking, and ensembling. The purpose of using the clustering method is to identify homogeneous subsets of data and assign them to similar groups. This allows similar data placements in a cluster to reduce variance and increase correlation within clusters, as well as to increase the prediction accuracy in order to create an optimal model. For this purpose, k-means clustering is used. The second part, which is related to the prediction algorithm, is responsible for predicting the initial trees with low error. The main idea is to use a set of weak predictors with high accuracy to make initial decisions that can be summed together to obtain the final prediction. To do this, RF is used. The reason to choose RF as a prediction algorithm is to produce low-variance trees and use them as an input for shrinkage methods. The next part is to use the shrinkage methods to reduce the number of RF trees within each cluster in order to perform the final prediction in each cluster without an initial selection of trees. The accuracy of the model can be increased compared to traditional RF. In the last part, the remaining trees are ensembled to improve the power of the learners.