# Machine Learning Models for Ecofriendly Optimum Design of Reinforced Concrete Columns
## 2 Materials and Methods
### Machine Learning Models
#### 2.3.2 Ensemble Methods  
Random forests (RFs) are effective in prediction and random forest (RF) regression is formed by growing trees depending on a random vector, such that the tree predictor takes on numerical values [64]. In ML, \"boosting\" is a way to combine multiple simple models into a single composite model. The term \"gradient\" in \"gradient boosting\" indicates that the algorithm uses gradient descent to minimize loss. Gradient boosting is used for regression, while gradient boosting is used to estimate a continuous value [65]. Histogram-based gradient boosting regression is much faster than the gradient boosting regressor for big datasets and has native support for missing values [66]. There are several ways to combine multiple learners to produce the final output in a model-building scheme. An example of using output produced by all core learners given input is voting and stacking [67]. These are ensemble methods. With regard to the voting regressor, voting provides a basic way to classify ensembles. The default scheme averages the probability or numerical estimates for classification and regression, respectively [68]. Voting is used to get a linear combination of learners (Figure 5) [67]. A voting ensemble increases the system's performance. It can be utilized for both classification and regression issues by integrating the results of numerous methods. For regression problems, voting regressors (VRs) are the estimators of all models that are averaged to obtain the final estimate [69]. The final estimate is less prone to error [70]. The idea in voting is to vote on models with high variance and low variance, ensuring that the variance remains small after combination and the variance is reduced by averaging. Even if the individual models are biased, decreasing the variance can balance this deviation and reduce the error [67]. We have  
\[y_{i}=\sum_{j}w_{j}d_{ji}\text{ where }w_{j}\geq 0,\sum_{j}w_{j}, \tag{5}\]  
where L is the number of independent voters. In weighted sum, \(d_{ji}\) is the vote of model \(j\) for class C\({}_{i}\), and \(w_{j}\) is the weight of its vote. Simple voting is a special case that all voters have equal weight, namely, \(w_{j}=1/\text{L}\). For regression, the outputs of the baseline regressors can be combined by using simple or weighted averages or medians. The median is more tolerant of noise than the average. Another possible way might be to assess the accuracies of the learners on a separate validation set and utilize that information to compute the weights so that more weights can be given to more accurate learners. These weights can also be learned from data [67]. Each estimator follows the true regression function. If the discrepancy function is treated as a random noise function added to the true regression function and these noise functions are not related to the mean zero, averaging the individual estimates reduces the mean noise to be mean-like. In this sense, voting is smoothing in the function space and can be seen as a modifier of the real regression function, assuming smoothing [71]. With regard to the stacking regressor, in data mining the outputs of several different models are combined to make decisions more reliable. Some ML techniques do this by learning a collection of models and using them together. Stacking is one of these schemes. It can often improve predictive performance over a single model. It is a general technique applicable to classification and numerical estimation problems [64]. Stacking is a way of combining multiple models. Stacking tries to find which classifiers are reliable by using another learning algorithm like metalearner to find the best way to combine baseline learner results [68]. Stacking employs a metalearning method to learn how to best consolidate the multiple classifiers from different ML techniques [72]. Stacking has the advantage of combining the capabilities of several well-implemented methods for a regression or classification and make more successful predictions than a single method in the ensemble [73].