# An Empirical Evaluation of Ensemble Models for Python Code Smell Detection
## 3 Proposed Research Framework
### Data Selection
#### 3.1.7 Ensemble Methods  
Ensemble learning is an ML technique that combines many algorithms to create a more precise prediction than a single algorithm. It trains multiple algorithms on the same data and then combines their predictions to improve the overall performance [14]. Ensemble learning is effective because it reduces overfitting and improves generalization. The underlying principle is that different algorithms will make uncorrelated errors, which cancel each other out when aggregated.  
Deep learning methods typically necessitate substantial datasets and significant computational resources. Due to the limited size of the dataset, comprising 1000 samples for both the LC and LM, ensemble learning methods are more suitable and effective in this context.  
Some common ensemble learning algorithms used in this study are Bagging, Gradient Boosting, Max Voting, AdaBoost, and XGBoost.  
**Bagging:** Alternatively referred to as Bootstrap aggregation, Bagging is an ensemble learning technique that helps to increase ML accuracy and performance. The use of bias-variance trade-offs in agreement serves to reduce the inaccuracy of a prediction model. The technique of Bagging is used in regression and classification models as a means to mitigate the risk of overfitting [14].  
**Gradient Boosting (GB):** This methodology is the most effective Ensemble ML method. The most prevalent forms of ML faults are bias error and variance error. The GB method is a boosting technique that may be used to enhance the bias accuracy of the approach. The GB method employs statistical techniques such as regression and classifiers to manipulate constant and categorical target variables. The cost function used in regression is the mean square error, whereas in classification, it is the log loss [14].  
**Max Voting:** It generates outcomes by selecting the class with the highest probability from a set of ensemble models. The Voting Classifier aggregates the outcomes of each classifier and predicts the class with the highest number of votes. In general, a unified predictive model is developed that incorporates many models and uses the aggregate number of votes for each output class to forecast output, as opposed to generating distinct individual models and assessing their performance [14].  
**AdaBoost:** It was the first boosting algorithm extensively used for binary classification and was developed by [PERSON] and [PERSON]. The boosting approach merges several \"weak classifiers\" into one \"strong classifier\" [14].  
\begin{table}
\begin{tabular}{l l l} \hline \hline
**Dataset** & **Number of Features** & **Extracted Features** \\ \hline LC & 10 & \
\begin{tabular}{l} n2, scloc, ln, multi\_comments, \\ single\_comments, comments, blanks, h1, LargeClass \\ \end{tabular} \\ \hline LM & 10 &
\begin{tabular}{l} n2, scloc, lh2, n1, multi\_comments, comments, \\ blanks, h1, Experience Based \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table}
Table 2: Extracted Feature by Chi-Square.  
**XGBoost:** The Extreme Gradient Boosting approach (XGBoost) is a tree-based ML model with better performance and speed. [PERSON] designed it, and the Distributed ML Community organization manages it. XGBoost has gained significant relevance in the field. It has proven its worth by consistently producing good results in structured and tabular data sets, making it a valuable tool for data scientists and ML practitioners.