# Enhancing Machine Learning Models Through PCA, SMOTE-ENN, and Stochastic Weighted Averaging
## 2 Materials and Methods
### Methodology
#### 2.2.2 Modeling Pipeline  
Seven classifiers were employed in this study to analyze their effectiveness in predicting survival. These classifiers were chosen for their strengths in handling various aspects of classification tasks [12].  
Logistic Regression: A statistical model widely used for binary classification problems where the goal is to predict the probability that a given input instance belongs to one of two classes. This model is particularly effective when the relationship between features and binary outcomes is approximately linear.  
Support Vector Machine (SVM): A powerful classification algorithm that finds a hyperplane that best separates classes in the feature space. SVM's primary objective is to create a maximum-margin hyperplane that separates data points from different classes, focusing on the nearest data points, termed support vectors. SVMs are particularly effective in high-dimensional spaces and are robust to overfitting, especially when the number of dimensions exceeds the number of samples [13].  
K-Nearest Neighbors (KNNs): KNNs is a simple, yet powerful, classification algorithm that makes predictions based on the majority class of the k nearest data points in the feature space. KNNs assumes that similar data points are close to each other, and it assigns a class label to a new data point by finding the majority label among its closest neighbors. One of the key strengths of KNNs is that it is non-parametric, meaning it makes no assumptions about the underlying data distribution, making it useful for various types of data [14].  
Random Forest: Random Forest is an ensemble technique that builds multiple decision trees to enhance predictive accuracy by averaging their predictions in regression or voting in classification tasks. Random Forest is highly effective in handling large, high-dimensional datasets and is resistant to overfitting due to its use of multiple trees. It also handles missing values and maintains accuracy [15].  
XGBoost (Extreme Gradient Boosting): An advanced implementation of the gradient boosting framework, designed for efficiency, flexibility, and high performance. Gradient boosting builds models sequentially, where each new model attempts to correct the errors introduced by the previous one. XGBoost introduces innovations such as regularization techniques and efficient sparse data handling, significantly improving performance. Its main advantages include handling missing data, parallel processing, and regularization techniques that prevent overfitting [16].  
LightGBM: A highly efficient gradient boosting framework optimized for both speed and performance. It uses a leaf-wise (best-first) tree growth algorithm to achieve deeper trees and better accuracy compared with level-wise (depth-wise) tree growth used in other boosting algorithms. LightGBM is particularly effective in scenarios where the dataset is very large and has high dimensionality [17].  
CatBoost: Another powerful gradient boosting algorithm specifically designed to efficiently handle categorical features. Based on the gradient boosting framework, CatBoost improves the way categorical variables are handled and reduces overfitting. It introduces a technique called 'order boosting' to reduce bias due to target leakage, ensuring that the learning process remains unbiased by splitting the data into separate training and validation sets at each iteration [18].