## 2 Theoretical Framework
### Supervised Machine Learning: Classification  
This section discusses the practical application of supervised machine learning algorithms, specifically focusing on classification techniques such as decision trees, bagging, AdaBoost, and K-nearest neighbors (KNN). Classification algorithms play a crucial role in predicting outcomes on new data by analyzing labeled training data, providing valuable insights, and supporting decision making across various disciplines. Each classification method offers unique working principles and benefits that we will explore further [18]. Decision trees, versatile machine learning models for classification and regression tasks, employ a hierarchical tree-like structure. Each internal node represents a feature and threshold, while leaf nodes represent classes or predicted values. This simple yet powerful structure facilitates the identification of key variables influencing customer segmentation. Decision trees create homogeneous subsets by recursively partitioning data based on feature thresholds, making them easy to comprehend and visualize [23]. Additionally, theirtransparency helps uncover patterns and customer groups more likely to be classified as \"high-value\". In cases where improved classification accuracy is required, ensemble methods like AdaBoost and bagging come into play. Both methods aim to enhance predictive accuracy and robustness by combining the predictions of multiple base models [18]. Bagging trains multiple instances of the same base model on different subsets of the training data [24; 25], while AdaBoost focuses on misclassified samples and assigns higher weights to them during training [24; 26]. Doing so allows weaker models to adapt and improve over time. Bagging reduces model variance while maintaining or improving bias, making it effective for small datasets or models prone to overfitting [25]. On the other hand, AdaBoost reduces model bias while maintaining or improving variance, making it suitable for more complex models [24]. The idea behind AdaBoost lies in iteratively assigning higher weights to misclassified instances, making them more influential in subsequent training rounds. Initially, all data points in the training dataset are given equal weights. The first weak learner is trained using the initial sample weights and evaluated on the trained data. Misclassified samples are identified, and their weights are increased. These weights emphasize the importance of each instance during the training process. Since AdaBoost focuses on improving weak learners' performance (such as shallow decision trees), the weights of misclassified samples are increased during each training cycle, making them more influential in the subsequent training rounds. Such a process is repeated for a predefined number of iterations or until the desired level of accuracy is achieved. The final prediction is obtained by averaging or voting on the predictions of each model, creating a weighted mixture of the weak learners' forecasts [24; 26]. Bagging, enhanced by random forests, significantly improves classification performance through ensemble methods. The process involves training multiple instances of the same base model on different subsets of the training data obtained through bootstrapping - a random sampling technique with replacement. The bagging algorithm randomly selects N instances with replacements from a training dataset with N instances. Each subset, called a \"bootstrap sample\", is applied to train a separate base model. Since each base model is trained on a different bootstrap sample, they are relatively independent. In bagging, diversity is essential, as it reduces overfitting and increases robustness overall. A final prediction is calculated by averaging the predictions made by each tree. Random forest is particularly effective for handling high-dimensional data and complex issues, as it lowers overfitting risk, improves accuracy, and ranks the priority features.