## 3 Results
### Smote  
Next, in order to address the dataset's class imbalance, we utilized the Synthetic Minority Oversampling Technique (SMOTE) algorithm. This approach involved generating synthetic instances within the minority class to augment its representation within the dataset. The application of SMOTE resulted in the generation of an adequate number of observations, thereby equalizing the representation of both classes. Consequently, the dataset comprised a total of 397,294 observations. The performance results of the trained models on the test set, in contrast with their performance on the training set after applying SMOTE in the dataset, can be seen in Table 5.  
The implementation of SMOTE yielded a significant improvement in the models' ability to detect individuals at risk of heart disease. Notably, the recall metric experienced a substantial surge, rising from 3% to an impressive 64% when employing logistic regression or random forest algorithms. This heightened sensitivity indicates an enhancement in accurately identifying positive cases, a critical aspect in the domain of heart disease prediction. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Model** & **Macro Avg. F1-Score** & **Weighted Avg. F1-Score** \\ \hline Logistic Regression & 0.52 & 0.89 \\ Decision Tree & 0.53 & 0.89 \\ Random Forest & 0.52 & 0.89 \\ Gradient Boosting & 0.52 & 0.89 \\ XGBoost & 0.58 & 0.80 \\ CatBoost & 0.58 & 0.80 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Macro and weighted average F1-score. Figure 4: Trade-off between accuracy and recall. However, it is imperative to acknowledge the trade-offs associated with this improvement. While the recall metric demonstrated a remarkable boost, accuracy and precision witnessed a decline. This implies that while the models become more proficient at capturing instances of heart disease, there is a corresponding increase in false positives and a potential reduction in overall predictive accuracy. Interestingly, the boosting algorithms did not demonstrate a proportional enhancement in performance with the introduction of SMOTE. Despite the augmentation of synthetic data, they maintained a level of performance comparable with that before oversampling the dataset. These variations in performance across different algorithms underscore the complexity associated with the utilization of SMOTE and its impact on various evaluation metrics. Subsequently, we proceeded to fine-tune our models, and the attained results are delineated in Table 6.  
The findings in Table 6 reveal that, despite an exhaustive grid search aimed at optimizing their hyperparameters, there was no discernible enhancement in the performance of logistic regression, decision trees, and random forest. Furthermore, the latter two algorithms appear to have potentially overfit the training data. However, our boosting algorithms have exceeded expectations. CatBoost achieved an impressive recall of 79%, while XGBoost attained a noteworthy 80%. Additionally, both models maintained a robust AUC score of 80%, underscoring their resilience and discriminative capabilities. As depicted in the confusion matrix presented in Figure 5, the optimized XGBoost model successfully identified 6078 out of the total 7477 positive cases. This outcome highlights the potential utility of our approach in the healthcare domain, showcasing its efficacy in accurately identifying a substantial proportion of positive cases.