# Heart Disease Prediction Using Concatenated Hybrid Ensemble Classifiers
## 3 Proposed Model
### Ensemble Learning  
The proposed approach introduces an ensemble learning technique, which combines the predictions of two hybrid ensemble classifiers:  
1. Hybrid Ensemble 1: This ensemble consists of SVM, Decision Tree, and KNN classifiers. Each base classifier is trained on the preprocessed dataset independently. 2. Hybrid Ensemble 2: This ensemble includes Logistic Regression, Adaboost, and Naive Bayes classifiers, each trained on the same dataset as in Hybrid Ensemble 1. The selection of base learners was conducted based on their robust performance in previous studies and their relevance to the specific characteristics of the dataset. While the ensemble method employs a majority voting scheme, the fusion of diverse classifiers with distinct decision boundaries enables the exploration of complementary aspects of the data, thereby enhancing the model's predictive capabilities. The inclusion of these basic classifiers in components 1 and 2 was chosen to leverage their individual strengths and ensure a diverse range of learning strategies within the hybrid ensemble framework. Specifically, in designing Hybrid Ensemble 1, we aimed to integrate classifiers with diverse capabilities to enhance the model's overall performance. Three base classifiers were selected based on their individual strengths:  
1. SVM: Linear kernels were chosen for their simplicity and robustness to linearly separable data. SVMs are known for producing efficient decision boundaries. 2. Decision Tree: Selected for its ability to represent nonlinear relationships in the data and for its interpretability. 3. KNN: Employed for recognizing local patterns and adjusting to the structure of the data. The integration of these three classifiers in Hybrid Ensemble 1 provides the model with the ability to handle linear and nonlinear patterns, contributing to its generalization and robustness. For Hybrid Ensemble 2, three distinct base classifiers were chosen:  
1. Logistic Regression: A straightforward yet powerful linear classifier suitable for binary classification tasks. 2. AdaBoost: An ensemble technique known for building a powerful classifier by combining weak ones, adapting to complex data. 3. Naive Bayes: A probabilistic classifier frequently used in various domains, particularly in text categorization. This ensemble design ensures a combination of classifiers with diverse attributes, enhancing the model's adaptability to different data features and improving the overall prediction accuracy. The rationale behind the selection aligns with established practices in the literature, promoting transparency and reproducibility of the proposed approach. The predictions from both hybrid ensembles are then concatenated and used as input to a Voting Classifier. This final step aggregates the predictions from all base classifiers, employing a majority voting scheme to make the final prediction. The main reason behind this concatenated hybrid ensemble approach is to exploit the diversity of base classifiers, each with its strengths and weaknesses. By combining two hybrid ensembles, we aim to enhance the model's overall predictive performance. The proposed hybrid ensemble classifier can be represented using the equation below (Equation (3)):  
\[f(x)=mode(C_{1}(x),C_{2}(x),\ldots,C_{N}(x)) \tag{3}\]  
where \(f(x)\) represents the decision function of the ensemble voting classifier, \(mode()\) returns the most frequent class among the predictions, and \(C_{i}(x)\) represents the class predicted by the \(i\)-th base classifier for the input feature vector \(x\). For the first ensemble classifier, \(C_{1}(x)\) represents the Support Vector Machine, \(C_{2}(x)\) is the Decision Tree classifier, and \(C_{3}(x)\) is the K-Nearest Neighbor classifier.