# Heart Disease Prediction Using Concatenated Hybrid Ensemble Classifiers
## 3 Proposed Model
### Feature Selection  
Feature selection plays a critical role in ML models, as not all features may positively contribute to decision-making. To address this, the Extra Tree classifier has been employed to select the most relevant features from the dataset. The Extra Trees Classifier, also known as Extremely Randomized Trees, is an ensemble learning method based on decision trees. Mathematically, we can represent the decision function of the Extra Trees Classifier as follows in Equation (2):  
\[f(x)=C_{k}(x) \tag{2}\]  
where \(f(x)\) represents the decision function and \(C_{k}(x)\) denotes the class assigned to the input feature vector \(x\) by the \(k\)-th decision tree.  
The strategic feature selection process using the Extra Trees Classifier enhances the efficiency, interpretability, and generalization of our heart disease prediction model. Before feature selection, the dataset comprised a comprehensive set of features, including age, sex, chest pain type (cp), resting blood pressure (tresbps), serum cholesterol (chol), fasting blood sugar (fbs), resting electrocardiographic results (restecg), maximum heart rate achieved during exercise (thalach), exercise-induced angina (exang), ST depression induced by exercise relative to rest (oldpeak), slope of the peak exercise ST segment (slope), number of major vessels colored by fluoroscopy (ca), and thalassemia type (thal), as depicted in Table 4.  
\begin{table}
\begin{tabular}{l l} \hline \hline
**Name of Features before Feature Selection** & **Name of Features after Feature Selection** \\ \hline Independent Features: age, sex, cp, tresbps, chol, fbs, restecg, thalach, exang, oldpeak, slope, ca, thal & Independent Features: age, cp, thalach, exang, oldpeak, slope, ca, thal \\ Dependent Feature: Target & Dependent Feature: Target \\ \hline \hline \end{tabular}
\end{table}
Table 4: Features before and after feature selection.  
After careful consideration, the feature selection process retained the most informative features, including age, chest pain type (cp), maximum heart rate achieved during exercise (thalach), exercise-induced angina (exang), ST depression induced by exercise relative to rest (oldpeak), slope of the peak exercise ST segment (slope), number of major vessels colored by fluoroscopy (ca), and thalassemia type (thal). This selection aligns with existing medical knowledge about factors that influence heart disease, ensuring that our model focuses on the most relevant aspects to achieve accurate predictions.  
The selected features contribute significantly to the model's predictive capabilities while eliminating redundancy and reducing the risk of overfitting. This strategic feature selection process not only improves the computational efficiency of our model, but also enhances its interpretability and generalization to new data. Certain features, such as sex, thalach, and exang, were deemed less clinically relevant to heart disease prediction and were discarded during the feature selection process.  
To visualize the significance of each feature, we apply the Extra Tree classifier, as shown in Figure 4. Notably, slope, age, oldpeak, thalach, thalach, exang, ca, and cp were identified as the most influential features in our model.  
These insights into feature selection aim to provide a deeper understanding of the variables influencing our model's predictions.