## 3 Methodology
### Combining Different Models (Ensemble Approach)  
The transition to a supervised learning approach produces two sets of models specialised in quantitative and qualitative data modelling, respectively. Many decisions are made by these models. Undoubtedly, one of these models alone cannot be selected because it specialises in a particular feature set space. The ensemble approach, therefore, had to be adopted to combine the different models and to produce a single composite decision. An ensemble-based system is obtained by blending diverse models, subsequently referred to as classifiers or experts. These systems are also known as multiple classifier systems or ensemble systems [41].  
There are many scenarios in which the ensemble approach makes statistical sense. In many applications requiring automated decision making, receiving data from various sources that may provide complementary information is not unusual. An appropriate combination of this information is known as data or information fusion [41]. It can improve the accuracy of a classification decision compared to a decision based on any of the individual data sources alone [41].  
It is clear that the scenario of data fusion has an excellent fit with the proposed model. The observed heart data set is considered to be heterogeneous data, coming from two different sources which represent the quantitative and qualitative features, respectively. Clustering algorithms specialised in processing these data types are used, which collaborate to improve their results through the previously described collaborative method. Finally, clusters are assigned to the data set classes, i.e., the transition from clustering to classification. This produces a set of models that are experts in modelling the heart data set features. In order to create a single decision, the decisions made by each expert are grouped by a specific combination rule, which, for the purposes of the current model, is described below.  
The ensemble approach used in the model consists of two levels of learning: (i) essential learning and (ii) meta-learning. At the first level, a set of primary models is trained on the features of the observed data sets in order to produce a group of expert models. Once the training is finished, the expert models create a new data set containing the results of their predictions. The meta-learner is then trained using this new data set, and ultimately used to classify the latest cases. Figure 4 illustrates the ensemble approach used in the model.  
Any meta-classifier can be used. However, LR was used as this performs well with binary classification problems. It is trained on the results of expert models' predictions for both quantitative and qualitative data types and used to predict the final decision.  
The stacking approach is in keeping with the current strategy of using a stacking rule for training a meta-classifier. The main difference between them is that the cross-validation type is usually used to prepare the first-level or basic models in stacking. In the current approach, it is trained on a set of data set feature sets, i.e., on different feature spaces.  
The basic idea of using a two-level ensemble approach to learning is to see whether or not the data sets were learned correctly. For example, if a given model knew a particular region of the feature space incorrectly and thus consistently misclassified states coming from that region, a second-level meta-model could potentially learn that behaviour, together with the learned behaviours of the model. The improper training of the original model can then be corrected.