# A Soft-Voting Ensemble Classifier for Detecting Patients Affected by COVID-19
## 2 Materials and Methods
### Ensemble Classifiers
#### 2.2.2 Blending Policies for Ensemble Methods  
In principle, training strategies and blending policies are independent of each other, whereas in practice, there is often a bias toward a specific output blending policy given the adopted training strategy. Some the most acknowledged voting policies are introduced hereinafter, which are followed by relevant comments on their use with sequential and parallel ensemble methods.  
Figure 2: Inception-V2 module is obtained factorizing filters kernel size \(5\times 5\) to two \(3\times 3\) and then factorizing \(n\times n\) convolutions to a stack of \(1\times n\) and \(n\times 1\) convolutions.  
Given an instance to be classified, several base classifier outputs must be blended together to give rise to the actual classification. Voting and stacking are the main blending methods acknowledged by the research community.  
Voting--Two main voting policies are feasible, namely hard and soft voting (see Figure 3).  
A schematic representation of both concepts follows hereinafter. Let \(m_{i}\) be the \(i\)-th model of an ensemble consisting of \(N\) individual classifiers aimed at predicting the class label from a set of \(k\) possible labels \(\{c_{1},c_{2},...,c_{k}\}\). For a given input \(\mathbf{x}\), the classifier \(m_{i}\) will generate a \(k\)-dimensional vector \([\,m_{i}^{j}(\mathbf{x}),m_{i}^{2}(\mathbf{x}),...,m_{i}^{k}(\mathbf{x})\,]\), where \(m_{i}^{j}(\mathbf{x})\) is the output of the \(i\)-th model for the \(j\)-th category label. All \(N\) individual classifiers are treated equally, and the overall classification is obtained by averaging the individual outputs with varying the class label and then selecting the label \(k^{*}\) corresponding to the highest vote. In symbols:  
\[k^{*}=\arg\max_{j}M^{j}(\mathbf{x})=\arg\max_{j}\frac{1}{N}\cdot\sum_{i=1}^{N} m_{i}^{j}(\mathbf{x})=\arg\max_{j}\sum_{i=1}^{N}m_{i}^{j}(\mathbf{x}) \tag{1}\]  
Note that the above formula can be adapted to both hard- and soft-voting policies, depending on whether base classifiers output a value in \(\{0,1\}\) (hard voting) or in \([0,1]\) (soft voting). In the former case, majority voting is actually enforced, meaning that the class that obtained most votes by the base models is selected, whereas in the latter case, \(m_{i}^{j}(\mathbf{x})\) represents an estimate of the posterior probability \(P(c_{j}|\mathbf{x})\) issued by the \(i\)-th base classifier.  
Weighted voting can be seen as a generalization of the previous cases, as here, base models are combined depending on a weight that accounts for their importance. Under the same hypotheses made above, and assuming that \(w_{i}\) represents the weight assigned to the \(i\)-th model \(m_{i}\), the weighted voting policy can be summarized as follows:  
\[k^{*}=\arg\max_{j}M^{j}(\mathbf{x})=\arg\max_{j}\sum_{i=1}^{N}w_{i}\cdot m_{i} ^{j}(\mathbf{x}) \tag{2}\]  
Note that weighted voting is the most general case of voting, as non-weighted voting can be easily simulated by setting each individual \(w_{i}\) to any given positive constant (e.g., \(w_{i}=1\)).  
Stacking--An ensemble made of base models can also be combined by training a meta-learner. The obtained model will output a prediction starting from those of the base models. There are no constraints on the type and technology of the stacking module, which in particular is not tied to the type of base classifiers.  
Figure 3: The figure shows with an example how both hard and soft voting work. The ensemble consists of three base models able to predict two possible classes, which are labeled with \(0\) and \(1\). Hard voting (**left** side) predicts the class that obtained the most votes by base models. In the example, the voting policy outputs the class labeled \(1\), which is predicted by two out of three models. Soft voting (**right** side) averages the class pseudo-probabilities of the base models (represented in red in the figure) so that the class with the highest average probability is selected.