## 5 Performance Evaluation
### Implementation and Evaluation
#### 5.3.2 Part II-Ensemble Learning algorithms  
After analyzing and comparing the performance of MLNN algorithms which are introduced in Section 4, here, we evaluate performance of different combinations of some machine learning algorithms. The combination technique is known as Ensemble Learning or Voting Classifier. Ensemble learning combines two or more learning algorithms in order to get a better performance. Table 5 shows results of different combination of our 7 algorithms in groups of 3 algorithms. In other words, we implemented 35 different combinations of the algorithms. As LSTM was implemented using Pytorch, due to technical limitation, we did not combine it with other algorithms. The results show that the ensemble learning methods including two neural network-based algorithms (i.e., CNB-SLP-MLP, DT-SLP-MLP, KNN-SLP-MLP, RF-SLP-MLP and SLP-SVM-MLP) have poor performance. However, as expected, a combination of a low-performance neural network algorithm with two high-performance algorithms (e.g., DT, RF and MLP) can give an acceptable performance. In general, it can be seen that there are several ensemble learning algorithms with an accuracy above 79%. Looking again at results of Part I (i.e., Table 4), we have only RF algorithm which its accuracy is above 79%. According to Table 5, a combination of DT, RF and CNB can provide the best cross validation scores (e.g., with accuracy equals to 79.81%) than other combinations. Regarding the execution time, we can see that the combinations which contain MLP consume more time than other combinations for the training phase. It is because training phase of MLP is time consuming. \begin{table}
\begin{tabular}{l l l l l l} \hline \hline
**VC** & **Accuracy** & **Precision** & **Recall** & **f1\_Score** & **Training (s)** & **Testing (s)** \\ \hline CNB,SLP,MLP & 76.40 & 78.55 & 76.40 & 74.20 & 0.5928 & 0.0056 \\ CNB,S,LPSVM & 76.21 & 78.68 & 76.21 & 73.53 & 0.0379 & 0.0197 \\ CNB,SVM,MLP & 76.65 & 79.05 & 76.65 & 74.11 & 7.3174 & 0.0206 \\ DT,CNB,MLP & 79.42 & 81.45 & 79.42 & 77.94 & 0.5881 & 0.0057 \\ DT,CNB,SLP & 78.72 & 80.68 & 78.72 & 77.34 & 0.0094 & 0.0049 \\ DT,CNB,SVM & 79.25 & 81.41 & 79.25 & 77.48 & 0.0343 & 0.0196 \\ DT,RF,CNB & 79.81 & 81.65 & 79.81 & 79.11 & 0.2205 & 0.0239 \\ DT,RF,MLP & 79.72 & 81.58 & 79.72 & 79.00 & 0.7966 & 0.0245 \\ DT,RF,SLP & 79.26 & 81.13 & 79.26 & 78.63 & 0.2182 & 0.0233 \\ DT,RF,SVM & 79.76 & 81.60 & 79.76 & 78.97 & 0.2416 & 0.0380 \\ DT,SLP,MLP & 76.84 & 78.84 & 76.84 & 75.06 & 0.5886 & 0.0056 \\ DT,SLPSVM & 76.69 & 78.87 & 76.69 & 74.50 & 0.0373 & 0.0197 \\ DT,SVM,MLP & 76.92 & 79.12 & 76.92 & 74.69 & 0.6160 & 0.0206 \\ KNN,CNB,MLP & 79.34 & 81.44 & 79.34 & 77.85 & 0.5870 & 0.0069 \\ KNN,CNB,SLP & 78.28 & 80.36 & 78.28 & 76.88 & 0.0092 & 0.0063 \\ KNN,CNB,SVM & 79.15 & 81.41 & 79.15 & 77.36 & 0.0346 & 0.0212 \\ KNN,DT,CNB & 79.60 & 81.58 & 79.60 & 78.98 & 0.0059 & 0.0062 \\ KNN,DT,MLP & 79.39 & 81.45 & 79.39 & 78.70 & 0.8575 & 0.0070 \\ KNN,DT,RF & 79.18 & 81.12 & 79.18 & 78.81 & 0.2245 & 0.0256 \\ KNN,DT,SLP & 78.84 & 80.96 & 78.84 & 78.26 & 0.0088 & 0.0062 \\ KNN,DT,SVM & 79.44 & 81.48 & 79.44 & 78.68 & 0.0336 & 0.0209 \\ KNN,RF,CNB & 79.40 & 81.21 & 79.40 & 78.79 & 0.2171 & 0.0250 \\ KNN,RF,MLP & 79.42 & 81.18 & 79.42 & 78.81 & 0.7988 & 0.0259 \\ KNN,RF,SLP & 78.93 & 80.79 & 78.93 & 78.37 & 0.2176 & 0.0247 \\ KNN,RF,SVM & 79.39 & 81.13 & 79.39 & 78.71 & 0.2427 & 0.0398 \\ KNN,SLP,MLP & 76.58 & 78.70 & 76.58 & 74.81 & 0.5883 & 0.0068 \\ KNN,SLP,SVM & 76.37 & 78.53 & 76.37 & 74.14 & 0.0373 & 0.0211 \\ KNN,SVM,MLP & 76.90 & 79.09 & 76.90 & 74.66 & 4.2762 & 0.0222 \\ RF,CNB,MLP & 79.37 & 81.44 & 79.37 & 77.86 & 0.7913 & 0.0245 \\ RF,CNB,SLP & 78.57 & 80.65 & 78.57 & 77.09 & 0.2214 & 0.0236 \\ RF,CNB,SVM & 79.18 & 81.40 & 79.18 & 77.39 & 0.2439 & 0.0379 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Ensemble Learning Algorithms (combinations of three algorithms). We can see, in Table 6, results of different combination of the seven algorithms in groups of 5 algorithms.