## 5 Error Analysis, Explainability and Meta-Evaluation
### Error Distribution by Class
\begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**SVM**} & \multicolumn{3}{c}{**CNN**} \\ \hline
**Clinical Pathway** & **Prec** & **Rec** & **F1** & **Prec** & **Rec** & **F1** & **Sup** \\ \hline Cough & 0.818 & 0.835 & 0.826 & 0.843 & 0.829 & 0.836 & 122,598 \\ Nausea and vomiting & 0.779 & 0.831 & 0.804 & 0.803 & 0.864 & 0.832 & 72,311 \\ Abdominal pain & 0.771 & 0.781 & 0.776 & 0.807 & 0.807 & 0.807 & 65,419 \\ Oropharynx problem & 0.769 & 0.712 & 0.740 & 0.793 & 0.741 & 0.766 & 56,327 \\ Rash & 0.875 & 0.906 & 0.890 & 0.888 & 0.897 & 0.892 & 47,756 \\ Flu syndrome & 0.586 & 0.634 & 0.609 & 0.562 & 0.663 & 0.608 & 41,827 \\ Diarrhea & 0.842 & 0.780 & 0.810 & 0.846 & 0.826 & 0.836 & 37,202 \\ Urinary problem & 0.897 & 0.873 & 0.885 & 0.897 & 0.880 & 0.888 & 37,757 \\ Body temp change pr. & 0.730 & 0.764 & 0.747 & 0.742 & 0.762 & 0.752 & 34,364 \\ Chest pain & 0.836 & 0.836 & 0.836 & 0.847 & 0.853 & 0.850 & 35,393 \\ \hline average (10 most freq) & 0.790 & 0.795 & 0.792 & 0.803 & 0.812 & 0.807 & \\ stdev (10 most freq) & 0.088 & 0.080 & 0.082 & 0.096 & 0.072 & 0.083 & \\ \hline Fainting or lipothymia & 0.647 & 0.688 & 0.667 & 0.701 & 0.720 & 0.710 & 3684 \\ Foreign body problem & 0.675 & 0.684 & 0.679 & 0.675 & 0.651 & 0.663 & 3220 \\ Emergency & 0.437 & 0.083 & 0.140 & 0.413 & 0.114 & 0.179 & 3201 \\ Breakfeeding problem & 0.595 & 0.505 & 0.547 & 0.597 & 0.457 & 0.518 & 1579 \\ Asthma or wheezing & 0.576 & 0.257 & 0.355 & 0.577 & 0.275 & 0.372 & 1519 \\ Elbow problem & 0.565 & 0.621 & 0.592 & 0.664 & 0.599 & 0.630 & 766 \\ Solar exposure pr. & 0.565 & 0.759 & 0.648 & 0.585 & 0.582 & 0.584 & 601 \\ Crisis adaptation pr. & 1.000 & 0.006 & 0.013 & 0.000 & 0.000 & 0.000 & 313 \\ Measles problem & 0.706 & 0.107 & 0.186 & 0.800 & 0.071 & 0.131 & 112 \\ Heat problem & 0.333 & 0.079 & 0.128 & 0.000 & 0.000 & 0.000 & 76 \\ \hline average (10 least freq) & 0.610 & 0.379 & 0.395 & 0.501 & 0.347 & 0.379 & \\ stdev (10 least freq) & 0.176 & 0.300 & 0.260 & 0.282 & 0.287 & 0.280 & \\ \hline minimum (full data) & 0.333 & 0.006 & 0.013 & 0.000 & 0.000 & 0.000 & \\ maximum (full data) & 1.000 & 0.922 & 0.919 & 0.929 & 0.932 & 0.931 & \\ average (full data) & 0.725 & 0.671 & 0.677 & 0.712 & 0.674 & 0.683 & \\ stdev (full data) & 0.136 & 0.213 & 0.199 & 0.185 & 0.224 & 0.212 & \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Test set precision, recall and F1 of 2017–2018 SVM and CNN models for the 10 most and least frequent pathways. allows us to conclude that despite the number of examples per class, some pathways are more difficult to predict than others. Table 5 presents the confusion matrix between \"Cough\" and \"Flu syndrome\", two problems that share symptoms using the 2017-2018 models. It can be observed that, for the CNN model, the number of \"Cough\" examples misclassified as \"Flu syndrome\" surpasses the total for all other pathways (10,636 vs. 10,359), being also true for \"Flu syndrome\" examples classified as \"Cough\" (7157 vs. 6922); for the SVM model this trend is also true for \"Flu syndrome\" examples classified as \"Cough\" (7821 vs. 7484), but there are less \"Cough\" examples misclassified as \"Flu syndrome\" when compared to all other classes (9271 vs. 10,988). Moreover, while SVM classifies more \"Flu syndrome\" cases as \"Cough\" (7821 vs. 7157), CNN classifies more \"Cough\" cases as \"Flu syndrome\" (10,636 vs. 9271).