## 5 Error Analysis, Explainability and Meta-Evaluation
### Explainability  
It is well known that explaining a ML model decision is not a straightforward task (except in very simple models). Presently, there is a considerable effort in the community to overcome this issue; in that sense, several tools are becoming available. Some of these tools intend to show the most relevant features contributing to the decision process, thus providing ways to explain black-box models. EL15 (v0.13) [34] is one such tool and was used here to illustrate how we can explain the output produced by the SVM model. Figure 11 shows the EL15 output for two examples corresponding to the following contact reasons:  
1. _dername ocular a direita hu 2 semanas aprox. e prurido ocular desde ontem_ right eye effusion 2 weeks ago approx. and itchy eyes since yesterday  
Figure 10: Per class performance: F1 value vs. support. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**SVM**} & \multicolumn{3}{c}{**CNN**} \\ \hline  & Cough & Flu & Other & & Cough & Flu & Other \\ \hline Cough & 102,339 & 9271 & 10,988 & Cough & 101,603 & 10,636 & 10,359 \\ \hline Flu & 7821 & 26,522 & 7484 & Flu & 7157 & 27,748 & 6922 \\ \hline Other & 14,989 & 9432 & 991,171 & Other & 11,728 & 11,015 & 994,040 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Confusion matrix for “Cough” and “Flu syndrome” pathways using 2017–2018 models. 2. _Congestao nasal, tosse com exectoracao ndo efica e secretoes oculares amarelas hd 24 horas_ Nasal congestion, coughing ineffective sputum and yellow eye secretions for 24 h Both texts above were assigned by the nurse to the pathway \"Ocular problem\"; they were classified by the SVM model as \"Ocular problem\" in the first case and as \"Cough\" in the second (with \"Ocular problem\" being the third guess). Figure 11 shows, for each example, the input words and corresponding weights for two pathways. For the first example, it shows the pathway chosen by the classifier the second most probable pathway; for the second example, it shows the pathways chosen by the classifier and by the nurse (the 3 rd most probable one). The green colour signals words contributing in favour of the pathway; the red colour, against. Looking at the examples and their explanations, it is noticeable that sometimes the most contributing words have similarities with the pathway name. Nonetheless, there are also cases in which different terms are chosen as important contributors to the decision. For the first example (on the left), the most relevant features for classifying it as \"Ocular problem\" are: \"ocular\" (being related to the algorithm denomination), followed by \"secretion\" (_dername_), \"right\" (_direita_) and \"itching\" (_purrido_); only 3 words count negatively. For the second most probable pathway, \"Rash\", \"itching\" (_purrido_ is the word contributing the most to the class, with six words contributing negatively (being \"ocular\" one of them). ELIS also states that the example has a belonging probability of 99.2% to \"Ocular problem\" and of 0.1% to \"Rash\". For the second example (on the right), classified as \"Cough\" by the ML model with a probability of 70.1%, \"cough\" (_tosse_) is the word with the highest positive weight, followed by \"expectoration\" (_expectoraedo_) and \"nasal\". The pathway chosen by the nurse, \"Ocular problem\", was the 3 rd most probable one, with a probability of 4.3% (after \"Flu syndrome\" with a probability of 12.6%); for this pathway, \"oculars\" (_oculares_), \"secretions\" (_secrecyes_) and \"yellow\" (_amarelas_) have considerable positive weights, but there are 5 words contributing negatively, being \"cough\" (_tosse_) and \"nasal\" the most important ones. Although contributing negatively, their weights in this pathway are not as large (absolute value) as the weights contributing positively to pathway \"Cough\" (for example, \(-1.118\) vs. +9.108 for word \"cough\"). From a clinical perspective, the ML model leads to the correct assessment of the clinical symptoms priority when choosing the algorithm.