###### Abstract  
This paper presents an analysis of the calls made to the Portuguese National Health Contact Center (SNS24) during a three years period. The final goal was to develop a system to help nurse attendants select the appropriate clinical pathway (from 59 options) for each call. It examines several aspects of the calls distribution like age and gender of the user, date and time of the call and final referral, among others and presents comparative results for alternative classification models (SVM and CNN) and different data samples (three months, one and two years data models). For the task of selecting the appropriate pathway, the models, learned on the basis of the available data, achieved F1 values that range between 0.642 (3 months CNN model) and 0.783 (2 years CNN model), with SVM having a more stable performance (between 0.743 and 0.768 for the corresponding data samples). These results are discussed regarding error analysis and possibilities for explaining the system decisions. A final meta evaluation, based on a clinical expert overview, compares the different choices: the nurse attendants (reference ground truth), the expert and the automatic decisions (2 models), revealing a higher agreement between the ML models, followed by their agreement with the clinical expert, and minor agreement with the reference. clinical triage; clinical pathways; SNS24; data analysis; machine learning; support-vector machines; deep neural networks; explainability +
Footnote †: journal:  
0000-0001-9901-9901-9901-9901-9 important role in the National Health Service since allows a close connection with citizens in their home places and, according to their symptoms, a specialist analysis upon their clinical situation and decision about the most appropriate referral. Clinical pathways are developed by health professionals and approved by the General Directorate of Health (DGS). During the period of this study, the number of existent clinical pathways was 54, ranging from \"Cough\" or \"Abdominal Pain\" to \"Allergy\". The SNS24 Scout.AI project aims at applying Artificial Intelligence methodologies in the construction of decision support tools to help nurses in selecting the most appropriate clinical pathway and to support DGS in the process of optimising the design of clinical pathways and respective referrals. This study presents the work developed during the second year of the project, which extends and elaborates on previous experiments using three years of receiving calls, in a total of around 2,577 millions of records. In [2], a preliminary study using different text representations and shallow ML classification algorithms is presented while in [3] a thorough analysis of Deep Neural Network (DNN) models is exposed. Both studies use a subset of data composed of 3 months calls (with around 270,000 records). Differently from these previous works, the current one presents a detailed characterisation of the data, a comparison between both approaches along with a new comprehensive analysis of the results, regarding errors per class, model explainability and a meta-evaluation made by an clinical expert. The main contributions of this paper are summarised as follows:  
* a characterisation of 3 years SNS24 calls;
* a thorough comparison of different Machine Learning models to select the most appropriate clinical pathway;
* a comprehensive analysis of the results including classification and execution time performance, per class error analysis, explainability of decisions and experts' meta-evaluation.