## 4 Material and Methods
### Machine Learning Algorithms and Performance Metrics  
To validate the best model's performance, 20% of the initial dataset was reserved randomly as a test set by stratifying on the \"admission status\" variable. Using the remaining 80% of the initial dataset, a set of models (such as artificial neural networks (ANNs), K-nearest neighbor (KNN), random forest (RF), and logistic regression (LR)) were evaluated using a five-fold cross-validation method and trained as a regressor (1 if admitted, and 0 if discharged). An ANN is a widely used ML algorithm composed of input, hidden, and output layers that are connected to each other by weighted edges. After input, feature vectors are processed sequentially by every layer via non-linear transformations in each neuron; an output (e.g., a class label for classification or regression) is generated by the final layer. In the KNN method, each object being classified is compared with its k-nearest training examples via a distance function, where k is an integer; its label is then assigned by a majority vote [57]. RFs are a combination of tree predictors such that each tree depends on the value of a random vector sampled independently from the same distribution for all trees in the forest [58]. Predictions from all trees are aggregated, and the final class is assigned by majority vote. In LR, categorical and continuous predictors predict a binomial outcome as a special case of a generalized linear model [59]. The 5-fold cross validation method consists of five random splits (four fifths for training and one fifth for validation) stratified on the outcome variable; this results in five models for each type of algorithm. The preprocessing consisted of value normalization for numerical variables and one-hot encoding for categorical variables. Given the limited medical data available at this stage in the care path, we used all the features available at triage. The ANN model consisted of a multilayer perceptron composed of four dense layers and three dropout layers. This architecture was based on the structured data branch of the preliminary experiments [27]. The hyperparameters (the number of neurons in the first three dense layers and the dropout rate) were tuned using the random search strategy, with the cross entropy as the loss function. A rectified linear unit activation function was used in the hidden layers, and a sigmoid function was used in the output layer. The training process was completed over 10 epochs, using the Adam [60] optimizer with its default parameters. The KNN algorithm was trained with up to 10 neighbors and the kd_tree algorithm driven by the Manhattan metric. The RF algorithm was set up using 100 estimators, the Gini criterion, \"min_samples_split\" = 2, and use of the square root to find the best split. The LR algorithm was set up using the penalty at L2 and the limited-memory Broyden-Fletcher-Goldfarb-Shanno solver [61].  
Each algorithm's performance was assessed in terms of the mean and the standard deviation (SD) AUROC for the 5-folds cross validation of the training set (80%). A Friedman test was used to compare the respective models' AUROC, with \(p\) < 0.05 as the threshold for statistical significance (Supplementary Material S1.2). Lastly, the performance metrics were evaluated once on the best model for each architecture, using the test set (20%). The performance of each of the best models was rated in terms of recall, precision, and accuracy, by applying a threshold (defined by the best F1 score). The models were trained and assessed using the Python library Scikit-Learn 0.21 [62] and Keras 2.11.0 [63].