## 3 Results  
### Best Model Selection  
The results of parameter tuning for the 98 models evaluated showed an improvement in performance with an increasing number of nodes and decreasing L2 regularisation (Figure 3). log loss was consistently lower than the non-informative threshold (0.693 for a binary classification), reaching the lowest value of 0.339 at 27 nodes and L2 parameter equal to \(5\times 10^{-3}\), when one hidden layer was used, and of 0.346 with 19 nodes, L2 equal to zero and two hidden layers. Log loss was consistently lower for models using one hidden layer rather than two, although the differences were small for models with more than 11 nodes and low L2 parameter. AUC was always higher in models with one hidden layer and less variable among models (Figure 3), suggesting that two hidden layers may introduce a degree of overfitting in the conditions analysed here. The highest AUC of 0.929 was obtained with one hidden layer, 11 nodes, and L2 equal to \(1\times 10^{-4}\).  
To choose the best model, we accounted for both log loss and AUC, also prioritising the least complex model to avoid overfitting. We picked the best performing model among those trained with one hidden layer because they consistently outperformed models with two hidden layers. Among those models, the lowest log loss and highest AUC were obtained with different parameter values, as reported above: 27 nodes and L2 of \(5\times 10^{-3}\), and 11 nodes and L2 of \(1\times 10^{-4}\), respectively. The performance of those two models differed negligibly: the difference in AUC and log loss was \(5\times 10^{-4}\) and \(2\times 10^{-3}\), respectively. Based on these results, we can confidently prioritise the least complex model among the ones performing best for AUC and log loss. The selected model had one hidden layer, 11 nodes, and L2 equal to \(1\times 10^{-4}\).  
The model chosen as the best performing (based on the criteria stated above) was among the models with the lowest log loss and highest AUC among those validated in the present study (Figure 4). The performance on cross-validation returned an accuracy of 0.867 \(\pm\) 0.022, AUC of 0.929 \(\pm\) 0.017, and log loss of 0.341 \(\pm\) 0.033. Furthermore, when  
Figure 3: Performance of the sex classification on the validation dataset during parameter tuning. The top graphs show the performance as log loss, while the bottom ones use the area under the receiver operating characteristic (ROC) curve (area under curve or AUC). The performance is compared across different values of the tuned parameters: L2 regularisation, number of nodes per layer, and number of hidden layers (the results for models trained with one hidden layer are shown on the left, with two hidden layers on the right). The performance improves from yellow to blue, specifically with log loss decreasing and AUC increasing. The blue sphere indicates the best performing model: it shows the maximum performance obtained and the relative values of the tuned parameters that produce that performance.  
we look at the learning curves for the best model (Figure 5), we can appreciate how the performance (log loss and AUC) of the training and validation sets had a comparable pattern and small difference, reaching a point of stability along the epochs. The pattern shown in Figure 5 suggests that the model selected had not undergone overfitting.