## 4 Machine Learning Techniques
### Multilayer Perceptron (MLP)  
[PERSON] and [PERSON] first introduced ANNs as a powerful predictive tool in [66]. ANNs are a programming paradigm which try to mimic the brain's structure. They are broadly employed in artificial intelligence problems, from simple pattern-recognition tasks to advanced symbolic manipulation. ANN-based models process the available data during a training stage, assessing the network output to efficiently and accurately match some available targets in case of a supervised approach, like the one here adopted based on the FELA results included in the dataset. ANNs can be used for modeling complex systems in virtually any science [67; 68]. MLP is a class of ANNs, and stands as an evolution of the perceptron neural network originally developed in the 1960s, see [69]. For a general problem, MLP is able to provide a nonlinear map \(f:\mathbb{R}^{k}\rightarrow\mathbb{R}^{h}\), in case of an input layer made of \(k\) neurons (input values) and an output layer made of \(h\) neurons (output values). It also consists of an arbitrary number of hidden layers, with a variable number of neurons. The neurons are storage cells for scalar values, obtained by an activation function applied to the neuron values in the previous layer. For our problem, a preliminary analysis has been performed to identify the optimal architecture of the MLP, in terms of the number of neurons in the hidden layers, to accurately describe the correlation between input and output of the analyzed dataset. In this regard, one-hidden-layer and two-hidden-layer ANNs have been investigated--to speed-up the optimization procedure, only 5% of the data has been adopted at this stage. To this aim, in [70], it was suggested that the ratio between the number of data and the number of input and output variables should be greater than five, to attain an appropriate performance of MLP--in the present case, the mentioned ratio results are \(1900/9=211\), which are much larger than the minimum value suggested. Figure 6 provides a comparison of the behaviors of all the considered architectures at the end of training. Results are reported in terms of the root mean square error (RMSE), given by:  
\[\text{RMSE}=\sqrt{\frac{1}{n}\sum_{i=1}^{n}(Q_{\text{OG2 }i}-Q_{\text{Model }i})^{2}} \tag{3}\]  
which has been assumed as the loss function to be minimized during the training. In Equation (3): \(n\) is the total number of data; and for the \(i\)-th model, \(Q_{OG2\ i}\) is the load bearing capacity obtained with the Optum G2 software, whereas \(Q_{\text{Model }i}\) is the value estimated through the MLP algorithm. To assess the effects of the hyperparameters on the ANN accuracy, the plot in Figure 6 shows the final value of RMSE as a function of the total number of neurons in the MLP. The continuous line represents the solution obtained with the one-hidden-layer ANN, which is obviously uniquely defined by the number of neurons in the layer itself. The dashed lines represent the solutions for the two-hidden-layer ANNs, and for each of them, the label in the chart represents the number of neurons in its first hidden layer. Each point in this graph has been computed as the average value out of ten repetitions of the training process, to also assure robustness against stochastic effects. What can be seen from this analysis is that the ANN featuring one hidden layer only, almost surely provides the bestperformance. Furthermore, for a number of neurons larger than 20, there is no noticeable improvement in the final value of the RMSE. Accordingly, and in order to minimize the computational costs, the 8-20-1 ANN architecture has been adopted henceforth. The effect of the proportioning of the data handled for training, validation, and testing of the MLP has been investigated next. As before, 5% of the data have been handled to evaluate the performance of MLP for different values of proportioning, and the results are shown in Figure 7 at varying values of the percentage of training data.