## 3 Methodology
### ANN Training and Regularization  
The supervised training process identifies connection weights and biases that minimize the difference between the ANN output \(\hat{\mathbf{y}}\) and the experimental target \(\mathbf{y}\), corresponding to the input feature vector \(\mathbf{x}\). This process is divided into 2 distinct phases: a forward and a backward pass. In the forward stage, the training feature vector \(\mathbf{x}\) is inputted to the network, and the neuron activations produce the output \(\hat{\mathbf{y}}\). After that, a backward comparison is made between the computed output \(\hat{\mathbf{y}}\) and the experimental target vector \(\mathbf{y}\) by a performance function \(F(\hat{\mathbf{y}},\mathbf{y})\), also called loss function, with the aim of defining the corrections to the weights and biases of the network. The training process involves the use of a learning rule that defines the update of network parameters \(\mathbf{W}\)(the matrix of weights and biases) according to the value assumed by the performance function for a fixed  
Figure 3: Exponential linear (ELU), hyperbolic tangent (TanH), and logistic sigmoid (LogS) activation functions. number of iterations \(E\). The mean sum of the error squares (or mean square error, MSE) is commonly accepted as \(F(\cdot)\) function and its gradient with respect to \(W\), calculated by means of a backpropagation algorithm [52], is used in the learning rule so that the network parameters are updated to minimize the loss value. For a better understanding of the training process implemented in this study, the analytical expression of the learning rule is presented in Equation (2):  
\[\mathbf{W}^{+1}=\mathbf{W}^{-}-\Big{(}\
abla^{2}F(\mathbf{W}^{*})\Big{)}^{-1}\
abla F(\mathbf{W }^{*}),e\in\{1,\cdots,E\} \tag{2}\]  
Such equation, assuming the Levenberg-Marquardt (LM) backpropagation algorithm [56], becomes:  
\[\mathbf{W}^{e+1}=\mathbf{W}^{e}-\Big{[}\mathbf{J}^{T}(\mathbf{W}^{e})\mathbf{J}(\mathbf{W}^{e})+\mu_{e} \mathbf{I}\Big{]}^{-1}\mathbf{J}^{T}(\mathbf{W}^{e})\mathbf{v}(\mathbf{W}^{e}),e\in\{1,\cdots,E\} \tag{3}\]  
where \(\mathbf{W}^{e}\) is the matrix of weights and biases at iteration \(e\), \(\mathbf{J}\) is the Jacobian matrix of the training loss \(F(\cdot)\) with respect to \(\mathbf{W}^{e}\), \(\mathbf{I}\) is the identity matrix, and \(\mathbf{v}(\mathbf{W}^{e})=\hat{\mathbf{y}}(\mathbf{W}^{e})-\mathbf{y}\) is the vector of network errors. \(\mathbf{W}^{e+1}\) are the updated values of network parameters to be used in the forward pass of iteration \(e+1\). While the direction towards the minimum is determined by the gradient \(\mathbf{J}^{T}(\mathbf{W}^{e})\mathbf{v}(\mathbf{W}^{e})\), the scalar \(\mu\) determines the step size taken in that direction at each iteration and, as a result, the convergence rate. In particular, a high \(\mu\) value corresponds to a small learning step and a very long convergence time; on the contrary, a low \(\mu\) allows the convergence speed to increase but the algorithm could jump over the minimum. In order to achieve faster convergence and, in parallel, avoid undesirable local minima, the parameter \(\mu\) is varied during training: the value \(\mu_{e+1}\) corresponds to \(\mu_{e}\) multiplied by \(\mu_{inc}>1\) (or \(\mu_{dec}<1\)) if the performance index has increased (or decreased) between iterations \(e-1\) and \(e\). In addition, if the parameter \(\mu\) becomes too large, \(\mu_{e+1}>\mu_{\max}\), the LM algorithm is stopped. At the end of the \(E\) iterations of the training process or when \(\mu_{max}\) is reached (a good indication that the algorithm has truly converged), the optimal weights and biases are kept fixed while the test feature vector is processed just in the forward manner to define the model's loss index on novel data. However, under the conditions set out above, a neural model may overfit the training data: such a situation is generally caused by too large values of connection weights, which increase the slope of the network function.