## 3 Methodology
### ANN Optimization  
The network parameters \(W\) are identified by means of a supervised training process, which is divided into two successive steps, i.e., a forward and a backward phase. In the latter, a backpropagation algorithm [58] is exploited to update the ANN's weight and biases:  
\[W^{(e)}=W^{(e-1)}-\alpha\
abla E\Big{[}W^{(e-1)}\Big{]},e\in\{0,\dots,E-1\} \tag{7}\]  
where \(E\) is the number of training iterations, \(\alpha\) is the learning rate and \(W^{(e-1)}\) are the parameters values at iteration \(e-1\). A detailed and comprehensive description of the Equation (7) has already been widely discussed, e.g., by [PERSON] et al. [29]. At the end of the training process (i.e., after the E iterations have been completed), the parameters \(W\) of the e-th iteration that produced the minimum of the loss function \(L\) are selected. Such loss function is usually defined as the 2-norm of the difference between the network output \(\mathbf{y}^{(d)}\) and the ground-truth vector \(\mathbf{y}^{(d)}\), thus being called Mean Squared Error (MSE):  
\[L\left(\mathbf{y}^{(d)},\mathbf{y}^{(d)}\right)=\parallel\mathbf{y}^{(d)}-\mathbf{y}^{(d)} \parallel_{2}^{2} \tag{8}\]  
The training process just outlined is known as Stochastic Gradient Descent (SGD). Such approach is commonly used in the well-known [60; 61] and widely used MATLAB(r) ANN Toolbox [22; 23; 29; 45; 46; 50].  
With the aim of increasing the convergence speed of the learning algorithm, several improvements have been proposed in recent years. The first remarkable one, called RAdam optimizer [62], faces the issue of implementing warmup heuristics to avoid falling in bad local minima. This problem can occur using adaptive optimizers such as Adam [63]. According to the authors, the variance of the adaptive learning rate in the initial weight updates can be intractable and direct the solution search towards local minima with poor performance. Adding a rectifier operation to the adaptive learning rate was shown to reduce the variance and lead to better solutions. The second remarkable improvement, Lookahead [64], is a suitable method in reducing the need for extensive hyperparameters search, and the variance of the parameter updates. The Lookahead algorithm maintains two copies of the parameters, respectively, the \"fast weights\" and \"slow weights\". The first is initially updated for k times using a standard SGD-based optimizer. Then, the second set of weights is updated in the direction (i.e., the gradient) of the last computed  
Figure 5: Activation functions: exponential linear (ELU), hyperbolic tangent (TanH), rectified linear (ReLU).  
fast weights. Intuitively, as the name states, the first kind of update looks ahead to acquire information of the loss function's surface. Once obtained, the gradients used for the actual weight update result in a more accurate direction towards the loss function's minimum. The combination of the strategies is known as Ranger optimizer. In this work, such optimization algorithm was used to calculate the optimal weights of the implemented ANN, in contrast with the common procedures of neural modeling used in previous literature studies [22; 23; 29; 31; 45; 46; 50].