## 4 Feedforward Neural Network and RMS  
An FNN is a supervised ANN where the information flows in a single direction from the input to the output nodes through hidden nodes [62, 63]. It is composed of the following:  
* An input layer with a number of neurons equal to the number of inputs;
* An output layer with a number of neurons equal to the number of outputs;
* A few hidden layers and corresponding nodes defined through a hyperparameter optimization process.  
Each node in every layer of the network is connected to all nodes in the previous and subsequent layers. Considering an FNN with a single hidden layer, the output \(z_{j}\) of \(j\)-th neuron is modelled as follows:  
\[z_{j}=\sum_{i=1}^{n}w_{ij}u_{i}+b_{j} \tag{2}\]  
where \(w_{ij}\) is the weight assigned to the input \(u_{i}\) in the \(j\)-th neuron and \(b_{j}\) is the bias associated with the \(j\)-th neuron. Applying the activation function (also subject to optimization) to the output of the next \(j\)-th neuron results in the following:  
\[y=\sum_{j=1}^{n}W_{j}a_{j}+B \tag{3}\]  
Figure 4: Model training, optimization, and testing workflow.  
where \(y\) is the output, \(W_{j}\) is the weight of the input \(a_{j}\) of output layer, and \(B\) is the bias.  
During the network supervised training phase using backpropagation for gradient descent, the FNN adjusts weights and biases based on the assigned input and output variables. This process is repeated over a few epochs or until the loss function reaches a predefined threshold: the goal is to minimize a loss function that measures the difference between the model predictions and the desired values. To achieve these results, it is necessary to iteratively update the model weights to reduce the loss. To optimize the loss function and iteratively update the model weights, the Quasi-Newton BFGS method [67] is applied: its advantage lies in its ability to quickly converge to the global minimum of the loss function, reducing the number of iterations required for convergence.  
Among the various activation functions available are ReLU (Rectified Linear Unit), hyperbolic tangent, and SoftMax. The first of these, ReLU, was chosen for the input layer:  
\[ReLU(x)=\begin{cases}x,&x>0\\ 0,&x\leq 0\end{cases} \tag{4}\]  
The hyperbolic tangent function was applied as the activation function for the hidden layers:  
\[Tanh(x)=\frac{sinh(x)}{cosh(x)} \tag{5}\]  
For the final layer, the SoftMax activation function, ideal for multi-class classification problems, was used:  
\[Softmax\ (x_{i})=\frac{e^{x_{i}}}{\sum_{j}e^{x_{j}}} \tag{6}\]  
The hyperparameters of an ML model are not learned during the training phase because the objective function is not differentiable with respect to them. To set the best model hyperparameters, it is necessary to use an optimization process such as Bayesian Optimization [64,65], which is a method used to optimize for a higher coast global function when the function is not in closed form. Assuming only observations of unknown function \(f(\cdot)\) are known,  
\[y=f(x)+\varepsilon \tag{7}\]  
where \(\varepsilon\) is the gaussian noise. The Bayesian Optimization process can be expressed below:  
\[x^{*}=\arg\underset{x}{\max}f(x) \tag{8}\]  
Using sequential optimization, only \(x_{t}\) at instant \(t\) is selected and the objective function \(f(\cdot)\) is modelled as a Gaussina Process.  
In the last part of this research, the RMS [66] statistical index was calculated. It is defined as follows:  
\[RMS=\sqrt{\frac{1}{n}\sum_{i=1}^{n}{x_{i}}^{2}} \tag{9}\]  
where \(x_{i}\) is the \(i\)-th component of signal \(x\) and \(n\) is the signal length.