## 2 Neural Model  
The global structure of an LRNN is like that of MLP, where neurons are organized in layers, but dynamic properties are achieved using neurons with internal feedback. In Figure 1, the assumed structure of NN is shown. For the sake of simplicity and without prejudice to generality, the NN has a single-input, single-output structure with only one hidden layer, where the dynamic element of the network is concentrated, and a linear activation function is assigned to the output neuron. In the rest of the paper, we will refer to this neural structure since such treatment has the advantage of simplicity and matches the exigencies of the paper. As shown in Figure 1, \(\{i\}\) represents the input vector signal, \(\underline{W}=[w_{1},\ w_{2},\ \ldots,\ w_{K}]\) is the weights matrix of the links between the input neuron and the hidden layer, \(K\) is the number of the hidden neurons, \(\{\underline{s}\}=[\{s_{1}\},\ \{s_{2}\},\ \ldots,\ \{s_{K}\}]\) and \(\left\{\underline{y}\right\}=[\{y_{1}\},\ \{y_{2}\},\ \ldots,\ \{y_{K}\}]\) are, respectively, the input and the output vector succession of the hidden layer, \(\underline{V}=[v_{1},\ v_{2},\ \ldots,\ v_{K}]\) is the weights matrix of the links between the hidden layer and the output neuron, and \(\{u\}\) is the output vector signal. The dynamic part of the network is an ARMA filter, where the parameters \(a_{i}\) create the IIR part and \(b_{i}\) the FIR part. The FIR part is a feedforward structure, for which the literature provides a layout of efficient methodologies; therefore, this work will focus only on the IIR part. The output \(u(t)\) of the NN is calculated as  
\[u(t)=\sum_{k=1}^{K}v_{k}\cdot y_{k}(t) \tag{1}\]  
Let us consider the calculation of the state \(x(t)\) of the \(k\) - th hidden neuron  
\[x(t)=\underline{a}^{T}\cdot\underline{x}(t)+s_{k}(t) \tag{2}\]  
where \(\underline{a}\) is the vector of feedback gains, \(\left(\cdot\right)^{T}\) indicates the transposal operator, \(\underline{x}(t)\) is the vector state of the delay line with depth \(r\), at the time \(t=0,\ 1,\ 2,\ldots\infty\), which includes samples of the state sequence from \(x(t-1)\) to \(x(t-r)\), \(s_{k}(t)\) is the current input of the neuron. By referring to Figure 1b, the output of the \(k\) - \(th\) hidden neuron is calculated as:  
\[y_{k}(t)=f\Big{[}b_{0}\cdot s_{k}(t)+\underline{b}^{T}\cdot\underline{x}(t) \Big{]} \tag{3}\]  
Figure 1: Structure of the LRNN. In (**a**) the global structure is shown, which is identical to that of MLP. The symbols between curl brackets are time sequences, the values without brackets indicate the weights. In (**b**) the internal structure of a hidden neuron is represented. The blocks with \(z^{-1}\) constitute the delay line. The battery of \(b_{i}\) gains are the weights of the FIR filter, while the \(a_{i}\) gains are the IIR weights. The gain \(b_{0}\) represents the a-dynamic connection with the output, while \(f\) is the nonlinear activation function of the neuron. where \(\underline{b}\) is the vector of forward gains, \(b_{0}\) is the a-dynamic weight, and \(f(\cdot)\) is the activation function of the neuron. Equation (3) allows to write a dynamic loss function to be used for the training (or adapting) of the NN. Let \(\{d\}\) be the desired output sequence of the NN as an answer to the input sequence \(\{i\}\). A loss function can be defined as the mean squared error of the output with respect to the desired sequence:  
\[J=\frac{1}{2}{\sum_{t=1}^{T}}[u(t)-d(t)]^{2} \tag{4}\]  
where \(T\) is the duration of the sequence for which the NN must be trained. The simplest procedure to minimize the (4) is based on the gradient of \(J\) calculated with respect to all the parameters of the NN. Referring to Figure 1, no difficulties arise in computing the derivatives of \(J\) with respect to the global parameters \(w_{k}\), as well as the internal parameters \(b_{j}\), with \(j=0,\ldots,r\).