## 2 Conceptual Model and Problem Procedure
### Enhanced LSTM Neural Network  
To implement a meaningful mapping between a set of observations and a response variable, artificial neural networks (ANNs) employ a combination of linear and nonlinear operations. A network's depth is the number of successive transformation layers that it uses. Using ANNs with multiple layers to implicitly extract underlying patterns from data at progressively higher abstraction levels is the aim of deep learning [37]. Because of this, deep neural networks (DNNs) typically do not need an additional feature extraction step, such as the one used in [38], to estimate the parameters of the sinusoidal model. It is necessary to estimate the weights of the linear operations used in DNN layers during the training phase based on observations, as they are currently unknown. A detailed discussion of the algorithms used to train DNNs can be found elsewhere [39; 40]. A schematic representation of this training procedure is shown in Figure 4.  
Recurrent neural networks (RNNs) function based on a set of internal states in addition to input observations, which sets them apart from feed-forward networks in a fundamental way [41; 42]. The internal states record historical data in sequences that the network has already processed. In other words, the network has a cycle that enables the retention of historical data for a period of time that is dependent on the observations and weights it receives as input rather than being fixed a priori. Therefore, an RNN is a dynamic system capable of learning sequential dependencies extended over time, unlike other common DNN architectures that implement a static input-output mapping. It has therefore been widely utilized for time-series analysis [43]. In this instance, we codify the RNN dynamics. The \(k\times 1\), \(m\times 1\), and \(L\times 1\) vectors of input observations, hidden states, and output vectors, respectively, can be represented by the symbols \(x_{t}\), \(h_{t}\), and \(y_{t}\) and \((h_{1},h_{2},\ldots,h_{T})\) and \((y_{1},y_{2},\ldots,y_{T})\) are computed given an input sequence \((x_{1},x_{2},\ldots,x_{T})\) by iterating through the following recursive equations for \(t=1,\ldots,T\)[44]:  
\[h_{t}=f_{h}(W_{xh}x_{t}+W_{hh}h_{t-1}+b_{h}) \tag{13}\]  
\[y_{t}=W_{hy}h_{t}+b_{y} \tag{14}\]  
where \(M\times K\), \(M\times M\), and \(L\times M\) weight matrices are represented by \(W_{xh}\), \(W_{hh}\), and \(W_{hy}\), \(M\times 1\) and \(L\times 1\) bias terms are represented by \(b_{h}\) and \(b_{y}\); and an element-wise nonlinear hidden layer function (such as the logistic sigmoid function) is represented by \(f_{h}\). An illustration of a fundamental RNN unit described by Equations (13) and (14) is shown in Figure 5.  
Figure 4: General Structure for Training a Deep Neural Network (RNN). Since a basic RNN unit unfolds over time to become a composite of several nonlinear computational layers, it is already temporally deep [39]. In addition to the temporal depth of an RNN, [PERSON] et al. investigated several alternative definitions of depth in RNNs, which allowed for the construction of deep RNNs [40]. Nonetheless, piling several recurrent hidden layers on top of one another is a standard method of defining deep RNNs [40]. In this instance, the output of the layer before it forms the input of that layer. With N hidden layers assumed, the hidden states for layer \(j\) are iteratively calculated for \(j=1,\ldots,N\) and \(t=1,\ldots,T\) using the following formula:  
\[h_{t}^{j}=f_{h}(W_{h^{j-1}h^{j}})h_{t}^{j-1}+W_{h^{j}h^{j}}h_{t-1}^{j}+b_{h}^{j}) \tag{15}\]  
\[y_{t}=W_{h^{N}y}h_{t}^{N}+b_{y} \tag{16}\]  
where the same hidden layer operates. It is assumed that \(f_{h}\) for every layer exists, where \(h_{t}^{0}=x_{t}\) and \(h_{t}^{j}\) represents the state vector of the \(j^{th}\) hidden layer. RNNs have a strong and intuitive structure, but when trained in real-world scenarios, they are unable to capture long-term dependencies.