# Deep Learning Models Applied to Prediction of 5G Technology Adoption
## 3 Methodology
### Deep Learning Models
#### 3.1.1 Long Short-Term Memory  
Among the many challenges that recurrent neural networks (RNN) must contend with, vanishing gradients and growing gradients are two that the LSTM is able to address in an effective manner [35]. Because of the error caused by the vanishing gradient problem, RNNs are unable to be trained when there is a delay of more than 5-10 timesteps between the events that are input and the signals that indicate success. On the other hand, the LSTM is able to connect up to a thousand timesteps despite having low temporal delays. The cells that are assembled here to form these specialised units are the ones that are to blame for the constant flow of errors. Multiplicative gate units control cell entry [36]. The memory nodes of a typical LSTM network are always situated in the hidden layer of the network [37]. A set of memory cells and a pair of multiplicative gate units are the only components of a memory block that are responsible for processing input and output data. The constant error carousels in a memory cell solve the problem of vanishing gradient error by preventing the local backflow error from decreasing or increasing when no input or error signals are present. Because of the input and output gates, the constant error carousel is safeguarded against faults that originate from both inside and outside the computer. The level of activity of the constant error carousels is what determines the state of the cell. The switching states of the input gate \(y(in)\) and the output gate \(y(out)\) at discrete times such as 1, 2,... make use of the Equations (1) and (2):  
\[net_{out_{j}}(t)=\sum_{m}w_{out_{j}}y^{m}(t-1),\;y^{out_{j}}(t)=f_{out_{j}} \Big{(}net_{out_{j}}(t)\Big{)} \tag{1}\]  
\[net_{in_{j}}(t)=\sum_{m}w_{in_{j}}y^{m}(t-1),\;y^{in_{j}}(t)=f_{in_{j}}\Big{(} net_{in_{j}}(t)\Big{)} \tag{2}\]  
where \(j\) is the memory block, \(f\) is the logistic sigmoid in the range \([0,\;1]\), and \(w_{in_{m}}\) denotes the connection weight from the unit \(m\) to the unit \(l\).  
To compute the internal state of a given memory cell \(S_{c}(t)\), the squashed gate input to the state at the recent time step \(S_{c}(t-1)\) where \((t>0)\) can be added by the following Equations (3) and (4):  
\[net_{c_{j}^{c}}(t)=\sum_{m}w_{c_{j}^{c}}y^{m}(t-1) \tag{3}\]  
\[S_{c_{j}^{c}}(t)=S_{c_{j}^{c}}(t-1)+y^{in_{j}}(t)g\Big{(}net_{c_{j}^{c}}(t) \Big{)} \tag{4}\]  
where \(c_{j}^{c}\) denotes cell \(v\) of the memory block \(j\), squashing of the cell input is performing by \(g\), and \(S_{c_{j}^{c}}(0)=0\). To determine the output of a cell \(y^{c}\), the internal state \(S_{c}\) is squashed using an output squashing function \(h\) and gating it with the activation of the output gate \(y^{out}\) expressed in Equation (5):  
\[y^{c_{j}^{c}}(t)=y^{out_{j}}(t)h\Big{(}S_{c_{j}^{c}}(t)\Big{)} \tag{5}\]  
where \(h\) denotes a centered sigmoid in the range \([-1,1]\).  
The output units \(K\) of a network with layered topology consisting of hidden layer with memory blocks, standard input, and output layer can be defined by the Equations (6) and (7):  
\[net_{k}(t)=\sum_{m}w_{kny}y^{m}(t-1) \tag{6}\]\[y^{k}(t)=f_{k}(net_{k}(t)) \tag{7}\]  
where \(f_{k}\) denotes the squashing function with logistic sigmoid in the range \([0,\ 1]\) and \(m\) ranges over all input units and the cells in the hidden layer. The LSTM is capable of solving tasks with complex long time-lags that were never solved by RNN.