# 1-D Convolutional Neural Network-Based Models for Cooperative Spectrum Sensing
## 3 Overview of Spectrum Sensing and Adopted Deep Learning Algorithms
### Bidirectional Long Short-Term Memory  
LSTM belongs to a category of ANNs known as recurrent neural networks (RNNs), and it was developed to tackle the problem of vanishing gradients that often arises in conventional RNNs. This problem occurs when the gradients used to modify the network's parameters become excessively small during the backpropagation process, causing difficulties in learning long-term dependencies. To overcome this issue, LSTM incorporates specialized memory cells that can selectively remember or forget information over time by including gates in the cell. An LSTM hidden layer consists of a set of recurrently connected  
Figure 1: The 1 DCNN architecture.  
blocks in a chain structure, known as memory blocks. Each of them contains one or more recurrently connected memory cells and three multiplicative units, called gates, that provide continuous analogues of write, read and reset operations for the cells. The architecture of the LSTM cell is shown in Figure 2.  
On the basis of the connections described in this figure, we can express the LSTM cell mathematically as follows [56]:  
\[\begin{split} f_{t}&=\sigma(W_{fh}h_{t-1}+W_{fx}x_{t }+b_{f})\\ i_{t}&=\sigma(W_{ih}h_{t-1}+W_{ix}x_{t}+b_{i})\\ \tilde{c}_{t}&=tanh(W_{dh}h_{t-1}+W_{cx}x_{t}+b_{c}) \\ c_{t}&=f_{t}*c_{t-1}+i_{t}*\tilde{c}_{t}\\ o_{t}&=\sigma(W_{oh}h_{t-1}+W_{ox}x_{t}+b_{o})\\ h_{t}&=o_{t}*tanh(c_{t})\end{split} \tag{4}\]  
At time step \(t\), we use the activation vectors \(i_{t}\), \(f_{t}\), \(o_{t}\) and \(c_{t}\) to represent the input, forget, output and cell state gates, respectively. We denote the weight matrix by \(W\), and the bias matrix by \(b\). The Sigmoid function is represented by the symbol \(\sigma\), and the element-wise multiplication is denoted by \"\(*\)\". The flow of information into and out of the cell is controlled by these gates, and the cell remembers values over arbitrary time intervals. More precisely, the input gate determines which values should be updated in the memory cell, the forget gate determines which values should be discarded from the memory cell, and the output gate determines which values should be used to produce the output.  
Bidirectional LSTM is a type of RNN that differs from the standard LSTM by its ability to handle input sequences in both the forward and backward directions. This unique characteristic enables the network to effectively capture and utilize a broader range of input information.  
It achieves this by adding a second set of hidden states that process the input sequence in reverse order, where one layer processes the input sequence in the forward direction, and the other processes it in the backward direction. The input sequence flows in both states, and the resulting outputs from each direction are merged to generate the final output. This means that the two states' outputs are not connected to the inputs of the opposite direction's states. This structure can be illustrated as a diagram unfolded over multiple time steps, such as the one shown in Figure 3. BiLSTM is primarily trainable using the same algorithms as a standard unidirectional LSTM. This is because the two types of state neurons do not interact and can be transformed into a general feedforward network. However, during the backpropagation through time (BPTT) process, additional steps are necessary since  
Figure 2: LSTM cell architecture.  
updating the input and output layers cannot be completed simultaneously. The training process for the unfolded bidirectional network is as follows: during the forward pass, the forward states and backward states are passed first, and then the output neurons are passed. During the backward pass, the output neurons are passed first, followed by the forward and backward states. Finally, after the forward and backward passes are complete, the weights are updated.