## 3 LSTM Attention Decoder
Thereafter, \(C_{t}\) is scaled by \(\tanh\) and multiplied by \(o_{t}\) to obtain the final output \(h_{t}\). LSTM saves the learned features as memory through the above operations and retains or forgets the saved memories according to the training process selectively. After several iterations, the important feature information is retained, which gives the network better performance in processing tasks with a long-time dependence. Figure 3: Diagram of the LSTM network. Both RNN and LSTM networks are designed to handle the problem of long time series. However, the network will forget the previous useful information because of the existence of a forget gate when LSTM dealt with the gradient explosion problem. This deteriorates the effect of long sequence training and the system performance. Fortunately, the attention mechanism is a great solution to this problem [26]. Variants of attention mechanisms include multi-head attention, hard attention, structured attention, and key-value pair attention [27]. Multi-head attention utilizes multiple queries to calculate in parallel to select multiple pieces of information from the input. Each attention focuses on a different part of the input. Hard attention can be implemented in two ways. One is to select the input information with the highest probability. Another is to randomly sample the distribution of attention. Structured attention involves picking out task-relevant information from input. Key-value pair attention employs a key-value pair format to represent input information, where \"key\" is utilized to calculate the attention distribution and \"value\" is utilized to generate the selected information. Considering its excellent performance, the key-value pair attention mechanism is employed in our proposal to perform LSTM attention decoder, and it can be utilized to process the received FTN hybrid signals. The diagram of the LSTM attention decoder is shown in Figure 4.  
As shown in Figure 5, the whole calculation process of attention can be summarized in three steps [28; 29; 30]. Firstly, \(\overline{S_{PD}}\) is sent to the LSTM network, the outputs of hidden layer are transmitted to the key module, and the time series signal of \(\overline{S_{PD}}\) are sent to the query module. Thereafter, the key (K) and query (Q) module perform a similarity calculation to obtain the weight of the attention module. The equation of similarity calculation can be written as  
Figure 3.0  
where \(s\) denotes the similarity calculation, \(T\) denotes the transpose operation, \(Q\) denotes the target matrix to be obtained, and \(K\) denotes the actual detected matrix. It should be noted that the goal of using a neural network for training and testing is to determine the mapping relationship corresponding to the original signal. However, when \(\overline{S_{PD}}\) is sent to the network, the output is not unique. It is difficult to determine the mapping relationship and not conducive to the back propagation of the network. Therefore, the Softmax function is employed to normalize the output of \(s(Q,K)\) (S) to (0,1). The probability formula of the Softmax function can be written as  
Figure 3.1  
Figure 3.2 Diagram of the LSTM attention decoder. where \(f\) denotes the output states of the network. As mentioned above, the value of the output state is 4. Therefore, each output signal can be denoted by a \(1\times 4\) matrix. \(\overline{S^{\prime}_{y}}\) denotes the y-th output of \(\overline{S_{PD}}\) and \(\overline{S^{\prime}_{q}}\) denotes the current value to be calculated in \(\overline{S^{\prime}_{y}}\). Finally, the output of the Softmax function (a) and the actual output \"value\" of the LSTM network are summed to obtain the attention value. \(\varepsilon\) denotes the weighted sum. This is the final result of the proposed LSTM attention decoder.