# Multi-User Joint Detection Using Bi-Directional Deep Neural Network Framework in NOMA-OFDM System
## 4 Proposed Deep-Learning Model
### Model Architecture
#### 4.2.2 Internal Structure of LSTM  
The LSTM network can learn information between time steps of sequence data and preserve relevant information. The time steps are treated equivalently as subcarriers in the OFDM system. By focusing on one time-step module in the LSTM layer, the DNN can be trained to realize multiuser detection for an arbitrary subcarrier. The internal cell structure and operation of LSTM network are shown in Figure 3b [34]. The output of the LSTM cell is generated according to the current input and the preceding cell state. The LSTM cell consists of three gates, such as the forget gate, the input gate, and the output gate. In addition, the LSTM is comprised of two states, namely cell state \(C_{t-1}\) and hidden state \(m_{t-1}\). The cell state works as a memory to cumulate information that is extracted from past inputs. On the other hand, to compute the output, the hidden state is utilized. From the Figure 3b, \(t\) is the time instant, the current input is \(x_{t}\), and finally, the current output channel coefficient of multiuser at time \(t\) is denoted by \(m_{t}\). The LSTM cell can add and remove information from the cell state at each time step, which is updated through operation of the gates. The operations of each gate can be summarized as follows:  
The control for the level of cell state that needs to be reset is performed by the forget gate. The forget gate \(fr_{t}\) can be expressed as follows [35]:  
\[fr_{t}=f_{\sigma_{c}}(w_{fr}x_{t}+T_{fr}m_{t-1}+b_{fr}), \tag{15}\]  
Figure 3: (**a**) The architecture of the Bi-LSTM model system with its different layers. (**b**) The internal cell structure of the LSTM model.  
where \(w_{fr}\) is the weight related to \(x_{t}\) and \(T_{fr}\) is the weight related to \(m_{t-1}\). The bias of the forget gate is \(b_{fr}\). The control for the level of cell state that needs to be updated is performed by the input gate. The input gate \(in_{t}\) can be expressed as follows [35]:  
\[in_{t}=f_{\sigma_{c}}(w_{in}x_{t}+T_{in}m_{t-1}+b_{in}), \tag{16}\]  
where \(w_{in}\) is the weight related to \(x_{t}\) and \(T_{in}\) is the weight related to \(m_{t-1}\). The bias of the input gate is \(b_{in}\). The addition of information to the cell state is managed by the candidate gate. The candidate gate \(cat_{t}\) can be expressed as follows [35]:  
\[ca_{t}=f_{\tanh}(w_{ca}x_{t}+T_{ca}m_{t-1}+b_{ca}), \tag{17}\]  
where \(w_{ca}\) is the weight associated with \(x_{t}\) and \(T_{ca}\) is the weight related to \(m_{t-1}\). The bias of the candidate gate is \(b_{ca}\). The updated cell state can be expressed as follows:  
\[up_{t}=(C_{t-1}\odot ft_{t})+(in_{t}\odot ca_{t}), \tag{18}\]  
where the element-wise multiplication is represented by \(\odot\). The control for the level of cell state to be updated is performed by the output gate. The output gate \(out_{t}\) can be expressed as follows:  
\[ou_{t}=f_{\sigma_{c}}(w_{ou}x_{t}+T_{ou}m_{t-1}+b_{ou}), \tag{19}\]  
where \(w_{ou}\) is the weight related to \(x_{t}\) and \(T_{ou}\) is the weight associated with \(m_{t-1}\). The bias of the output gate is \(b_{ou}\). \(f_{\sigma_{c}}(z)=[1/(1+e^{z})]\) is the sigmoid function, which is responsible for computing the gate activation function. The estimated output coefficient of the hidden state at time step \(t\) can be expressed as follows:  
\[m_{t}=ou_{t}\odot f_{\tanh}(up_{t}), \tag{20}\]  
where \(f_{\tanh}(z)=[(e^{2z}-1)/e^{2z}+1)]\) is the hyperbolic tangent function, which is responsible for computing the state activation function.  
Figure 3a shows the Bi-LSTM model, which is constructed with two LSTM layers in opposite directions. The output \(V_{t}\) of the two hidden Bi-LSTM state layers can be calculated as follows [36]:  
\[V_{t}=f(w_{V\overrightarrow{T}}\overrightarrow{T}_{t}+w_{V \overleftarrow{T}}\overleftarrow{T}_{t}+b_{Z}), \tag{21}\]  
where \(\overrightarrow{T}\) and \(\overleftarrow{T}\) are the forward and backward sequences.