# A Deep Learning Approach for MIMO-NOMA Downlink Signal Detection
## 3 Deep-Learning Scheme Based on the MIMO-NOMA System
### DNN Design  
As mentioned above, a DNN is a deep version of a neural network. The adjustable hyperparameters include the weights, bias, regularization parameter, learning rate, and drop-out. The DNN model we designed for MIMO-NOMA detection comprises seven layers: one input layer, one output layer, and five  
Figure 7: Construction of the MIMO-NOMA-DL model. DL: deep learning.  
hidden layers. The input layer and hidden layer are fully connected, whereas the output layers are divided into groups to decode the signals of multiple antennas in a slot.  
The input layer is where the MIMO-NOMA signal is received. Suppose that the numbers of transmit and receive antennas of the BS and UE are \(N_{t}\) and \(N_{r}\), respectively. The complex receive signal is decomposed into the real part and the imaginary part, so the number of input layer cells is \(2N_{r}\). The input signal is a two-dimensional vector of a slot and multiple antennas. That is, the \(2N_{r}\) data are sent to the network in one slot as a column vector.  
The hidden layers are composed of five fully connected layers. To avoid the vanishing gradient problem of the sigmoid function, the ReLU function (9), an effective nonlinear function, is used to activate the neurons after the linear operation.  
The output layer is used to report the final detection results. The normal DNN output layer is usually fully connected, and has one-hot encoding with the softmax function. In MIMO-NOMA signal detection, however, signals from multiple antennas should be decoded in a single slot. So, the proposed output layer was designed to form groups. The number of groups is equal to the number of transmitting antennas \(N_{t}\), and the number of neurons in each group is equal to the number of one-hot encodings. For example, the output layer structure of a \(4\times 4\) MIMO-NOMA system is shown in Figure 8. Because the label is not in the traditional form, but the group one-hot encoding, the output data adopt the soft-decision form with the sigmoid function (7).  
Additionally, the choice of loss function and optimization algorithm is another key point for the MIMO-NOMA-DL network. The loss function measures the distance between the predictions and labels. The classic loss function is the mean square error (MSE) function. In logistic regression, MSE performs in terms of accuracy. However, in multiclass classification problems, MSE has a slow convergence speed. Here, we consider the cross-entropy function. In [PERSON]'s information theory, the Kullback-Leibler divergence (KLD) can be used to represent the difference between two probability distributions, and the expression is written as:  
\[D_{KL}\left(P||Q\right)=\sum_{i}P(i)\log\frac{P(i)}{Q(i)}(x). \tag{31}\]  
The process of minimizing the KLD is equivalent to minimizing the cross-entropy \(H(P,Q)\), which is defined as:  
Figure 8: Design of the output layer.  
\[H(P,Q)=D_{\text{KL}}\left(P||Q\right)+H(P). \tag{32}\]  
The cross-entropy function has fast convergence and low complexity in the iterative optimization process.  
Moreover, considering the self-adaptation of the learning rate and the robustness, the Adam [20] method is used as the optimization algorithm. The Adam method improves upon the Momentum and RMSPro algorithms, and is more robust in terms of the hyperparameters. Details of the optimization algorithms and their comparison can be found in [21].  
To avoid overfitting, we add the L2 regularization term \(\lambda\cdot\sum_{i}\|\omega_{i}\|_{2}\) to the loss function to effectively decrease the sensitivity of the parameters to parametric variation so that the performance of the test result is close to the training result.  
The proposed DL algorithm based on the above DNN architecture design is summarized in Algorithm 1.