# A Deep Learning Approach for MIMO-NOMA Downlink Signal Detection
## 2 Fundamentals of MIMO-NOMA and the Deep Learning System
### Deep Learning Basics  
The main techniques of deep learning include DNN, CNN, and RNN. In this section, the basics of these DL-based approaches are briefly introduced. The DNN is a deeper version of the neural network that generally consists of three types of layers: input, hidden, and output. The input layer and output layer are single layers, whereas the hidden layers can be extended to multiple layers depending on the complexity of the signal-processing algorithm. Each layer contains multiple nodes, and the effects are exerted only on adjacent layers. The construction of DNN model can be seen in Figure 4.  
There are two components of the relation between adjacent layers: linear and nonlinear. The linear component is responsible for the linear relation between the input and output for each layer. It includes two types of operation: multiplication, represented by the weight \(w\), and addition, represented by the bias \(b\). In most practical scenarios, however, we face nonlinear problems that cannot be solved by the linear method. Therefore, the nonlinear component is addressed via the activation function \(f(\cdot)\). Suppose that the output of the \((n-1)\)th layer is \(\mathbf{y_{n-1}}\), the weight matrix of the \(n\)th layer is \(\mathbf{w_{n}}\), the bias vector is \(\mathbf{b_{n}}\), and the output of the \(n\)-th layer \(\mathbf{y_{n}}\) can be denoted as:  
\[\mathbf{y_{n}}=f\left(\mathbf{w_{n}}\cdot\mathbf{y_{n-1}}+\mathbf{b_{n}}\right). \tag{6}\]  
A classic DNN activation function is the sigmoid function (7). The range of the function is limited to \([0,1]\) and can approximately represent the probability. The \(\tanh\) function (8) is also a classic activation function. The range of the \(\tanh\) function is extended to \([-1,1]\), and the center of each layer's output is 0, which results in faster convergence via stochastic gradient descent (SGD). Another powerful activation function is the rectified linear unit (ReLU) function (9). Instead of restricting the value to \([0,1]\) or \([-1,1]\), the ReLU function increases linearly when \(x\geq 0\) and is zero when \(x<0\). The gradient does not disappear after repeated nonlinear operations. Figure 4: Construction of the deep neural network (DNN) model. \[f_{sig}\left(x\right)=\frac{1}{1+\exp^{-x}} \tag{7}\]  
\[tanh(x)=\frac{\exp x-\exp-x}{\exp x+\exp-x} \tag{8}\]  
\[ReLU(x)=\left\{ \begin{array}{cc}x&,x\geq 0\\ 0&,x<0\end{array} \right. \tag{9}\]  
For the multiple hidden layers, assuming that the bias is 0 for simplicity, the transmission expression can be defined as:  
\[y_{n}=f(w_{n}f(w_{n-1}f\cdots w_{2}f\left(w_{1}y_{0}\right)) \tag{10}\]  
For the output layer, the most popular choices are the sigmoid function expressed in (7) and the softmax function. The softmax function is used mainly for multiclass classification and can be defined as:  
\[f_{soft}\left(\textbf{x}\right)_{i}=\frac{\exp\left(x_{i}\right)}{\sum_{j}\exp \left(x_{j}\right)}. \tag{11}\]  
In deep learning algorithms, we often need to feed the system considerable data, called the training set, so that the system can adjust itself adaptively to the optimal status offline. During the training process, correct data should be used to rectify the output. Then, a connection between the input and the output can be established in a supervised manner. Afterwards, the trained system can be applied to the test set to assess the performance of the DNN. In addition, CNNs also play an important role in DL, especially in the realm of computer vision. In contrast to the traditional NNs, CNNs have a special structure. A classic CNN is LeNet-5, which can be seen in Figure 5. Layer C1 is a convolution layer, S2 is a pooling layer, C3 is another convolution layer, and next is a pooling layer again in S4. C5 can be identified as a dense layer together with the F6 layer. Obviously, before the fully connected layers, the input data are processed by multiple convolution layers and pooling layers.