# DIR-Net: Deep Residual Polar Decoding Network Based on Information Refinement
## 2 Background
### Neural Network Decoder  
Unlike the traditional decoding schemes which decode the codewords with analytical or iterative methods, deep learning decoding methods implement decoding tasks by deep neural networks. The decoding network is trained offline with huge amounts of data and used online in the real world. To obtain a high-quality decoder, large amounts of training data need to be collected first. The training data consist of noisy polar codes and the original decoded signal. The former is the input of the decoding network, and the latter is the corresponding label. The data can be collected in the communication system with polar codes or by computer simulation. The latter is the method most used in the literature. In general, training data are specified with a code length \(k\), and there are \(n\) code words \(\mathbf{X}=\{\mathbf{x}_{1},\mathbf{x}_{2},\ldots,\mathbf{x}_{n}\}\) under the code length requirements, where each code word is \(\mathbf{x}_{n}=\{x_{n}^{1},x_{n}^{2},\ldots,x_{n}^{N}\}\). Subsequently, the data are encoded using the polar encoding algorithm. To make the decoding algorithm resistant to noise interference in the channel, the encoded signal is usually processed with noise. The whole process is expressed as  
\[\mathbf{z}_{n}=\left(\mathbf{x}_{n}\right)^{*}+\mathbf{\mu}_{n}, \tag{3}\]  
where \(\mathbf{z}_{n}\) is the received codeword, \(\mathbf{\mu}_{n}\) is noise, and \((\cdot)^{*}\) is the codeword after encoding. The training data pair \((\mathbf{z}_{n},\mathbf{x}_{n})\) is obtained through this process, where \(\mathbf{z}_{n}\in\mathbf{Z}\) is the input of the network, and \(\mathbf{x}_{n}\in\mathbf{X}\) is the training label.  
The structure of the neural network decoder (NND) can be expressed as \(\{\mathbf{z}_{n},S_{1},S_{2},\ldots,S_{L},\hat{\mathbf{x}}_{n}\}\), where \(L\) is the number of network layers, \(S_{l}\) refers to the \(l-\)th layer and \(1\leq l\leq L\), and \(\hat{\mathbf{x}}_{n}\) denotes the output. For each layer, there is an associated \(S_{l-1}\times S_{l}\) weight matrix \(\mathbf{w}^{(l)}\) and a \(1\times S_{l}\) bias vector \(\mathbf{b}^{(l)}\)[32]. The output vector of layer \(l\) is denoted as  
\[\mathbf{z}_{n}^{(l)}=f_{l}\left(\mathbf{z}_{n}^{(l-1)}\right)=\rho\left(\mathbf{z}_{n}^{(l -1)}*\mathbf{w}^{(l)}+\mathbf{b}^{(l)}\right), \tag{4}\]  
where \(\rho(*)\) is the activation function [32]. When \(1\leq l\leq L-1\), it can be expressed as  
\[\rho(m)=\max(0,m). \tag{5}\]  
When \(l=L\), it can be expressed as  
\[\rho(m)=\frac{1}{1+e^{-m}}. \tag{6}\]  
It is the sigmoid function, which restricts the values of the output layer to the range (0, 1). The NND can be denoted as  
\[\mathbf{p}_{n}=f(\mathbf{z}_{n})=f_{L}^{\rho}(f_{L-1}^{\rho}\left(\ldots(f_{1}^{\rho} (\mathbf{z}_{n})\right))), \tag{7}\]where \(\mathbf{p_{n}}\) refers to the vector of probabilities. \(f_{l}^{p}\) refers to the activated output of the \(l-\mathrm{th}\) layer. Binarize \(\mathbf{p_{n}}\) to obtain \(\hat{\mathbf{x}}_{n}\).  
Next, the parameters are updated utilizing backpropagation algorithm [33]. In NND, the commonly used loss function is MSE or cross-entropy (CE) [32] as  
\[\begin{split} Loss_{MSE}=\frac{1}{K\times Z_{num}}\sum\limits_{i= 0}^{K-1}\sum\limits_{j=0}^{Z_{num}-1}\left(x_{j}^{i}-p_{j}^{i}\right)^{2},\\ Loss_{CE}=-\frac{1}{K\times Z_{num}}\sum\limits_{i=0}^{K-1}\sum \limits_{j=0}^{Z_{num}-1}\left[\left(1-x_{j}^{i}\right)\log\left(1-p_{j}^{i} \right)+x_{j}^{i}\log\left(p_{j}^{i}\right)\right],\end{split} \tag{8}\]  
where \(Z_{num}\) and \(K\) are the mini-batch size and information bits of each codeword, respectively. \(x_{j}^{i}\in x_{n}\) and \(p_{j}^{i}\in p_{n}\) represents the bit value and output probability of the \(i-\mathrm{th}\) bit in the \(j-\mathrm{th}\) codewords.