# RNN-Based Sequence to Sequence Decoder for Run-Length Limited Codes in Visible Light Communication
## 3 Learning to RLL Decode
### Seq2 seq Decoder  
Analysis illustrates that the RLL decoding problem is the mapping of a received signal sequence to another bit sequence. It is available to be converted into a sequence-to-sequence problem with a many-input many-output structure. RNN has memories to process a sequence of inputs, and hence has strong capabilities for the time series tasks. A typical RNN-based model, namely, the seq2 seq model, is suited to dealing with the sequence-based \"\(N\) inputs and \(K\) outputs\" tasks (\(N\
eq K\)) and learn mappings between unequal sequences due to its specific architecture [18; 19]. Thus, we adopted the seq2 seq model to perform our RLL decoding task. Figure 2 shows the model, which includes two RNN structures, i.e., RNN-encoder and RNN-decoder, respectively. (1) _Training Phase:_ In this phase, by training the adopted seq2 seq decoder, we aim to make the output of the decoder close to the real decoded sequence. In the following, we introduce the input and output of the decoder, the loss function, and the training procedure. * **Encoder-Input:** Denote \((\boldsymbol{y},\boldsymbol{x})\) as a training sample of dataset \((\boldsymbol{y},\boldsymbol{x})\), with \(\boldsymbol{y}\) being the the received signal and \(\boldsymbol{x}\) being the corresponding label. Thus, the \(\boldsymbol{y}=\{y_{1},y_{2},\ldots,y_{N}\}\) are fed into RNN-encoder as the inputs. The labels of the inputs \(\boldsymbol{y}\) can be given by \(\boldsymbol{x}=\{x_{1},x_{2},\ldots,x_{K}\}\), where \(x_{k}\in\{0,1\}\). Here, the labels \(\boldsymbol{x}\) are the encoded sequences corresponding to the inputs \(\boldsymbol{y}\). * **Encoder-Hidden States:** The encoder has two hidden layers of size \(5N\). In the hidden layers, the hidden state is computed as \(\mathbf{h}_{j}^{(1)}=g(\mathbf{h}_{j-1}^{(1)},y_{j})\), where \(g(\cdot)\) is a Relu activation function and \(\mathbf{h}_{j}^{(1)}\) is the \(j\)-th hidden unit of the first hidden layer. Then, \(\mathbf{h}_{j}^{(1)}\) is used to compute the second hidden layer, given by \(\mathbf{h}_{j}^{(2)}=g(\mathbf{h}_{j}^{(1)},\mathbf{h}_{j-1}^{(2)})\). For the RNN cell, long short-term memory (LSTM) cell and gated recurrent unit (GRU) cell are widely used to avoid gradient vanishing as the network deepens. Moreover, GRU has less training parameters than LSTM [20]. Thus, we employ GRU cell in our work. * **Encoder-Output:** The encoder has an output layer of size 1, which only outputs the result \(c\) of the final hidden state in the second hidden layer. To realize the single output of RNN-encoder, we set \(\mathbf{return\_sequences}=\mathbf{False}\) in Keras. Note that the final output \(c\) contains all information of the received signal \(\mathbf{y}\) so that the RNN-decoder can catch the information for further training. * **Decoder-Input:**\(K\) repeated \(c\) are fed into RNN-decoder as its inputs, which are given by \(c=\{c_{1},c_{2},\ldots,c_{K}\}\). Here, the \(c_{i}(i=1,2,\ldots,K)\) are the same. * **Decoder-Hidden States:** The decoder has two hidden layers of size \(5K\). The working steps and its configure are the same as encoder-hidden states. * **Decoder-Output:** The output layer is a fully-connected layer with the sigmoid activation function, given by \(\sigma_{\text{sigmoid}}\left(z\right)=1/(1+e^{-z})\), with \(z\) being the input of the sigmoid function and \(\sigma_{\text{sigmoid}}\left(z\right)\in[0,1]\). Thus, the decoder outputs the final \(K\) soft estimates \(\tilde{\mathbf{x}}=\{\tilde{x}_{1},\tilde{x}_{2},\ldots,\tilde{x}_{K}\}\) of transmitted data \(\mathbf{x}\). Here, \(\tilde{x}_{k}\in(0,1)\), which indicates the probabilities of the RLL-decoded bit \(\tilde{x}_{i}\) being a \"0\" or a \"1\". Therefore, we can output the final decoded sequence based on the rule (2).