Venue: ICLR 2018 conference

Reviews:

final_score: rating: 9: Top 15% of accepted papers, strong accept
review_content: This paper shows how RNNs can be used to decode convolutional error correcting codes. While previous recent work has shown neural decoders for block codes results had limited success and for small block lengths. 
This paper shows that RNNs are very suitable for convolutional codes and achieves state of the art performance for the first time. 
The second contribution is on adaptivity outside the AWGN noise model. The authors show that their decoder performs well for different noise statistics outside what it was trained on. This is very interesting and encouraging. It was not very clear to me if the baseline decoders (Turbo/BCJR) are fairly compared here since better decoders may be used for the different statistics, or some adaptivity could be used in standard decoders in various natural ways. 

The last part goes further in designing new error correcting schemes using RNN encoders and decoders for noisy feedback communication. 
For this case capacity is known to be impossible to improve, but the bit error error can be improved for finite lenghts. 
It seems quite remarkable that they beat Schalkwijk and Kailath and shows great promise for other communication problems.

The paper is very well written with good historical context and great empirical results. I think it opens a new area for information theory and communications with new tools. 

My only concern is that perhaps the neural decoders can be attacked with adversarial noise (which would not be possible for good-old Viterbi ). It would be interesting to discuss this briefly. 
A second (related) concern is the lack of theoretical understanding of these new decoders. It would be nice if we could prove something about them, but of course this will probably be challenging. 


review_title: RNNs decode convolutional codes very well

final_score: rating: 2: Strong rejection
review_content: In this paper the authors propose to use RNNs and LSTMs for channel coding. But I have the impression the authors completely miss the state of the art in channel coding and the results are completely useless for any current communication system. I believe that machine learning, in general, and deep learning, in particular, might be of useful for physical layer communications. I just do not see why it would be useful for channel coding over the AWGN channel. Let me explain.

If the decoder knows that the encoder is using a convolutional code, why does it need to learn the decoder instead of using the Viterbi or BCJR algorithms that are known to be optimal for sequences and symbols, respectively. I cannot imagine an scenario in which the decoder does not know the convolutional code that it is being used and the encoder sends 120,000 bits of training sequence (useless bits from information standpoint) for the decoder to learn it. More important question, do the authors envision that this learning is done every time there is a new connection or it is learnt once and for all. If it is learnt every time that would be ideal if we were discovering new channel codes everyday, clearly not the case. If we learnt it one and for all and then we incorporated in the standard that would only make sense if the GRU structure was computationally better than the BCJR or Viterbi. I would be surprise if it is. If instead of using 2 or 3 memories, we used 6-8 does 120,000 bits be good enough or we need to exponentially increase the training sequence? So the first result in the paper shows that a tailored structure for convolutional encoding can learn to decode it. Basically, the authors are solving a problem that does not need solving. 

For the Turbocodes the same principle as before applies. In this case the comments of the authors really show that they do not know anything about coding. In Page 6, we can read: “Unlike the convolutional codes, the state of the art (message-passing) decoders for turbo codes are not the corresponding MAP decoders, so there is no contradiction in that our neural decoder would beat the message-passing ones”. This is so true, so I expected the DNN structure to be significantly better than turbodecoding. But actually, they do not. These results are in Figure 15 page 6 and the solution for the turbo decoders and the DNN architecture are equivalent. I am sure that the differences in the plots can be explained by the variability in the received sequence and not because the DNN is superior to the turbodecoder. Also in this case the training sequence is measured in the megabits for extremely simple components. If the convolutional encoders were larger 6-8 bits, we would be talking about significantly longer training sequences and more complicated NNs.

In the third set the NNs seems to be superior to the standard methods when burst-y noise is used, but the authors seems to indicate that that NN is trained with more information about these bursts that the other methods do not have. My impression is that the authors would be better of focusing on this example and explain it in a way that it is reproducible. This experiment is clearly not well explained and it is hard to know if there is any merit for the proposed NN structure. 

Finally, the last result would be the more interesting one, because it would show that we can learn a better channel coding and decoding mechanism that the ones humans have been able to come up with. In this sense, if NNs can solve this problem that would be impressive and would turn around how channel coding is done nowadays. If this result were good enough, the authors should only focus in it and forget about the other 3 cases. The issue with this result is that it actually does not make sense. The main problem with the procedure is that the feedback proposal is unrealistic, this is easy to see in Figure 16 in which the neural encoder is proposed. It basically assumes that the received real-valued y_k can be sent (almost) noiselessly to the encoder with minimal delay and almost instantaneously. So the encoder knows the received error and is able to cancel it out. Even if this procedure could be implemented, which it cannot be. The code only uses 50 bits and it needed 10^7 iterations (500Mbs) to converge. The authors do not show how far they are from the Shannon limit, but I can imagine that with 50 bit code, it should be pretty far.  

We know that with long enough LDPC codes we can (almost) reach the Shannon limit, so new structure are not needed. If we are focusing on shorter codes (e.g. latency?) then it will be good to understand why do we need to learn the channel codes. A comparison to the state of the art would be needed. Because clearly the used codes are not close to state of the art. For me the authors either do not know about coding or are assuming that we do not, which explains part of the tone of this review. 

review_title: Deep learning for channel coding 

final_score: rating: 6: Marginally above acceptance threshold
review_content: Error-correcting codes constitute a well-researched area of study within communication engineering. In communication, messages that are to be transmitted are encoded into binary vector called codewords that contained some redundancy. The codewords are then transmitted over a channel that has some random noise. At the receiving end the noisy codewords are then decoded to recover the messages. Many well known families of codes exist, notably convolutional codes and Turbo codes, two code families that are central to this paper, that achieve the near optimal possible performance with efficient algorithms. For Turbo and convolutional codes the efficient MAP decodings are known as Viterbi decoder and the BCJR decoder. For drawing baselines, it is assumed that the random noise in channel is additive Gaussian (AWGN).

This paper makes two contributions. First, recurrent neural networks (RNN) are proposed to replace the Viterbi and BCJR algorithms for decoding of convolutional and Turbo decoders. These decoders are robust to changes in noise model and blocklength - and shows near optimal performance.

It is unclear to me what is the advantage of using RNNs instead of Viterbi or BCJR, both of which are optimal, iterative and runs in linear time. Moreover the authors point out that RNNs are shown to emulate BCJR and Viterbi decodings in prior works - in light of that, why their good performance surprising?

The second contribution of the paper constitutes the design and decoding of codes based on RNNs for a Gaussian channel with noisy feedback. For this channel the optimal codes are unknown. The authors propose an architecture to design codes for this channel. This is a nice step. However, in the performance plot (figure 8), the RNN based code-decoder does not seem to be outperforming the existing codes except for two points. For both in high and low SNR the performance is suboptimal to  Turbo codes and a code by Schalkwijk & Kailath. The section is also super-concise to follow. I think it was necessary to introduce an LSTM encoder - it was hard to understand the overall encoder. This is an issue with the paper - the authors previously mentioned (8,16) polar code without mentioning what the numbers mean. 

However, I overall liked the idea of using neural nets to design codes for some non-standard channels. While at the decoding end it does not bring in anything new (modern coding theory already relies on iterative decoders, that are super fast), at the designing-end the Gaussian feedback channel part can be a new direction. This paper lacks theoretical aspect, as to no indication is given why RNN based design/decoders can be good. I am mostly satisfied with the experiments, barring Fig 8, which does not show the results that the authors claim.

review_title: An interesting paper that brings in the tools of recursive neural networks to error-correcting codes for communication 

Meta-review: This paper studies trainable deep encoders/decoders in the context of coding theory, based on recurrent neural networks. It presents highly promising results showing that one may be able to use learnt encoders and decoders on channels where no predefined codes are known.

Besides these encouraging aspects, there are important concerns that the authors are encouraged to address; in particular, reviewers noted that the main contribution of this paper is mostly on the learnt encoding/decoding scheme rather than in the replacement of Viterbi/BCJR. Also, complexity should be taken into account when comparing different decoding schemes.

Overall, the AC leans towards acceptance, since this paper may trigger further research in this direction. 
Decision: Accept (Poster)