Paper:
Sequence prediction models can be learned from example sequences with a variety of training algorithms. Maximum likelihood learning is simple and efficient, yet can suffer from compounding error at test time. Reinforcement learning such as policy gradient addresses the issue but can have prohibitively poor exploration efficiency. A rich set of other algorithms, such as data noising, RAML, and softmax policy gradient, have also been developed from different perspectives. In this paper, we present a formalism of entropy regularized policy optimization, and show that the apparently distinct algorithms, including MLE, can be reformulated as special instances of the formulation. The difference between them is characterized by the reward function and two weight hyperparameters. The unifying interpretation enables us to systematically compare the algorithms side-by-side, and gain new insights into the trade-offs of the algorithm design. The new perspective also leads to an improved approach that dynamically interpolates among the family of algorithms, and learns the model in a scheduled way. Experiments on machine translation, text summarization, and game imitation learning demonstrate superiority of the proposed approach.

Review:
This paper claims to propose a general entropy regularized policy optimization paradigm. MLE and RL are special cases of this training paradigm. Paper is well written, and the experimental results are convincing enough.  
However, there are still some minor problems in the paper. For the optimization framework ERPO (shown in Equation 1), it consists of three parts, a cross-entropy term (Shannon entropy), a $p,q$ KL divergence term, and a reinforcement learning reward loss item. From the framework point of view, it is not like the author claim that is supposed to present a general optimization framework, including various optimization algorithms. Instead, it is just a combined loss through weight control and the selection of corresponding functions. It may not really theoretically work to unify various types of optimization algorithms for general cases, let alone claiming that this is a general optimization algorithm framework. 
For the interpolation algorithm (I regard this is the true technical contribution of this paper), the authors used an annealing mechanism to use different weights and functions at different stages of training. The essence is that after MLE pre-training, different optimization algorithms are used in different stages, and this should be the focus of the article. The annealing settings used is only introduced in the appendix simply. Without more comparison experiments, we cannot clearly get the conditions for the annealing algorithm to be effective and ineffective. 
For the title of connecting the dots between MLE and RL, this paper did not do so, MLE and RL are only used collaboratively, and this has also been mentioned in previous work.
typo
Page 6 Paragraph “Other Algorithms & Discussions”: We We show in the appendix… -> We show in the appendix…