Task: Language Identification
Input (Spanish, English): This paper considers the problem of model-based policy search. The authors  consider the use of Bayesian Neural Networks to learn a model of the environment and advocate for the $\alpha$-divergence minimization rather than the more usual  variational Bayes.   The ability of alpha-divergence to capture bi-modality however  comes at a price and most of the paper is devoted to finding tractable approximations.  The authors therefore use the approach of Hernandez-Lobato et al. (2016) as proxy to the alpha-divergence .   The environment/system dynamics is clearly defined as a well as the policy parametrization  (section 3) and would constitute a useful reference point for other researchers.  Simulated roll-outs, using the learned model, then provide samples of the expected  return. Since a model of the environment is available, stochastic gradient descent  can be performed in the usual way, without policy gradient estimators, via automatic  differentiation tools.   The experiments demonstrate that alpha-divergence is capable of capturing multi-model  structure which competing methods (variational Bayes and GP) would otherwise struggle with. The proposed approach also compares favorably in a real-world batch setting.  The paper is well-written, technically rich and combines many recent tools  into a coherent algorithm. However, the repeated use of approximations to original  quantities seems to somehow defeat the benefits of the original problem formulation.  The scalability and computational effectiveness of this approach is also questionable  and I am uncertain if many problem would warrant such complexity in their solution.  As with other Bayesian methods, the proposed approach would probably shine in low-samples  regime and in this case might be preferable to other methods in the same class (VB, GP).
Output (English): en