Task: Language Identification
Input (Spanish, English): Paper Summary This paper proposes an unsupervised learning model in which the network predicts what its state would look like at the next time step (at input layer and potentially other layers).  When these states are observed, an error signal is computed by comparing the predictions and the observations. This error signal is fed back into the model. The authors show that this model is able to make good predictions on a toy dataset of rotating 3D faces as well as on natural videos. They also show that these features help perform supervised tasks.  Strengths - The model is an interesting embodiment of the idea of predictive coding   implemented using a end-to-end backpropable recurrent neural network architecture. - The idea of feeding forward an error signal is perhaps not used as widely as it could   be, and this work shows a compelling example of using it.  - Strong empirical results and relevant comparisons show that the model works well. - The authors present a detailed ablative analysis of the proposed model.  Weaknesses - The model (esp. in Fig 1) is presented as a generalized predictive model   where next step predictions are made at each layer. However, as discovered by running the experiments, only the predictions at the input layer are the ones that actually matter and the optimal choice seems to be to turn off the error signal from the higher layers. While the authors intend to address this in future work, I think this point merits some more discussion in the current work, given the way this model is presented. - The network currently lacks stochasticity and does not model the future as a   multimodal distribution (However, this is mentioned as potential future work).  Quality The experiments are well-designed and a detailed analysis is provided in the appendix.  Clarity The paper is well-written and easy to follow.  Originality Some deep models have previously been proposed that use predictive coding. However, the proposed model is most probably novel in the way it feds back the error signal and implements the entire model as a single differentiable network.  Significance This paper will be of wide interest to the growing set of researchers working in unsupervised learning of time series. This helps draw attention to predictive coding as an important learning paradigm.  Overall Good paper with detailed and well-designed experiments. The idea of feeding forward the error signal is not being used as much as it could be in our community. This work helps to draw the community's attention to this idea.
Output (English): en