Paper:
Recurrent Models of Visual Attention
Applying convolutional neural networks to large images is computationally expensive because the amount of computation scales linearly with the number of image pixels. We present a novel recurrent neural network model that is capable of extracting information from an image or video by adaptively selecting a sequence of regions or locations and only processing the selected regions at high resolution. Like convolutional neural networks, the proposed model has a degree of translation invariance built-in, but the amount of computation it performs can be controlled independently of the input image size. While the model is non-differentiable, it can be trained using reinforcement learning methods to learn task-specific policies. We evaluate our model on several image classification tasks, where it significantly outperforms a convolutional neural network baseline on cluttered images, and on a dynamic visual control problem, where it learns to track a simple object without an explicit training signal for doing so.

Review:
The paper describes a method for sequentially learning sequences of action for classification and regression tasks in the context of visual scenes. The motivation comes from attentional models in vision where decisions are made by considering sequentially selected information.  The model is formulated as a POMDP and the algorithm is implemented using a recurrent neural network. The hidden layer of this network serves as the state of the POMDP and the approximated policy is learned via the neural network. Experiments are performed on image classification tasks (noisy digits) and on toy control problem.The paper is well motivated and clear. Some precisions could be added especially for the dynamic environment setting (both for the model, e.g. LSTM units and for the experiments). The paper originality, in the context of this application, is in the use of a recurrent net for dynamically updating the state of the model in its hidden layer and in the regression model for the prediction of the next place to look for.The test beds on the other hand concern only very simple problems (both classification and dynamic prediction), so that the state space is probably not that big for all these problems. Experiments on stronger benchmarks are needed in order to effectively assess the capacity of this method to handle more realistic problems. Also, the proposed system could be compared with other sequential approaches. You could at least compare with a random policy + see the references below. The complexity of the method should be discussed, there is no mention of this aspect, and it should be compared to the complexity of other methods, both sequential and non-sequential.  There is neither discussion nor analysis on the usefulness of the hidden layer recurrence. Did you made experiments with different memory sizes or decay rate (weights of the recurrent connections) or did you attempt to learn the weight values?Detailed comments: It seems that for the classification, the number of steps is fixed in advance and is the same for all the images. This is probably a useless assumption (by the way, how do you set up the value of this variable?).Below are some references which could be interesting for this work.Karayev, Sergey, Baumgartner, Tobias, Fritz, Mario, and Darrell, Trevor. Timely object recognition. In NIPS, pp. 1–9, 2Cuccu, Giuseppe, Luciw, Matthew, Schmidhuber, Jurgen, and Gomez, Faustino. Intrinsically motivated neuroevolution for vision-based reinforcement learning. In Development and Learning (ICDL), 2011 IEEE International Conference on, volume 2, pp. 17. IEEE, 2011.012.Contardo, Gabriella , Denoyer, Ludovic , Artieres, Thierry,  Gallinari, Patrick, Learning State Representations in POMDP, ICLR 2014Dulac-Arnold, Gabriel , Denoyer, Ludovic, Thome, Nicolas, Cord, Matthieu, Gallinari, Patrick, Sequentially Generated Instance-Dependent Image Representations for Classification, ICLR 2014