Paper:
One of the key challenges of session-based recommender systems is to enhance users' purchase intentions. In this paper, we formulate the sequential interactions between user sessions and a recommender agent as a Markov Decision Process (MDP). In practice, the purchase reward is delayed and sparse, and may be buried by clicks, making it an impoverished signal for policy learning. Inspired by the prediction error minimization (PEM) and embodied cognition, we propose a simple architecture to augment reward, namely Imagination Reconstruction Network (IRN). Specifically, IRN enables the agent to explore its environment and learn predictive representations via three key components. The imagination core generates predicted trajectories, i.e., imagined items that users may purchase. The trajectory manager controls the granularity of imagined trajectories using the planning strategies, which balances the long-term rewards and short-term rewards. To optimize the action policy, the imagination-augmented executor minimizes the intrinsic imagination error of simulated trajectories by self-supervised reconstruction, while maximizing the extrinsic reward using model-free algorithms. Empirically, IRN promotes quicker adaptation to user interest, and shows improved robustness to the cold-start scenario and ultimately higher purchase performance compared to several baselines. Somewhat surprisingly, IRN using only the purchase reward achieves excellent next-click prediction performance, demonstrating that the agent can "guess what you like" via internal planning.

Review:
The paper aimed at improving the performance of recommendation systems via reinforcement learning. The author proposed an Imagination Reconstruction Network for the recommendation task, which implements an imagination-augmented policy via three components: (1)  the imagination core (IC) that predicts the next time steps conditioned on actions sampled from an imagination policy; (2) the trajectory manager (TM) that determines how to roll out the IC under the planning strategy and produces a set of imagined item trajectories; (3) the imagination-augmented executor (IAE) that aggregates the internal data resulting from imagination and external rewarding data to update its action policy.
Strengths of the paper:
(1) The research problem that the performance of recommendation systems needs to be improved is of great value to be investigated, as recommendation systems play crucial role in people’s daily lives. 
(2) Experiments were conducted on a publicly available dataset. 
(3) Robustness to cold-start scenario was tested and evaluated in the experiments.
Weaknesses of the paper:
(1) The motivations of applying reinforcement learning techniques are not convinced to me. There are a lot of supervised learning algorithms to the task of recommendations. Why do the authors utilize reinforcement learning to the task but not other supervised learning techniques? Is it because reinforcement learning based methods work better than traditional machine learning based ones? The motivations of integrating A3C (Asynchronous Advantage Actor-Critic) but not other techniques into the proposed model are not convinced to me as well. 
(2) State-of-the-art reinforcement learning algorithms were not taken into account for baselines in the experiments. As the proposed method is built based on reinforcement learning, it would be better if the authors could include state-of-the-art reinforcement learning algorithms as their baselines.
(3) Some details are missing, resulting in the fact that it is hard for other researchers to fully capture the mechanism of the proposed algorithm. In equations (2) and (3), what is theta_v? How is theta_v associated with the parameters in LSTM. Is theta_v denoted the parameters of LSTM? How do the authors define the loss functions, i.e., \mathcal{L}_{A3C} and \mathcal{L}_{IRN}? What are the relationships among \mathcal{L}_{A3C}, \mathcal{L}_{IRN} and the one defined in equation (4)?
(4) The contributions of the paper in terms of theory are somewhat not significant. It seems that the proposed algorithm is built based on and combined by existing algorithms such as A3C. 
Minor comments:
(1) It would be better if the authors can test the proposed model on more datasets. There are many publicly available datasets for testing the performance of recommendation systems.
(2) Figure 2 is not straightforward. It would be better if the authors can draw the figure in other ways. (I am not sure if the authors have expressed the underlying ideas clearly with Figure 2).