Instructions: In this task, you are given a paper review. Based on the review, your job is to identify language and generate "en" if the review is in English or generate "es" if the review is in Spanish. Note that URLs in the text have been replaced with [Link].
Input: The paper presents a hierarchical DRL algorithm that solves sequences of navigate-and-act tasks in a 2D maze domain. During training and evaluation, a list of sub-goals represented by text is given to the agent and its goal is to learn to use pre-learned skills in order to solve a list of sub-goals. The authors demonstrate that their method generalizes well to sequences of varying length as well as to new combinations of sub-goals (i.e., if the agent knows how to pick up a diamond and how to visit an apple, it can also visit the diamond). Overall, the paper is of high technical quality and presents an interesting and non-trivial combination of state-of-the-art advancements in Deep Learning (DL) and Deep Reinforcement Learning (DRL). In particular, the authors presents a DRL agent that is hierarchical in the sense that it can learn skills and plan using them. The skills are learned using a differential temporally extended memory networks with an attention mechanism. The authors also make a novel use of analogy making and parameter prediction. However, I find it difficult to understand from the paper why the presented problem is interesting and why hadn't it bee solved before. Since the domain being evaluated is a simple 2D maze, using deep networks is not well motivated. Similar problems have been solved using simpler models. In particular, there is a reach literature about planning with skills that had been ignored completely by the authors. Since all of the skills are trained prior to the evaluation of the hierarchical agent, the problem that is being solved is much more similar to supervised learning than reinforcement learning (since when using the pre-trained skills the reward is not particularly delayed). The generalization that is demonstrated seems to be limited to breaking a sentence (describing the subtask) into words (item, location, action). The paper is difficult to read, it is constantly switching between describing the algorithm and giving technical details. In particular, I find it to be overloaded with details that interfere with the general understanding of the paper. I suggest moving many of the implementation details into the appendix. The paper should be self-contained, please do not assume that the reader is familiar with all the methods that you use and introduce all the relevant notations. I believe that the paper will benefit from addressing the problems I described above and will make a better contribution to the community in a future conference.
Output:
en