This paper presents MAd-RL, a method for decomposition of a single-agent RL problem into a simple sub-problems, and aggregating them back together. Specifically, the authors propose a novel local planner - emphatic, and analyze the newly proposed local planner along of two existing ones - egocentric and agnostic. The MAd-RL, and theoretical analysis, is evaluated on the Pac-Boy task, and compared to DQN and Q-learning with function approximation. Pros: 1. The paper is well written, and well-motivated. 2. The authors did an extraordinary job in building the intuition for the theoretical work, and giving appropriate examples where needed. 3. The theoretical analysis of the paper is extremely interesting. The observation that a linearly weighted reward, implies linearly weighted Q function, analysis of different policies, and local minima that result is the strongest and the most interesting points of this paper. Cons:1. The paper is too long. 14 pages total - 4 extra pages (in appendix) over the 8 page limit, and 1 extra page of references. That is 50% overrun in the context, and 100% overrun in the references. The most interesting parts and the most of the contributions are in the Appendix, which makes it hard to assess the contributions of the paper. There are two options:   1.1 If the paper is to be considered as a whole, the excessive overrun gives this paper unfair advantage over other ICLR papers. The flavor and scope and quality of the problems that can be tackled with 50% more space is substantially different from what can be addressed within the set limit. If the extra space is necessary, perhaps this paper is better suited for another publication? 1.2 If the paper is assessed only based on the main part without Appendix, then the only novelty is emphatic planner, and the theoretical claims with no proofs. The results are interesting, but are lacking implementation details. Overall, a substandard paper. 2. Experiments are disjoint from the method’s section. For example:2.1 Section 5.1 is completely unrelated with the material presented in Section 4. 2.2 The noise evaluation in Section 5.3 is nice, but not related with the Section 4. This is problematic because, it is not clear if the focus of the paper is on evaluating MAd-RL and performance on the Ms.PacMan task, or experimentally demonstrating claims in Section 4. Recommendations:1. Shorten the paper to be within (or close to the recommended length) including Appendix. 2. Focus paper on the analysis of the advisors, and Section 5. on demonstrating the claims. 3. Be more explicit about the contributions. 4. How does the negative reward influence the behavior the agent? The agent receives negative reward when near ghosts. 5. Move the short (or all) proofs from Appendix into the main text. 6. Move implementation details of the experiments (in particular the short ones) into the main text. 7. Use the standard terminology (greedy and random policies vs. egoistic and agnostic) where possible. The new terms for well-established make the paper needlessly more complex. 8. Focus the literature review on the most relevant work, and contrast the proposed work with existing peer reviewed methods. 9. Revise the literature to emphasize more recent peer reviewed references. Only three references are recent (less than 5 years), peer reviewed references, while there are 12 historic references. Try to reduce dependencies on non-peer reviewed references (~10 of them). 10. Make a pass through the paper, and decouple it from the van Seijen et al., 2017a 11. Minor: Some claims need references: 11.1 Page 5: “egocentric sub-optimality  does not come from the actions that are equally good, nor from the determinism of the policy, since adding randomness…” - Wouldn’t adding epsilon-greediness get the agent unstuck? 11.2 Page 1. “It is shown on the navigation task ….” - This seems to be shown later in the results, but in the intro it is not clear if some other work, or this one shows it. 12. Minor:12.1 Mix genders when talking about people. Don’t assume all people that make “complex and important problems”, or who are “consulted for advice”, are male. 12.2 Typo: Page 5: a_0 sine die 12.3 Page 7 - omit results that are not shown 12.4 Make Figures larger - it is difficult, if not impossible to see 12.5 What is the difference between Pac-Boy and Ms. Pacman task? And why not use Ms. Packman?