This paper proposes a new method for reverse curriculum generation by gradually reseting the environment in phases and classifying states that tend to lead to success.  It additionally proposes a mechanism for learning from human-provided "key states". The ideas in this paper are quite nice,  but the paper has significant issues with regard to clarity and applicability to real-world problems: First, it is unclear is the proposed method requires access only high-dimensional observations (e.g. images) during training or if it additionally requires low-dimensional states (e.g. sufficient information to reset the environment).  In most compelling problems settings where a low-dimensional representation that sufficiently explains the current state of the world is available during training, then it is also likely that one can write down a nicely shaped reward function using that state information during training, in which case, it makes sense to use such a reward function.  This paper seems to require access to low-dimensional states, and specifically considers the sparse-reward setting,  which seems contrived. Second, the paper states that the assumption "when resetting, the agent can be reset to any state" can be satisfied in problems such as real-world robotic manipulation.  This is not correct.  If the robot could autonomously reset to any state, then we would have largely solved robotic manipulation.  Further, it is not always realistic to assume access to low-dimensional state information during training on a real robotic system (e.g. knowing the poses of all of the objects in the world). Third, the experiments section lacks crucial information needed to understand the experiments.  What is the state, observation, and action space for each problem setting?  What is the reward function for each problem setting?  What reinforcement learning algorithm is used in combination with the curriculum and tendency rewards?  Are the states and actions continuous or discrete?  Without this information, it is difficult to judge the merit of the experimental setting. Fourth, the proposed method seems to lack motivation, making the proposed scheme seem a bit ad hoc.  Could each of the components be motivated further through more discussion and/or ablative studies? Finally, the main text of the paper is substantially longer than the recommended page limit.  It should be shortened by making the writing more concise. Beyond my feedback on clarity and significance, here are further pieces of feedback with regard to the technical content, experiments, and related work:I'm wondering -- can the reward shaping in Equation 2 be made to satisfy the property of not affecting the final policy?  (see Ng et al. '09)  If so, such a reward shaping would make the method even more appealing. How do the experiments in section 5.4 compare to prior methods and ablations?  Without such a comparison, it is impossible to judge the performance of the proposed method and the level of difficulty of these tasks.  At the very least, the paper should compare the performance of the proposed method to the performance a random policy. The paper is missing some highly relevant references.  First, how does the proposed method compare to hindsight experience replay?  [1] Second, learning from keyframes (rather than demonstrations) has been explored in the past [1].  It would be preferable to use the standard terminology of "keyframe". [1] Andrychowicz et al. Hindsight Experience Replay. 2017 [2] Akgun et al. Keyframe-based Learning from Demonstration. 2012 In summary, I think this paper has a number of promising ideas and experimental results,  but given the significant issues in clarity and significance to real world problems, I don't think that the current version of this paper is suitable for publication in ICLR. More minor feedback on clarity and correctness:- Abstract: "Deep RL algorithms have proven successful in a vast variety of domains" -- This is an overstatement. - The introduction should be more clear with regard to the assumptions.  In particular, it would be helpful to see discussion of requiring human-provided keyframes.  As is, it is unclear what is meant by "checkpoint scheme",  which is not commonly used terminology. - "This kind of spare reward, goal-oriented tasks are considered the most difficult challenges" -- This is also an overstatement.  Long-horizon tasks and high-dimensional observations are also very difficult.  Also, the sentence is not grammatically correct. - "That is, environment" -> "That is, the environment" - In the last paragraph of the intro, it would be helpful to more clearly state what the experiments can accomplish.  Can they handle raw pixel inputs? - "diverse domains" -> "diverse simulated domains" - "a robotic grasping task" -> "a simulated robotic grasping task" - There are a number of issues and errors in citations, e.g. missing the year, including the first name, incorrect reference - Assumption 1: \mathcal{P} has not yet been defined. - The last two paragraphs of section 3.2 are very difficult to understand without reading the method yet - "conventional RL solver tend" -> "conventional RL tend",  also should mention sparse reward in this sentence. - Algorithm 1 and Figure 1 are not referenced in the text anywhere, and should be - The text in Figure 1 and Figure 3 is extremely small - The text in Figure 3 is extremely small