Paper:
Video prediction models combined with planning algorithms have shown promise in enabling robots to learn to perform many vision-based tasks through only selfsupervision, reaching novel goals in cluttered scenes with unseen objects. However, due to the compounding uncertainty in long horizon video prediction and poor scalability of sampling-based planning optimizers, one significant limitation of these approaches is the ability to plan over long horizons to reach distant goals.
To that end, we propose a framework for subgoal generation and planning, hierarchical visual foresight (HVF), which generates subgoal images conditioned on a goal image, and uses them for planning. The subgoal images are directly optimized to decompose the task into easy to plan segments, and as a result, we observe that the method naturally identifies semantically meaningful states as subgoals. Across four simulated vision-based manipulation tasks, we find that our method achieves more than 20% absolute performance improvement over planning without subgoals and model-free RL approaches. Further, our experiments illustrate that our approach extends to real, cluttered visual scenes.

Review:
This paper introduces a hierarchical extension to existing work in vision-based model predictive control. Here, a hierarchical model is optimised to find sub-goals that minimise the planning cost (bottleneck states), so as to allow for improved planning to goal states expressed in higher dimensional state spaces. As expected, results show that this hierarchy improves tasks execution success rates. 
The paper is clearly written, although many sub-components of the architecture are described in other articles. While I appreciate that the work is incremental, and that it relies on a number of previously modules, this brevity and the architectural complexity means that the work will be challenging to reproduce or adapt to different tasks. 
Nevertheless, I believe the idea is interesting, and the paper does achieve its stated goals of allowing for entirely self-supervised learning without motion primitives, demonstrations and rewards, so makes a useful contribution here.  Compositional planning through sub-goals is a good idea, and this paper adds a useful mechanism to identify suitable sub-goals in a latent space and plan through these.
However, I don't believe that this approach is in any way practically feasible at present, and the paper makes a number of hyperbolic claims that should be toned down to give a more balanced perspective (I appreciate that the authors attempted to do so in their limitations section, but this needs to be done throughout the paper).
Specifically:
1.) Please avoid the use of relative improvement percentages (200% performance improvements in the abstract - the 20% absolute performance is strong enough to stand alone).
2.) The paper oversimplifies the importance of exploration and data collection in the proposed approach. The paper states that a uniform random policy in the continuous action space of the agent works well, and offsets the data collection to "any exploration policy", but this is a key requirement if the proposed approach is to be useful. At present the method seems to be relying on typewriting monkeys writing Shakespeare as a precursor to control. An exploration policy using identified sub-goal of interest may be a particularly valuable extension in future work, but for now the paper glosses over this aspect.
3.) The use of the term "long-horizon tasks" and "manipulation". In line with point 2, my personal feeling is that any task or state that can conceivably be explored and accessed by uniformly sampling from a continuous action space does not qualify as a long-horizon task. From the results and videos, it seems that all of the tasks could be solved by following a single trajectory in 3D space, ie. swing left to knock objects off the table, then right to close the door. Terming this "long-horizon" is a bit of a stretch, as is the notion that flopping about a table bashing into objects counts as "manipulation". Motion planning and trajectory following are not long-term horizon manipulation tasks, solving towers of Hanoi is. Moreover, I appreciate that experiments were conducted on real robot datasets, but this seems to be more of an exercise in latent space anthropomorphism than practical evidence of a feasible control policy.
4.) Limitations regarding latent space expressiveness.  As mentioned in the limitations section of the paper, the proposed approach is heavily reliant on a latent space that fully captures the scene and can roll-out future states sensibly. This is an extremely challenging problem, and one which appears to affect the proposed approach, for example, in the accompanying video (3:.40 - 3:59), the desired goal state contains two objects standing on the table and a closed door, but the policy only closes the door, while objects are knocked over. This seems to indicate that the latent space and planning is unable to learn good object embeddings and spatial representations, and that the type of task that can be solved is along the lines of move forward and to the right when the black blob is on the left of the image, or move down to the left, when there are some coloured blobs on the table. 
5.) Limitations around goal state representations . Following on from 4, the identification of important aspects in a given image or latent space represents a major challenge to the proposed approach. Given an image of a desired state, how can the proposed approach identify which elements in the scene are required for success, and which are simply distractors? At present, I see no mechanism by which this could ever be learned in a self-play setting or through an image-based goal state. Eg. what if the image indicated that I want the two objects to be in a specific position, and I never cared about the state of the door?
Despite these limitations, the granularity of the solution is fine for a proof of concept work like this, and is itself a commendable achievement. Unfortunately, the paper's choice of language, relying on terms like "long term horizon" and "manipulation" for simple reaching and pushing tasks exaggerates the state of robot learning, and is potentially misleading to those less familiar with the field.
Despite my gripes, the paper definitely meets the ICLR threshold of "accept if you'd share it with a colleague", and I believe it is is a useful approach to sub-goal identification and a nice piece of work, so I recommend acceptance.