reviews: 
Comments: This paper introduces an exploration approach, LS-Imagine, to improve model-based reinforcement learning from pixels on open-world environments (with large state spaces). 

The paper aims to address the short-hoirzon limitation of model-based approaches relying on repeated autoregressive single timestep prediction. This is achieved by first generating affordance maps of observations by zooming into different patches and determining if there are objects associated with the current task in those patches using a vision-language reward model (MineCLIP in this work) distilled into an affordance map predictor. This affordance map is used to generate an intrinsic reward associated with the observation. The kurtosis of the affordance map is then used to determine if the object associated with reward is near or far away (binary, based on a threshold). If the object is deemed to be near, a standard 'short-term' world model using autoregressive single timestep prediction is used to generate trajectories for an actor and critic to learn from. Alternatively, if the object deemed to be far away, a 'long-term' world model using autoregressive multi-timestep interval prediction is used (although ignoring actions). This enables the world model to generate a trajectory over a larger number of environment timesteps in the same number of autoregressive prediction steps, therefore potentially enabling the generated trajectory to get closer to the object of interest. This long-term trajectory is used to train the critic but not the actor. The combination of these models is termed a Long Short-Term World Model, giving the approach its name: LS-Imagine.

LS-Imagine is applied to 5 MineDojo tasks and outperforms the baselines in terms of success rate and per-episode steps after training. Ablations demonstrate the benefit of both affordance-based intrinsic reward and the use of long-term imagination.

In summary, the paper attempts to tackle the important problem of long horizon world modelling and provides an interesting and novel approach to address this problem in 3D environments. While it is a relatively complex method and somewhat limited to Minecraft, I believe the ideas and insights warrant acceptance.
Strengths: - **Significance**
  - Long-horizon world modeling and reinforcement learning in open-world environments are important problems.
  - The proposed approach is insightful and successfully addresses these problems.
- **Originality**
  - The proposed approach involves the combination of multiple novel and inisightful components.
- **Quality**
  - Overall the quality of the paper is relatively high, with the method reasonably clearly explained and analyzed.
Weaknesses: - **Clarity**
  - Some aspects of the paper are not particularly clear. The main one is the use of the word 'jumpy' throughout the paper. The meaning of this word is assumed, but is not defined in the paper or standard usage as far as I'm aware, and is relatively unscientific, so I feel it is not the right word to use. 'Multi-step' state transitions seems more appropriate. If the authors were attempting to highlight that the number of steps can vary, then 'variable-step' transitions would be better. At the very least, 'jumpy' should be properly defined at the beginning of the paper.
  - Similarly affordance maps may not be familiar to all readers and the exact meaning of this term can vary. For example, a short clarification early in the paper such as ''...affordance maps, that elucidate which parts of an observation may be associated with reward, ..." would be helpful.
  - Some other unclear aspects/minor mistakes include:
    - L326: "employs *an* actor-critic algorithm to learn behavior *from* the latent state sequences..."
    - L351: Grammar is slightly wrong and confusing, should be: "Notably, since long-term imagination does *not* involve actions, we do not optimize the the actor when long-term imagnation is adopted." 
    - L354: Also worth highlighting the difference with the DreamerV3 loss. "The loss of the actor is therefore equivalent to DreamerV3, with an additional factor of $(1-\hat j_t)$ to ignore updates from long-term imagination:"
    - L361: "on the chellenging..."
    - L500: Doesn't make sense. Maybe "Our work is also related to affordance maps for robotics." is sufficient?
    - "*Learning to Move with Affordance Maps*" [1] should likely also be compared and cited here.

- **Limitations**

  - This approach has important limitations that are not mentioned. In particular, the approach is limited to embodied agents navigating a 3D environment in which there are objects associated for which reward is obtained by approaching them. Therefore the approach assumes, for example:
    - Observations are of a 3D environment
    - Actions are navigation actions of an embodied agent
    - Rewards are assoiated with identifying or moving towards objects
    - A reward model is available to identify high reward regions of observations
  - The experiments are limited to Minecraft for which these assumptions hold. This approach would likely not work as well even in Crafter [2] for example, which provides a 2D 'open-world' analogue of Minecraft, since objects do not become larger as you move towards them.
  - The approach also relies on both the long-term and short-term models being used, given only the short-term model is able to update the actor. While the thresholding of $P_{jump}$ can partially be used to address this, this is not particularly robust, and still requires some close-up objects in initial exploration for the standard one-step world model to be used, so the approach may not work as well in very sparse environments.
  - There is still significant value of the approach despite these limitations, and the paper is reasonably scoped, but they should be included in the limitations at the end of the paper, which are currently overly brief and narrow.

  **References:**

  [1] "*Learning to Move with Affordance Maps*", Qi et al., 2020

  [2] "*Benchmarking the Spectrum of Agent Capabilities*", Hafner, 2021
Recommendation: 8
Reviewer confidence: 4
Soundness/correctness: 3
Clarity: 3