Paper:
In reinforcement learning, we can learn a model of future observations and rewards, and use it to plan the agent's next actions. However, jointly modeling future observations can be computationally expensive or even intractable if the observations are high-dimensional (e.g. images). For this reason, previous works have considered partial models, which model only part of the observation. In this paper, we show that partial models can be causally incorrect: they are confounded by the observations they don't model, and can therefore lead to incorrect planning. To address this, we introduce a general family of partial models that are provably causally correct, yet remain fast because they do not need to fully model future observations.

Review:
This paper tackles the issue of identifying the causal reasoning behind why partial models in MBRL settings fail to make correct predictions under a new policy. The novel contribution was a framework for learning better partial models based on models learning an interventional conditional, rather than an observational conditional. The paper tried to provide both theoretical and experimental reasoning for this framework. 
I vote to (weak) reject the paper due to the major issues with section 2. Furthermore, the paper hard or at times almost impossible to understand as too many assumptions are made and too little is explained.
Recommendations
Because your graphs are not MDPs, you are not framing your example as an RL problem. This is causing a number of issues with notation and lack of clarity in the argument you're making.
1. It is unclear to me that the FuzzyBear example is correctly constructed as a RL example, reasons being that:  
- Figure 1 (a) & (b) do not correspond to an MDP as two different states, teddy vs grizzly, are both designated as s_1 and similarly, the two possible actions, hug or run, are both designated as a_1 and thus are not distinct.
- Note your terminal state for the episodes
- Have a reward for (s0, a0) as every s-a pair should have a reward
- It would be helpful to note that the environments in Figure 1 are stochastic
2. Clarify notation. There are a number of assumptions about what background knowledg the reader should have. Given the bridging of disciplines in the paper, it would be useful to provide more detail on notation in Section 3.
3. Add a section on reinforcement learning in Section 3. If it's the last subsection in section 3, you could describe the relationship between the various causal reasoning and RL principles. This would further clarify how you're bridging these subtopics.
4. For sentence,
"Fundamentally, the problem is due to causally incorrect reasoning: the model learns the observational condi- tional p(r|s0, a0, a1) instead of the interventional conditional given by p(r|s0, do(a0), do(a1)) = s1 p(s1|s0, a0)p(r|s1, a1)."
As you don't cover the meaning of the do() operator until a later paragraph, provide a quick description of it as it is not common knowledge to a general AI audience, e.g., where do() indicates that the action was taken.
5. Correct the following sentences,
"Mathematically, the model with learn the following conditional probability:"
"In Section 3, we review relevant concepts from causal reasoning based on which we propose solutions that address the problem."
6. I recommend putting the interventional conditional equation, p(r|s0, do(a0), do(a1)) = 􏰀s1 p(s1|s0, a0)p(r|s1, a1), on its own line as the reader is doing a comparison of it with the previous equation, p(r|s0, a0, a1), given on page 2.
7. Strengthen your abstract by aligning more with claims you make in your conclusion.
8. The experiments in Figure 5 are averaged over 5 seeds. This is not enough to be statistically significant - furthermore, there are no error bars in the Figure.
Question(s):
1. You've indicated two policies for Figure 1 (a):
- pi1: the agent knows it is encountering a teddy bear, so it will hug
- pi2: the agent knows it is encountering a grizzly bear, so it will run
Is this the "change in the behavior policy" that you're referring to? If so, make this clearer, this currently requires a lot of work by the reader to make sense of it.
2. What are the partially observable parts of the environments in Figure 1 (a) & (b)? Make this clear.