DMWM: Dual-Mind World Model with Long-Term Imagination
Authors: Lingyi Wang
Rashed Shelim
Walid Saad
Naren Ramakrishnan
Venue: NeurIPS 2025 spotlight ()
Keywords: World model
long-term planning
logical reasoning
dual-process system
Primary area: reinforcement_learning

Abstract: Imagination in world models is crucial for enabling agents to learn long-horizon policy in a sample-efficient manner. Existing recurrent state-space model (RSSM)-based world models depend on single-step statistical inference to capture the environment dynamics, and, hence, they are unable to perform long-term imagination tasks due to the accumulation of prediction errors. Inspired by the dual-process theory of human cognition, we propose a novel dual-mind world model (DMWM) framework that integrates logical reasoning to enable imagination with logical consistency. DMWM is composed of two components: an RSSM-based System 1 (RSSM-S1) component that handles state transitions in an intuitive manner and a logic-integrated neural network-based System 2 (LINN-S2) component that guides the imagination process through hierarchical deep logical reasoning. The inter-system feedback mechanism is designed to ensure that the imagination process follows the logical rules of the real environment. The proposed framework is evaluated on benchmark tasks that require long-term planning from the DMControl suite and robotic environment. Extensive experimental results demonstrate that the proposed framework yields significant improvements in terms of logical coherence, trial efficiency, data efficiency and long-term imagination over the state-of-the-art world models.

DMWM: Dual-Mind World Model with Long-Term Imagination
Lingyi Wang!" Rashed Shelim! ! Walid Saad Naren Ramakrishnan"
Department of Electrical and Computer Engineering, Virginia Tech, USA
2 Department of Computer Science, Virginia Tech, USA Emails: (lingyiwang, rasheds, walids, naren] @ vt.edu
Abstract
Imagination in world models is crucial for enabling agents learn long-horizon policies in a sample-efficient manner. Existing recurrent state-space model (RSSM)- based world models depend on single-step statistical inference to capture the envi- ronment dynamics, and, hence, they cannot effectively perform long-term imagina- tion tasks due to the accumulation of prediction errors. Inspired by the dual- suol-psocess theory of human cognition, we propose novel dual-ming muil- world model (DMWM) framework that integrates logical reasoning to enable imagination with logical consistency. DMWM is composed of two components: an RSSM-based System (RSSM-S1) component that handles state transitions in intuitive manner and a logic-integrated neural network-basee System 2 (LINN-S) component that guides the imagination process through hierarchical deep logical reasoning. The inter- system feedback mechanism is designed to ensure that the imagination process follows the logical rules of the real environment. The proposed framework evaluated on benchmark tasks that require long-term planning from the DMControl suite and the robotic platforms. Extensive experimental results demonstrate that the proposed framework yields significant improvements in terms of logical coherence, trial efficiency, data efficiency and long-term imagination over the state-of-the-an world models. The code is available http::/ggghubb.co//www/////..b
Introduction
Imagination is a core capability of world models that allows agents to predict and plan effectively
within internal virtual environments by using real-world knowledge [1, 3, 4]. By predicting future scenarios in latent spaces, agents can evaluate potential outcomes without frequent real-world interactions thereby significantly improving data efficiency and minimizing trial-and-error costs. For complex tasks requiring long-term planning, imagination capabilities allow world models to evaluate the long-term consequences of diverse strategies and identify the optimal action plans. Consequently, the effectiveness of model-based decision-makiin approaches, such as model-based reinforcement learning (RL) [5, 6, 8] and model predictive control (MPC) [9, 10, 11]. can heavily depend on the quality of their imagination abilities.
One of the most widely used frameworks for world models is the so-called recurrent state-space model (RSSM) [2, 9, 12] and its variants [13, 14, 15, 16] that combine deterministic recurrent structures with stochastic latent variables model environmental dynamics in compact latent space. By doing so, RSSM models can capture the sequential dependencies and uncertainty of their target environment. However, RSSM cannot provide reliable, long-term predictions over extended imagination horizons due to the accumulation of prediction errors and the limitations of statistical inference [17, 18, 19]. In particular, although existing RSSM-based solutions can generate accurate short-term predictions
Conresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

in single-step manner, small enrors inevitably propagate over longer time horizons [20], gradually amplifying over each step and resulting in significant deviations between the imagined and actual states. Moreover, RSSM schemes often optimize state-space representations through reconstruction loss or regression. This approach can lead to overfitting to observed patterns and cannot properly capture latent dynamics [18, 20, 21], particularly in complex, dynamic environments.
Several models [17, 18, 20, 22, 23, 24J have been proposed for long-term planning. For instance, trajectory- based models [20, 23] learn long-temn -teg dynamics by directly predicting future states to reduce compounding emrors compared to single-step prediction models. Latent space-enhanced mod- els [17, 18] incorporate abstracted long-term information, such as high- level goals and global knowledge, to mitigate error propagation in each single-step prediction. However, all of these approaches [17, 18, 20, 22, 23] still rely on statistical inference, which inevitably leads to prediction error accumulation and drift over long horizons. By skipping intermedi- ate reasoning steps, these approaches [17, 18, 24] lose the logical consis- tency and interpretability required for complex, high-precision planning. Hence, existing approaches cannot provide robust and reliable imagination over an extended horizon size.
System 2: Slow Thinking With Logic Reasoning
Logical Steps
From SI to $2 Real-World Representation
From S2 Logic Guidance
tt, st
Ot, Tt
at
t
System I: Fast Thinking
Direct Ouiput
With Intuition
DualMind World Model
Observation
Decision-Makingg
Observable
at
Objects
Environment
Figure 1:
The proposed framework for DMWM.
Motivated by the challenges of long-term imagination and the limitations of existing long-tern planning approaches, the main contribution of this paper is a novel dual-mind world model (DMWM) framework based the drac-ssocess theory of human cognition [25, 26, 27J. The proposed DMWM framework can achieve reliable and efficient imagination by synergistically combining the comple- mentary strengths of System and System 2 learning processes [28, 29]. By designing this framework shown in Figure 1, we make the following key contributions:
We propose DMWM, a novel world model framework that integrates the dual-process theory of human cognition (System 1 and System 2) to endow agents with robust, long-tern imagination capabilities driven by both data and logical reasoning. Particularly, based on the RSSM-based System 1 (RSSM-SI) component that predicts the state transitions in a fast, intuitive-driven manner, we further propose logic-integrated neural network-band System 2 (LINN-S2) component that is used, for the first time, to guide imagination through logical reasoning a higher level.
In LINN-S2, we introduce logical regularization rules to conduct logical reasoning within the state and action spaces by using operations such as A, V, - and -. The logical rules allows the logical consistency and interpretability of the world model. Additionally, we propose new recursive logic reasoning framework that extends local reasoning into globally consistent long-term planning. enabling the modeling of logical sequence dependencies in complex tasks.
We design an effective inter-system feedback mechanism. In particular, LINN-S2 provides logical constraints to guide RSSM-SI so to ensure that predicted sequences consistent with domain- specific logical rules. For the feedback based on real- world observations and latent representations from RSSM-S1, it updates the domain-specific logic of LINN-S2, thereby allowing dynamic refinement and adaptation. This novel inter-system feedback mechanism can thus allow the human-like, duoc-psocsss cognitive abilities for agents.
We evaluate the proposed DMWM with actor-critic based RL and MPC in extensive experiments including DMControl and robotie tasks. Simulation results demonstrate that DMWM is able to respectively provide 14.3%, 5.5-fold, 32% and 120% improvement in logic consistency, trial efficiency, data efficiency and reliable imagination over an extended horizon size compared to baselines in complex tasks.

---

2 Proposed DMWM Framework
In this section, we introduce the proposed DMWM framework inspired by the dual-process theory of human cognition. DMWM consists of RSSM-S1 and LINN-S2. First, we introduce RSSM-S1, which builds upon the RSSM architecture to learn the environment dynamics in a latent space and perform fast, intuitive state representations and predictions. Next, we introduce novel LINN-S2 to capture the intricate logical relationships between the state space and the action space. By employing hierarchical deep reasoning framework, LINN-S2 facilitates structured reasoning and enforces logical consistency over extended horizons. Finally, we explain the proposed inter-system feedback mechanism. The pipeline of the proposed DMWM framework is shown in Figure 1.
2.1 RSSM-based System 1
The RSSM-SI component is based on DreamerV3 [7]. which is represented by
Deterministic State:
h f (h--1, 2t,1 1,
Encoder:
21 ~ 9p (Zt
h4, 0t):
Stochastic State:
& ~ Pe (2g
h4)
(I)
Reward Predictor:
F. ~ Pep (Fr
h4 2c)
Decoder:
Or Pe (ōr h4, 2x);
with the deterministic state h1s observation o,, predicted observation o, stochastic state o. predicted stochastic state 2,, action a, and the predicted reward f, at time step t. RSSM-SI achieves effective balance between deterministic and stochastic states and enables efficient data-driven prediction similar to the intuitive and automatic processes of System 1. The deterministic state captures data patterns, and the stochastic state models inherent uncertainty and dynamics for complex environments.
System 1 loss. RSSM-S1 is optimized by using the loss function of DreamerV3 [7]:
Lsi (4p poed (1)) + Ty Layn (4p) + Sp Cup (4),
pped (vp) ln (ox 2t, h4) In Pep (re z2, he),
(2)
Ldyn (4p = KL [sg (gp (Zt hes or)) [p (zt [x)]s
Lrep (1p) KT. [gp (z+ he, oc) || || (pp (zx h+))],
where way and uap are respectively the weight factors of the dynamic loss Layn and the representation loss Crep, and sg(-) represents the stop-gradien gradient operator.
System 1 limitations. The RSSM framework can viewed as a computationa counterpart of the System 1 component in the dual-process theory of cognition [25] characterized by fast, intuitive-driven reasoning. However, it is inher- ently constrained by two critical limitations: the absence of explicit logical reasoning and the inability to maintain coherence over extended temporal horizons. While RSSM-S1 enables efficient and immediate response through pat- tern recognition and intuitive prediction, these strengths come at the cost of enforcing logical consistency or inferring causal relationships in complex scenarios. Moreover, RSSM-S1's 's fo- cus on short-term processing limits its ability to integrate information across long periods, resulting in fragmented or inconsistent predictions in tasks that demand global planning long-term foresight. These inherent limitations demonstrate the need to complement the capabilities of RSSM-S1 with mechanisms of structured reasoning and sustained temporal coherence, similar to System 2, to construct a more robust, cognitive world model framework.
Swhay Azg
ayhas
A-82
Los
ASS
cras
22'02
+22
11
a1
2/2
Q2
23
Figure 2: Logic reasoning for LINN-S2.
2.2 Logic-integrated neural network-based System 2
Next, we introduce the LINN-S2 component whose deep logical reasoning is shown in Figure 2. In LINN-S2, the states and actions are encoded as logic vector inputs, enabling logical deduction through operations such as negation (-), conjunction (A), disjunction (V), and implication (-). To establish these logical operations, the LINN framework [30] serves as the foundational module for
3

---

System 2. We propose novel hierarchical logical reasoning framework and extended regularization rules for implication operations capture and reason about structural relationships between the state and action spaces, thus endowing the world model with logical inference capability.
Neural modules for basic logic operations. In logical reasoning for world models, it is essential to uncover structural information and semantic relationships across different embedding spaces, specifically the action space and the state space. In [30], the authors proposed straightforward concatenation-base method for LINN. However, the approach of [30] is limited by the fact that it limits that imput vectors are from the same source space. Moreover, the straightforward concatenation- based method overlooks the semantic disparities between states and actions, risking the loss of critical information and failing to capture complex cross-space logical relationships.
To address the need for cross-space logical reasoning in world models, we propose to explore the action embeddings and apply the Kronecker product for cross-space feature alignment, which preserves logic integrity and captures second-order relationships. The (imagined) state logical embedding and the state logical embedding m are obtained with multilayer perception (MLP) by
W (Wi(=) + b,,,, m / (Wf(z @ a) b
where @ is the operation of vector concatenation, and @ a aims to capture the action logic within the state context. The formulations of operations (AND, OR, NOT) are respectively given by
AND(U, m) = f (WI O m) + b + Conv2D(u @ m, K() + by.
(4)
OR(e, m) Wif (WT(2 4 m) + b%) + Conv2D y @ m, KO) + bg.
(5)
NOT(v) = + f [W"'++ bw),
(6)
where V E Rd, m E RR, w E dx22d W E Raxd By E R b). E R are the parameters of the logical neural networks, I nd, Conv2D(.) represents the convolution neural network, K is the convolution core, @ is the operation of Kronecker product, and f(-) is the activation function.
Logical operations capture intricate logical correlations that cannot be fully encapsulated by the geometric properties of the embedding space. For instance, the logical independence NOT(u), reflected in the relationship between and -u, does not correspond precisely vector orthogonality. Logical operations are governed by logical axioms (c.g., identity, annihilator, and Complement), which impose algebraic constraints that transcend purely geometric interpretations, which are intro- duced in the logic regularization part.
Implication relationship for state reasoning. Based on the basic logical operations (AND, OR, NOT), we implement the implication operation (p.- + q,)) to enable logical reasoning within the state and action logical embedding spaces. The implication operation is critical for assessing the rationality a predicted state given the current imagination and action embeddings. which is derived through the equivalence relationship p q < "p V g. Hence, the operation IMPLY for - m is realized based on the fundamental operations of negation (-) and conjunction (A), represented by
IMPLY (v, m) = OR(NOT()),))) m).
(7)
Table 1: Partital Logical Regularizers and Rules for Implication -
Logical Rule
Equation
Logic Regularizer r;
Identity
w + T = T
T11 WEEW Sim (OR(T((((()))))) T), T)
Annihilator
w + = -w
F12 =
KwEW (OR(OO (w), F), NOT(w))
Idempotence Complement W + W = w
w + T
713 WEW (im(OO((OT((w)) w), T)
F14 wwe - (NO(T((((()))))) NOT NOT(w))
Logical regularizations. To ensure that neural modules accurately perform logical operations, the work in [31] introduced logic regularizers {ri} that enforce adherence to fundamental logical rules, thereby constraining module behavior. While neural networks can implicitly learn logical operations from data, the explicit incorporation of logical constraints improves model consistency, interpretability, and robustness while maintaining efficient neural computation. Derived from principles of AND, OR, and NOT, logic regularizers establish a unified inference framework with fundamental and advanced operations. This approach aligns model outputs with neural networks and formal logic, and enabling effective learning, representation, and inference of logical formulas for interpretable reasoning in world models.
reasoning, bridging

---

We extend the foundational framework of logical regularization in [30] by introducing implication regularizers to address advanced logical rules. As summarized in Table 1, the implication rules refine basic logical structures to handle intricate constructs. By explicitly encoding principles such as contraposition (w + = -U + -we and complement (w + -w = -w), LINN-S2 improves its capacity to manage compound logical expressions while ensuring consistency across implication operations. The extended logical regularizers for IMPLY (T11-114) are given in Table I, combined with the basic logical regularizers for AND, OR, and NOT (ri-r10), collectively define the regularization loss function Creg. N. Σ, Ti, where v; represents individual logical regularizers and N, presents the total number of the regularizers. The complete table of the logical regularizers given in Appendix G.
+
Proposed symbolic representation of hierarchical logical reasoning. We now present the symbolic representation of LINN-S2 for deep logical reasoning. Specifically, the logical reasoning process is formalized by using symbolic logic to represent logical relationships in sequences. By leveraging logical operations (A, +), LINN-S2 constructs hierarchical logical reasoning framework that facilitates systematic and interpretable reasoning over sequential dependencies. The framework of the hierarchical logical reasoning is illustrated in Figure 2, which includes three key procedures given as follows.
1) Local logical composition: Each (imagined) state logical embedding e, and action logical embed- ding my are combined by using the logical conjunction (^) to capture the intrinsic logic as
: A me,
Vt < T.
(8)
The composition et establishes localized logical features based on the existing i/ and me
2) Recursive implication reasoning: To ensure logical consistency across (imagined) states, each composition (c.) undergoes an implication operation subsequent (imagined) state
(-> that aligns the logical information with the
@:: : Ct - 2++1
Vt T.
(9)
The recursive formulation (9) encodes sequential dependencies and enforces consistency in predictive state transitions across the hierarchy. However, Φ: provides only single-step logical inference, and lacks reasoning depth and comprehensiveness. To address this limitation, propose incorporating deterministic historical information into the logical reasoning framework. The deep recursive implication reasoning is represented by
of :t-a - 1 A + 2441, Va < <T
(10)
where a is the inference depth. complex scenarios with long-term dependencies, the model utilizes logical relationships from past states to ensure coherent reasoning. Capturing logic across time steps retains historical information, strengthens sequential dependencies and enhances global consistency. This prevents inference bias from information loss during multi-step reasoning and ensures the reliability of long-term imagination.
3) Global logical chain: The global reasoning process integrates local consistency and recursive reasoning into unified global logical chain LT within the time period T
L = of A of A o ... 0- + T,
(11)
where T represents the consistency condition, ensuring that the (imagined) state sequences align with the reasoning objectives. This formulation consolidates local information into globally consistent reasoning framework.
In this way, LINN-S2 enhances the formal interpretation and mathematical rigor of world models along with several key features. First, the hierarchical structure encodes deep logical reasoning as a layered process, which allows logical consistency between state and action representations. Second, logical composition leverages logical operations for binding states and actions in a logical manner. This composition facilitates formal logic principles and enables modular, interpretable reasoning. Finally, the recursive implication ensures robustness and interpretability for long-term imagination.
System 2 loss. The loss function of the logic reasoning with inference depth is given by
bing
1
Σ Sim of T) 10-Sim(65 F)) T - 1 Σ Sim(d, T)-Simi Sim(m, (O,NO)))) (12)
T
5

---

where the function Sim is the logic similarity metric that takes value between and 1. In practice, T a randomized fixed vertor, and F is obtained by NOT(T). We use the cosine similarity since the logical information of action and state spaces has been aligned in the logical vector space, given by
Sim (v. m) = a (x((v m)/ (l
(13)
To ensure the logical consistency of basic operations of AND and OR by order-independence, i.e..
v A m m A e and v V m m V e, the imputs of AND and OR are randomly disrupted. Moreover, the fg-regularization term C, [30] is employed to prevent the vector lengths from exploding, which could otherwise lead to trivial solutions (e.g., logical rules becoming ineffective) during optimization, and constraint on the model parameters to mitigate the risk of overfitting, represented by
Li
Σ
10112
Σ
mmm11 + Σ /wl2:
(14)
NEV
MEM
wEW
where W is the model parameter of LINN-S2. The loss function of System is expressed as
Ls2(wr = Σ Ling + Pregfrey Beg Li2.
Sing
(15)
a=
where Breg and Be, represents the weight factors, and A denotes the maximum reasoning depth. 2.3 Inter-System Feedback Mechanism
Next, we develop novel inter-system feedback mechanism to enable communications of observation signals and logical signals between System and System 2.
Feedback from S1 to S2. During the real-environment interactions, LINN-S2 updates the domain- specific logical relationships by using the actual state transitions from RSSM-S1. Particularly, the state-action sequence {2t, at. *++1 V captured by RSSM-SI based on observations {or]T=o serves as labeled data, which is fed into LINN-S2 with the objective of minimizing the inference loss (12).
logical reasoning into RSSM-S1, we propose logic feedback that utilizes LINN-S2's logical consistency mechanisms to guide RSSM-S1. By unifying high-level logical structures with low-level representations, the proposed approach fosters a more robust and coherent world model for imagination. We particularly introduce the logical rules and rederive the variational evidence lower bound (ELBO) [5] with logic inference term by
Feedback from S2 to S1. To embed LINN-S2-inspil
T
Po(01.T Z1:T
(1:T) = II (o( 1=1
24 20) 1--100t1),
(16)
where (25 01-1, & (x( 21-1 C(é(z, 21
a-1)) and C(Φ) = Sim(Ø, T) is
the logical consistency function that measures whether the logical rule ø satisfied by imagination. The logical ELBO of the observation loss can be expressed as (Derivation in Appendix C)
T
In P.p (over (Ir))= = / Pup (oL:T, Fver (1:)) d2y. 2
(gg [In (or 2+)
Decoding
(17)
+ Fn [In C(b(z) a4-1,
Rap KT. (gp (zt
Oggs ace) 1pp (z+ 2111 11))))
Logic Inference
Prediction
where 41 9p (2x Oge: act): 92 9u (2--1 Oge-1, ac+-1), and the state priors can be approxi- mately obtained by past observations and actions qo (21:T) 01:7, d1.r) = II 9x (2, he, or).
With the proposed inter-system feedback, DMWM enables the agents to think in human-like, dual-praa duoc-ss cognitive way for more robust, reliable and long-term imagination.
3 Experimental Results and Analysis
In this section, we conduct extensive experiments to address the following key questions: (a) Can our model effectively capture logical relationships in dynamic environments?, (b) Does enhanced logical consistency enable our model to achieve higher task rewards under limited environment trials and data?, and (c) Over extended horizon, can our model generate reliable long-term imagination?.
Experimental setup. The training environments consist of continuous control tasks from DeepMind control (DMC) suite, 4 robotic tasks from ManiSkil12 platform, and 4 robotic tasks from

---

MyoSuite platform. We evaluate DMWM using two model-based decision makiing approaches: An actor-critic reinforcement learning method and gradient-based model predictive control (Grad- MPC) approach [11], referred to as DMWM-AC and DMWM-GD,, respectively. Training details for DMWM-AC and DMWM-GD are provided in Appendix D. For comparison purposes and benchmarking, we include Dreamer [7]. Dreamer-enabled Grad-MPC [11], and TD-MPC2 [13] as baselines. To highlight the limitations of using a single RSSM for world cognition, we compare our method against two state-of-the-ar RSSM variants: Hieros [3] and HRSSM [15]. HRSSM improves representation robustness via masking and bisimulation to mitigate visual noise interference, while Hieros enhances long-temn modeling and exploration efficiency through S5WM and hierarchical strategies. Details on environment and model settings are provided in Appendix E.
Logical consistency. Figure shows the logical correlations between individual state- action pairs (s,A aj) and the target state ($30). The diagonal shows strong correlations for one-step pairs 8; A Aa, - $30. which is the key path of localized reasoning. The off-diagonal elements show interdependency across differ- ent state-action pairs (s; A aj,i + j) that propagate global logical information. The ob- served patterns highlight the need for deep logical reasoning capture both short-term and long-term logical dependencies.
22
0.7782
Azzion
0.739
85
Table 2 compares the logical consistency of the proposed DMWM against various base- lines on DMC tasks. Logical consistency data for 20 tasks with different horizon sizes is provided in Appendix H.1. The proposed
0.739
41
55 530 515 520
Figure 3: Heatmap of deep logic correlations for se- quential imagination so A ag A ...829 A 0229 -> 830 with reasoning depth = 30. The horizontal axis indicates the past states s; and the vertical axis indi- cates past actions aj. The color of points represents the logical strength of long-term state-action pairs.
DMWM achieves state-of-the-art logical con- sistency in both mean logic loss and stability across all DMC environments. DMWM re- spectively achieves 14.3%, 2.6% and 3.3% improvement in logic consistency compared to Dreamer, Hieros, and HRSSM. For instance, while the masking and hierarchical strategies in Hieros and HRSSM can reduce single-step propagation errors by mitigating environment noise, they still have difficulty in addressing long-term imagination due to predictive deviation in statistical inference and error accumulation. This highlights the limitations of relying solely on System 1 and the need for logical consistency from System 2 for robust imagination.
Table 2: Performance comparison of our approach with various baselines on DMC tasks in terms of logic consistency. The mean and variance of logical consistency are reported over 100 test episodes with the horizon size H 30. Complete results of logical consistency over 20 tasks with varying horizon size are concluded in Appendix H.I
Env
Dreamer [7]
Hieros [3]
(15)
DMWM (Proopsed)
Cartpole Balance
0.683 i 0.057
0.711 + 0.032
0.713 1 0.041
0.727 1 0.023
Pendulum Swingup
0.611 + 0.137
0.709 + 0.054
0.699 0.079
0.730 + 0.037
Reacher Hard
0.608 + 0.121
0.702 + 0.062
0.703
+730000004
Finger Turn Hard
0.627 0.131
0.698 = 0.061
0.703
0.725+0.000
Cheetah Run
0.643 + 0.131
0.689 = 0.113
0.695+0.00
0.725 0.049
Cup Catch
0.652 + 0.087
0.701 + 0.072
0.714
0.728 1 0.021 0.021
Walker Walk
0.612 + 0.140
0.696 + 0.063
0.701 + 0.073
0.730 + 0.034
Quadruped Walk
0.656 + 0.092
0.701 + 0.067
0.703 + 0.072
0.723 + 0.039
Hopper Hop
0.633 + 0.127
0.704 + 0.087
0.701 + 0.092
0.722 + 0.038
Trial efficiency. Figure 4 presents test returns under limited environment trials. The number of environment trials serves as the x-axis to quantify exploration efficiency by measuring performance under the same environment exploration opportunities, which is a crucial factor for real-world applications and high-cost simulations. The complete results for 20 DMC tasks under limited environment trials are provided in Appendix H.2. The results show that the proposed DMWM- AC and DMWM-MPC significantly outperform baseline methods across most tasks with limited
7

---

environment trials, particularly in complex environments such as Cheetah Run and Finger Turn Hard, and Quadruped Run. contrast, Dreamer and GD-MPC exhibit limited performance in high- dimensional control tasks with lower learning efficiency and stability. As shown in Figure 4, DMWM approaches achieve an average 5.5-fold improvement in test return under limited environment trials compared to baseline methods. These observations highlight that leveraging logical information from the environment enhances exploration efficiency, especially in complex tasks, by enhancing DMWM's capability in long-term imagination.
Cartpole Valance
Finger turn basy
Hopper Stand
Welker Stand
003
8.0
690
620
Tea Rauum
003
650
090
420
203
200
200
200
605
620
D00
1000
2000
420
603
320
1000
200
4.00
620
DX0
2000
200
4000
500
000
1000
Envirooment Flads
Enviorment
Erverooment
Riiads
Enwiormenn Trials
CDAMFC
Figure 4: Performance comparison results on DMC tasks under environment trials that indicate the number of times that models explore the environments. The vertical axis indicates the average return over 100 test episodes. Complete results on 20 DMC tasks are concluded in Appendix H.2.
Finger Turn Easy
Quadruped Run
Reacher Hard
Hopper Hop
1000
1000
Not Patum
200
200
200
L0U
1.0
Endirommnnt Stops 2055
Ensironmennt Stapp 30s
Encrronmen Staps 105
Environment Stops 1.:5
MPC
HPCZ
Figure 5: Perfooraacce comparison DMC tasks under environment steps that indicate the number of environment interactions. The vertical axis denotes the average test return over 100 episodes. Complete test results on 20 DMC tasks are provided in Appendix H.3.
Cubee
Pick Cube
Stack Cube
Turn Foucet
100
120
LC0
75
75
75.
Nox: tatum
50-
50-
50-
23
25-
25
Erokunmmntt Staga
Ersincmmmet Stapp Juc
Staze
Ensiforment
Creamer
GD-MPC
Figure 6: Performance comparison of test results on ManiSkil12 robotic tasks under environment steps. The vertical axis denotes the average test return 100 episodes.
Data efficiency. Figures 5-7 present test results under limited environment steps for the purpose of quantifying the data efficiency. The complete results for 20 DMC tasks under limited environment steps are provided in Appendix H.3. DMWM approaches consistently outperform Dreamer and GD-MPC across most tasks in terms of convergence speed and final returns, particularly in high- dimensional dynamics such as Cheetah Run and Quadruped Run. These results highlight DMWM's superior long-term planning and dynamic modeling capabilities, enabling more efficient utilization of environment data. For tasks that require simple dynamic modeling like Cartpole Balance and Cup Catch, all methods converge rapidly. In contrast, for complex tasks, such as Reacher Hard and Hopper Hop, the proposed DMWM approaches achieve stable performance and an average improvement of 32% in test return under limited data compared to Dreamer and GD-MPC.

---

Twirl
100
100
1.00
1.00
Not Nathuuur
1/0
1.0
0.2
1.5
Stopo
Stops
Ervironment
DANAN-IC
DHWM-MPC
Dreamer
CDAPPC
TD-NFC2
Figure 7: Performance comparison of test results MyoSuite robotic tasks under environment steps. The vertical axis denotes the average test return over 100 episodes.
Cup
Sparse
Hopper
1000
1000
1020
400
820
Notur
200
Tea
400
430
100
200
200
200
40
20
80
20
80
60
Pendulum
Roacher
Reacher
Walker
1000
1000
1000
1000
800
810
8:0
800
Tot futur
6:0
420
200
200
200
40
20
Morizon
-d
Figure 8: Performance comparison on 8 DMC tasks across different horizon size. Complete results on 20 DMC tasks across varying horizon size provided in Appendix H.4.
Imagination ability over extended horizon size. As shown in Figure 8, although a long-horizon prediction introduces cumulative errors, DMWM-AC and DMWM-MPC consistently achieve high test returns across most tasks and remain stable performance even over extended horizons. In stability- critical tasks, such Cartpole Balance and Pendulum Swingup, DMWM approaches maintain strong performance across a wide range of prediction horizons, whereas Dreamer and GD-MPC are more susceptible degradation due to prediction errors. For extended horizon size of H 30 in complex control tasks, DMWM approaches achieve an average 120% improvement in test return compared to Dreamer and GD-MPC. These results emphasize the crucial role of logical reasoning in long-term imagination. Across most tasks, DMWM approaches demonstrate superior performance over long-term horizons.
Impact of logic inference depth. Figure presents test returns with inference depth a = 10,30, 50 and the complete results for 20 DMC tasks are provided in Appendix H.5. From Figure 9, we observe that a bigger reasoning depth a can produce performance gains long-horizon decision making along with diminishing marginal returns from increasing logical inference depth and increased computational overhead. Moreover, Figure also shows that the proposed deep logic inference can effectively capture the logic over the long-horizon trajectories for more robust imagination.
Related Works
World models are critical components model-based intelligent systems, which enable learning, reasoning and decision-making in complex environments [5, 6, 7, 32, 33, 34, 35, 36, 37]. By

---

Cup
Swingup
1000
1000
1030
900
Nox fattrm
000
830
020
700
600
630
620
300
20
40
60
20
40
60
80
20
50-
40
60
Horizon Sizc
Harizon Sizc
Horizon Siec
Horizon Size
+ -3
Figure 9: Performance comparison of results on DMC tasks under environment trials that indicate the number of times that models explore the environments. The vertical axis indicates the average return over 100 test episodes. Complete results on 20 DMC tasks are concluded in Appendix H.5.
simulating the environment, these models imagine future states, plan behaviors and optimize strategies without relying heavily on real-world trial and error. Existing frameworks for world modeling, such as RSSM [5, 6, 7]. generative models [32, 33, 34], and large language models (LLM) [35, 36, 37], have demonstrated varying strengths in tasks, such as prediction and efficient behavior planning. The Dreamer series [5, 6, 7], for instance, has advanced model- based reinforcement learning by improving task generalization and sample efficiency through latent space modeling and planning. Additionally, diffusion model-based world models [32, 33, 34] can generate diverse and high-quality outputs but cannot ensure long-term logical consistency. On the other hand, I.LM-based models [35, 36, 37] can perform limited logical reasoning and task decomposition but face challenges in dynamic environment modeling and resource consumption.
Furthermore, logic neural reasoning promising approach to enhance the generalization ability and long-term imagination of world models [38, 29, 39]. Previous research has explored logic neural networks and probabilistic logic [40, 30, 41], but these methods have struggled with dynamic enviromments and evolving logical variables. Our approach, by contrast, enables the automatical and implicit logie inference, thus providing superior generalization and robustness complex, dynamic task settings. Hence, in contrast to the existing world models, our work introduces a novel dual-mind framework that combines the efficient sampling RSSM with logic-driven reasoning enhance long-term imagination with logical consistency, thus addressing critical research gap for logic-driven robust world models.
Conclusion and Limitations
Conclusion. In this paper, we have proposed DMWM, novel world model framework for reliable long-term imagination inspired by the dual-process theory of human cognition. The proposed DMWM combines the fast, intuitive-driven S1 with the structured, logic inference-driven S2. We have designed an efficient inter-systent feedback mechanism that enhances the logic consistency and adaptability imagination trajectories. Extensive evaluations on DMControl and robotic tasks across diverse benchmarks demonstrated significant improvements in logical consistency, data-efficiency and reliable imagination over an extended horizon size.
Limitations and future work. A key limitation of our approach lies in its reliance on predefined simple domain-specific logical rules, which restricts its adaptability environments where such rules are ambiguous or constantly evolving. This dependency limits the framework's 's ability to generalize to novel tasks. Future work could focus on enabling the model autonomously learn and adapt logical rules from data by exploring causal relationships. This, in turn, can reduce the need for explicit manual definitions and enhance flexibility in dynamic and complex environments.
Acknowledgments
This work was supported by the US National Science Foundation under Grants CNS-2225511, IIS- 2509636, DBI-2412389, CMMI-2240402, IIS-2312794, and CCF-1918770. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the sponsor(s).
10

---

References
[1] Zhixuan Lin, Yi-Fu Wu, Skand Peri, Bofeng Fu, Jindong Jiang, and Sungjin Ahn. Improving generative imagination object-centric world models. In International conference on machine learning, pages 6140-6149. PMLR, 2020.
[2] Guangxiang Zhu, Minghao Zhang, Honglak Lee, and Chongjie Zhang. Bridging imagination and reality for model-based deep reinforcement learning. Advances in Neural Information Processing Systems, 33:8993-9006 2020.
[3] Paul Mattes, Rainer Schlosser, and Ralf Herbrich. Hieros: Hierarchical imagination on struc- tured state space sequence world models. arXiv preprint arXiv:2310.05167 2023.
[4] Lior Cohen, Kaixin Wang. Bingyi Kang, and Shie Mannor. Improving token-based world models with parallel observation prediction. arXiv preprint arXiv:2402. 00644000066,,... 2024.
[5] Danijar Hafner, Timothy Lillicrap. Jimmy Ba, and Mohammad Norouzi. Dream to control:
Learning behaviors by latent imagination. arXiv preprint arXiv:19/2.01600 2019.
[6] Danijar Hafner, Timothy Lillicrap, Mohammad Norouzi, and Jimmy Ba. Mastering atari with discrete world models. arXiv preprint arXiv:2010.022 109...0,.1....... 2020.
[7] Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse control tasks through world models. Nature, pages 1-7, 2025.
[8] Yucen Wang, Shenghua Wan, Le Gan, Shuai Feng, and De-Chuan Zhan. Ad3: Implicit action is the key for world models to distinguish the diverse visual distractors. arXiv preprint arXiv:2403.09976 2024.
[9] Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, and James Davidson. Learning latent dynamics for planning from pixels. In International conference on machine learning, pages 2555 2565. PMLR, 2019.
[10] Nicklas Hansen, Xiaolong Wang, and Hao Su. Temporal difference learning for model predictive control. arXiv preprint arXiv:2203. 04955, 04955
2022.
[11] Jyothir SV, Siddhartha Jalagam, Yann LeCun, and Vlad Sobal. Gradient-based planning with world models. arXiv preprini arXiv:2312. 2023.
[12] Yilun Du, Sherry Yang, Bo Dai, Hanjun Dai, Ofir Nachum, Josh Tenenbaum, Dale Schuurmans, and Pieter Abbeel. Learning universal policies via text-guided video generation. Advances in Neural Information Prucessing Systems, 36, 2024.
[13] Nicklas Hansen, Hao Su, and Xiaolong Wang. Td-mpe2: Scalable, robust world models for continuous control. arXiv preprint arXiv:2310.
2023.
[14] Qi Wang, Jumming Yang, Yunbo Wang. Xin Jin, Wenjun Zeng, and Xiaokang Yang. Making offline rl online: Collaborative world models for offline visual reinforcement learning. Advances in Neural Information Processing Systems,
01197203-972340 2024.
[15] Ruixiang Sun, Hongyu Zang. Xin Li, and Riashat Islam. Learning latent dynamic robust representations for world models. arXiv preprint arXiv:2405.06260 2024.
[16] Qi Wang, Zhipeng Zhang, Baao Xie, Xin Jin, Yunbo Wang, Shiyu Wang, Liaomo Zheng, Xiaokang Yang, and Wenjun Zeng. Disentangled world models: Learning transfer semantic knowledge from distracting videos for reinforcement learning. arXiv preprinl 1rX1v:2503.08751, 2025.
[17] Nan Rosemary Ke, Amanpreet Singh, Ahmed Touati, Anirudh Goyal, Yoshua Bengio, Devi Parikh, and Dhruv Batra. Modeling the long term future in model-based reinforcement learning. In International Conference on Learning Representations, 2018.
[18] Anthony Simeonov, Yilun Du, Beomjoon Kim, Francois Hogan, Joshua Tenenbaum. Pulkit Agrawal, and Alberto Rodriguez. A long horizon planning framework for manipulating rigid pointcloud objects. In Conference on Robot Learning, pages 1582-1601. PMLR, 2021.
[19] Joseph Clinton and Robert Lieck. Planning transformer: Long-horizon offline reinforcement learning with planning tokens. arXiv preprint arXiv:2409. 09513 2024.
[20] Nathan Lambert, Albert Wilcox, Howard Zhang, Kristofer SJ Pister, and Roberto Calandra. Learning accurate long-term dynamies for model-based reinforcement learning. In 202/ 60th IEEE Conference decision and control (CDC), pages 2880-2887, IEEE, 2021.
11

---

[21] Shiqian Li, Kewen Wu, Chi Zhang, and Yixin Zhu. On the learning mechanisms in physical reasoning. Advances in Neural Information Processing Systems, 35:28252-2888 22265, 2022.
[22] Jianping Zhu, Xin Guo, Yang Chen. Yao Yang, Wenbo Li, Bo Jin, and Fei Wu. Adaptive meta-learning probabilistic inference framework for long sequence prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 17159-17166, 2024.
[23] Artyom Sorokin, Nazar Buzun, Leonid Pugachev, and Mikhail Burtsev. Explain my surprise: Learning efficient long-term memory by predicting uncertain outcomes. Advances in Neural Information Processing Systems, 35:36875-36888, 2022.
[24] Jiajian Li, Qi Wang. Yunbo Wang. Xin Jin, Yang Li, Wenjun Zeng, and Xiaokang Yang. Open-world reinforcement learning over long short-term imagination. arXiv preprint drXiv:2410.03618, 2024.
[25] Jonathan St BT Evans and Keith E Stanovich. Dual-process theories of higher cognition: Advancing the debate. Perspectives psychological science, 8(3):223-241, 2013.
[26] Keith Frankish. Dual-process and dual-system theories of reasoning. Philosophy Compass, 5(10):914-926, 2010.
[27] Jonathan St BT Evans. In two minds: dual-process accounts of reasoning. Trends in cognitive sciences, 7(10):554 2003.
[28] Wenyue Hua and Yongfeng Zhang. System I+ system 2= better world: Neural-symbolic chain of logic reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 601-612, 2022.
[29] Shiqian Li, Yuxi Ma, Jiajun Yan, Bo Dai. Yujia Peng, Chi Zhang, and Yixin Zhu. A simulation- heuristics dual- proc-ss model for intuitive physics. arXiv preprint arXiv:2504.09544 2025.
[30] Shaoyun Shi, Hanxiong Chen, Weizhi Ma, Jiaxin Mao, Min Zhang, and Yongfeng Zhang. Neural logic reasoning. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pages 1365-1374, 2020.
[31] Hanxiong Chen, Shaoyun Shi, Yunqi Li, and Yongfeng Zhang. Neural collaborative reasoning. In Proceedings of the Web Conference 2021, pages 1516-1527,200
[32] Chang Chen, Yi-Fu Wu, Jaesik Yoon, and Sungjin Ahn. Transdreamer: Reinforcement learning with transformer world models. arXiv preprint arXiv:2202.00448 2022.
[33] Xiaofeng Wang, Zheng Zhu, Guan Huang, Xinze Chen, Jiagang Zhu, and Jiwen Lu. Drive- dreamer: Towards werl-reoddn world models for autonomous driving. arXiv preprint arXiv:2309.09777, 2023.
[34] Eloi Alonso, Adam Jelley, Vincent Micheli, Anssi Kanervisto, Amos Storkey, Tim Pearce, and François Fleuret. Diffusion for world modeling: Visual details matter in atari. arXiv preprint arXiv:2405. /2399, 2024.
[35] Kolby Nottingham, Prithviraj Ammanabrolu, Alane Suhr, Yejin Choi, Hannaneh Hajishirzi, Sameer Singh, and Roy Fox. Do embodied agents dream of pixelated sheep: Embodied decision making using language guided world modelling. In International Conference on Machine Learning, pages 26311-26333 PMLR, 2023.
[36] Lionel Wong, Gabriel Grand, Alexander K Lew, Noah D Goodman, Vikash K Mansinghka, Jacob Andreas, and Joshua B Tenenbaum. From word models to world models: Translating from natural language to the probabilistic language of thought. arXiv preprint arXiv:2306.12672. 2023.
[37] Jiannan Xiang, Tianhua Tao, Yi Gu, Tianmin Shu, Zirui Wang, Zichao Yang, and Zhiting Hu. Language models meet world models: Embodied experiences enhance language models. Advances in neural information processing systems, 36, 2024.
[38] Joao Ferreira, Manuel de Sousa Ribeiro, Ricardo Gonçalves, and Joao Leite. Looking inside the blx:k-box: Logic- explanations for neural networks. In Proceedings of the international conference principles of knowledge representation and reasoning, volume 19, pages 432-442, 2022.
[39] Tao Li and Vivek Srikumar. Augmenting neural networks with first-order logic. arXiv preprint arXiv: www...00600.0.. 06298, 2019.
12

---

[40] Meng Qu and Jian Tang. Probabilistic logic neural networks for reasoning. Advances in neural information processing systems, 32, 2019.
[41] Ryan Riegel, Alexander Gray, Francois Luus, Naweed Khan, Ndivhuwo Makondo, Ismail Yunus Akhalwaya, Haifeng Qian, Ronald Fagin, Francisco Barahona, Udit Sharma, et al. Logical neural networks. arXiv preprint arXiv:2006. 13155, 2020.
[42] Quentin Garrido, Mahmoud Assran, Nicolas Ballas, Adrien Bardes, Laurent Najman, and Yann LeCun. Learning and leveraging world models in visual representation learning. arXiv preprint arXiv:2403.00500 2024.
[43] Aidan Scannell, Mohammadreza Nakhaei, Kalle Kujanpää, Yi Zhao, Kevin Sebastian Luck, Arno Solin, and Joni Pajarinen. Discrete codebook world models for continuous control. arXiv preprint arXiv:2503.00655 2025.
[44] Yann LeCun. A path towards autonomous machine intelligence version 0.9. 2, 2022-06-27.
Open Review, 62(1):1-62, 2022.
[45] Weirui Ye, Shaohuai Liu, Thanard Kurutach, Pieter Abbeel, and Yang Gao. Mastering atari games with limited data. Advances in neural information processing systems, 34:25476-25488, 2021.
[46] Julian Schritwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Si- mon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, et al. Mastering atari, go, chess and shogi by planning with a learned model. Nature, 588(7839))604-600 2020.
[47] Michel Ma, Tianwei Ni, Clement Gehring, Pierluca D'Oro, and Pierre-Lue Bacon. Do trans- former world models give better policy gradients? arXiv preprint (ttb:2402205299 2024.
[48] Jan Robine, Mare Höftmann, and Stefan Harmeling. Simple, good, fast: Self-supervised world models free of baggage. In The Thirteenth International Conference Learning Representations, 2025.
[49] Maxime Burchi and Radu Timofte. Learning transformer- based world models with contrastive predictive coding. arXiv preprint 26344:2503.04416, 2025.
[50] Jonathan Feldstein, Paulius Dilkas, Vaishak Belle, and Efthymia Tsamoura. Mapping the neuro- symbolic ai landscape by architectures: A handbook on augmenting deep learning through symbolic reasoning. arXiv preprint arXiv:2410. 22077, 2024.
[51] Chaojun Ni, Guosheng Zhao, Xiaofeng Wang. Zheng Zhu, Wenkang Qin, Guan Huang, Chen Liu, Yuyin Chen, Yida Wang, Xueyang Zhang, et al. Recondreamer: Crafting world models for driving scene reconstruction via online restoration. arXiv preprinl arXiv:241 1. 1.19548, 2024.
[52] Sicheng Zuo, Wenzhao Zheng, Yuanhui Huang, Jie Zhou, and Jiwen Lu. Gaussianworld: Gaussian world model for streaming 3d occupancy prediction. arXiv preprint arXiv:2412.1037 2024.
[53] Chenyang Cao, Yucheng Xin, Silang Wu, Longxiang He, Zichen Yan, Junbo Tan, and Xue- qian Wang. Fosp: Fine-tuning offline safe policy through world models. arXiv preprint arX1v:2407.04992. 2024.
[54] Yang Yue, Yulin Wang, Haojun Jiang, Pan Liu, Shiji Song, and Gao Huang. Echoworld: Learning motion-aware world models for echocardiography probe guidance. arXiv preprint arXiv:2504. /3065, 2025.
[55] Yichao Liang, Nishanth Kumar, Hao Tang, Adrian Weller, Joshua B Tenenbaum, Tom Silver, João F Henriques, and Kevin Ellis. Visualpredicator: Learning abstract world models with neuro-symbolic predicates for robot planning. arXiv preprint arXiv:2410.23/55 2024.
[56] Leonardo Barcellona, Andrii Zadaianchuk, Davide Allegro, Samuele Papa, Stefano Ghidoni, and Efstratios Gavves. Dream to manipulate: Compositional world models empowering robot imitation learning with imagination. arXiv preprint arXiv:2412. 14957, 2024.
[57] Ignat Georgiev, Varun Giridhar, Nicklas Hansen, and Animesh Garg. Pwm: Policy learn- ing with multi-task world models. In The Thirteenth International Conference on Learning Representations, 2025.
[58] Richard Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018.
13

---

[59] Kenneth R Muske and James B Rawlings. Model predictive control with linear models. AlChE Journal, 39(2):262 1993.
[60] Sanket Kamthe and Mare Deisenroth. Data-efficient reinforcement learning with probabilistic model predictive control. In International conference on artificial intelligence and statistics, pages 1701-1710. PMLR, 2018.
[61] Zhe Wu, David Rincon, Quanquan Gu, and Panagiotis D Christofides. Statistical machine learning in model predictive control of nonlinear processes. Mathematics, 916):1912, 2021.
[62] Vincent Micheli, Eloi Alonso, and François Fleuret. Efficient world models with context-aware tokenization. arXiv preprint arXiv:2406. 19320, 2024.
[63] Siyuan Zhou, Yilun Du, Jiaben Chen, Yandong Li, Yan Yeung, and Chuang Gan. Ro- bodreamer: Learning compositional world models for robot imagination. arXiv preprint UrXiv:2404.12377, 2024.
[64] Philipp Wu, Alejandro Escontrela. Danijar Hafner, Pieter Abbeel, and Ken Goldberg. Day- dreamer: World models for physical robot learning. In Conference robot learning, pages 2220- PMLR, 2023.
[65] Jessy Lin, Yuqing Du, Olivia Watkins, Danijar Hafner, Pieter Abbeel, Dan Klein, and Anca Dragan. Learning to model the world with language. arXiv preprint arXiv:2308.01 01399, 2023.
[66] Yunhai Feng, Nicklas Hansen, Ziyan Xiong, Chandramouli Rajagopalan, and Xiaolong Wang. Finetuning offline world models in the real world. arXiv preprint arXiv:2310.1660 2023.
[67] Ignat Georgiev, Varun Giridhar, Nicklas Hansen, and Animesh Garg. Pwm: Policy learning with large world models. arXiv preprint -xXix22407002466 2024.
[68] Anthony Hu, Lloyd Russell, Hudson Yeo, Zak Murez, George Fedoseev, Alex Kendall, Jamie Shotton, and Gianluca Corrado. Gaia-1: A generative world model for autonomous driving. arXiv preprint arXiv:2309. 17800, 2023.
[69] Lunjun Zhang, Yuwen Xiong, Ze Yang, Sergio Casas, Rui Hu, and Raguel Urtasun. Learning unsupervised world models for autonomous driving via discrete diffusion. arXiv preprint 0rXiv:2311.01011 2023.
[70] Marc Rigter, Jun Yamada, and Ingmar Posner. World models via policy-guidde trajectory diffusion. arXiv preprint arXiv:23/2.08533 2023.
[71] Zihan Ding, Amy Zhang, Yuandong Tian, and Qinqing Zheng. Diffusion world model. arXiv preprint arXiv:24022 035700 2024.
[72] Jake Bruce, Michael D Dennis, Ashley Edwards, Jack Parker-Holder Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, et al. Genie: Generative interactive environments. In Forty-first International Conference on Machine Learning, 2024.
[73] Haoyu Zhen, Xiaowen Qiu, Peihao Chen, Jincheng Yang, Xin Yan, Yilun Du, Yining Hong, and Chuang Gan. 3d-vla: A 3d lision-lanuuage-action generative world model. arXiv preprint urXiv:2403.0963 2024.
[74] Shibo Hao, Yi Gu, Haodi Ma. Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and Zhit- ing Hu. Reasoning with language model is planning with world model. arXiv preprint arXiv:2305. 14992, 2023.
[75] Jialong Wu, Shaofeng Yin, Ningya Feng, Xu He, Dong Li, Jianye Hao, and Mingsheng Long. ivideogpt: Interactive videogpts are scalable world models. arXiv preprint MXXi:2405.15222 2024.
[76] Jiannan Xiang, Guangyi Liu, Yi Gu, Qiyue Gao, Yuting Ning, Yuheng Zha, Zeyu Feng, Tianhua Tao, Shibo Hao, Yemin Shi, et al. Pandora: Towards general world model with natural language actions and video states. arXiv preprint 0rX55:2400009455, 2024.
[77] Samy Badreddine, Artur d' Avila Garcez, Luciano Serafini, and Michael Spranger. Logic tensor networks. Artificial Intelligence, 303: 103699, 2022.
[78] Prithviraj Sen, Breno WSR de Carvalho, Ryan Riegel, and Alexander Gray. Neuro-symbolic in- ductive logic programming with logical neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pages 2212-8219, 2022.
14

---

Contents of Appendix
A Impact Statement
16
Background
16
B.1 Actor-Critic
16
B.2 Model Predictive Control
16
Derivation of Logical ELBO for Observation Loss
17
D Algorithms
18
D.I DMWM With Actor-Critic
18
D.2 DMWM With MPC
19
E Hyperparameters
19
E.1 Environment Setting
19
E,2 Model Setting
20
F Further Related Work
21
F.I World Model
F22 Logic Neural Reasoning
Complete Logic Regularization and Rules Table
23
H Additional Experiments
23
H.l Logical Consistency
................................
23
H.2 Trial Efficiency
25
H.3 Data Efficiency
26
H.4 Long-term Imaginations Over Extended Horizon Size
27
H.5 Impact of Logic Inference Depth Over Extended Horizon Size
28
15

---

A
Impact Statement
Inspired by the human cognitive model, our work proposes a novel DMWM architecture that integrates intuitive RSSM-S1 with logic-driven LINN-S2 for the first time, which addresses long-horizon imagination for model-based RL and MPC. By enhancing logical consistency of the architecture, DMWM improves both efficiency and robustness in complex tasks. Because of these benefits, DMWM provides a practical and interpretable solution for real-world applications such as autonomous driving and robotic planning.
Moreover, this work presents a generalized dual-mind framework for world models that can serve solid foundation for future rescarch general world models. We believe this approach marks a meaningful step toward artificial general intelligence (AGI) by bridging intuitive processes with logical reasoning. Hence, DWMW paves the way for more robust, logical and human-like decision- making systems.
B
Background
B.1
Actor-Critic
The actor-critic model is a widely used algorithmic framework in Reinforcement Learning (RL), which combines the advantages of policy gradient methods and value function approximation. It addresses RL tasks through the collaborative learning of two primary components: the Actor (policy network) and the Critic (value network), given follows:
Action model:
0, 2o (a- (87)
(18)
Value model:
(s = By((s,)
In the world Model, the target of the actor-critic model is to maximize the reward over the imagined trajectories with the horizon size H. Specifically, the action model seeks to maximize estimated value, while the value model strives to accurately predict the value estimate, which evolves as the action model updates. The training target of the Actor and the Value given follows [58, 5, 11].
+++ Σ
0r = max E
V3(++)
(19)
9
H+
30 min EE
99:99
Σ
(up(s)) v2())))²
(20)
10
#11
V((a+) = =(1
Σ
- n
(8+) + + (87)
(21)
VIN (sT) E Ego.gs
Σ
In + - (
(22)
where h = min(T + k., t + H), & represents the parameters of the Actor &e represents the parameters of the Critic, and À is the discount factor.
B.2 Model Predictive Control
Model predictive control (MPC) an optimization- based control method extensively applied in engineering and industrial control systems [13, 59, 60, 61]. By leveraging the system's 's dynamic model, MPC predicts future behavior through rolling optimization, generating optimal control inputs at each time step to achieve the desired system objectives. However, due to the fact that MPC strongly relies on the system model and the requirement for online optimization, has difficulty in effective decision decision-making performance if the world model fails to provide stable and reliable
16

---

imagined trajectories [9, 11]. The gradient- MPC framework [1 [] for world model is presented
++
G max RO),
R = Σ
E
(gslrr |
(23)
(a) Gtt+
T=H+1
{
(1)
J
diaggo)))
(24)
1=1
++
b++H++1 ~ go(or
(
(1--1
(
II Po(or
- 1
(25)
r=++1
a bi+n
(
01
VR
(26)
()) where autH and ( the timestep + H.
56++ are sequences of states and actions of the candidate j from the timestep to
C Derivation of Logical ELBO for Observation Loss
The ELBO with logic rules for the observation loss can be derived by
/
Pe (our (a.T)) Pp (op(o 2u.T
(1:T) dzy.
Po 21.7 a1.))
9p (31:T 01. 01:r)
Po 2e)p ( a=-1)
In (nimllaimm
a0(2 Oge, age)
2
Σ In pop 2)) + Inpo s0-1,a--1)
+ 2-,, 11-1) In lnq as)))
(z1loraaa))) [m (or 2x) + E, qu (zilognaze) [In 21 11)1
Docoding
Logic Inforence
E
2 (=--1 (
KL 1q (2) oge, ase) Ilpo (2t 2-11
4-1))
Prediction
(27)
where C(ò(zz: (2-1,4-11) = Sim(0( 21-1, a1-1), T) is the logical consistency function that measures whether the logical rule 2--1 ac-1 - Zt is satisfied by imagination.
Appendices continue next page
17

---

D Algorithms
D.1 DMWM With Actor-Critic
The training process of DMWM with actor-critic- criticcrri- decision module is shown in Algorithm
Algorithm 1 DMWM With Actor-Critic
Hyper Parameters: seed episode S, training episodes N, batch size B, collect interval C, sequence length L, imagination horizon H, learning rate 1).
Initialize dataset D with S seed episodes.
Initialize DMWM parameters Ø; w. Initialize Actor-Critic parameters 0, v/.
for Training step n - do for Collect interval -> C do
// System Training
Sample B Sequences {(or, at, Te) D.
Compute he = (1i- Zt ar-1).
Predict & ~ Ppp (2+ he) Or ~ Pe (ōr z).
Update S1 4- 1b Th V (v))
// System 2 Training
Learn Logic Regularizations Leeg // S1's Guidance on S2 Based on Truth {(hs at hx41))
Compute L = Φी A Φ5 A Φ3 ..----r-
Update S2 w +- w Tu V Lsi (w))
// Actor- CriitCC Training
Imagine {(zr, ) from each 2t.
[++
Predict rewards (r Rt,, 27) and values (87).
Compute value estimates (s(s)).
Update 0 9 + n/ V Σ 54+ (8+).
Update a + 인/ ny V Eit (( (a(s)) )| I/ S2's Guidance on S1 Based on Imagination ((2,, a,))
Compute Logic Consistency of {(2,, a+)) With T.A.
Update S1
ui + 3b - no V (t))
end for
// Real Environment Interaction & Data Collection Start environment enc.reset()
for Time step t + T do
Compute h = ff (ht-1, 34- 2- 01-). Compute x, 9 (z, h4p o.)
Obtain a go (a: 21) from decision-making model.
Interact 0t+1 + env.step(a.).
end for
Add experience to dataset D + DU (or, at, TT)
end
18

---

D.2 DMWM With MPC
The training process of DMWM with Grad-MPC for decision-making [11] is shown Algorithm
Algorithm 2 DMWM With Grad-MPC
Hyper Parameters: iterations I, candidate Size J, learning rate T)-- Initialize dataset 'D with seed episodes.
Initialize DMWM parameters Φ; w. for Training step n -> N do
// Real Environment Interaction & Data Collection Start environment env.reset()
for Time step t + 1 do
Compute h = fpp (h--1, ( at-1). Compute 2 9p (2t he, 01)
// MPC Decision Making
Sample Actions
()) ()) H
~ NNW,, diag(o?)).
for Iteration - do for Candidate sequence j + J do
+ (8t. 01t "1:1-1111111
(1)
(5) (5)
po(or 6 (
RG) Σ T-t+1
+++++ E [p(r+ 8 }.
Update Action 1447 end for end for
6.t4 1R V
V
Gbu+H = max (9) R(5)
Interact rx, Oc+1 + env.step(a.).
end for
Add experience to dataset D <- DU ((0,, at T
end
E Hyperparameters
E.1
Environment Setting
The action repeat setting for different DMControl tasks and robotic tasks is presented in TABLE 2.
Table 3: Action Repeat Setting and Action Dim
Eny
Task
Action Dim
Action Repeat
Cartpole Swingup
Pendulum Swingup
6
Reacher Easy
4
Finger Spin
2
DMC
Cheetah Run
4
Cup Catch
2
6
Walker Walk
2
Quadruped Walk
12
2
Hopper Hop
2
Key Turn Hard
39
1
Object Hold Hard
1
MyoSuite
Pen Twirl
39
Reach Hard
1
Lift Cube
2
Pick Cube
2
ManiSkill2
Stack Cube
2
Turn Faucet
7
2
19

---

E.2
Model Setting
The hyperparameters of models are presented in TABLE 3.
Table Hyperparameter Setting
Parameter
Symbol
Value
Dual-Mind World Model (General)
Replay memory size
le6
Batch size
B
50
Sequence length
L
64
Seed episode
S
5
Training episodes
N
1e3
Collect Interval
C
100
Max episode length
500
Exploration noise
0.3
Imagination horizon
H
30
Gradient clipping
100
RSSM-S1
Activation function
Relu
Embedding size
1024
Hidden size
200
Belief size
200
State size
30
Overshooting distance
50
Overshooting KL-beta
0
Global KL-beta
0
overshooting reward scale
0
Free nats
3
Bit-depth
5
Weights
dya
Optimizer
Adam
Adam epsilon
le-4
Learning rate
7/e
le-3
LINN-S2
Reasoning depth
α
30
Logic vector size
|o]. m
64
L.2 weight Regularization weight
Big
le-5
Preg
1
Logic MLP number
3
Optimizer
SGD
Learning rate
The
le-2
Actor-Critic [5]
Return lambda
X
0.95
Planning horizon discount
0.99
Optimizer
Adam
Adam epsilon
le-4
Learning rate
170. Th
le-4
Grad-MPC [1T]
Iterations
7
40
Candidate Size
J
1000
Learning Rate
//
0.1 0.1-0.01-0.005-00.000
TD-MPC2 (refer to [13])
20

---

F
Further Related Work
E1
World Model
World models are fundamental building blocks for model-based intelligent systems to make decisions, learn, and reason in complex environments. enables prediction, efficient behavior planning through environment simulation, and strategy optimization using virtual simulations, thus reducing reliance on real-world trial and error [5, 7. 62, 63J. Existing world modeling frameworks can be categorized as recurrent state space models (RSSMs) [2, 5, 6, 7, 10, 64, 65, 13, 66, 3, 15, 67], generative models [32, 33, 68, 69, 70, 34, 71, 72, 12, 73], and large language model (LLM) [35, 36, 74, 75, 37, 76].
The Dreamer series [5, 6, 7] lays an important foundation for general model-based reinforcement learning (MBRL) by continuously improving sample efficiency and task generalization ability through dynamic modeling and planning in the latent space. To adapt the RSSM-based world model for real- world environments, [2] optimized confidence and entropy regularization for the gradient to mitigate discrepancies between virtual simulations and real-world environments. [1] studied an object-centric world model, and introduced an object-state recurrent neural network (OS-RNN) to follow the object states. [12] extended multimodal RSSM to support joint text and visual inputs. [7, 13, 67] explored the multiple tasks with RSSM-based world models. [13] utilized SimNorm to sparse and normalize the potential states, which projected the latent representations to simplices of fixed dimensions, thus mitigating the gradient explosion and improving the training stability. [15] introduced masking- based latent reconstruction with a dual branch structure to handle the exogenous noise in the complex environment. [3] introduced the hierarchical imagination with parallel processing and proposed efficient time-balanced sampling.
The diffusion model-based world model offers high-quality and diverse generations with tem- poral smoothness and consistency, achieved through denoising and time inversion processes [32, 33. 68, 69, 70, 34, 12]. However, its reliance on multi-step denoising significantly slows downsampling, inference, and computation compared to RSSM, making it less suitable for real- time tasks and challenging to maintain long-term consistency [68, 69, 70, 12]. Additionally, unlike RSSM's ' latent space modeling, diffusion models cannot extract task-relevant features and accurately capture environment dynamics, particularly in complex environments with long-tailes distributions [32, 68, 70, 71, 12]. The LLM--aseed world model serves as powerful tool for complex task planning and execution through its logical reasoning capability and dynamic controllability [35, 36, 74], and able to realize task decomposition and cross-domain knowledge transfer through natural language instructions [76]. However, the model cannot perform in dynamic modeling, and the generated states and actions often cannot accurately reflect the dynamic changes in the environment. In addition, its high reliance on linguistic representation may lead to insufficient quality of modal alignment, posing the risk of information loss misinterpretation, while the significant consumption of compu- tational resources during training and inference limits the application of LLM-based world models in essurce-constrriied environments [35, 36, 74, 75].
Unlike all of the aforementioned works, we focus on the long-term imagination of world models and proposes to enable the logical consistency of state representations and predictions over an extended horizon for the first time. We retain the efficient sampling and representation capabilities of RSSM as System 1 and integrate logic-integrated neural network (LINN) as System 2 to imbue the world model with logic reasoning capabilities. The proposed DMWM thus delivers reliable and efficient long-term imagination with logical consistency, addressing critical research gap in existing approaches.
F.2 Logic Neural Reasoning
Logic neural reasoning enhances the logical consistency, long-term imagination, and generalization ability of world models by embedding logical rules and reasoning capabilities [38, 29]. By combining the interpretability of symbolic logic with the expressive power of neural networks, it provides robust and rapid reasoning for complex task planning [39, 40, 30, 41, 31, 77, 78]. However, explicitly modeling the logical structure of the world presents significant challenges due to the inherent ambiguity of logical rules, the prevalence of noisy data, and the dynamic and complex nature of logical interactions. [39] converted first-order logic rules into computational graphs for neural networks, enhancing their learning capabilities in low-data environments while providing limited support for complex temporal logic. [40] integrated Markov logic networks with knowledge graph
21

---

embeddings to address uncertainty in logical reasoning. In a logical neural network (LNN) [41], neurons are mapped neurons to logical formulas, enabling each neuron function logical operator. However, L.NS rely predefined and static logical rules, facing limitations in generalization when confronted with uncertainty or evolving logical variables. The work [78] introduced inductive logic programming and automatically learned logic rules. However, the aforementioned approaches make hard to capture the environment dynamics and extend the long-term imagination capability of the world model. In contrast, the concept of LINN [30, 31] introduces paradigm shift by dynamically constructing computational graphs and employing neural modules to learn logical operations. Thus, LINN can automatically infer implicit logical rules, circumventing the need for explicit specification. By integrating neural flexibility with logical rigor [30, 31], LINN achieves superior generalization capacity and enhanced robustness noise, making it well-suited for reasoning tasks in dynamic and complex environments.
In this work, we enhance the long-term imagination capabilities of world models through logical reasoning, critical yet unexplored area in world model research, with significant implications for creating advanced and general artificial intelligence (AGI).
Appendices continue on next page-
22

---

G
Complete Logic Regularization and Rules Table
The complete logic rules and the corresponding regularizers are given TABLE 5 [30] for negation
7, conjunction A, disjunction V and implication -,
Table Complete Logical Regularizers and Rules for System 2
Logical Rule
Equation
Logic Regularizer
Negation
W + T T
T1 () Sim (), W)
Negations
- (--W = w
12 WEW (OimNNOTNNOT (w)): w)
Identity
W A T
F3 WEW 1 -Sim(AND (u) T). w)
A Idempotence
Annihilator
W A T
T4
WEW 1 -Sim(AND( W., F),
W A F =
WEW 1
Sim W. w)
Complement
W A W
T6 NEW 1
Sim(AND(ww (OT(w)), F)
Identity
MI V F
17
WEW 1
Sim OR F)
Annihilator
W V T T
T8
NEW
Sim OR W, T),
Idempotence
W V
NEW 1 Sim OR W1, w), W)
Complement
W. TW T
T10
WEWW Sim (OR (w NOT ( T
Identity
W + T
F11
WEW
(OO(OR(NOT(() T),
Annihilator
+ F
T12
NEW
Sim (On(O(OOT(()) F), NOT(w)))
Idempotence
W0 + w = T
T13
WEW
Sim(OR (OT))RO)()))))))) T)
Complement
W W
T14
Sim (im(O NOT NOT(W))
H
Additional Experiments
H.1
Logical Consistency
Table 6: Logical consistency comparison between our proposed DMWM and various RSSM baselines
across 20 DMC tasks. We report the mean and variance of logical consistency from imagination 100 test episodes with the horizon size of 10, 30, 50, 100.
Env
H
Dreamer
Hieros
HRSSM
DMWM (Ours)
10
0.713 = 0.031
0.730 0.012
0.722 0.017 0.017
0.733 = 0.007
30
0.667 + 0.063
0.704 +: 0.032
0.712 2112:0.00 0.044
0.731 + 0.017
Acrobot Swingup
50
0.568 + 0.129
0.692 0.058
4.687+00083
0.715 + 0.032
100
0.485 + 0.167
0.672 + 0.112
0.651 0.132
0.699 0.078
10
0.721 0.023
= 1730000.000
0.729 0.011
0.730 0.008
30
0.683 + 0.057
0.711 + 0.032
0.713 + 0.041
0.727 + 0.023
Cartpole Balance
50
0.574 0.112
0.687 + 0.084
0.695 0.072
0.717 0.045
100
0.491 + 0.171
0.663 0.142
0.655 0.121
0.701 + 0.092
10
0.705 = 0.132
0.719 = 0.032
0.034
0.722 = 0.026
Cartpole Balance Sparse
30
0.602 = 0.167
0.682 : 0.101
0.693 0.081
0.695 = 0.093
50
0.432 + 0.203
0.632 + 0.178
0.643 + 0.162
0.652 + 0.135
100
0.321 + 0.286
0.571 + 0.232
0.589 + 0.213
0.603 + 0.182
10
0.719 + 0.031
0.729 0.011
0.723 0.023
0.730 + 0.011
30
0.678 0.062
0.698 + 0.043
0.703 + 0.082
0.723 0.032
Cartpole Swingup
50
0.563 + 0.091
0.672 + 0.124
0.662 = 0.145
0.702 + 0.078
100
0.474 0.158
0.621 + 0.191
0.602 = 0.191
0.672 0.152
10
0.703 = 0.142
0.717 + 0.036
1.15500003
0.720 0.030
Cartpole Swingup Sparse
30
0.613 = 0.187
0.669 = 0.121
0.671 = 0.098
0.699 = 0.087
50
0.443 = 0.213
0.621 + 0.185
0.621 + 0.173
0.669 + 0.121
100
0.403 + 0.252
0.532 + 0.241
0.559 + 0.231
0.627 + 0.162
10
0.709 0.085
0.723 1 0.031
0.721 = 0.023
0.730 0.016
30
0.643 + 0.131
0.689 + 0.113
0.695 + 0.087
0.725 + 0.049
Cheetah Run
50
0.527 0.165
0.651 + 0.147
0.667 0.131
0.703 0.092
100
0.428 + 0.221
0.606 = 0.186
1627
0.676 + 0.142
10
0.711 + 0.049
0.728 + 0.010
0.726 + 0.014
0.732 0.008
30
0.652 + 0.087
0.701 0.072
0.714 0.061
0.728 + 0.021
Cup Catch
50
0.534 + 0.113
0.681 + 0.098
0.683 + 0,112
0.712 0.053
100
0.465 + 0.181
0.647 + 0.182
0.633 + 0.172
0.692 - 0.112
23

---

10
0.772
0.727 0.011
0.728
0.733 0.009
30
0.64
0.705
0.752
0.011
Finger Spin
50
0.117
0.687
0.679
0.710
100
0.435 0.211
0.636 0.165
0.645
1.84400122
0.708
0.098
0.726 0.013
0.724
0.018
0.732 + 0.015
30
0.634 + 0.113
0.702 + 0.051
0.705 + 0.067
0.728 + 0.023
Finger Turn Easy
50
0.512 = 0.157
0.681 = 0.117
0.682 0.132
0.712 0.067
100
0.412 = 0.191
0 0.194
1617=0.199
0.683 = 0.148
10
0.704 1004:0.00 0.118
0.723 0.014
0.724=0.022
0.731 0.011
30
0.627 + 0.131
0.698 0.061
0.703 = 0.073
0.725 0.029
Finger Turn Hard
50
0.162
0.673 - 0.112
0.673 + 0.152
0.705 + 0.073
100
0.385 + 0.231
0.623 + 0.201
0.601 = 0.205
0.675 + 0.142
10
0.703 + 0.092
0.729 + 0.023
0.726 + 0.023
0.731 + 0.017
0.633 0.127
0.704 0.087
0.701 0.092
0.722 + 0.008
Hopper Hop
50
0.506 + 0.191
0.664 + 0.132
0.673 + 0.132
0.698 + 0.092
100
0.407 + 0.237
0.612 0.237
0.623 = 0.201
0.689 = 0.143
10
0.709 = 0.056
0.728 = 0.021
0.725 = 0.019
0.732 = 0.013
30
0.645 = 0.112
0.699 = 0.057
0.704 0.081
0.724 = 0.039
Hopper Stand
50
0.523 + 0.168
0.671 + 0.121
0.689 = 0.137
0.703 + 0.080
100
0.421 + 0.197
0.632 + 0.211
0.642 + 0.189
0.692 + 0.139
10
0.715 = 0.054
0.728 = 0.017
0.719 0.023
0.732 0.015
30
0.611 + 0.137
0.709 + 0.054
0.699 + 0.079
0.730 0.037
Pendulum Swingup
50
0.526 + 0.161
0.684 0.112
0.664 + 0.146
0.721 + +0.088
100
0.359 + 0.224
0.641 + 0.198
0.612 0 0.204
0.686 + 0.131
10
0.718 = 0.042
0.727 = 0.023
=
0.731 = 0.013
30
0.642 = 0.107
0.701 = 0.072
0.726 = 0.042
Quadruped Run
50
0.556 0.143
0.679 = 0.135
0.683 1 0.137 0.137
0.705 = 0.078
100
0.398 + 0.217
0.644 + 0.193
0.629 + 0.184
0.682 + 0.129
10
0.719 = 0.045
0.728 = 0.027
0.732 = 0.011
30
0.656 = 0.092
0.701 0.067
0.703 0.072
0.723 = 0.039
Quadruped Walk
50
+ 0 0.129
0.682 + 0.114
0.685 + 0.132
0.701 + 0.082
100
0.413 = 0.212
0.654 0.183
0.634 0.199
0.687 0.137
10
0.711 + 0.065
0.729 + 0.012
0.724 0.013
0.732 + 0.008
30
0.634 0.102
0.706 = 0.061
0.705 0.067
0.731 0.031
Reacher Easy
50
0.464 = 0.167
0.675 = 0.104
0.678 0.121
0.720 0.075
100
0.373 = 0.221
0.631 = 0.175
0.614 0.163
0.705 = 0.125
10
0.712 0.072
0.729 0.013
0.724
0.014
0.732 = 0.010
30
0.608 + 0.121
0.702 = 0.062
0.703 1703=00 0.072
0.730 0.042
Reacher Hard
50
0.501 = 089
0.663 = 0.114
0.674 0.132
0.717 0.082
100
0.349 + 0.233
0.602 - 0.183
0.607 = 0.172
0.701 + 0.165
10
1.702.0.11
0.720 0.023
0.713 0.027
0.729 + 0.0013
30
0.554 + 0.158
0.676 + 0.067
0.682 = 0.092
0.711 + 0.072
Walker Run
50
0.424 0.213
0.613 = 0.132
0.658 = 0.188
0.672 = 0.128
100
0.323 = 0.261
0.532 = 0.191
0.645 = 0.175
10
0.706 + 0.087
0.728 0.012
0.721 - 0.012
0.734 + 0.008
30
0.598 + 0.128
0.690 0.051
0.692 0.113
0.722 + 0.067
Walker Stand
50
0.494 + 0.373
0.648 - 0.
0.658
1658&00110 0.162
0.701 + 0.138
100
0.369 + 0.241
0.568 0.172
0.662 - 0.185
10
0.701 0.188
0.727 0.015
0.723 0.022
0.731 + 0.028
0.612 + 0.140
0.696 0.063
0.701 0.073
0.730 + 0.034
Walker Walk
50
0.474 + 0.181
0.664 + 0.123
0.667 0.112
0.718 + 0.079
100
0.371 + 0.251
0.598 = 0.194
1601+
0.675 + 0.142
24

---

H.2 Trial Efficiency
Swingup
Cartpole
Campola Sparse
1030
1200
920
800
E00
420
Tett 90hmm
630
600
300
100
230
400
620
000
1000
400
602
800
1000
200
600
80C
1000
200
800
1000
Cortoole Swinngup
Carppote Swingup Sparse
Cheeteh Run
Finger Spin
420
100t Retum
330
230
50-
200-
200
400
620
1900
1000
200
420
600
820
1000
200
4.0
620
300
1000
200
400
500
800
1000
Finger Turn Easy
Fingeer Turn Hard
Hupper Hop
Hopper Stand
820
603
600
400
420
230
230
400
600
800
1000
200
400
600
800
1000
200
400
600
800
1000
200
400
600
800
1000
Pendulum Swingup
Quadruped Run
Quadruped Walk
Reacher Easy
Ted berumn
620
470
2:00
220
402
620
4200
8003
820
1.000
6000
DXC
1000
200
800
1000
Meacher Hard
Vie Icerr
Welker Stand
Welker Wisk
500
020
620
500
Ret 2.03
400
400
300
730
103
1:0
800
1000
Eroironment Triaks
Enairsmment Titsk
Tesfrmmment
Einiinnmment
Figure 10: Performance comparison of test results on 20 DMC tasks under limited environment steps, where the standard error is shaded in the distraction setting. The horizontal axis indicates the number of environment data that is used to train the models. The vertical axis represents the average test return over 100 test episodes.
25

---

H.3 Data Efficiency
Swingup
Cup
Cartpole
Cartpole
1000
1000
1000
1920
850
600
7251 200m
10
Caripole Swingup
Caripole Swingup Sparse
Cheetah Run
Fingeer Spin
Not Nuuu
600
600
403
400
Turn
Turn Herd
Hopper
Hopper Stand
1.00
1000
500
1000
Tet Nazum
203
200
100
Swingup
Quadruped
Easy
1.00
1000
1000
1000
2030
1900
800
600
Text Maum
Reacher Hard
Walker Run
Walker Stand
Walker Walk
Text Mauum
600
600
600
600
402
400
400-
Envtromment Stepe 165
Ensrronment Stope 165
Envrronment Stops 165
Envionment Steps 165
TD-NPC
Figure 11: Performance comparison of test results on 20 DMC tasks under limited environment interactions, where the standard error is shaded in the distraction setting. The horizontal axis indicates the number of times that the models explore the environments. The vertical axis represents the average test return 100 test episodes.
26

---

H.4 Long-term Imaginations Over Extended Horizon Size
Acrobot Swingup
Cup Catch
Cartpole Balance
Cartpole Balance Sparse
155 8:0
600
200
420
400
100
200
220
200
Sparse
Spin
1000
800
1030
800
1920
600
830
Tol Potum
600
6:00
400
230
20
40
20
80
20
80
60
Finger Turn Easy
Finger Turn Hard
Happer Hop
Hopper Stand
330
Text Ratum
600
200
400
120
Perdulum Swingup
Quadruped Run
Quadruped Welk
Reachee Easy
1000
1000
1030
1000
8000
000
830
800
Text Ratum
620
6:00
6:00
400
420
400
200
200
200
40
20
Reacher Hard
Walker Run
Walker Stand
Waller Walk
Tex: Rabum
400
40
20
20
Morizon
Size
DHAMAAC
300MM NFC
Dreamer
GD-MPC
+ MPC2
Figure 12: Performance comparison of test results 20 DMC tasks over different horizon size of imagination. The horizontal axis indicates the horizon size of each imagination. The vertical axis represents the average test return over 100 test episodes.
27

---

H.S
Impact of Logic Inference Depth Over Extended Horizon Size
Swingupp
Cartpole Balance
Sparse
500
1000
1030
1000
401
900
920
900
705 Rotum
-00
800
8.0
800
700
720
700
630
20
40
20
40
80
20
60
80
40
60
Cartpo e Swiingup
Cartoo e Swvingup Sparse
Cheetah Run
Finger Spin
Toxt Patum
700
630
420
500
200
Finger Turn Easy
Finger
Hopper
Hopper Stand
900
1000
520
1000
0000
900
400
900
Thst Robum
700
800
300
800
600
700
200
700
600
1.00
600
20
Perdulum Swingup
Quadrruped Run
Quadruped Walk
Reacher Easy
Tox 7aaburn
600
520
20
20
Reacher Hard
Walker Run
Wolker Stand
Walker Walk
900
1020
1000
800
800
900
900
R:t Rebum
700
700
800
8.00
800
620
730
700
500
500
600
600
500
500
40
20
00
Hortzo
Hortzon
Horioo Size
Figure 13: Performance comparison of test results on 20 DMC tasks with different logic inference depth over extended horizon size of imagination. The horizontal axis indicates the horizon size of each imagination, and the vertical axis represents the average test return over 100 test episodes.
28

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We explain our contributions and scope in detail in the introduction (Section
l).
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations in Section 5,
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]

---

Justification:
We derive the logical ELBO in Appendix C.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Please refer to Section 3, Appendix D, and Appendix reproduce our results. We also release all the code and data.
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
30

---

Answer: answerYes
Justification:
We provide the code and data in the supplemental material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: //t////ooaemsssseeeeeee
/nips. wwtppbccc//aaeeccccccccccccc CttaaigggddeeeeeeSuniiieeeeeee sssssssssiiiyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification: We described the training and test details in Section 3, Appendix D, and Appendix E.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification:
We report error bars for Figure 4-14, and in Table 2 and 6.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
31

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: See Section
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics fcttss//neuurpp: ccpppublcc/thhiicciueeeeeeeeeee
Answer: [Yes]
Justification: The research conducted in the paper conforms, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: See Appendix A.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
32

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The creators or original owners of assets used in the paper are properly credited and the license and terms of use explicitly mentioned and are properly respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
33

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
34

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in our research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA
that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https:/ http://eess..//eeeee htt/e//rps..cc/oooeeeeeeeeeeeeee http:////ooeeeeeee//2//////11//0 for what should or should be described.
35


Reviews:
[{"review_id": "rnIQKT4TxO", "reviewer": "Reviewer_YoYe", "summary": "This work proposes a neural-symbolic world model consists of system I and II for long-term imagination and planning. The system I is built upon on  RSSM from DreamerV3. The system II is based on Logic-integrated neural network that implement hand-crafted propositional logical rules using neural networks, incorporating logical consistency across imagination states. System I and system II are trained jointly via inter-system feedback: system II is trained with system I's predicted latent dynamics; system I minimizes observation loss and the logical constraint loss from system II. The authors conduct experiments on robotics and control tasks. The results show that the proposed method outperform existing world models in terms of test return and logical consistency.", "questions": "How did you measure the logical consistency for evaluation? did you use the trained system II?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n\n* This work is well motivated: incorporating logical constraints into world model is an important research problem.\n\n* It presents an interesting and novel approach to modeling world dynamics through neural-symbolic learning.\n\n* The experiment results show the effectiveness of the proposed method.\n\n## Weaknesses\n\n* As discussed in the limitation section, the method relies on hand-crafted rules, which are difficult to scale or fully cover in more complex environments.\n\n* The logical rules considered are restricted to propositional logic. It's nontrivial to extend this approach to more expressive logic with existential quantifiers and uncertainty.\n\n* (minor) The authors claim that their application of Kronecker product can facilitate action-state alignment, but there lacks an ablation study to support this claim.", "was_revised": true, "final_justification": "I'm more convinced that this paper should be accepted.", "rebuttal": "We are grateful to Reviewer YoYe for spending time on reviewing our manuscript and raising many valuable improvement suggestions. In the response and our updated manuscript, we have made every effort to answer and address your insightful concerns. Our detailed responses are given as follows.\n\n> **W1:** As discussed in the limitation section, the method relies on hand-crafted rules, which are difficult to scale or fully cover in more complex environments.\n\n**A**: Thank you for this feedback. Our proposed method is general and scalable since the logical thinking ability of the System 2 component can be obtained by a learning-based method instead of hand-crafted rules. In particular, the System 2 component is designed to learn logical rules from the actual environment trajectories $s_t \\land a_t \\rightarrow s_{t+1}$. This method is applicable to any environment and any task without reliance on prior knowledge of logic. With regards to our claim regarding the reliance on domain-specific logic, this relates to the fact that action $a_t$ in $s_t \\land a_t \\rightarrow s_{t+1}$ follows the specific task $g$, i.e., $a \\in \\mathcal{A}_g \\subsetneq \\mathcal{A}$, and it leads to $s \\in \\mathcal{S}_g \\subsetneq \\mathcal{S}$, where $S$ is the state space of the open world, $\\mathcal{A}$ is the action space of the agent, and $\\mathcal{S}_g$ and $\\mathcal{A}_g$ are respectively the subset of the state space and the action space with the task $g$. In this context, we can use a goal-conditioned method, similar to TD-MPC2 [1], to overcome this particular limitation and scale to different tasks. To demonstrate the generalization and scalability of the proposed method, we have added two experiments that extend DMWM to multi-task environments (scalability) and few-shot learning for new tasks (generalization) (Please see our response to W1 of Reviewer Y1Jx). The simulation results demonstrate that DMWM can achieve a performance improvement of 8% on DMC tasks, 30% on ManiSkill2, and 6% on MyoSuite compared to TD-MPC2 at 1M environment steps by using the same hyperparameters across all tasks. Moreover, we have added one paragraph that discusses, in detail, the limitations of our method in the appendix for clarifying these points. In a nutshell, our approach can be general and scalable for complex environments. \n\nReferences:\n\n[1] N. Hansen, H. Su, and X. Wang. Td-mpc2: Scalable, robust world models for continuous control. International Conference on Learning Representations, 2024.\n\n> **W2:** The logical rules considered are restricted to propositional logic. It's nontrivial to extend this approach to more expressive logic with existential quantifiers and uncertainty.\n\n**A**: We have adopted the propositional logic system since it is enough to capture the general environment logical rules by $s_t \\land a_t \\rightarrow s_{t+1}$. However, DMWM  can also be extended to existential quantifiers ($\\forall$ and $\\exists$) and probabilistic logic ($\\mathbb{P}(\\varphi)$). In particular, the universal quantification $\\forall$ can be extended by   $\\forall x: \\varphi(x) \\Rightarrow \\min_{x \\in D} \\text{Sim}(\\varphi(x), \\text{True})$, and the existential quantification $\\forall$ can be extended by   $\\exists x: \\varphi(x) \\Rightarrow \\max_{x \\in D} \\text{Sim}(\\varphi(x), \\text{True})$, where $D$ represents the domain of variable $x$ and $\\varphi(x)$ represents a logical formula evaluated over $x$. The min-pool and max-pool can be added to the logic loss to train $\\forall$ and $\\exists$, respectively. To enable DMWM with logic uncertainty, the existing logic similarity metric $\\text{Sim}(\\varphi, \\text{True}) \\in [0,1]$ in (12) can be considered as a probabilistic estimate $\\mathbb{P}(\\varphi)$ of the truth value of a logical proposition $\\varphi$. In this way, probabilistic logic can be incorporated by introducing soft constraints of the form $\\mathbb{P}(\\varphi) \\geq \\tau$, where $\\tau$ is a required confidence threshold. These constraints can be integrated into the logic loss through regularization terms or logic-enhanced variational objectives that reflect the probabilistic consistency of imagined trajectories. \n\n> **W3:** The authors claim that their application of Kronecker product can facilitate action-state alignment, but there lacks an ablation study to support this claim.\n\n**A**: We agree with the need for an ablation study.  We must first point out that we had a typo in the manuscript. In particular, $\\otimes$ represents the outer product rather than the Kronecker product since the state embedding $v \\in \\mathbb{R}^d$ and the action embedding $m \\in \\mathbb{R}^d$ are vectors. In other words, for vectors, $\\text{OP}(v,m) = \\text{KP}(v,m).\\text{reshape}(d,d) \\in \\mathbb{R}^{d \\times d}$, where $\\text{OP}$ is the outer product operator, and $\\text{KP}$ is the Kronecker product operator. Following your suggestion, in the updated appendix, we have added an ablation study to support the advantages of the outer product for action-state alignment compared to the concatenation method as follows.\n\n| Env        | *H*  | Outer Product (DMWM) | Concatenation      |\n|------------|------|-----------------------|---------------------|\n| DMC Tasks  | 10   | 0.729 ± 0.014         | 0.713 ± 0.032       |\n|            | 30   | 0.719 ± 0.042         | 0.697 ± 0.075       |\n|            | 50   | 0.698 ± 0.084         | 0.672 ± 0.128       |\n|            | 100  | 0.687 ± 0.140         | 0.654 ± 0.195       |\n| ManiSkill2 | 10   | 0.721 ± 0.022         | 0.702 ± 0.052       |\n|            | 30   | 0.710 ± 0.049         | 0.681 ± 0.091       |\n|            | 50   | 0.689 ± 0.112         | 0.661 ± 0.183       |\n|            | 100  | 0.669 ± 0.149         | 0.627 ± 0.238       |\n| MyoSuite   | 10   | 0.718 ± 0.031         | 0.698 ± 0.072       |\n|            | 30   | 0.705 ± 0.062         | 0.676 ± 0.125       |\n|            | 50   | 0.682 ± 0.121         | 0.649 ± 0.194       |\n|            | 100  | 0.658 ± 0.163         | 0.617 ± 0.285       |\n\nThe simulation results demonstrate that the outer product of DMWM can improve logical consistency compared to the concatenation method across all tasks. In particular, the outer product-based method achieves 3.9% improvements on DMC Tasks, 4.2% on ManiSkill2, and 5.1% on MyoSuite in terms of logical consistency. This is because, compared to the concatenation method that overlooks structural dependencies, the outer product can capture second-order interactions between latent state and action embeddings, which enables cross-space logic modeling.\n\n> **Q:** How did you measure the logical consistency for evaluation? Did you use the trained system II?\n\n**A**: Yes, we use the trained System 2 components to evaluate and verify the logical consistency across different approaches during the test stage. In particular, for a fair comparison, all the LINN-S2  components are trained with the same network architecture, parameter scale, and training steps. For the baselines, including Dreamer, Hieros, and HRSSM, the LINN-S2 components only capture and verify the logic of state transitions and will not be used to impose the logical constraints on the System 1 components during training."}, {"review_id": "lkx9voyveg", "reviewer": "Reviewer_Jdzu", "summary": "Reinforcement learning algorithms have become increasingly important in the last years. One of the major challenges to deep reinforcement learning algorithms is sample efficiency, especially for long-term tasks. To address this issue many approaches learn a world model to train an agent entirely in imagination, eliminating the need for direct environment interaction during training. However, these methods often suffer from either a lack of imagination accuracy (accumulation of prediction errors), exploration capabilities, or runtime efficiency.\n \nThe paper proposes DMWM, a model framework that seeks to integrate logical reasoning to enable imagination with logical consistency to improve training in imagination. The model consists of (i) a component (RSSM-S1 based on DreamerV3) that handles state transitions and (ii) a logic-integrated neural network-based System (LINN-S2) that guides the imagination process by including logical reasoning.\n \nNumerical experiments are performed for the DeepMind control suite as well as 4 robotic tasks from ManiSkill2 platform and 4 robotic tasks from MyoSuite platform. Results are compared to SOTA baseline solutions (Dreamer, Hieros, HRSSM).", "questions": "1. What are the main insights? Under which conditions (e.g. size/dimensions of the state space or action space) the framework performs good / not good? Where does the good performance come from? Which components are responsible?\n2. Does the proposed technique scale to larger problems (dimensions of the state space or action space)? How many training will be required? Please discuss.\n3. What are the required runtimes for the performed experiments?\n4. How problematic is the tuning of the hyperparameters used?\n5. Could the operations used to describe logical rules be extended?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths:\n * The proposed model is novel and creative. The investigated problem is timely and highly relevant.\n * The provided benchmark experiments against existing state-of-the-art models are rich and a strong performance is demonstrated.\n * The paper is well written and well executed.\n \n \nWeaknesses:\n * The logical rules have to be known and predefined, which is fine for some applications but a limiting factor for others.\n * Limitations of the approach could be better discussed. When does the framework perform particularly good / not good?", "was_revised": true, "final_justification": "The authors addressed all questions I had but given I already rated this paper \"Accept\", I will not change my rating. It's not exceptional but incremental (in a good way).", "rebuttal": "> **W1:** The logical rules have to be known and predefined, which is fine for some applications but a limiting factor for others.\n\n**A**: Thank you for this comment. Our proposed method is general and scalable since the logical thinking ability of the System 2 component can be obtained by a learning-based method instead of hand-crafted rules. In particular, the System 2 component is designed to learn logical rules from the actual environment trajectories $s_t \\land a_t \\rightarrow s_{t+1}$. This method is applicable to any environment and any task without reliance on prior knowledge of logic. To demonstrate the generalization and scalability of the proposed method, we have added two experiments that extend DMWM to multi-task environments and few-shot learning for new tasks (Please see our response to W1 of Reviewer Y1Jx). \n\n> **W2:** Limitations of the approach could be better discussed. When does the framework perform particularly good/ not good?\n\n**A**: We first detail the cases in which DMWM performs well:\n- Environments with Clear Structural Rules: DMWM uses logic rules most effectively in domains where domain knowledge (e.g., physical constraints or control dependencies) can be formalized through first-order logic. For instance, in the *Cartpole Balance* task from the DMControl suite, domain rules like “if the pole angle exceeds a threshold, the agent must apply a force in the opposite direction to maintain balance” can be encoded as logical implications, i.e., `angle > θ → force = left`. In such environments with clear structural rules, logical regularization and learned logical rules can improve both interpretability and generalization.\n- Complex Tasks Requiring Long-Horizon Planning: RSSM-S1 performs well in short-term prediction but cannot handle the error accumulation over longer horizons. LINN-S2 enhances recursive implication reasoning to construct long-term logical chains and aligns the predicted trajectories with logical inference, as shown in (10) and (11). This reliable imagination ability is important for complex tasks with the requirements of long-horizon planning. For instance, in tasks such as *Quadruped Run* in Figure 13 and *Key Turn Hard* in Figure 12, DMWM achieves more stable long-horizon planning compared to Dreamer.\n- Limited-Data or Limited-Trial Settings: Unlike conventional RSSM-based models that rely solely on statistical patterns in observed transitions, LINN-S2 imposes logical constraints that function as supervisory signals. These logic-driven regularizations make the model generalize effectively from sparse or partial observations, which enables more efficient learning of long-horizon dynamics. As shown in Figures 9–12, DMWM outperforms baselines with substantially fewer environment interactions.\n\nNext, we discuss some of the scenarios in which DMWM may have limited performance gains:\n- Simple Tasks with Short-Term Dependencies: In simple tasks that are characterized by short-term dependencies and low-dimensional dynamics, such as *Cartpole Balance* or *Cup Catch*, DMWM takes limited advantage over baseline models, as shown in Figure 10. The additional logical reasoning from LINN-S2 does not significantly enhance prediction quality, as the short-term dynamics can be effectively captured by RSSM-S1 alone. In such cases, the complexity and computational overhead introduced by hierarchical logical inference are over the task demands. \n- Highly Stochastic Environments: In highly stochastic environments, LINN-S2 faces difficulty in establishing effective logical dependencies between states and actions. Although LINN-S2 learn logical rules from general environment transitions $s_t \\land a_t \\rightarrow s_{t+1}$, this still relies on the presence of stable, interpretable relationships, which are often absent in high-stochastic settings with sparse logic rules. In this context, the logic consistency loss $C(\\phi)$ can be noisy and unreliable. To address this, future work can incorporate uncertainty-aware logic modeling or probabilistic logic reasoning to selectively apply logical constraints only where they are supported by sufficient structural confidence.\n- Non-Stationary Environments: In non-stationary environments where underlying logical dependencies evolve over time, the static formulation of LINN-S2 fails to maintain accurate inference since it cannot adapt to abrupt changes in environment dynamics. This failure can lead to outdated or incorrect logical constraints being imposed on RSSM-S1 and lead to degraded imagination quality, along with unstable planning. Future extensions can incorporate continual learning to dynamically update logic rules.\n\nWe will add this discussion to the appendix.\n\n> **Q1** What are the main insights? Under which conditions (e.g. size/dimensions of the state space or action space) the framework performs good / not good? Where does the good performance come from? Which components are responsible?\n\n**A**: We agree that this issue deserves more discussions, as explained next. Our work proposes a dual cognitive system with fast, data-driven System 1 and logic-driven System 2, which enables robust long-horizon imagination by mitigating the accumulated prediction errors. This is achieved through a recursive logic reasoning framework that imposes local and global logical consistency on temporal predictions to improve long-term planning. The inter-system feedback mechanism ensures that symbolic logical rules can refine intuitive transitions and adapt logic based on environmental feedback. The conditions of performance can be seen in our response to W2. Performance gains primarily arise from the LINN-S2 component, which learns logical rules from the environment through hierarchical deep reasoning, thereby ensuring logical consistency in imagined rollouts. The inter-system feedback mechanism enables the intuitive predictions of RSSM-S1 to align with logical constraints, thus creating a closed-loop dual-mind learning process. Overall, the dual-mind design and the inter-system feedback are responsible for performance gains.\n\n> **Q2:** Does the proposed technique scale to larger problems (dimensions of the state space or action space)? How many training will be required? Please discuss.\n\n**A**: Yes, DMWM can scale to larger problems due to its modular dual-system architecture. In particular, RSSM-S1 captures environment dynamics through compact latent representations to generalize across high-dimensional observations. Meanwhile, the LINN-S2 enhances scalability by reasoning over logic-structured embeddings rather than raw state-action inputs, which reduces the dimensional complexity of inference. The use of logic operations in LINN-S2 abstracts high-dimensional dependencies into compact, symbolic representations. It maintains reasoning tractability even as the state/action dimension increases. Moreover, the proposed hierarchical reasoning mechanism (10) enables the model to recursively infer over long horizons without unrolling full trajectories, which also alleviates the computational overhead in large-scale environments. \n\n> **Q3:** What are the required runtimes for the performed experiments?\n\n**A**: All experiments are conducted on a single NVIDIA RTX 3090 GPU. Training a single DMC task with DMWM requires 1.2 GPU days, compared to 0.35 GPU days for a standard RSSM. It reflects the additional cost of dual-system optimization and inter-system feedback. However, DMWM introduces no additional computational overhead for inference after training since S1 and S2 function as a unified model, with logical reasoning distilled into RSSM-S1. Hence, DMWM maintains runtime complexity comparable to RSSM. \n\n> **Q4:** How problematic is the tuning of the hyperparameters used?\n\n**A**: The tuning of hyperparameters is moderately complex. In particular, hyperparameters stem from RSSM-S1,  LINN-S2, and inter-system feedback. While RSSM-S1 builds on existing hyperparameters from architectures like DreamerV3, LINN-S2 introduces additional tuning needs, such as reasoning depth and logic regularization weights. However, these are guided by logical rules, which reduce arbitrariness and improve stability. The inter-system feedback mechanism requires balancing, but is supported by clearly defined loss components. Overall, although tuning DMWM involves more components than standard world models, the process remains feasible and justified.\n\n> **Q5:** Could the operations used to describe logical rules be extended?\n\n**A**: Yes, they could. As we have implemented the propositional logic system with $\\neg$, $\\land$, $\\lor$, and $\\to$ in a modular way, it is easy to form more complex, compositional logical operations within propositional logic.  Moreover, the operations can also be extended to first-order logic ($\\forall$ and $\\exists$) and probabilistic logic ($P(\\varphi)$). In particular, the universal quantification $\\forall$ can be extended by $\\forall x: \\varphi(x) \\to \\min_{x \\in D} \\text{Sim}(\\varphi(x), \\text{True})$, and the existential quantification $\\forall$ can be extended by   $\\exists x: \\varphi(x) \\Rightarrow \\max_{x \\in D} \\text{Sim}(\\varphi(x), \\text{True})$, where $D$ represents the domain of variable $x$ and $\\varphi(x)$ represents a logical formula evaluated over $x$. The min-pool and max-pool can be added to the logic loss to train the $\\forall$ and $\\exists$, respectively. To extend operations to probabilistic logic, the existing logic similarity metric $\\text{Sim}(\\varphi, \\text{True}) \\in [0,1]$ for DMWM can be considered as a probabilistic estimate $P(\\varphi)$ of the truth value of a logical proposition $\\varphi$. In this way, the probabilistic logic can be incorporated by introducing soft constraints of the form $P(\\varphi) \\geq \\tau$, where $\\tau$ is a required confidence threshold. These constraints can be integrated into the logic loss through variational objectives that reflect the probabilistic consistency of imagination."}, {"review_id": "28uCDfB0Im", "reviewer": "Reviewer_2YL1", "summary": "This paper presents a neurosymbolic model of dual-process theory in human cognition, introducing DMWM, which combines an RSSM-based System 1 for fast, intuitive state prediction with a logic-integrated neural network (LINN)-based System 2 for higher-level logical reasoning. LINN-S2 imposes logical constraints (via regularization rules) on the state and action spaces to ensure that RSSM-S1's imagined trajectories comply with domain-specific logic. The model is evaluated in both actor-critic reinforcement learning and model predictive control settings.", "questions": "Q1: Regarding the use of the Kronecker product in LINN-S2, this work applies Kronecker product to combine the latent state and action vectors before logic processing. What motivated this choice over more conventional fusion strategies like concatenation or bilinear pooling? Did you experiment with alternatives, and how does the Kronecker product impact downstream logical reasoning or model stability?\n\nQ2: Did you explore how DMWM performs when some logic rules are systematically incorrect, redundant, or even contradictory?\n\nQ3: Why apply logical regularization on joint embeddings rather than over predicted transitions? Could you elaborate on why you chose to apply logic at this representational level, and whether other points of intervention (e.g., such as over $(s_t, s_{t+1})$ pairs or full imagined trajectories) were considered or tested?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths \n\n* **Mutual feedback**: Unlike many symbolic-augmented systems where logic is only used post hoc, DMWM features mutual feedback. System 1 provides latent rollouts that inform rule discovery, while System 2 enforces logical constraints that correct and regularize System 1's behavior. This interleaving helps maintain coherence, especially as authors argue about long-term imagination. This is very good design philosophy.\n* **Experiments**: The experiments show that DMWM improves consistency and robustness, suggesting the logic-regularized component improves extrapolation under uncertainty.\n\n## Weaknesses\n\n* Heavy reliance on manually defined logic rules. The logic component depends on domain-specific rule definitions, which may not generalize across environments. While authors briefly mentioned this and gave a discussion on the limitations, this paper can benefit from more analysis on how the system would perform with imperfect, noisy, or missing rules, and no ablations to test sensitivity to these conditions. Solutions (e.g., fuzzy logic?) are not expected, but it would be a plus to see the failure patterns of this method.\n* The dual-module design, including Kronecker-product feature interactions and recursive logical rollouts, adds significant architectural and computational complexity. The paper does not explore how well this framework scales to high-dimensional or more open-ended tasks where symbolic abstraction may be harder to define. It can be hard to scale this work up to a modern scale.\n* Authors should provide more quantitative and qualitative comparisons to the line of work mentioned in Appendix F2 (complex task planning [71, 72, 25, 73, 26, 74, 75]), including Logic Tensor Networks. Also LS‑Imagine (Li et al., ICLR 2025) and other jumpy state-space methods offer similar long-range planning benefits.\n\n- - -\n\nOpen-World Reinforcement Learning over Long Short-Term Imagination. Jiajian Li, Qi Wang, Yunbo Wang, Xin Jin, Yang Li, Wenjun Zeng, Xiaokang Yang. ICLR 2025 Oral.", "was_revised": true, "final_justification": "My concerns have been cleared with the additional experiments.\n\nSystem 1/2 is an overloaded term. This work sets a much better standard of what the machine learning community would expect in a technical paper. Unlike many symbolic-augmented systems where logic is only used post hoc, DMWM features mutual feedback. System 1 provides latent rollouts that inform rule discovery, while System 2 enforces logical constraints that correct and regularize System 1's behavior. This interleaving helps maintain coherence, especially as authors argue about long-term imagination. I recommend acceptence of this work.", "rebuttal": "We are grateful to Reviewer 2YL1 for spending time on reviewing our manuscript and raising many valuable improvement suggestions. Our detailed responses are given as follows.\n\n> **W1:** Heavy reliance on manually defined logic rules.\n\n**A**: Thank you for this feedback. First of all, our method is general since the logical thinking ability of the System 2 component can be obtained by a learning-based method instead of hand-crafted rules. In particular, the System 2 component learns logics from the actual environment changes with one-step logic $s_t \\land a_t \\rightarrow s_{t+1}$ and multi-step logical chain $s_{t-\\alpha} \\land a_{t-\\alpha} \\cdots \\land s_t \\land a_t \\rightarrow s_{t+1}$. This learning-based method is applicable to any environment and any task without reliance on known and predefined logical rules. To demonstrate the generalization and scalability of the proposed method, we have added two experiments that extend DMWM to multi-task environments (scalability) and few-shot learning for new tasks (generalization) (Please see our response to W1 of Reviewer Y1Jx). Moreover, we provide more analysis of when DMWM performs well/not well (Please see our response to W2 of Reviewer Jdzu).\n\n> **W2:** The dual-module design, including Kronecker-product feature interactions and recursive logical rollouts, adds significant architectural and computational complexity. The paper does not explore how well this framework scales to high-dimensional or more open-ended tasks where symbolic abstraction may be harder to define. It can be hard to scale this work up to a modern scale.\n\n**A**: While the integration of Kronecker product-based feature interactions and recursive logical reasoning adds architectural complexity, these operations are confined to compact logical embeddings rather than raw high-dimensional spaces. Moreover, System 2 operates at an abstract symbolic level that scales with task logic rather than input dimensionality. Hence, the architectural and computational complexity of DMWM is controllable and moderate. Moreover, as mentioned in response to W1, the logical rules learned by the System 2 component are general and can be extended to an open-ended environment while keeping the scalability for specific rules.\n\n> **W3:** Authors should provide more quantitative and qualitative comparisons to the line of work mentioned in Appendix F2 (complex task planning [71, 72, 25, 73, 26, 74, 75]), including Logic Tensor Networks. Also, LS‑Imagine (Li et al., ICLR 2025) and other jumpy state-space methods offer similar long-range planning benefits.\n\n**A**: We are grateful for this feedback. In the revised manuscript, we have first added more detailed introductions of S‑Imagine (Li et al., ICLR 2025) and other jumpy state-space methods as the important complex planning methods in Appendix F2. Regarding to your concerns about quantitative and qualitative comparisons with [71, 72, 25, 73, 26, 74, 75], one of our main contributions is to propose a general dual-mind world model, in which we can replace LINN with any logical method for the System 2 component. Hence, the choice of the best logical inference method is not our research focus. The jumpy state-space methods are important to realize the long-term planning, especially the long short-term imagination proposed in (Li et al., ICLR 2025). For control tasks that require accurate single-step actions,  the quantitative comparison of DMWM and LS-imagine via environment trails in 4 representative DMC tasks is shown as follows, where environment trials represent the number of environment exploration opportunities.\n\n| Method           | Env Trials | DMWM (Proposed) | LS‑Imagine       |\n|------------------|--------------|------------------|------------------|\n| Cartpole Balance | 200     | 574 ± 24 | 321 ± 32   |\n|  | 400 | 643 ± 26 | 423 ± 27 |\n|  | 600 | 721 ± 18 | 532 ± 28 |\n|  | 800 | 787 ± 29  | 654 ± 23 |\n|  | 1000 | 837 ± 17 | 764 ± 19 |\n| Finger Turn Easy | 200 | 531 ± 22 | 192 ± 17 |\n|  | 400 | 712 ± 19 | 571 ± 23 |\n|  | 600 | 749 ± 14   | 621 ± 18 |\n|  | 800 | 789 ± 19  | 678 ± 13 |\n|  | 1000 | 765 ± 21   | 702 ± 28  |\n| Hopper Stand | 200 | 568 ± 29 | 178 ± 17 |\n| | 400 | 721 ± 21 | 506 ± 25 |\n| | 600 | 802 ± 25 | 623 ± 19 |\n| | 800 | 812 ± 14 | 713 ± 28 |\n| | 1000 | 821 ± 16 | 732 ± 28 |\n| Walker Stand | 200  | 278 ± 31 | 98 ± 28 |\n| | 400  | 409 ± 33  | 226 ± 35 |\n| | 600 | 482 ± 23| 310 ± 27 |\n| | 800 | 598 ± 31| 387 ± 40 |\n| | 1000 | 685 ± 33| 517 ± 28 |\n\nThe simulation results demonstrate that the proposed DMWM has better exploration efficiency compared to LS-imagine since LS-image is still a pattern-driven Intuitive approach, i.e., System 1 defined in our paper. Next, the qualitative comparison is given as follows.  DMWM and LS‑Imagine both aim to improve long-horizon imagination, with different paradigms. LS‑Imagine relies on jumpy transitions and affordance maps to extend the imagination horizon in visually grounded environments. It enables efficient exploration through value-guided behaviors for an open world. In contrast, DMWM introduces a dual-mind architecture inspired by human cognition, which combines intuitive RSSM-based modeling with a logic-integrated neural network that enforces symbolic reasoning and consistency over an extended horizon. While LS‑Imagine performs spatially-aware exploration using visual semantics, DMWM provides interpretable and semantically coherent predictions through structured logical inference. \n\n> **Q1:** Regarding the use of the Kronecker product in LINN-S2, this work applies Kronecker product to combine the latent state and action vectors before logic processing. What motivated this choice over more conventional fusion strategies like concatenation or bilinear pooling? Did you experiment with alternatives, and how does the Kronecker product impact downstream logical reasoning or model stability?\n\n**A**: We agree that the motivation needs a more detailed discussion. We must first point out that we had a typo in the manuscript. In particular, $\\otimes$ represents outer product instead of Kronecker product since the state embedding $v \\in \\mathbb{R}^d$ and the action embedding $m \\in \\mathbb{R}^d$ are vectors. In other words, for vectors, $\\text{OP}(v,m) = \\text{KP}(v,m).\\text{reshape}(d,d) \\in \\mathbb{R}^{d \\times d}$, where $\\text{OP}$ is the outer product operator and $\\text{KP}$ is the outer product operator. In this context, our method of the LINN-S2 component that uses outer product and the convolutional neural network (CNN)  to capture the cross-space logical relationships is similar to bilinear pooling [1], which also captures multiplicative interactions between feature pairs. Moreover, in the appendix, to verify the stability and logical consistency of the outer product, we have added an ablation study to demonstrate the superior performance of the outer product compared to concatenation  (Please see our response to W1 of Reviewer YoYe). The simulation results demonstrate that the outer product of DMWM can improve logical consistency compared to the concatenation method across all tasks. This is because, compared to the concatenation method that overlooks structural dependencies, the outer product captures second-order interactions between latent state and action embeddings for cross-space logic modeling. Hence, the cross-space representation by outer product and CNN can enable the System 2 component to express more complex logical dependencies essential for long-horizon imagination. As demonstrated in the experiment results, this design is effective and stable for logic processing in our framework.\n\nReferences: \n[1] Kong S, et al. Low-rank bilinear pooling for fine-grained classification. Proc IEEE CVPR, 2017: 365–374.\n\n> **Q2:** Did you explore how DMWM performs when some logic rules are systematically incorrect, redundant, or even contradictory?\n\n**A**: Yes. As our response to W1, the System 2 component is a learning-based method to capture logic rules from the environment changes. Hence, if there are systematically incorrect, redundant, or contradictory logic rules, it means that the observations from the environment are noisy or extremely stochastic. In this context, we can incorporate meta-learning or continual-learning mechanisms to dynamically update logic rules or introduce probabilistic logic to reflect the probabilistic consistency of trajectories. We have provided more details and potential solutions for this discussion in the appendix (Please see the response to W2 of Reviewer Jdzu).\n\n> **Q3:** Why apply logical regularization on joint embeddings rather than over predicted transitions? Could you elaborate on why you chose to apply logic at this representational level, and whether other points of intervention (e.g., such as over pairs $\\boldsymbol{(s_t, s_{t+1})}$ or full imagined trajectories) were considered or tested?\n\n**A**:  To clarify, the logic learning can be divided into two parts: one is logical rules from actual environment trajectories, and the other is logical regularization from basic logical laws. In particular, the purpose of logical regularization is to ensure that the neural modules implement logical operations (i.e., $\\text{AND}$, $\\text{OR}$, and $\\text{NOT}$) following basic logical laws such as *identity*, *annihilator* and *idempotence*, as shown in Appendix G. The logical regularizers are applied over individual vectors $w$ rather than over joint embeddings. Theoretically, $w$ can be any vector, since logical regularization only aims to ensure that the logical networks behave consistently under logical transformations, regardless of the specific origin of input vectors. Hence, logical operations require semantically structured inputs, while the raw state lacks learned logic structures. Moreover, embedding both actions and states into a shared logic space enables compositional reasoning across spaces, which can be explicitly optimized for symbolic logical rules."}, {"review_id": "OySxF0TtoU", "reviewer": "Reviewer_Y1Jx", "summary": "The paper proposes a novel framework called Dual-Mind World Model (DMWM) to enhance the imagination capabilities of agents in complex environments. By integrating a recurrent state-space model (RSSM) with a logic-integrated neural network (LINN), DMWM aims to achieve logical consistency and long-term planning. The framework is evaluated on various benchmark tasks, demonstrating significant improvements in logical coherence, data efficiency, and trial efficiency compared to existing state-of-the-art models.", "questions": "1. How do the authors plan to extend the DMWM framework to environments with ambiguous or evolving logical rules?\n2. What strategies will be adopted to mitigate the complexity introduced by the dual-system architecture in real-world applications?", "limitations": "The paper acknowledges limitations regarding the model's reliance on predefined logical rules and its focus on specific tasks. Future work should aim to address these gaps and explore broader applications.\n\nI would consider raising my score if the above concerns are properly addressed.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The dual-process framework effectively combines intuitive and logical reasoning, addressing limitations of traditional models that struggle with long-term predictions.\n2. Comprehensive experiments on DMControl and robotic tasks showcase the framework's robustness and efficiency, providing a solid basis for its claims.\n3. The ability to enhance exploration efficiency and performance in low-data regimes makes the proposed model relevant for real-world applications.\n\nWeaknesses:\n1. The reliance on predefined logical rules may limit adaptability to dynamic environments where such rules are not clear-cut. Although the authors have mentioned that in limitations, this remains a big concern about the generalizability and scalibility of the proposed dual-process methods.\n2. Missing references. There are more relevant studies about dual-process models from human cognition and physical inference [1-4].\n3. While the model performs well in controlled environments, its effectiveness in dynamic and unpredictable real-world scenarios remains uncertain.\n\n[1] Li S, Wu K, Zhang C, et al. On the learning mechanisms in physical reasoning. NeurIPS 2022.\n\n[2] Smith K, Battaglia P, Tenenbaum J. Integrating heuristic and simulation-based reasoning in intuitive physics. 2023.\n\n[3] Sosa F A, Gershman S J, Ullman T D. Blending simulation and abstraction for physical reasoning. Cognition 2025.\n\n[4] Li S, Ma Y, Yan J, et al. A Simulation-heuristics Dual-process Model for Intuitive Physics. CogSci 2025.", "was_revised": true, "final_justification": "As authors have addressed my concerns and proposed ideas about future work to address the limitations. I have raised my score correspondingly.", "rebuttal": "We are grateful to Reviewer Y1Jx for spending time on reviewing our manuscript and raising many valuable improvement suggestions. In the response and our updated manuscript, we have made every effort to answer and address your insightful concerns. Our detailed responses are given as follows.\n\n> **W1:** The reliance on predefined logical rules may limit adaptability to dynamic environments where such rules are not clear-cut. Although the authors have mentioned that in limitations, this remains a big concern about the generalizability and scalability of the proposed dual-process methods.\n\n**A**:  We thank you for this comment that deserves further discussion. We first note that our method is general since the logical thinking ability of the System 2 component can be obtained by a learning-based method. In particular, the System 2 component be designed to learn logical rules from the actual environment changes with one-step logic $s_t \\land a_t \\rightarrow s_{t+1}$ and deep logical chain $s_{t-\\alpha} \\land a_{t-\\alpha} \\cdots \\land s_t \\land a_t \\rightarrow s_{t+1}$. This learning-based method is applicable to any environment and any task without reliance on known and predefined logical rules. If there is an environment with clear rules, such as traffic rules for vehicles in autonomous driving, these rules can also scale to our proposed method as the logical regularization. Hence, our learning-based System 2 component is general and scalable to more complex applications. With regards to our claim regarding the reliance on domain-specific logic, this relates to the fact that action $a_t$ in $s_t \\land a_t \\rightarrow s_{t+1}$ follows the specific task $g$, i.e., $a \\in \\mathcal{A}_g \\subsetneq \\mathcal{A}$, and it leads to $s \\in \\mathcal{S}_g \\subsetneq \\mathcal{S}$, where $S$ is the state space of the open world, $\\mathcal{A}$ is the action space of the agent, and $\\mathcal{S}_g$ and $\\mathcal{A}_g$ are respectively the subset of the state space and the action space with the task $g$. In this context, we can use the goal-conditioned method, similar to TD-MPC2 [1], to overcome this particular limitation and scale DMWM to different tasks with generalization.  In particular, DMWM will learn a task vector $g$ as an additional input to the System 1 component, such as $\\hat{z}_t \\sim p\\left(\\hat{z}_t \\mid h_t, g\\right)$. To demonstrate the generalization and scalability of the proposed method, we have added two experiments that extend DMWM to multi-task environments (scalability) and few-shot learning for new tasks (generalization). Due to the image and link limitations, we present data tables below for the experiment results. We first investigate the task performance in multi-task complex environments.\n\n| Method                | Env Steps | TD-MPC2     | DMWM (Proposed) |\n|----------------------|-------------|-------------|-----------------|\n| DMC Tasks (20 Tasks) | 1M          | 729 ± 11    | 785 ± 14        |\n|                      | 2M          | 787 ± 13    | 843 ± 12        |\n| ManiSkill2 (4 Tasks) | 1M          | 47 ± 3      | 61 ± 5          |\n|                      | 2M          | 56 ± 5      | 72 ± 5          |\n| MyoSuite (4 Tasks)   | 1M          | 53 ± 5      | 56 ± 3          |\n|                      | 2M          | 62 ± 6      | 68 ± 4          |\n\nIn Table 1, the value for DMC tasks is episode return, and the value for ManiSkill2 and MyoSuite is success rate. Moreover, to support the multi-task environments, varying action dimensions across tasks by zero-padding are handled to a unified maximum dimension, and the action masking is applied to ignore invalid dimensions during training and inference [1]. The simulation results demonstrate that the proposed DMWM can be extended to multi-task environments based on the goal-conditioned method [1]. Compared to TD-MPC2, DMWM can achieve a performance improvement of 8% on DMC tasks, 30% on ManiSkill2, and 6% on MyoSuite at 1M environment steps by using the same hyperparameters across all tasks.\n\nNext, we investigate the generalization of the DMWM to new tasks, where the DMWM is trained on 16 DMC tasks and then finetuned to 4 new DMC tasks. The task vector $g$ is initialized by a semantically similar seen task.\n\n| Method           | *Env Steps* | From scratch | Finetuned      |\n|------------------|-------------|---------------|----------------|\n| Finger Turn Hard | 10k         | 11 ± 7        | 27 ± 14        |\n|                  | 20k         | 17 ± 9        | 43 ± 13        |\n| Quadruped Run    | 10k         | 9 ± 3         | 33 ± 12        |\n|                  | 20k         | 26 ± 15       | 51 ± 23        |\n| Reacher Hard     | 10k         | 13 ± 5        | 36 ± 11        |\n|                  | 20k         | 21 ± 8        | 41 ± 17        |\n| Hopper Hop       | 10k         | 3 ± 5         | 21 ± 19        |\n|                  | 20k         | 15 ± 6        | 27 ± 15        |\n\nThe performance improvement across unseen tasks with few-shot learning demonstrates that DMWM can generalize effectively to novel environments. Compared to training from scratch, the finetuned model achieves higher scores under limited environment steps.\n\nReferences:\n[1] N. Hansen, H. Su, and X. Wang. Td-mpc2: Scalable, robust world models for continuous control. International Conference on Learning Representations, 2024.\n\n> **W2:** Missing references. There are more relevant studies about dual-process models from human cognition and physical inference [1-4].\n\n**A**: We have carefully read these references and have introduced them in our updated manuscript. These references are also valuable to our future work.\n\n> **W3:** While the model performs well in controlled environments, its effectiveness in dynamic and unpredictable real-world scenarios remains uncertain.\n\n**A**: While our experiments focus on controlled benchmarks such as the DMControl suite and robotic platforms, such as MyoSuite and ManiSkill2, we acknowledge that these environments, despite their complexity, may not fully capture the variability and stochasticity present in open-ended real-world scenarios. We have provided a more detailed discussion of this limitation and the potential solutions in the appendix (Please see the response to W2 of Reviewer Jdzu). In particular, for future work, to address the effectiveness of WMWM, we plan to develop the probabilistic logic to handle the ambiguous logical rules in real-world scenarios and develop the mechanisms for online logical rule adaptation to handle evolving and uncertain environments.\n\n> **Q1:** How do the authors plan to extend the DMWM framework to environments with ambiguous or evolving logical rules?\n\n**A**:  For ambiguous logical rules, we can further develop the probabilistic logic for the System 2 component. In particular, the existing logic similarity metric $\\text{Sim}(\\varphi, \\text{True}) \\in [0,1]$ for DMWM can be considered as a probabilistic estimate $\\mathbb{P}(\\varphi)$ of the truth value of a logical proposition $\\varphi$. In this way, the probabilistic logic can be incorporated by introducing soft constraints of the form $\\mathbb{P}(\\varphi) \\geq \\tau$, where $\\tau$ is a required confidence threshold. These constraints can be integrated into the logic loss through regularization terms or logic-enhanced variational objectives that reflect the probabilistic consistency of imagined trajectories. To address the evolving logical rules, DMWM can incorporate meta-learning or continual-learning mechanisms to dynamically update logic rules. \n\n> **Q2:** What strategies will be adopted to mitigate the complexity introduced by the dual-system architecture in real-world applications?\n\n**A**: One of the most promising strategies can be selective logical Inference through uncertainty estimation. In particular, to reduce the computational overhead of always-on logical reasoning, we can use uncertainty-aware control mechanisms, such as the mentioned probabilistic logic in the response to your Q1 comment, to selectively activate the System 2 component. In particular, when the System 1 component exhibits high epistemic or aleatoric uncertainty in prediction (e.g., via ensemble variance or Monte Carlo dropout), logical reasoning from the System 2 component will be activated to guide or correct the imagined trajectory. This adaptive triggering mimics the cognitive allocation of deliberative reasoning in humans and significantly reduces redundant logical inference in seen situations. Moreover, we can adopt sparse and pruned logic representations. In particular, as the logic heatmap in Figure 3, we have found that many logical rules can be sparsely activated since the logical operations within the System 2 component are executed over a structured set of vectorized embeddings. This enables effective pruning or attention-based selection of relevant rules during inference, which can provide a reduced computation graph without losing consistency."}]

Author rebuttal:

Meta-review: The paper proposes DMWM, a dual-system world model: System-1 is an RSSM-based predictor; System-2 (LINN-S2) performs neural logical reasoning to enforce logical consistency over imagined trajectories. The two systems interact via inter-system feedback. Experiments on DMControl, ManiSkill2, and MyoSuite evaluate both task return and a logical-consistency metric. 

During rebuttal the authors: 
(i) clarified the learning-based nature of System-2 (rules are learned from environment traces), 
(ii) added ablations motivating the outer/Kronecker product vs. concatenation for state–action alignment, 
(iii) explained how logical consistency is measured, 
(iv) provided analyses of when the method helps or struggles, 
(v) added multi-task and few-shot generalization results, and 
(vi) compared exploration efficiency to LS-Imagine, plus expanded related work.

All four reviewers recommend accept after discussion; two explicitly express increased convincing (YoYe, Y1Jx). 

Core criticisms (proposed by almost all reviewers), hand-crafted rules and limited logic expressivity, were substantially mitigated by clarifying the learning-based System 2 and by outlining/implementing extensions (probabilistic/first-order).

Methodological clarifications and new experiments (outer/Kronecker ablation; LS-Imagine comparison; multi-task & few-shot generalization; evaluation protocol for logical consistency) directly address the specific asks from reviewers.

Some limitations remain (complexity in highly stochastic settings; broader open-world evaluations), as noted by Jdzu and Y1Jx, but the empirical evidence and clarified scope justify publication.

Overall, this paper is well-organized and high-quality enough for publication. 

### Suggestions for the camera-ready
- Retain the explicit “when it works / when it struggles” guidance and training/runtime details in the main text; keep the limitations section prominent.
- Keep the outer/Kronecker vs. concatenation ablation and the logical-consistency evaluation protocol clearly presented.
- Summarize the LS-Imagine comparison and the motivation for applying logic on joint embeddings in the main paper (not only appendix).
- Include the added references and the discussion of probabilistic logic and uncertainty-gated System 2 to make the generalization story concrete. 
- Other tips proposed by reviewers.
Decision: Accept (spotlight)