Title: Curious Causality-Seeking Agents in Open-ended Worlds
Venue: NeurIPS 2025
Accepted: yes
Keywords: Causality

World Model
Abstract: When building a world model, a common assumption is that the environment has a single, unchanging underlying causal rule, like applying Newton's laws to every situation. However, in truly open-ended environments, the apparent causal mechanism may drift over time because the agent continually encounters novel contexts and operates within a limited observational window. This brings about a problem that, when building a world model, even subtle shifts in policy or environment states can alter the very observed causal mechanisms.
In this work, we introduce the Meta-Causal Graph as world models for open-ended environments, a minimal unified representation that efficiently encodes the transformation rules governing how causal structures shift across different latent world states. A single Meta-Causal Graph is composed of multiple causal subgraphs, each triggered by meta state, which is in the latent state space. Building on this representation, we introduce a Causality-Seeking Agent whose objectives are to (1) identify the meta states that trigger each subgraph, (2) discover the corresponding causal relationships by agent curiosity-driven intervention policy, and (3) iteratively refine the Meta-Causal Graph through ongoing curiosity-driven exploration and agent experiences. Experiments on both synthetic tasks and a challenging robot arm manipulation task demonstrate that our method robustly captures shifts in causal dynamics and generalizes effectively to previously unseen contexts.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on the representation and identification of a world model in a causal lens. \n\nIn the first part, it introduces the key concept -- Meta-Causal Graph. It is a collection of causal subgraphs, and each subgraph corresponds to a distinct meta status. The identifiability of both meta status and Causal Subgraphs is presented in detail.\n- The identifiability of meta status relies on the existence of the (state)-to-(meta state) mapping: that is, one can distinguish observations from different meta-subgraphs by observing their values.\n- The identifiability of subgraphs relies on sufficient intervention targets applied on at least one directed edge.\n\nIn the second part, it develops a Causality-Seeking Agent to (1) exploit existing experience to estimate the Meta-Causal Graph, including the status mapping and each subgraph; (2) actively explore the environment by taking actions. The key design here is a curiosity-driven reward. It is defined by the Shannon entropy of the learnable distributions on edges. This makes sense because higher entropy implies higher uncertainty.\n\nThe paper combines the two parts together: the agent decides what actions (or interventions) to take based on the current estimated Meta-Causal Graph; and the Meta-Causal Graph would be updated based on the collected data from actions. The Meta-Causal Graph serves as a world model for the agent.\n\nIn evaluation, it uses the Chemical environment setting. (1) OOD states in the Chemical environment; (2) rewards on the downstream task. In this part, while the results seem promising, some details are missing, for example, what are the downstream tasks in the Chemical environment? What does it mean to \" predict transition dynamics for model-based reinforcement learning\"? I checked the appendix. I cannot open the link to the anonymous code repo.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- The paper proposes a Meta-Causal Graph to serve as a world formulation. This setting considers environmental conditions shift by introducing the (status)-(meta status) mapping component.\n- Section 3 presents identifiability on both meta status and Causal Subgraphs.\n\n\n**Weaknesses**\n- I have concerns that the proposed Meta-Causal Graph may still have limitations on serving as a world model. (1) The assumption on the existence of such (status)-(meta status) mapping needs to be further justified. (2) The model assumes a fixed set of variables, which may not hold in an open world: one may encounter new variables/objects during exploration.\n- The details in the experimental setting are not sufficient. For example, what are the downstream tasks in the Chemical environment? What does it mean to \" predict transition dynamics for model-based reinforcement learning\"? Also,  it might be necessary to evaluate the causal discovery metrics (SHD, SID, F1 score, etc) compared with other baselines (methods that learn multiple causal graphs from data).\n\n\n**Explanation on the decision**:\n\nThis paper contains sufficient technical details, like the identifiability of each component in the Meta-Causal Graph model, the parameterization, the reward model, and the loss function in the agent part.\n\nThe paper presents insight on how to combine a world model (based on a collection of causal graphs ) and an agent that can take actions to collect new data from actively applying interventions.\n\nLimitation includes insufficient details in the experiment section, and the assumptions on the existence of (status)-(meta status) mapping, and a fixed set of variables across all environments. \n\nI recommend a Borderline acceptance if the experiment details can be clarified, and the mentioned limitation can be fairly discussed in the rebuttal phase. Otherwise, it is possible to decrease the score.\n\nQUESTIONS:\n1. In Definition 3 and Assumption 1, it assumes the meta state $u$ can be determined from observations $\\mathbf{x}$. Wouldn't this be a very strong assumption? \n\n2. It seems not to include the usual assumptions for causal discovery, like faithfulness and Markovness conditions. Am I missing something?\n\n3. While the experiment provides a careful evaluation of downstream tasks, it might be necessary to evaluate the causal discovery metrics (SHD, SID, F1 score, etc) compared with other baselines (methods that learn multiple causal graphs from data).\n\n4. In reality, conducting interventions can be expensive. How would the performance be influenced if: (1) the round of Algorithm 1 is limited, (2) the size of samples from the intervened distribution is small?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a theoretically grounded, curiosity-driven agent that actively discovers a set of context-dependent causal graphs, which are called meta-causal graphs, leading to more robust world models and improved downstream performance over existing passive or single-graph approaches.\n\nThe agent observes state-trajectories and occasionally intervenes on individual variables through a curiosity (entropy reduction) mechanism. From this mix of passive observations and active single-variable interventions it learns by (i) clustering observations into a higher level meta-state, (ii) estimate the causal skeleton and edge directions of observation variables given the inferred meta-state, and (iii) fit a transition predictor whose factorisation follows the learned graph (which can be used for model-based planning; cf world model).\n\nThe paper backs the method with theoretical results that show the agent can, in principle, identify the structure of the environment: Meta-state identifiability (Theorem 1), and Causal-edge identifiability via interventions (Proposition 1). This is a priori not possible by methods which utilize a single causal graph, i.e., which are insensitive to the context variable for the environment. \n\nThe paper benchmarks the learned world models on two environments: a Chemical environment and a more challenging Magnetic robot-arm task. The authors benchmark out-of-distribution prediction accuracy on the chemical environment and cumulative reward in both environments, showing that the proposed method yields the highest performance on all metrics and against several competing baselines.\n\nSTRENGTHS AND WEAKNESSES:\nClarity: I found the abstract not very clear. Generally, I find the paper clear to a causality audience, but less so for a reinforcement learning audience. I added some suggestions to improve this in the next section. \n\nQuality: the paper combines experimentation with nice theoretical results regarding identifiability and clearly acknowledges some of the limitations of the proposed approach. The curiosity-driven intervention approach is a refreshing addition to the classical causal framework. Curiosity in causal learning is not a new idea and indeed early works from the early 2000s considered this, e.g. [1]. However, it seems to have fallen out of fashion, and most works in causal learning do not consider uncertainty (only maximum likelihood inference) and therefore cannot leverage uncertainty for interventions. The proposed implementation of curiosity for uncertainty reduction is novel to me. \n\n[1] Active Learning of Causal Bayes Net Structure by Murphy (2001).\n\nOriginality: the originality is hard to fully assess. Certainly many things are original, but a clear contextualization within the model-based reinforcement learning literature that does not use the causal framework and a mention of related work that learns latent variable dynamic models (such as but not limited to POMDPs) through curiosity-driven actions would help contextualizing and emphasizing the unique contributions of this paper to the Broad RL Literature.\n\nSignificance: the experiments and theoretical results are compelling. The method achieves the best out of distribution prediction accuracy in the first experiment, and significantly boosts downstream episode reward in both experiment, beating all competing methods across all metrics considered. The experiments are rigorous: Ablations show that removing either the curiosity-based reward or the causal mask loss hinders both metrics, confirming their necessity, and comparison is done with several baselines. Furthermore, theoretical results such as Theorem 2, which predicts that overparameterisation never hurts (even if the agent allocates more codebook slots than there are real contexts, every true context still gets its own slot and extra slots fold onto them, leaving the recovered sub-graphs equivalent to ground truth), is validated in practice. One criticism it is hard to fully determine the significance of the paper since there are no discussion on efficiency, run-time and potential feasibility to scale to more complex tasks. In particular, it is unclear whether the method would really apply to real-world problems.\n\nQUESTIONS:\nSuggestion: Make the paper more accessible to the reinforcement learning literature that does not involve causality. Taking the example of Figure 1, for instance, the paper explains this with the causal graph push -> open door or push -> door remains closed. Therefore, it makes the point that we need a contextual meta-state that tells us which of these graphs is applicable in any given situation. A standard model-based reinforcement learning model of this would be a POMDP where state of door at time t (which could be open or closed) transitions to state of door at time t+1 (which could be open or closed) based on the action push. This (potentially stochastic) transition would depend on an extra state (the meta-state in the author's words; door locked or unlocked), at time t, that determines the consequence of push at time t+1. In other words, it seems to me that we can recapitulate this simple example in a standard POMDP, where one of the states is a contextual state. Perhaps one advantage of this formulation is that we can have fuzzy causal arrows in the sense that the push action could have a stochastic consequence on the state of the door. (Not that this is needed here, but it's useful in other types of situations). It seems to me that the causal formulism is handy because it produces these nice interpretable diagrams that do not have to invoke stochastic processes like POMDPs. The reader from model-based reinforcement learning will certainly wonder at the differences/respective features between these two frameworks. One classic paper, for instance, shows the equivalence between causal diagrams and interventions with probabilistic graphical models and intervention nodes (like POMDPs). \n\nCausality with Gates, Winn, John, Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, 2012.\n\nI do not mean to criticize the causal framework used in the paper, just to note that the exposition may be unnecessarily divorced from standard model-based reinforcement learning, which limits understanding/clarity by a broader audience and potential impact. Perhaps one path of least action would be to clarify these discrepancies/equivalences in the related works section or somewhere else. \n\nIncrease emphasis of unique contribution: a clear contextualization within the model-based reinforcement learning literature that does not use the causal framework and a mention of related work that learns latent variable dynamic models (such as but not limited to POMDPs) through curiosity-driven actions would help contextualize and emphasize the unique contributions of this paper in the broader RL Literature.\n\nSome words on efficiency and scale: one strength of the causal approach is parameter and compute efficiency (promoted by your sparsity regularizer and curiosity-driven exploration). Unless I missed it, there is no discussion of model efficiency in the paper. Is there anything that can be said here? Same goes with scalability. The results on the environments tested are impressive. So it begs the question: what are the limitations to scaling to more complex environments? When would you expect the method to scale and when would you expect it not to scale? This would be very helpful for future work.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to discover context-specific causal structures by learning a meta-causal graph based on meta-state estimation and targeted interventions that trigger specific meta-states. The estimation process relies on learning a quantized latent meta-state space via VQ-VAE, while intervention targets are selected based on uncertainty measurements of the graph edges. Together with likelihood matching (i.e., dynamics prediction of future states) and sparsity constraints, the framework jointly learns these components in an iterative loop to discover context-specific causal graphs.\n\nIn the experiments, the authors demonstrate the effectiveness of the approach on simulated benchmarks with chain and fork causal structures, as well as on RobotSuite tasks involving different object interactions to validate its practical utility. Overall, the approach presents a clear and straightforward design. \n\nIn addition, the paper provides theoretical guarantees on the identifiability of meta-states and subgraphs under certain assumptions, along with an analysis of intervention reachability.\n\nIn general, I believe this paper makes meaningful contributions to the field of context-specific causal discovery. However, I have some concerns and questions, which I'll specify in the following sections\n\nSTRENGTHS AND WEAKNESSES:\n### **Strengths**\n\n(+) [about problem setting] The problem setting of context-specific causal discovery is an important and meaningful direction within the broader field of causal discovery. The setup in this paper, where the agent can actively intervene in the system, is interesting and potentially impactful though some prior works have explored similar ideas.\n\n(+) [methodology] The methodology is clear and straightforward, and the proposed objectives align well with the problem setting. These objectives could also potentially be useful for other tasks like MBRL or general world models. \n\n(+) [Presentation] The presentation is clear and easy to follow. \n\n### **Weaknesses**\n\n**W1**: [Overall technical contribution] The first weakness concerns the methodological novelty. To be clear, I generally do not favor raising this point (as I find excessive focus on \"lack of novelty\" unconstructive in many reviews), but in this case, the algorithmic framework appears to be heavily based on prior works: FCDL [Inwoo et al., 2024] and NCD [Inwoo et al., 2023]. I understand that this paper introduces a curiosity-based intervention module, but this primarily serves as a mechanism for selecting intervention targets. In my view, the core components of learning the meta-causal state representation and identifying subgraphs via VQ-VAE are identical to FCDL. That said, I do not consider this a critical issue that would warrant rejection. However, I believe the paper should be better structured to explicitly acknowledge this connection, for example, by clearly stating that the approach builds on FCDL and extends it with a curiosity-driven intervention selection strategy. Such clarification would likely require a revision of the paper. Also, maybe adding one table to compare them and expanding the related work part to discuss the details. \n\n**W2**: [Interventional design] As this could be the most critical module in this paper, I would suggest touching on a more detailed analysis on that part to make sure it is technically robust and correct. \n\n- The intervention selection is based on entropy. How do you ensure that this is tractable in practice? Specifically, what happens if the posterior distribution estimation is inaccurate during the early stages of learning?\n\n- Did you consider other commonly used curiosity-based measures for intervention selection, such as prediction error-based curiosity (Pathak et al., 2017), model disagreement (Kim et al., 2020),  or curiosity-driven replay (Kauvar et al., 2023)? These are widely adopted in world model literature and could be adapted to your setting as well. It would be worthwhile to include a comparison or at least a discussion to better show the difference between this approach and these alternatives to make it clear.  \n\n- A minor suggestion: I believe the reachability analysis should be included in the main paper, as it is crucial to justifying the objective function and its alignment with the theoretical guarantees.\n\n**W3**: [Empirical parts] Following FCDL, this paper evaluates the proposed method on environments including Robosuite and simulated chain and fork causal graphs. I believe these settings do demonstrate the effectiveness of the approach. However, from the perspective of the broader ambition expressed in the paper, I would encourage the authors to conduct more extensive evaluations to further strengthen the work.\n\n- The paper frequently highlights the potential of this approach for world models. However, there is no experimental validation in this direction. In world model settings, it is common to consider an observation function, whereas the current experiments seem to focus only on low-dimensional state inputs. While this is not a major issue, it is worth acknowledging and clarifying.\n\n- I believe it would be valuable to show the learned causal graphs in manipulation tasks with more complex structures, as done in CDL (see Fig. 22: https://proceedings.mlr.press/v162/wang22ae/wang22ae.pdf). This would help validate the method's ability to recover causal structures when more variables are involved.\n\n- I suggest including additional baselines that infer time-varying or context-specific causal structures directly from learned model components, such as transformer attention weights or the Jacobian matrices of MLP layers. These alternatives can often be more flexible within the world model paradigm, as they do not require learning a separate latent space. Comparisons with such baselines would provide stronger empirical support for the proposed approach. In particular, works like [Pitis et al., 2020], [Urpí et al., 2024], and potentially [Lei et al., 2024] (though I understand there is no code available for this one) appear relevant and could likely be tested with minimal additional effort.\n\n- Any empirical validation of the reachability of the intervention?\n\n**W4**: [Discussions on related works] First, to echo my earlier point, I believe FCDL and NCD deserve more explicit acknowledgement to make the contributions of this work clearer. Additionally, you mentioned that [Pitis et al., 2020] relies on strong prior knowledge. However, after reading that paper myself, I found the approach to be quite flexible. Could you clarify specifically what strong priors are required in their method?\n\n**W5**: [Minor] 1. The authors could consider including the corresponding graphical model and specifying additional assumptions on the graph structure (e.g., no instantaneous relationships) to make the framework more self-contained and easier to follow. 2. A typo in the expression for $\\mathcal{L}{\\text{quantization}}$? I suspect that in the second term $e_u$ should be ${E}_\\phi(X)$ instead? Correct me if I am mistaken.\n\n------\n\n**References**\n\nHwang, Inwoo, et al. \"Fine-grained causal dynamics learning with quantization for improving robustness in reinforcement learning.\" arXiv preprint arXiv:2406.03234 (2024).\n\nHwang, Inwoo, et al. \"On discovery of local independence over continuous variables via neural contextual decomposition.\" Conference on Causal Learning and Reasoning. PMLR, 2023.\n\nPathak, Deepak, et al. \"Curiosity-driven exploration by self-supervised prediction.\" International conference on machine learning. PMLR, 2017.\n\nKim, Kuno, et al. \"Active world model learning with progress curiosity.\" International conference on machine learning. PMLR, 2020.\n\nKauvar, Isaac, et al. \"Curious replay for model-based adaptation.\" arXiv preprint arXiv:2306.15934 (2023).\n\nPitis, Silviu, Elliot Creager, and Animesh Garg. \"Counterfactual data augmentation using locally factored dynamics.\" Advances in Neural Information Processing Systems 33 (2020): 3976-3990.\n\nLei, Anson, Bernhard Schölkopf, and Ingmar Posner. \"SPARTAN: A Sparse Transformer Learning Local Causation.\" arXiv preprint arXiv:2411.06890 (2024).\n\nUrpí, Núria Armengol, et al. \"Causal action influence aware counterfactual data augmentation.\" arXiv preprint arXiv:2405.18917 (2024).\n\nQUESTIONS:\nAs I have already listed most of my questions in the weaknesses section to keep them coherent and easier to address during the rebuttal, I will only raise two additional, more general questions here:\n\nQ1. How can the proposed context-specific causal discovery approach be effectively integrated into world models, as motivated in the introduction? More concretely, how can one efficiently learn and estimate changing causal structures in world model environments? Are there any additional structural assumptions needed? Furthermore, how should actions be incorporated into the framework? Is it possible to leverage this structure discovery as part of reward learning, for example, by treating it as an intrinsic reward for skill discovery?\n\nQ2. I have reviewed the theoretical analysis, and overall, the assumptions appear reasonable with no major issues. However, the theoretical guarantees seem to rely on the assumption of observational equivalence. That is, assuming access to a perfect generative model for learning dynamics, which ensures identifiability of meta-states and subgraphs. While I understand this is a common assumption in the causality literature, it offers limited direct guidance for practical methodological design. It would strengthen the work to provide more discussion on this connection. Additionally, in practice, the learned representation is often lossy to some extent. Have you considered further theoretical analysis on how such imperfect (lossy) representations may affect the identifiability of meta-states and the causal graph?\n\n- **final note**: as mentioned in the weaknesses and questions above, I feel that the current version of the paper does not yet meet the acceptance bar.  The main concern is the significant overlap in objectives and theoretical foundations with FCDL, also the experimental validation is lacking in terms of comparisons with other intrinsic motivation methodologies and more comprehensive benchmarks. \nThat said, I am optimistic about the direction and potential of this work and would like to see the authors' response. For points W1–W5, I believe further discussion and clear plans for revision are necessary. Additional experiments are welcome but not strictly required for all points, given the short rebuttal window. Based on the current version, I lean toward a reject, but I would be happy to raise my score if these concerns are adequately addressed and clarified.\n\n***Given the short rebuttal window, I would be happy to see: (1) a formal discussion clarifying the contributions beyond FCDL, particularly regarding the critical role of the curiosity term (see W1-W2) and the empirical attainability of the curiosity-driven objective; (2) additional evaluations, or at least a clear explanation of what is feasible to provide and what is not within the current scope (see W3); (3) anwers to other questions listed above.***\n\nLIMITATIONS:\nYes, the authors provided a discussion on limitations.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 1, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposed a new definition of Meta-Causal Graph as a world model, and used curiosity-driven intervention policy for causal discovery. They conduct experiments in chemical and magnetic tasks to verify. \n\nFrom my perspective, the main contributions are two parts: 1. Use Meta-Causal Graph for causal discovery. 2. Design a curious reward fothe r exploration intervention. The proposed learning process is reasonable. But the experiments are not convincing.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n\n1. Theoretically, they derive sufficient conditions ensuring that strategically chosen families of interventions uniquely identify meta-causal graphs, overcoming observational limitations.\n\n2. They use meta meta-causal graph for causal discovery, and design a curious reward. The learning process is reasonable.\n\nWeaknesses:\n\n1. Need more experiments to verify the performance of the proposed method. Maybe other experimental environments can be conducted, not just chemical and manipulation. Moreover, you can revise the Magnetic tasks for generalization verification. The experiments in generalization are not enough.\n\n2. Clear the words for the method description. In lines 51-56 of the method description, the learning process is not described clearly. \n\n3. Explain the Meta-causal graph: from my understanding, the meta-causal graph is a combination of some graphs and their specific indicators. Right? For policy learning, learn those graphs and indicators, and then use them for downstream policy learning? \n\n4. Explain the world. The author mentions that the method is a meta-causal world. I think the model learned in this work is a dynamics model, like CDL, Grader. It is not a world model. \n\n5. More recent works should be discussed, for example:\n\n        [1] Wang Z, Wang C, Xiao X, et al. Building minimal and reusable causal state abstractions for reinforcement learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2024, 38(14): 15778-15786.\n\n        [2] Cao H, Feng F, Fang M, et al. Towards empowerment gain through causal structure learning in model-based reinforcement learning[C]//The Thirteenth International Conference on Learning Representations. 2025.\n\n        [3] Yu Z, Ruan J, Xing D. Learning causal dynamics models in object-oriented environments[J]. arXiv preprint arXiv:2405.12615, 2024.\n\nI hope that the author can explain the above questions.\n\nQUESTIONS:\nPlease see the pros and cons.\n\nLIMITATIONS:\nPlease see the pros and cons.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}