Learning Human Habits with Rule-Guided Active Inference
Authors: GONG ZHIREN
Chao Yang
Wendi Ren
Shuang Li
Venue: ICLR 2026 Poster (2026)
Keywords: Human Behavior Modeling
Active Inference
Logic Rule
Wake-Sleep Inference
Primary area: interpretability and explainable AI
TL;DR: A novel cognitive framework in which agents iteratively update their internal world models based on historical data through optimization, and use these models to plan and select future actions.

Abstract: Humans navigate daily life by combining two modes of behavior: deliberate planning in novel situations and fast, automatic responses in familiar ones. Modeling human decision-making therefore requires capturing how people switch between these modes. We present a framework for learning human habits with rule-guided active inference, extending the view of the brain as a prediction machine that minimizes mismatches between expectations and observations, and computationally modeling of human(-like) behavior and habits. In our approach, habits emerge as symbolic rules that serve as compact, interpretable shortcuts for action. To learn these rules alongside the human models, we design a biologically inspired wake--sleep algorithm. In the wake phase, the agent engages in active inference on real trajectories: reconstructing states, updating beliefs, and harvesting candidate rules that reliably reduce free energy. In the sleep phase, the agent performs generative replay with its world model, refining parameters and consolidating or pruning rules by minimizing joint free energy. This alternating rule–model consolidation lets the agent build a reusable habit library while preserving the flexibility to plan. Experiments on basketball player movements, car-following behavior, medical diagnosis, and visual game strategy demonstrate that our framework improves predictive accuracy and efficiency compared to logic-based, deep learning, LLM-based, model-based RL, and prior active inference baselines, while producing interpretable rules that mirror human-like habits.

Published as a conference paper at ICLR 2026
LEARNING HUMAN HABITS WITH RULE-GUIDED AC- TIVE INFERENCE
Zhiren Gongl Chao Yang', Wendi Ren', Shuang Li
'The Chinese University of Hong Kong (Shenzhen) Nanyang Technological University zhiren001@e ntu. edu. sg, 2220430116link aahk.edu.ce wendirene 1 inkk cuhk. edu. cn, Alshuang@ccuk.edu..
ABSTRACT
Humans navigate daily life by combining two modes of behavior: deliberate plan- ning in novel situations and fast, automatic responses in familiar ones. Modeling human decision-maie making therefore requires capturing how people switch between these modes. We present a framework for learning human habits with rule- sule-guided active inference, extending the view of the brain as a prediction machine that min- imizes mismatches between expectations and observations, and computationally modeling of human(-like) behavior and habits. In our approach, habits emerge symbolic rules that serve compact, interpretable shorteuts for action. To learn these rules alongside the human models, we design a biologically inspired wake- sleep algorithm. In the wake phase, the agent engages in active inference on real trajectories: reconstructing states, updating beliefs, and harvesting candidate rules that reliably reduce free energy. In the sleep phase, the agent performs generative replay with its world model, refining parameters and consolidating or pruning rules by minimizing joint free energy. This alternating rule-model consolidation lets the agent build a reusable habit library while preserving the flexibility to plan. Experiments basketball player movements, car-following behavior, medical diagnosis, and visual game strategy demonstrate that our framework improves predictive accuracy and efficiency compared to logic-based, deep learning, L.M-- based, model-based RI,, and prior active inference baselines, while producing interpretable rules that mirror human-like habits.
INTRODUCTION
Understanding human behavior in complex environments has long been central goal in both cognitive science (Pylyshyn. 1980) and artificial intelligence (Leichtmann et al.) 2023). A large body of work suggests that behavior in humans and other mammals is supported by least two complementary modes of control: a goal-directed system that evaluates actions based on their consequences, and habit system that relies on learned stimulus response routines, with evidence for partially distinct corticostriatal circuits underlying each (Balleine & O' O'doherty| (2010; Dolan & Dayan 2013). In novel situations, they engage deliberate planning. drawing on internal models of the world to simulate possibilities and anticipate outcomes. In familiar contexts, they shift effortlessly into habitual control, relying on rules or shortcuts (Neal et al. (012) that bypass heavy deliberation and allow rapid, efficient action. This smooth interplay between flexible reasoning and automatic habits is a hallmark of human intelligence- and capturing it in a biologically plausible way remains key challenge for building models that aspire to human-like adaptability.
Active inference (AIF) (Mazaaglia al. 2022). framework rooted in neuroscience and Bayesian principles, offers biologically inspired, brain-like account of adaptive behavior. It portrays the person as a prediction-driven mind that minimizes free energy (Parr & Friston 2019 Millidge et al. 202T]: through variational free energy it makes sense of sensations by inferring hidden causes, and through expected free energy it imagines plausible futures- favoring scenarios that both reduce uncertainty (curiosity-driver understanding) and align with the person's 's preferences (goal-congruent intentions). In doing so, AIF maintains an internal generative (world) model that continually refined to reduce surprise providing a principled, unified lens on perception, learning, and action.
Work done as a research assistant at The Chinese University of Hong Kong, Shenzhen. 1 "oorrspondiig author.

---

Published as a conference paper at ICLR 2026
Biologically
World Model
Human Behavior
Query LLM Intention
Environmmnn 0g
D2
States muy Decide hew many
latent states should be usad.
Generative Model
Provide interpretation....
By (O S My (ar-)
Habitual Policy and Action Selection
Habitual Rule Construction
Activation
Hybrid Policy
Rule Action
Rule-1: not move farward--a step directly
Rule-f active time iff (5 .s,) tr maaP mp
enemy
EFE Planning
By
Rule-2 priortt na shoolinge an enerry vis ble in the same row
Rule Hit??
(Bear Search
82
Wake Phose
m Eioaar Baads,nn,, (VFE m,;;; $, $.0)
0₁
Create New
Goal: Minimize Free Energy
Brain
Rules Confidence
Rule-1
New
Cleogs Phase
Wokk Phose
3.00.
Sleep
ming R. (V 0) 7BFE(a;0. (x) +y0x001
VFt + (7FF;+70aL)
Generate
Replayed
Retined
Retinad
Rule-1
Pi
Human Action
Trajectories
Rule-2
Figure I: Model framework. policy and action selection.
Biologically human behavior,
World model. "
Habitual
Wake phase.
r: Sleep phase.
Overall goal.
In this work, we use AIF in the control-as-inferenn sense as a modeling framework for human behavior (Levine| 20188] Toussaint (2009), rather than as a new -araaddmmximmzinggg control agent. Yet, classical AIF largely operationalizes behavior via prospective planning at each step, and thus specifsse three ingredients that are central to human behavior: habit acquisition. habir consolidation, and meta-control over when to plan versus when to act automatically (Han et al. [024]] Dung Nguyen et al. 2024). Concretely, (7) it lacks a mechanism to compress repeated successes into compact, reusable rules with confidence; (ii) it lacks principled way to switch modes modes-using instant, rule-based actions in familiar situations, and calling on costly look-ahead only when uncertainty is high; and (iii) it offers offline process to consolidate, prune, semantically anchor such rules. These gaps motivate augments AIF with habitual policies learned and refined through a biologically inspired wake-sleep process (Hinton et al. [995] Hewitt et al. [2020] Ellis et al. 2023). Our aim is to this framework objective. In the wake phase, the agent harvests candidate rules from real experience by identifying state- iveetion-action triples that consistently reduce free energy. In the sleep phase, it performs generative replay consolidate, prune, and semantically anchor these rules, so that useful ones are reinforced while spurious ones are discarded. Each rule is grounded in latent state prototypes and interpretable discrete intentions, forming neural syyrall unit that bridges continuous world models with symbolic decision decision-making. This hybrid structure enables instant action in familiar scenarios through high-confidence rules, while retaining flexible planning via expected free energy in novel cases.
approach. We propose rule-guided active inference framework that
to fit and explain human (and human- -ike)) action sequences via a control-as-infee
Beyond efficiency, the learned rules provide interpretable structure that facilitates knowledge transfer and offers insights into the agent's behavior. Altogether, our contributions are threefold: i) biologically inspired extension of active inference tailored to computational modeling of human(-like) habits via rule-guided policies, ii) novel wake-sleep algorithm that jointly learns generative models and symbolic rules under a unified free-energy objective, and iii) empirical evidence on human action prediction tasks such NBA player trajectories, car-following dynamics, medical diagnosis, and visual game strategy, where our framework improves both predictive performance and interpretability compared to deep learning, logic-based, and prior AIF baselines
2 RELATED WORK
Human Behavior Modeling. Modeling human behavior is central applications in public health (Ferguson| 2007 Marsch 20211) I), crime analysis (Savage & Vila 2003), and human robot
collaboration (Drgan Srinivasa] 20033 Maeda et al., (017). Probabilistic and deep approaches often focus on predicting dynamics of discrete events: [hhnn et al. (2018) developed deep models for spatio-temporal events, /houu et al. (202) combined neural networks with spatio-temporal point
processes, and Chen et al. (2020) proposed continuous-time normalizing flows. These methods
AAaiilable htgpaauuhhccpegggghtt...o.oooeae
inference
2

---

Published as a conference paper at ICLR 2026
achieve strong predictive performance but largely act as black boxes, offering limited insight into the structure of human decisions.
A complementary line emphasizes the role of habit in behavior maintenance (Rothman et al. 2009).
Examples include HAT (Serra et al. 2018) for cross-task stability, Markovian models of decision- making (Pentland & Liu 1999). and imitation learning (Ho & Ermon] 2016) Li et al.) 2017] Duan al. 2017) for reproducing expert routines. Yet these approaches encode regularities implicitly, making habits difficult to interpret or arbitrate against planning. Our method addresses this gap by learning compact, interpretable rules within probabilistic framework that supports both rapid habitual action and flexible planning in novel contexts.
Logic- and Rule-Based Explanations. To improve interpretability, recent work incorporates logic rules into predictive models. Li et al. (202I) extract symbolic rules from irregular events, Cao et al. 2023) integrate logic with trajectory models, and Yang et al. (2025) guide temporal point processes with logic priors. Neuro-symbolic methods extend this trend: Li et al.] (2023b)
impose compositional logic constraints on Transformers for human-object interaction, while Xu ct al. (2023) introduce LogicMP for efficient integration of first-order constraints via mean-fiell field inference. d' Avila Garcez et al. (2019) survey neural-symbolic computing, highlighting principled integration of machine learning and reasoning. In hierarchical RL., Bacon et al. (2017) proposed the Option-Critic architecture, which learns options (temporally extended actions) end-to-end, providing a connection between our highlight the value of logic and hierarchical control, rules are often static or imposed post hoc, limiting adaptivity. In contrast, our framework embeds rules directly into the generative process, coupling them with latent states and intentions, optimizing under the free-energy principle, and updating them dynamically through a wake-sleep cycle. This yields interpretable, biologically plausible rules that guide long- long-horizon planning.
controllers and the options literature. While these approaches
Active Inference. Active inference (AIF) offers a unifying framework for perception, action, and learning (Mazzaglia et al. 2022), with applications ranging from psychology (Goette et al. 2023
Demekas et 2020) and economics (Henriksen) (020) to scene construction (Mirza et al.
2016
Heins et al. 2020). Neural extensions have expanded its representational power (Ueltzhöffer 2018
while subsequent work emphasized action selection via expected free energy Friston et al. 2016
2015; 202T Millidge et al., 2020), efficient objectives (Mazzaglia et al.) 2021). and amortized
planning through habit networks (Fountas et al. 2020). Tschantz et al. (2020) explored action- oriented representation learning in active inference, focusing on how agents learn representations that
support effective action selection. Yet, these approaches largely center on prospective planning, with only ad hoc treatment of habits and no principled mechanism for transitioning between deliberate and routine behavior.
Our Contribution. We focus on human behavior modeling and address this gap by embedding symbolic rules directly into the AIF framework. During the wake phase, rules are extracted from experience; during sleep, they are refined via generative replay; and inference, the inferred active rules guide action selection together with expected-frre-cnery free-energy planning. This captures human --lke)) habits as reusable, rule-based shorteuts while retaining a principled planner for novel situations, providing biologically inspired account of their interaction in human action prediction.
3 BACKGROUND: SEQUENTIAL
RECISION-MAKING VIA ACTIVE INFERENCE
Active inference (AIF) provides unifying account of perception, learning, and action under single objective: the minimization of free energy. In sequential decision-making an agent uses a generative model to explain past observations, update beliefs about hidden states, and plan actions that bring about preferred future outcomes. This dual role naturally leads to two complementary objectives: the variational free energy (VFE) for inference and model learning, and the expected free energy (EFE) for planning and action selection.
We consider discrete steps 1,..., T. At each step the agent observes O E Rdd, selects an action a, E AA and accumulates history H, = (O1:t, (1= Latent states Z, E 2 summarize hidden structure relevant for decision making. A generative model with parameters Φ specifies
decision-making.
T
ps (Oi., Zr. o1.T) pa (Z:) II p. Z:) (Z: Z,-1, Pa Zt).
(1)
Variational Free Energy (VFE). Exact inference over ZL is intractable, so we introduce an amortized variational posterior qo (7, HH) with parameters &. At each time t, the per-step
3

---

Published as a conference paper at ICLR 2026
variational free energy is defined as
(2) The first term is prediction error, ensuring latent states explain observations; the second enforces temporal consistency with the transition prior. The VFE serves as the learning signal for updating both the generative model parameters and the inference network d.
VFE, == - log 2-)] + Dra( (Z- | HH-) Po 2--1,a--1))
Expected free energy (EFE). In contrast, action selection is prospective. Given a candidate action dg, the agent rolls out its generative model over horizon H to simulate possible futures. We denote the resulting predictive distribution by 9 which depends on á and the chosen action sequence. At each future step T, the expected free energy is
EFF (a) ag log (Ote Z(+r) P (o Z 1 (ctr-1))
Here Poret specifies which future observations are preferred: the agent aims to keep its trajectory within preferred outcomes while maintaining accurate internal model. The first term measures risk deeviation from preferred outcomes), and the second term encourages epistemic value by reducing uncertainty about hidden states. We adopt the standard control-as-ivg -al--aas---ffneee view, where preferences are encoded as a biased likelihood over outcomes (Levine) 2018 Toussaint 2009). Our goal is to model human behavior from demonstrations: we assume humans act approximately optimally under some latent preference distribution Ppref- but do not try to recover this distribution (or the underlying reward) explicitly. Instead, we learn the generative model and policy so that the resulting EFE-based controller assigns high likelihood to the observed human actions. In this setting, we use the special case
(deviation
Prva(O L+r Z.+-) x Po Zi++);
so that preferred futures coincide with high-probability outcomes under the learned observation model, and minimizing EFE favors actions whose predicted outcomes match human behavior. The cumulative expected free energy of action a over horizon H is
(FE
EFE..-(a).
Active inference couples these two objectives into an iterative loop. At each time step: (i) given new data, the agent updates its beliefs and generative model by minimizing (per-step) VFE, aligning latent states with observations; (ii) from this belief state, the agent forecasts future trajectories under candidate actions, evaluates their cumulative EFE, and exccutes the first action from the trajectory with lowest expected free energy. This integration of retrospective inference and prospective planning defines active inference as a general framework for sequential decision-making.
Computational Challenges. Although conceptually elegant, active inference is computationally demanding. Minimizing VFE requires efficient amortized inference for complex latent structures. Minimizing EFE is even more costly, since multi-step rollouts scale rapidly with horizon H and action space size. We will show later that in this paper, we propose augment AIF with compact rules distilled from experience, which can bypass expensive rollouts in familiar contexts while preserving full VFE/EFE reasoning in novel ones.
OUR APPROACH: RULE- HABITUAL POLICY
We extend AIF by introducing compact, aatent-groundee rules that capture habitual responses in familiar contexts. Rules act as symbolic triggers: when recognizable patterns (including the external world patterns and mental states occur in our setting), they prescribe actions directly, yielding fast and interpretable habitual policies. If no rule is triggered (novel scenario), the agent reverts to minimizing EFE through
rollouts, as in standard AIF. This hybrid arbitration enables seamless switching between fast habits in familiar situations and deliberative planning in novel ones.
Our approach has two main key components: (A) the representation of rules and their integration into a hybrid policy that combines habitual and planning- based control, and (B) a wake-sleep learning algorithm that jointly learn generative model (decoder), recognition network (encoder), and rules under a unified free-energy objective. We will discuss the two components one by one.
Latent State Representation First propose to split the (previously generic) latent Z, into two parts:
Z: = (S,, m.),
where St E S denotes the continuous external world state, compact din-eineniional embedding of the environment that supports accurate prediction of observations. In contrast, m, E [l,..., discrete mental state that encodes intentions, modes, or subgoals.
[l....,K] K} is a
4

---

Published as a conference paper at ICLR 2026
Given the new latent state representation, rewrite the generative model ①
Po(Or, Sur, mu., Mur,
01T) Po(S)) II po(O: St) po St-1, 0c-1)
x po(me) m.-1, Po., (a: Se,
(4)
with parameters d. Here the world model links external world states St observations through Po (O, / St), while the transition prior (s, St-1. captures how the world evolves under actions. The discrete mental state my evolves more slowly via po(m, m4-1, St), providing an interpretable bottleneck of intentions or modes. Finally, the policy Pø. (at St, m4) selects actions conditioned jointly on the external state and mental state, Later we will show how this policy can be parameterized with compact symbolic rules, allowing habitual responses to emerge inside the same probabilistic framework.
Given new latent state representation, approximate the posterior over latent states with an encoder qo(S., me H.), which maps the history H, (O14, a1:1-1) into distributions over both continuous world states and discrete mental states.
4.1 (A) RULE REPRESENTATIONS AND THE HYBRID HABITUAL. POLICY
Building the latent split Zı (St, m,), we introduce symbolic rules as compact carriers of habitual knowledge.
We define rule f as anchored condition-action pair: f: (57.m²) = as: S E E, m. E {l,..., K). ay E A, where the continuous anchor Sÿ encodes prototype of the external environment, mj specifies the intention mode, (Sj, mz)) together describe when the rule becomes active, and a is the prescribed action to take. Each rule has a confidence weight Pj E [0, 1]. reflecting its reliability. The full rule set is F {(S). my. Pf.Pr)} and is treated as part of the policy parameters Φ.. The rule library can be viewed as an amortized mixture over context-action pairs, where each rule defines a prototype component. Our current implementation is efficient, engineering-drive approximation to full variational learning over (m,) and parameters of p(S, m²) in a mixture model. A detailed probabilistic view given in Appendix
Rule Definition.
Rule Interpretation. The continuous component Sj summarizes the external world in a compact latent embedding. Although this representation is not directly numan-interrpeeti its encoded meaning can be probed through the generative world model: by decoding S back into observable space through (O, 5j), we can visualize or simulate the prototypical situation it represents. The discrete component m is categorical and designed to carry semantic meaning (e.g.. cautious, aggressive, conserve energy), often initialized or anchored with interpretable labels.
Together, (S7, m)) provide the condition under which a rule applies, yielding policies that are both context-sensitive (through the continuous embedding) and mental-state-driven (through the discrete mode). This design mirrors cognitive science accounts in which habits are grounded jointly in environmental context and internal goals: in familiar scenarios, learned associations trigger rapid action without deliberation.
For instance, driving agent might acquire the rule brake + (S) car ahead very close, mi cautious). Here, the latent world prototype Sj encodes the spatial situation of nearby cars and discrete mode m denotes cautious intention. When this familiar combination recurs, the action brake is triggered immediately bystiiatelyy-bpsiinnniign-
bypassing costly rollouts and enabling fast, interpretable habitual control.
Rule Activation and Recognition of Familiarity, Given new history H4, the encoder produces posterior distributions over latent states. We use MAP estimates for efficient matching:
MAP arg max (S. = Ht), m4 arg max qa (m = k S., H.).
MAP
A rule r is active if both the discrete mode and the continuous context are sufficiently close: k(SMAP .S)) 2 Tr, where x(-, is a Gaussian similarity kernel and T, a threshold. Under the Gaussian-mixture view (Appendix B). (SSMAP SS;) can be interpreted as (proportional to) the posterior responsibility of rule r given a Gaussian prior over S,, and T, simply truncates very small responsibilities. This soft matching allows rules to work robustly under noisy data. We adopt MAP estimates instead of sampling because they yield fast, deterministic recognition of familiar situations, consistent with how humans rapidly "pattern match" to known contexts.
m, MAP
m
5

---

Published as a conference paper at ICLR 2026
When multiple rules suggest the same action, their contributions are combined by weights: x(a St SMMPP,
mig
MAP)
k(S, .S,) 1/m MAP
1mMMAP
mm Pri
normalized across actions.
Hybrid Policy. The final action distribution blends the rule prior with EFE-based planning: (5)
Pd. (ot St, m²) x [a, SMMP,, , MAP)) + (1-1nid hit) exp( - (a))) If reliable rules fire, their prior dominates and habitual actions are executed directly. Otherwise, the agent falls back on deliberative planning via multi-step EFE minimization. The temperature parameter I controls how sharply the planning fallback discriminates between actions.
Connection to Cognition. This hybrid arbitration is inspired by neuroscientific accounts of the brain's dual systems: habits stored as cummulssrrsppone associations in basal ganglia circuits, and deliberative planning supported by prefrontal and hippocampal structures. This flexible switching between cached habits and on on-the-fl fly planning is a general mammalian capability, consistent with dual-system accounts of dorsomedial vs. dorsolateral striatum and prelimbic vs. infralimbic PFC, as demonstrated in devaluation and oontingency-degraaaiiee
paradigms (Dolan & Dayan 2013
Cushman & Morris 2015).
Our framework instantiates this duality: learned rules serve as compact, interpretable "habit circuits" grounded in latent world states and internal modes, while the generative model provides a flexible substrate for foresight and adaptation when novelty arises.
4.2 (B) LEARNING ALGORITHM: WAKE-SI.EEP
Joint Objective (Total Free Energy). We jointly train the generative model pa, inference model qo, and policy parameters On (including rule prototypes) by minimizing a unified total free-energy objective:
Fe(d, 0, 6.) (OGG() + EFE, 2.) + DKi(go (m)- (H.1)) (m HH))
(6)
lit observed data
appliicd to nollouts
consistncyy
Here (FE; db ()) measures how well pa explains the actual data Or, EFE, (1), Φx) is the expected free energy accumulated over a horizon H (with the same form as in planning but used for training on replayed trajectories), and the KL term regularizes discrete mental states. The KL term implements a sticky prior over the discrete mental state, encouraging slow, interpretable mode changes. This can be viewed as arising from a prior p(m; m,-1) ) that favors persistence, which naturally appears in the VFE expansion when minimizing free energy. The coefficients ? balance the contributions, allowing early training to emphasize world-model reconstruction while later phases prioritize accurate reasoning with appropriate regularization.
Wake Phase (Real Data). In wake, the agent processes real trajectories Dreat and updates (d), 19) by minimizing free energy on this dataset:
min 0.8
B, (8 [VFE(O,, me; Φ. 0)].
At the same time, we grow rules from real data: when triplet (SMAP, m .m. MAP aa) recurs often and yields low free energy, we either
create new rule (S;, m,, a,) with initial confidence pr > 0, or increase the confidence Pr of an existing nearby rule. Continuous anchors are updated as centroids of their assigned latents:
Esegppal w(S)
S
w(S) x exp( VFE(O, S, m(:
EsE urSS)
This update can be viewed as an EM-style M-step on the Gaussian means in the mixture view, with w(S) playing the role of (reweighted) responsibilities (see Appendix B).
Sleep Phase (Replay Data). In sleep, the agent generates replayed trajectories (S,O, m, a) ~ Po and jointly updates (d, Φx) by minimizing
min (s. VFE(O, m; 6, v)) + EFE(a; o, Φx) +ry (K)()))
Here VFE ensures consistency of pa and go under imagination, and EFE provides training signal for dr. Rules are refined during sleep: centroids S are updated on replayed latents, confidences pr adjusted, and won-coniieecce rules are pruned. Both phases share the same free-energy objective, differing only in their data source. This mirrors human learning, where waking experience updates models and dreaming replay consolidates them.

---

Published as a conference paper at ICLR 2026
4.3 OVERALL ALGORITHM
The overall framework alternates between two coupled processes with details provided in Appendix A
1. Learning (wake-sleep):
Wake: update models and grow new rules from real data.
refine models, consolidate/prune rules, and adjust confidences using replay. 2. Plamning (hybrid policy): At decision time, if rule is triggered, the agent acts habitually; otherwise it falls back on model-based search for action selection (c.g., MCTS, A", or rollout sampling) minimize expected free energy.
Sleep:
For efficiency, first run blockwise pretraining bootstrap the world model by minimizing VFE only (Stage 1), which provides a fast warm-up up and stabilizes later training, and then performfull wake-sleep cycles with replay using the joint objective in Eq. 6 (Stage 2).
EXPERIMENTS
We target cross-domain generality: shared latent (S,, m.) combines continuous cognitive state S, with an inner discrete mental state my E .M (intentions/sub-gaal goal-g that drives rule triggering and planning. When datasets do not provide salient mental labels, we obtain LLM-guided interpretable candidates and select K states via a lightweight matching routine with their detailed prompts and sensitivity study over K deferred Appendix Bad semantic lists deferred Appendix [C.6 Model backbones and training schedules are aligned across domains; full architectural settings and optimizer details appear in Appendix C.7
5.1
EXPERIMENTAL SETUP
Dataset We evaluate on four domains spanning structured sequences and temporal vision: (i) ~9.8k train 2.5k val clips with 7 action classes after
NBA SportVU (Kambhamettu et al. 2024 P
LLM-guided feature construction; (ii) Car-Following (Li et al. 2023a 3 199 train / 2.5k val samples with 7 driving modes under an action-centric world model; (ii) DDXPlus
165k train ~ 16.5k train / 16.5k val grayscale frames with 18 actions. Details of dataset, full preprocessing pipelines, feature construction, and action distributions are all provided in Appendix C.8
/ 25k val URTI trajectories with 225 actions (ASK/DIAG); (iv)
Baselines We choose state-of the----rtth art baselines considering following different fields: i) Logic based Models: RNNLogic (Qu et al., 2020) and STLR (Cao et al.] (023). ii) Deep Neural Models: Re-Net (Jin et al. 2019), iui Active Inference Models: DAI (Catal et al. (2200) and DAI-MC (Fountas) et al. 2020p, iv) Model-based RI.: DreamerV2 (Hafner et al.) 2020 and v) L.LM based Models:
Lalle (Song et al. 2024) and Qwen-0.5B (Team (024b) Team (2024a) (a pure L.M baseline that
processes observations through a learned encoder and generates action predictions via direct LM).
Metrics We evaluate performance along three dimensions: (i) Accuracy: (1) Ace@k measures the proportion of correct actions within the next k prediction steps 1d=1,3,5 5) rather than single-step classification; (2) High-Hit Action Ratio (HHAR) measures accuracy specifically on low-frequency critical actions (e.g., marginal maneuvers in each domain), and is considered satisfactory only if it reaches at least ~80% of the overall Acc. (ii) Efficiency: (1) Latency, average inference time per step (ms); (2) Convergence Time (CT), total training time to reach convergence (hours). (iii) Resource Cost: Peak Memory (PM), maximum memory usage during training and inference (MB).
5.2 EXPERIMENTAL RESULTS AND ANALYSIS
Results with best-configuration and baseline comparisons are summarized in Table 1 And the additional experimental results not shown below (especially for Car-Following and DDXPlus)are recorded in the Appendix D
Drreamer2 is adapted as model-based behavioral predictor: we keep the standard world-model architecture but replace the environment reward with supervised next-action objective, training from fixed replay buffer built from human trajectories (offline setting).
7

---

Published as a conference paper at ICLR 2026
Table 1: Overall results under the best configuration (mean across random seeds). Acc is
reported as Acc@ 1/3/5; Lat/CT in ms / h.
NBA Sp
Car-Following
Category
Method
Ace (%)
Lat/CT
Acc (%)
Lat/CT
RNNLogic
67.20/60.55551. 83
26.90/1.20
72.33/68. 22.33/68.14/57.555
7.58/0.54
Logic-based
(20.00
(#0.82/ 20.00)
(#1.26/ 2T..6//1111//11.
(#1.08 #0.00)
STLR
75.25/74,67770.20
174.18/3.35
78.90/76.37/75003
58,29/1.28
(+0.45 / =0.72/
(+0.75 / #0.33)
(+1.50 #2.35)
(#0.63 #0.20)
DeepNN
Re-Net
72.18/68 24118/8.45/62.00
218.42/2.34
76.32/70.71/67 .28
72.23/1.15
(20.67 =0.67)
(43.25 20.000
(20.83/ #2.06)
(22.18/ 20.00)
75.36/70.58/62.33
262.33/1.24 .24
78.86/73.35/68.50
146.33/0.62
DAI
Active Inf.
41..12/=148/=113
(+4.33 #0.001
(40.42/ 10.28 20,53)
(#1.11
82.33/80.61/76.77
386,50/1
84.54/82.87880.22
189.75/0. 75/0.9
DAI-MC
(20.87 =0.55/
(+3,62 #0.05)
(#0.34 /0.36/ #0.76)
(#1$5 #0,0))
78.50/73.32/64455
1244.20/4.6
82.36/77..75771. 1.82
528.33/2.46
LaTee
LLM-based
(+0.88 11 1.00)
(+10.07 / +0.90)
(+1.65) /
(+8.15 70,76)
Qwen-0.5B
71,25/64. 0/55/64.1//6.44
2845.35/N/A
44.85/68.32/62.11
1256.82/N/A
(+1.12/ /
(+12.551
(+1.88 +1,45/ =1.62)
(+9.25
Model-based RL D2eamerV2
86.42/88.557/8
52.73/1.75
08,433/85.88/8.0
38.57/0.92
(+0.47 / =0.66 =0.72)
(+3.25/
(40.54 +0. 78 +1,00)
(#1.04/ +0,00)
17,00//1.32/85..4
35,92/2.59
04.77/95.77994.16
10.44/0
Ours
(20.51 / =0.79 =0.89)
(+2.78 20.22)
(40.34/ 20 =0.47)
(e0.59/ +0.08)
DDXPlus (URTT)
Atari-Berzerk
Category
Method
Acc (%)
La//CT
Acc (%)
LaUCT
18.75/16.29/11 20/.2991333 .28
124.32/4.36
0..86//7.5///
72.46/2.04
RNNLogic
Logic-hased
(20.00 13)
(+5.71 20.39)
(a0.34/ 20.57 20.37)
(23.27 20.00)
22.45/18.33/15555
872.00/10.25
45.50/38. 72/38..727.22 224
432.35/7.18
STLR
/20.00/00550/
(210.53/20.77)
(#0.67 21.83/ 22.25)
(28.20/ 20.58)
DeepNN
Re-Net
27.33/20. 11/120.111111177
1112.42/16.38
10.69/32.48/29 .933
723.02/4.42
(#1.32/21.54) #1.76)
(#10.62 20.42)
(20.75 20.50/ 21.84)
(#5.48 20.30)
DAI
46,82/39,27/34
2033,25/3,88
59,97/52.28/41
977.24/3.67
Active Inf.
(+0.82 +1 +1.33)
(+4.59 +0.27)
(+0.72 #0.95 / =1 .30)
(#5.51 #0.93)
DAIMMC
57, 57,20/52. 12/4..5//3.611 66
2304,23/4.75
66, 82/58/ 20/48 20
1429 00/4 95
(40.67 +/ +1.49)
196.99/ +0.47)
(+0.64 04.44/+..771 #0.77/ =0 =0.90)
(+4.87 +0.74)
16.16/ 14//2...
95028. 72/28.7
62. 12.148/4..1/49 ,28
3230.43/1 1.64
LaTee
LLM-based
(40.32/ +0.67) +0.92)
+0.82) (445.88
(40.46/ #0.68 =0.94)
(+7.61 +0.84)
Qwen-0.5B
24.85/19.62// 11119/96667..1 .35
125842.
15/N/A
58.42/51 25/46.18 18
4856.72/N/A
(a0.42 / =0,78 =1.05)
(#52.35/
40.85/11.22) (40.58
(=8.75
Model-based
D2eamerV2
64.05/61 188611.4888..1
452.25/10.08
76.33/72. 11/.18//69// ,47
108.02/6.87
(20.8) / 20.99 1.3))
(45 1.5.66/6 20.800
(40.67 21.08 1.1.33)
(20.98 s0.24)
79.63/73.58/68.0 07
159.45/8.73 73
92.63/3.53 85.55/77.20772.44
Ours
(a1.54 22.62/222.2///22 22.6222.00)
(14.45 20.29)
(20.87 10.21/ =1.05)
(a2.29/ 20.15)
Sequential Dataset NBA SportVU. Figure 2a illustrates: (i) general decreases of AF, VFE, EFE, and KL, evidencing steady improvement in fit and decision quality; (ii) rule envelopes in latent space, enabling interpretable reconstructions; (iii) test-set curves (Ace@k @k,kk HHAR, latency) with stable convergence and strong accuracy; and (iv) rule-guided inference case where world-model overlays show rules lowering free energy and improving decisions.
Car-Following, This dataset exhibits a small-rule-high-payoo pattern: even compact rule sets saturate accuracy (Ace @3 x96%) while keeping latency very low ms). Training traces show consistent decreases in all free energy components.
DDXPlus. On URTI with 225 actions, rule envelopes reliably capture low-frequency edge actions, yielding pronounced gains on HHAR while sustaining strong Ace Rule-triggered choices also reduce per-step inference relative to model-only planning, though absolute latency remains higher due to the large action space.
Temporal Visual Dataset Atari-Berzerk. With visual inputs (128 x 128 frames, 18 actions). To further assess generalization, we conducted additional experiments on three Atari-100k games (Pong, Breakout, Qbert) with varying complexity levels (see Appendix [D6).. Fig. 2b shows that our encoder extracts task-relevant signals structuring (Se, m.) accurately with clear explanations after decoder, and rule- rule-guided inference examples demonstrate steady predictive rollouts and consistent decisions.
Baseline Limitations. Our baselines face distinct challenges: (i) Deep predictive models (Re-Net, Transformer/BiGRR world models) lack rule libraries and explicit habit mechanisms, struggling with rare actions and highly imbalanced action distributions (e.g., DDXPlus's 225 actions with many rare
8

---

Published as a conference paper at ICLR 2026
5000
pretrataing
Fall-raieing
Acuude Span Explerenn
Samooda Spat Liphen
VFE
1000
(corze))
No nil (centnt)
302
40300
1008
Other muthos
Hale costt
Ralt
(ms)
Trainingstops spochs
Training dynamics: F, VFE, EFE, KI..
(ii) Rule visualization: encoder/decoder spaces.
100
falHiriiiing
Lavett Space
Recoestacion
AxX955%)
Prosstige
Imaxy(nn)
Inietery
homer
Tinling
(iii) Test-set curves: Acc@k, HHAR, Latency.
(iv) Rule-guided inference with world-model.
(a) NBA composite: training dynamics, rule visualization, test-set metrics, and inference with world-model overlays. Additional qualitative figures full curves other datasets are in the Appendix D
2-8 (cspleio
z-1 (explorc)
2-2 (daager)
Lanst
3(MavcFirr)
Action: 1(Ep)
Action: PPPniccFin
Arc:
Latent rules and reconstruction (visual scenes).
(ii) Rule-guided inference with world-model overlays.
(b) Atari Berzerk: rule visualization and inference example, others deferred to the Appendix.
Figure 2: Composites for sequential (NBA) and temporal visual (Atari) datasets.
but critical diagnostic operations). (ii) Active inference baselines (DAI/DAI-MC) have only implicit habit policies without explicit latent mental states and symbolic rules, making them inefficient for multimodal behaviors and rare actions, with high computational costs (c.g.. DAI-MC latency: 2304ms on DDXPlus). (iii) Model-based RL (DreamerV2) limited by offline settings and sparse rewards, underperforming on pure prediction tasks. (iv) Logic-based methods (RNNLogic, STLR) use static or post-hoe extracted rules without joint optimization with world models. (v) LLM-based methods (LaTee, Qwen-0.5B) suffer from high latency (e.g.. LaTee: 95028ms, Qwen-O.5B3 125842ms on DDXPlus), poor performance (Qwen-0.5B Acc@3: 19.62% vs ours: 73.58%), and lack of generative world models active inference mechanisms. Our rule-guided active inference addresses these limitations by jointly learning rules and world models under unified free-energy objective.
5.3 RULE- PERFORMANCE TRADE-OFF
Figure shows the Pareto behavior between accuracy and latency as the rule bank grows. Four consistent patterns emerge:
(1) Rules speed inference; accuracy inverted-U in rule size. As rule count (RC) grows, reference latency drops since cheap rule triggers replace costly planning. Accuracy first rises then falls: compact

---

Published as a conference paper at ICLR 2026
100
Actunay Nay cllghte Bentee
IN Cowagge
(B0mg)
Opicall Mo CDCN
20
10148
Latracy gmo (TTnner Beiter)
20
60
(Lowee
(a) Performance Trade-off Analysis for NBA
(b) Performance Trade-off Analysis for Car- Follooio
Acann (h) anpaer 14 saner
Accorees d9g (ffigee Batte))
3
3
Past Aeggl goc-in
PV criit
120
143
1.0
103
223
04
Latiny [Lawer Better)
110
120
(c) Performance Trade-off Analysis for DDXPlus
(d) Performance Trade-off Analysis for Atari-Berzerk
Figure 3: Rule-Performance trade-off across datasets. Each point corresponds rule-bank size (RC/EN). Y-axis: accuracy (higher is better); X-axis: latency (lower is better). Bubble size encodes peak memory (PM), color encodes HHAR, and vertical bars denote EN coverage. Stars indicate the Pareto knees used in Table []
banks (e.g., NBA at RC=6, Car- Hol-oollowing at RC=3 4) capture reliable intents and boost Acc@k, but excessive rules introduce spurious hits and conflicts, degrading decisions despite faster inference. (2) Coverage vs. precision diverge after the knee. RC and rule-hit rate (RHR) keep increasing even as Acc@t @K k declines (HHAR may plateau). Thus which rules fire matters more than how many: too many rules bias toward noisy or redundant envelopes that compete with the model (e.g., DDXPlus peaks near RC=376, Atari near RC*64).
(3) Memory grows with RC; Pareto knee is optimal. Peak memory (bubble size) increases with RC. The practical operating point the knee, balancing accuracy, latency, and memory.
(4) Rules complement active inference. EFE-guided planning arbitrates between rollouts and rule triggers via AF. With a compact, semantically grounded bank, triggers reduce AF and depth, yielding large latency gains with robust accuracy. Oversized banks cause overlapping envelopes, weakening arbitration and explaining post-peak accuracy drops.
5.4 ABLATION STUDY
We ablate four factors relative to the full model: (i) removing rules; (ii) removing the latent intention mi (iii) dropping generative consistency (VFE, or both VFE and KL); and (iv) greedy rule selection. Key findings. Rules are essential: without them, both accuracy and latency degrade (e.g., NBA Ace 3 31.4-79.55 latency 36 + 73 ms). Latent intention m, organizes precision: removing low- ers accuracy and increases latency (e.g., Atari Acc@3 77.3 - 70.1). Generative consistency critical: - VFE or Only EFE keeps latency low but causes large accuracy drops (e.g., Car- Car-Following Ace 95.9 + 78.5). Greedy rule selection yields the fastest inference (as low as 2.6 ms) but sacrifices accuracy and HHAR, showing instability. Complete ablation study is provided in Appendix D.5 Therefore, each ablated component removes a distinct capability: speed (no rules), precision (no 2), cognition grounding (no VFE), or stability (greedy).
CONCLUSION
We present cognitive framework that jointly learns a world model and uses it to plan and select future actions via active inference, while rule engine provides fast, interpretable habitual control. universal mental-state set enables a single formulation across diverse domains. Experiments sports tracking, driving, clinical diagnosis, and Atari show strong accuracy under low latency, clear Pareto trade-offs, and rule envelopes that align with human strategies. Overall, the framework captures key aspects of human behavior and substantially enhances interpretability.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
We have made extensive efforts to ensure the reproducibility of our results. The complete description of both synthetic dataset generation and real-world dataset preprocessing methods are illustrated in Appendix Details of the computational setup, including hardware configuration and software environment, as well as the choice of syper-parameee are documented in Appendix C.7 release code.
We also
ACKNOWLEDGMENTS
This work was in part supported by the Key Program of the NSFC under grant No. 72495131, Shenzhen Stability Science Program 2023, the Shenzhen Science and Technology Program No. JCY122250004411000000. and Longgang District Key Laboratory of Intelligent Digital Economy Security.
REFERENCES
Rishabh Agarwal, Dale Schuurmans, and Mohammad Norouzi. An optimistic perspective on offline reinforcement learning. In Proceedings of the 37th International Conference Machine Learning, 2020.
Pierre-Luc Bacon, Jean Harb, and Doina Precup. The option-critic architecture. Proceedings of the AAAI Conference Artificial Intelligence, 31(1):1226-1344 2017.
Bernard W Balleine and John P O'doherty. Human and rodent homologies in action control: corticostriatal determinants goal-directed and habitual action. Neumosychapharmucclgs 35(1):48-69, 2010.
Chengzhi Cao, Chao Yang, Ruimao Zhang, and Shuang Li. Discovering intrinsic spatial- -eripall logic rules to explain human actions. Advances Neural Information Processing Systems, 36: 771488679557 2023.
Ozan Çatal, Tim Verbelen, Johannes Nauta, Cedric De Boom, and Bart Dhoedt. Learning perception and planning with deep active inference. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3952-3956. IEEE, 2020.
Ricky TQ Chen, Brandon Amos, and Maximilian Nickel. Neural spatio-temporal point processes. arXiv preprint arXiv:201 2l.::203/.00883, 2020.
Fiery Cushman and Adam Morris. Habitual control of goal selection in humans. Proceedings of the National Academy of Sciences. (((45):13817-388)) 2015.
Artur S. d' Avila Garcez, Marco Gori, Luis C. Lamb, Luciano Serafini, Michael Spranger, and Son N. Tran. Neural-symbolic computing: An effective methodology for principled integration of machine learning and reasoning. Journal of Applied Logics, 6(4):61 1-632, 2019.
Daphne Demekas. Thomas Parr, and Karl J Friston. An investigation of the free energy principle for emotion recognition. Frontiers Computational Neuroscience, 14:30, 2020.
Ray J Dolan and Peter Dayan. Goals and habits in the brain. Neuron, 80(2):312-325, 2013.
Anca D Dragan and Siddhartha S Srinivasa. A policy-blending formalism for shared control. The International Journal of Robotics Research, 3 805, 2013.
Yan Duan, Marcin Andrychowicz, Bradly Stadie, OpenAl Jonathan Ho, Jonas Schneider, llya Sutskever, Pieter Abbeel, and Wojeiech Zaremba. One-shot imitation learning. Advances in neural information processing systems, 30, 2017.
Viet Dung Nguyen, Zhizhuo Yang, Christopher L Buckley, and Alexander Ororbia. R-aif: Solving sparse-reward robotic tasks from pixels with active inference and world models. arXiv e-prints, pp. arXiv-2409, 2024.
Kevin Ellis, Lionel Wong, Maxwell Nye, Mathias Sable-Meyer, Luc Cary, Lore Anaya Pozo, Luke Hewitt, Armando Solar-Lezama, and Joshua B Tenenbaum. Dreamcoder: growing generalizable, interpretable knowledge with wake-sleep bayesian program learning. Philosophical Transactions of the Royal Society A, 381(2251):0020005) 2023.

---

Published as a conference paper at ICLR 2026
Neil Ferguson. Capturing human behaviour. Nature, 446(7137):733 (131573))33333333-.0. 2007.
Zafeirios Fountas, Noor Sajid, Pedro Mediano, and Karl Friston. Deep active inference agents using monte-carlo methods. Advances neural information processing systems, 33:1 1662-11675, 2020.
Karl Friston, Francesco Rigoli, Dimitri Ognibene, Christoph Mathys, Thomas Fitzgerald, and Giovanni Pezzulo. Active inference and epistemic value. Cognitive neuroscience, 6(4):187-214, 2015.
Karl Friston, Thomas FitzGerald, Francesco Rigoli, Philipp Schwartenbeck, Giovanni Pezzulo, et al. Active inference and learning. Neuroscience & Biobehavioral Reviews, 68:862-879, 2016.
Karl Friston, Lancelot Da Costa, Danijar Hafner, Casper Hesp, and Thomas Parr. Sophisticated inference. Neural Computation, 33(3):713-765 2021.
William F Goette, Anne R Carlew, Jeff Schaffert, Ben K Mokhtari, and C Munro Cullum. Validation of a bayesian diagnostic and inferential model for evidence-based neuropsychological practice. Journal of the International Neuropsychological Society, 29(2):182-19)) 2023.
Danijar Hafner, Timothy Lillicrap. Mohammad Norouzi. and Jimmy Ba. Mastering atari with discrete world models. arXiv preprint arXiv:2010.02/99 /93, 2020.
Dongqi Han, Kenji Doya, Dongsheng Li, and Jun Tani. Synergizing habits and goals with variational bayes. Nature Communicutions, 15(1):4461 2024.
R Conor Heins, M Berk Mirza, Thomas Parr, Karl Friston, Igor Kagan, and Arezoo Pooresmaeili. Deep active inference and scene construction. Frontiers in Artificial Intelligence, 3:509354, 2020.
Morten Henriksen. Variational free energy and economics optimizing with biases and bounded rationality. Frontiers in Psychology, 11:549187, 2020.
Luke Hewitt, Tuan Anh Le, and Joshua Tenenbaum. Learning to learn generative programs with memoised wake-sleep. In Conference on Uncertainty Artificial Intelligence, pp. 1278-1287. PMLR, 2020.
Geoffrey E Hinton, Peter Dayan, Brendan J Frey, and Radford M Neal. The" wake-sleep" algorithm 158-1161, for unsupervised neural networks. Science, 1588111)):1 1995.
Jonathan Ho and Stefano Ermon. Generative adversarial imitation learning. Advances in neural information processing systems, 29, 2016
Woojeong Jin, Meng Qu, Xisen Jin, and Xiang Ren. Recurrent event network: Autoregressive
structure inference over temporal knowledge graphs. arXiv preprint arXiv:1904.05533 2019.
Lukasz Kaiser, Mohammad Babaeizadeh, Piotr Milos, Blazej Osinski, Roy H. Campbell, Konrad Czechowski, Dumitru Erhan, Chelsea Finn, Piotr Kozakowski, Sergey Levine, al. Mosel-based reinforcement learning for atari. In International Conference on Learning Representations, 2020.
Archit Ramanasai Kambhamettu, Abhinav Shrivastava, and Matthew Gwilliam. Quantifying nba shot quality: deep network approach. In Proceedings of the 7th ACM International Workshop on Multimedia Content Analysis in Sports, pp. 91-95, 2024.
Benedikt Leichtmann, Christina Humer, Andreas Hinterreiter, Mare Streit, and Martina Mara. Effects of explainable artificial intelligence on trust and human behavior in a high-risk decision task. Computers in Human Behavior, 139:107539, 2023.
Sergey Levine. Reinforcement learning and control probabilistic inference: Tutorial and review. arXiv preprint arXiv: 1800110000000 009099 2018
Guopeng Li, Yiru Jiao, Victor L Knoop, Simeon Calvert, and JWC van Lint. Large car-following data based on lyft level-5 open dataset: Following autonomous vehicles vs. human-driven vehicles. arXiv preprint arXiv:2305. 1 1892/, 2023a.
Liulei Li, Jianan Wei, Wenguan Wang, and Yi Yang. Neural-logic human- objeet interaction detection. Advances in Neural Information Processing Systems, 36:21 151-21171, 2023b.
12

---

Published as a conference paper at ICLR 2026
Shuang Li, Mingquan Feng, Lu Wang, Abdelmajid Essofi, Yufeng Cao, Junchi Yan, and Le Song. Ex- plaining point processes by learning interpretable temporal logic rules. In International Conference Learning Representations, 2021.
Yunzhu Li, Jiaming Song, and Stefano Ermon. Infogail: Interpretable imitation learning from visual demonstrations. Advances in neural information processing systems, 30, 2017.
Guilherme Maeda, Gerhard Neumann, Marco Ewerton, Rudolf Lioutikov, Oliver Kroemer, and Jan
Peters. Probabilistic movement primitives for coordination of multiple human-robot collaborative tasks. Aulonomous Robots, :1:59-66 2017.
Lisa A Marsch. Digital health data-driven approaches understand human behavior. Neuropsy- chopharmacology, 46(1):191-196, 2021.
Pietro Mazzaglia. Tim Verbelen, and Bart Dhoedt. Contrastive active inference. Advances in neural information processing systems, 34:13870 13882, 2021.
Pietro Mazzaglia, Tim Verbelen, Ozan Çatal, and Bart Dhoedt. The free energy principle for perception and action: A deep learning perspective. Entropy, 24(2):301, 2022.
Beren Millidge, Alexander Tschantz, Anil K Seth, and Christopher L Buckley, On the relationship between active inference and control as inference. In International workshop on active inference, pp. 3-11. Springer, 2020.
Beren Millidge, Alexander Tschantz, and Christopher Buckley, Whence the expected free energy? Neural Compulation, 33(2):447-482, 2021.
M Berk Mirza, Rick A Adams, Christoph D Mathys, and Karl J Friston. Scene construction, visual foraging, and active inference. Frontiers in computational neuroscience, 10:56, 2016.
David T Neal, Wendy Wood, Jennifer S Labrecque, and Phillippa Lally. How do habits guide behavior? perceived and actual triggers of habits in daily life. Journal of experimental social psychology, 48(2):4922-9) 498, 2012.
Thomas Parr and Karl J Friston. Generalised free energy and active inference. Biological cybernetics, 113(5):495-35 2019.
Alex Pentland and Andrew Liu. Modeling and prediction of human behavior. Neural compulation, 11(1):229-24 242, 1999.
Zenon W Pylyshyn. Computation and cognition: Issues in the foundations of cognitive science. Behavioral and Brain sciences, 3(1): 1980.
Meng Qu, Junkun Chen, Louis-Pascal Xhonneux, Yoshua Bengio, and Jian Tang. Rnnlogic: Learning logic rules for reasoning on knowledge graphs. arXiv preprint arXiv:2010. 04009,, 2020.
Alexander J Rothman, Paschal Sheeran, and Wendy Wood. Reflective and automatic processes in the initiation and maintenance of dietary change. Annals of behavioral medicine, ((ssupp..)::44905 2009.
Joanne Savage and Bryan Vila. Human ecology, crime, and crime control: Linking individual behavior and aggregate crime. Social Biology, 50(1-2):77-101, 2003.
Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention the task. In International conference machine learning, pp. PMLR, 2018
Bilong Shen, Xiaodan Liang, Yufeng Ouyang, Miaofeng Liu, Weimin Zheng, and Kathleen M Carley. Stepdeep: A novel spatial-temporal mobility event prediction framework based on deep neural network. In Proceedings of the 24th ACM SIGKDD international conference knowledge discovery & data mining, pp. 733-733 2018.
Zitao Song, Chao Yang. Chaojie Wang, Bo An, and Shuang Li. Latent logic tree extraction for event sequence explanation from llss. arXiv preprint 122 2024.
13

---

Published as a conference paper at ICLR 2026
Qwen Team. Qwen2 technical report. arXiv preprint arXtv:2407.10671 2024a
Qwen Team. Qwen2.5: A party of foundation models, September 2024b. URL hwtphhhttp/eeeeee..
github. Crathub.i//bl wtt.b////g/ww...5555555
Marc Toussaint. Robot trajectory optimization using approximate inference. In Proceedings of the 26th annual international conference on machine learning. pp. 1049-1056, 2009.
Alexander Tschantz, Anil K. Seth, and Christopher L.. Buckley, Learning action-oriented models through active inference. PLOS Computational Biology, 1007)0c 2020.
Kai Ueltzhöffer. Deep active inference. Biological cybernetics, 12(6))547--57. 2018.
Weidi Xu, Jingwei Wang, Lele Xie, Jianshan He, Hongting Zhou, Taifeng Wang, Xiaopei Wan, Jingdong Chen, Chao Qu, and Wei Chu. Logiemp: A neuro-symbolic approach for encoding first-order logic constraints. arXiv preprint wrXiv:2309115456 2023.
Chao Yang, Shuting Cui, Yang Yang, and Shuang Li. Evolving minds: Logic-informed inference from temporal action patterns. In Forty-second International Conference on Machine Learning, 2025.
Zihao Zhou, Xingyi Yang, Ryan Rossi, Handong Zhao, and Rose Yu. Neural point process for learning spatiotemporal event dynamics. In Learning for Dynamics and Control Conference, pp- 777-789. PMLR, 2022.
14

---

Published as a conference paper at ICLR 2026
APPENDIX OVERVIEW
Section A presents the rule-guided Wake-Sleep framework and online reasoning. It unifies
Wake and Sleep into a single pseudocode and details how state reconstruction, belief updating, candidate rule extraction, and model optimization integrated (Algorithms) I
3.. All definitions (model factorization, VFE/EFE, joint objective) strictly follow the
main text.
Section B shows that our model naturally equivalent to a mixture over latent states, and that rule learning and activation can be interpreted as lightweight EM-style approximation to inference and parameter updates in this mixture.
Section C describes dataset preprocessing and feature construction for NBA SportVU, Car- Following, DDXPlus, and Mari-BBrrzer, including the action spaces and the world-model inputs. It also specifies the semantic interpretations of the discrete internal state m (Sec- tion Cion consolidates key hyperparameters for data/model and optimizationhhannnngnul
(Tables 2 and reports action distribution plots across all datasets (Figure 4
Section D provides additional experimental results beyond the main text: full training dynamics (AF, VFE, EFE, KL; Figure ( testing metrics on held-out sets (Acc@ K, rule-hit rate, latency; Figure 6 rule envelopes and visualizations across domains (Figures) 7 and 8
end-to-end trajectory visualizations (Figures 2 and 10), full ablation study (Table and
Atari-100k supplementary experiment (Section D.6 Table 5).
Section E conducts sensitivity analyses and lists the exact LLM prompts used: NBA action-
parameter sensitivity curves (Figure 11), the effect of the latent mental-state cardinality m
(Figures DDXPlus has fixed m)) LLM prompts for datasets (Section E.5), hy-
perparameter sensitivity analysis (Section E.3 Tables 7 and LLM-guided component ablation study (Section E.4).
Section discusses limitations and broader impact, including domain specificity of mined rules, sensitivity thresholds and hyperparameters, computational considerations, depen- dence on world-model quality, and potential directions such as hierarchical timescales and real-world deployment.
A WAKE-SLEEP ALGORITHM
Algorithm 1 Rule-Guided Active Inference: Unified Wake Sleep Cycle
Require:
D. generative model Ps. encoder ga. policy Φr, rule set F {(S}, a7. 1: Initialize parameters (o, 0, Φx); set 0
Dataset
2: for each epoch do
3: Wake phase (real trajectories)
4: for trajectory D do
5: for time = 1:7 do
6:
Infer (S,, m²) qo (S,, me H₂)
7:
Compute per-step free energy VFE, (Eq. 2 accumulate AF
8:
AF < -ōp recurrently for (S m, a.) then
9:
Create or update rule (St m² a, with confidence p p dcon Ee w(8)S Es "(S w(S) (VFO,,,
10:
Update centroid S
11:
end if
12:
end for
13:
end for
14:
Update (o, 0) on real data by minimizing VFE (Eq.
15:
Sleep phase (replay)
for mini-batch (S, 0, a) ps do Minimize joint objective F. VFE, + EFE, + KL(-) (Eq. Update (Φ. Φx) by VF; keep qo consistent
17:
19:
Refine S*, update p. prune rules with < 8conf
end for
21: end for
This section provides algorithmic details of our framework. We unify the Wake and Sleep phases into single pseudocode (Algorithm) I) describe the hybrid online reasoning procedure (Algorithm) 2
15

---

Published as a conference paper at ICLR 2026
and summarize the two-stage training schedule (Algorithm 3 All definitions and objectives strictly follow the main text: the generative model factorization (Eq. the variational free energy (VFE,
Eq. 2), the expected free energy (EFE, Eq. 3), the rule fusion policy (Eq. 5), and the joint objective
(Eq.
Algorithm 2 Online Reasoning with Rule Fusion and Planning
Require: History Hr, encoder qo, model Ps, rule F, horizon H. beam width K
1: Infer (Sr, qe (St, m4 H.)
2: Rule prior: for each E F. k(St, ST) 2 Tr and m, = m,. add weighted vote pr for a,
3: If rules triggered, form (a St, my); else set Trule =
4: Planning fallback: evaluate candidate actions by EFE (Eq. via beam search (width K,
horizon H)
5: Fuse distributions: Pe. (oe St, me: x (as) + (1 rule bit) exp (FE)((0)) (Eq.
A.1 WAKE& & SLEEP PHASES
Wake. Operates real trajectories: latent inference by ga, per-step free energy from Eq, 2 and candidate rules from recurring (S, m, a) triplets that stably reduce free energy. Rule anchors are updated weighted centroids (exp(-VFE) as weights).
Sleep. Uses replay samples from Pý, minimizing the composite loss J, (Eq. (5)). Rules are refined (anchor shift, confidence update) and pruned if redundant or low-confidence.
This division follows the classic sake-sleep paradigm (Hinton et al. 1995 Friston et al. 2015 but adapted rule-guided active inference.
A.2 TRAINING SCHEDULE
We adopt a two-stage schedule to stabilize learning.
Algorithm 3 Two-Stage Training: Blockwise Pretraining and Full Wake-Sleep
Require: Dataset D. blocks (Do) Stage 1: Blockwise pretraining
2: for block = 1:B do
3: for mini-batch (O, .) Dr do
4: Infer S ~ (S O); reconstruct O
5: Update (s) : by minimizing VFF only (Eq. 2
6: end for
7: end for
8: Stage 2: Full Wake-Sleep training
9: for epoch 1:E do
10: Run Wake phase real data (Alg. lines 3-12)
11: Run Sleep phase with replay (Alg. lines 14-18)
Refine/prune rule pool; update confidences
13: end for
A.3 SUMMARY
Wake extracts rules via AF--aaed improvements and updates confidences. Sleep uses replay to refine rules and jointly minimize (). Online reasoning integrates rule priors and planning (Eq. 5). realized via beam search or MCTS. Training proceeds in two stages: blockwise pretraining for fast bootstrapping, then full Wake Sleep for convergence.
B MIXTURE MODEL OVER LATENT STATES AND CONNECTION TO EM-STYLE UPDATES
In the main text (Sec. we introduce rules as compact, latent-grovd grounded carriers of habitual knowledge. Here we make explicit probabilistic interpretation of the rule library as Gaussian mixture over latent contexts, and clarify how our rule scores and updates correspond to approximate EM steps.
Gaussian mixture over latent contexts. Recall that the latent state factorizes as Zr (St, m.). with continuous S, and discrete m, E [I,..., K). Each rule stores a prototype
(S m,, ar Pr),
16

---

Published as a conference paper at ICLR 2026
where Sÿ is continuous anchor in latent state space, m is discrete mental-state state label, a, is the associated action, and Pr E [0, is a nonnegative rule weight. The full rule set
F= {(S m,, ar: pr)}r
can be viewed as the parameters of a Gaussian mixture over latent contexts.
Introduce latent rule index E (I, , at each time step, with categorical prior
p(r: = r) Tr:
-T
and component-specific priors over (5,,m)) of the form
((S. rt =) = (Se;; pr, Er); p(me = r) 8(mm, m.): where (i: Mr: Exp is Gaussian over the continuous latent state and 8(-) is Kronecker delta fixing the discrete mental state. In our implementation, we tic the mean to the stored anchor, H4 = S and use shared covariance E, o² so
I p(S; r. = r) = WS; 1²,0") & exp( - 20 || - -7 Under this mixture, the posterior over rules factorizes as q(r: = m,) & p(ri
= =) p(S, rc =)= p(m re r & Tr S; ,², [m, m,).
Thus the posterior responsibility of rule is Gaussian function of the distance between S₃ and the anchor S,, modulated by whether m, matches m
Connection to the encoder and rule scores. In the main text, given new history He the encoder (z HH2) produces a posterior over (S,, my), and we use its MAP estimate S = max (S; H.). m,
MAP arg max go (m, = k S. H4))
& Assuming (S. H4) approximately Gaussian with mean SMAP, combining this approximate posterior with the Gaussian prior p(S, rı = r) yields ros-rssposss that are quadratic in ||SMAR - S
|l. i.e. Gaussian distance.
instantiate this by defining Gaussian kernel scores
(t) x< x(S MAP. .S,) 1mm MAP m ssSSt MMP .,)) == exp
26 ||S S 1T'
These scores act unnormalized responsibilities of rule r for the current context. Comparing with the mixture posterior above, this corresponds to the approximation
q(r- = S MMP
m MAP) X म, exp(- - || s;,l²) 1 (m MAP = m
i.e. a Gaussian posterior over the rule index as function of the latent state. In our implementation, we absorb ग.. into the learned rule weight and treat rx(,.) as the dominant distance-basse term. A rule is declared active if
max w. (t) Tr:
so the threshold r, simply truncates very small posterior responsibilities. When multiple rules suggest the same action a, define rule-induced action distribution
Ta SMAP ..
MAP) x Σ wrr(t) Pr.
Taap=a
normalized over a. Under the Gaussian mixture view, w, (t) approximates the context- context-dependent responsibility q(r, = r SMP m, ...AP)) MAP), and pr the learned mixture weight of rule r.
Connection to EM--tyle updates. In the wake phase, we grow and refine rules from real trajectories. When triplet (SMAP, vn. MAP ax) recurs often with low free energy, we either (i) create a new rule (St, m2, a,) with initial weight pr O, or (ii) increase the weight Pr of an existing nearby rule. Continuous anchors are updated as weighted Gaussian centroids:
Esegroal u(S)$
S
u{S) exp( VFE m, d, 0)):
Esestoa u(S)²
where Steal collects latent states assigned to rule on real data. This update is an EM-style M- step on the Gaussian means μ, in the mixture model, with u(S) playing the role of (reweighted) responsibilities.
During sleep, rules are further refined using replayed trajectories: anchors S; updated replayed latents, rule weights p, are adjusted based on how often and how well they explain latent contexts, and rules with persistently low effective weight pruned. Birth (when rule's responsibility
17

---

0, otherwise if (h,-1 h.) 2 0. Published as a conference paper at ICLR 2026
exceeds the threshold) and pruning (when rule receives negligible responsibility) play the role of cluster creation/deletion nonparametric Gaussian mixture models (e.g.. me-mss.)..
Although we present the rule mechanism in algorithmic terms (kernels, thresholds, centroids), admits natural Bayesian interpretation: rules correspond Gaussian mixture components latent contexts (St, me), with Gaussian priors over St and fixed labels for my; (x)(() approximate posterior responsibilities over the rule index; and the anchor and weight updates correspond to EM-style M-steps. This interpretation clarifies that the rule library is not an ad-hoc heuristic, but structured, lightweight and computationally efficient approximation to Gaussian mixture modeling in latent AIF space.
C DATASET PREPROCESSING AND FEATURE CONSTRUCTION
C.I DATASET
NBA SportVU. We extract frame-level coordinates of the ball and ten players from SportVU event data, filtering invalid samples (e.g.. missing entities or frames with fewer than 11 tracked objects) and sampling up to 20 clips per game. Since raw coordinates are insufficient for rule construction. we construct new parameterized features under the guidance of large language models (LLMs) and conduct rationality analysis validate feature choices. This yields interpretable features such as relative velocity, spacing, and formation compactness. The final dataset contains about 9.8k training samples and 2.5k validation samples, with an action space of classes (e.g., straight run, turn, dribble).
Car-Following. The original traffic data do not explicitly provide environmental dynamics. We therefore define an action world model, where observation features are constructed from action sampling statistics (e.g., acceleration and headway patterns) to characterize driving dynamics. The resulting dataset consists of about 19k training samples and 2.5k validation samples, with driving modes such cruise, follow, and accelerate.
DDXPlus. The DDXPlus dataset consists of diagnostic trajectories generated by multi diseese Naive Bayes teacher model. We select URTI, the most frequent disease, as the target condition for diagnostic inference. Each trajectory contains sequence of ASK actions (doctor's inquiries) followed by a final DIAG action. After preprocessing and formatting, we obtain about 165k training samples and 25k validation samples, with an action vocabulary of 225 classes.
Atari-Berzerk. We use high-score human demonstration trajectories in the Atari Berzerk game (representing strong human intelligence). Raw RGB frames are converted into 128 x 128 grayscale images. Each frame is paired with the corresponding human action, drawn from 18 discrete classes (movement, positioning, firing, etc.). The processed dataset contains about 16.5k training samples and 16.5k validation samples.
C.2 NBA SPORTVU
Action space. We follow the analytic definitions and symbolic feature construction described in the main text and experimental log. Seven discrete basketball actions defined from player-ball relations. Let Pa E 2²² denote the player's position, b, E R2 the ball position, and h3 E R² the unit heading vector (h, IIr: PP- IT) Define constants
Pc
Dariblle = 2 ft,
Dretense 6 ft, Dreceive 2 ft.
Then the discrete action a, given by
4, lpyt by|l S Dariblee
5,
Pp bt-11 1|| < Dariblle
Ilp: - bull 2 Drelease min, !!y1 - b,l < D Dreseives
6,
||pt-1 b.-1ll < Dambilee
llp: bull 2 Dreleuse min, !/v! - b, > D receivee
0z
3,
(h1-1 he) >
2,
(h4-1 he) < -E,
1, otherwise if (he-1 h4) < 0.
where 1/18 (109) and v are defender positions.
Symbolic feature extraction.
For each historical frame / 1,..., H:
18

---

Published as a conference paper at ICLR 2026
(i) Soft direction-kernel features. ((1,0), (0, 1), (-1,0))) -1)}, define
For opponent
and basis direction b,
1l 1/11 20²
Pi = exp
max (b, u),
where u/
v and σ 10. | p
(ii) Relational distance/angle features. Let dj llpt - v''l and sort d(i) S d(2) S
d(3)- Define
dim lpp - rll. frim arctan 2(ry - Py: ps), dmean 1
Σ dj.
|01-1
jEO X (handler)
Then form
of [d() d(2). d(aj= dim- Prims dwean
Further sensitivity analysis of LLM guided action parameters, as well the prompts used for generating symbolic actions, are provided in Appendix
LLM-guided
C.3 CAR-FOLLOWING
For the car- car-following domain, use the open-source trajectory dataset where each run is recorded as sequence of driving regimes. Data are extracted from HDF5 files and organized into fixed-length training samples.
Preprocessing.
Each sample is represented by:
Previous-K K one-hot encoded regimes (history of executed actions).
dt: time interval between consecutive frames.
rum_len: cumulative length of the current driving run.
Jast time elapsed since the last regime change.
This yields a structured observation vector per frame.
Action space. We adopt seven discrete regimes (e.g., constant speed, acceleration, deceleration, free-flow, car-following, closing-in, and emergency braking), directly encoded in one-hot form.
C.4 DDXPLUS
We use the DDXPlus dataset, which consists of synthetic diagnostic dialogues covering multiple pathologies. Each trajectory represented sequence of evidence acquisition (ASK) and diagnostic (DIAG) actions.
Preprocessing. Each record in the original dataset contains a set of evidences with associated ground-truth diagnoses. We construct training samples
Evidence parsing: convert raw evidences into tokenized observations.
ASK/DIAG sequence construction: generate trajectories where the agent sequentially asks for evidence or outputs diagnosis.
Subset selection: restrict to URTI pathology for controlled experiments, with limits on train/val/test sizes as documented in the experimental log.
World model representation. For each evidence e, compute embedding E2V(e) (Evi- dence2Vec). At each step, the state representation concatenates:
The Evidence2Ves Vec embedding the most recent evidence.
A Top-K posterior vector over candidate diagnoses.
The entropy of the posterior distribution as an uncertainty measure.
The action vocabulary consists of ASK tokens (corresponding to medical ev- idences) and DIAG tokens (candidate diagnoses). This yields discrete action set comparable to multi-class classification.
Action space.
C.5
ATARI- BERZERK
We use human gameplay trajectories on the Atari BERZERK environment, where each frame a raw image and actions correspond to discrete joystick commands.
19

---

Published as a conference paper at ICLR 2026
Preprocessing. Game episodes unpacked into frame sequences. Each frame is preprocessed by:
Resizing to 128 x 128 pixels.
Converting grayscale.
Normalizing pixel intensities to [0, 1].
Sequences are then segmented into fixed horizons with stride, producing training samples aligned with action labels.
World model representation. A vision meooer-decoded architecture is used to reconstruct frames and predict latent states. The encoder extracts spatial features, while the decoder ensures faithful reconstruction for VFE minimization. Temporal dependencies are modeled by Transformer-basee dynamics module.
Action space. We adopt the original Atari action set with 18 discrete joystick commands (e.g.. move directions, fire, stay). Each action treated as a one-hot token in training.
C.6 INTERNAL STATE DEFINITIONS
LLM-guided mental-state matching. We construct uniform matching method for mental states M = {m...... m,), which correspond to interpretable intentions or sub-goals and, together with the external continuous state St, form the latent pair (S) that drives rule triggering and planning. Unless dataset already provides salient labels that can directly play the role of mental states (e.g., severity judgments in DDXPlus), we rely on large language models (LLMs) as expert guidance to generate discrete candidate mental states conditioned on task-specific context, yielding semantically grounded labels (e.g., defensive/offeensive sub-goals in sports, conservative/aggrssiive modes in driving). The exact prompts and semantic candidate lists are given in Appendix E.5 while a sensitivity analysis over the number of mental states K appears in Appendix E.2
At the optimal number of states, we interpret each m as follows:
NBA SportVU. Three mental states are used:
m = 0: Habitual/exploit - stable ball handling or passing routines.
* m = 1: Explore - probing maneuvers or less frequent moves.
* 2: Subgoal switching transitions between attack patterns (e.g., dribble shoot).
Car-Following. Two mental states used:
0: Exploit stable regimes such as constant speed smooth following.
m Explore - rare or abrupt switching regimes (e.g., sudden braking, acceleration).
DDXPlus. Five mental states are used:
m = Early exploration of evidences.
m = 2: Focused questioning around relevant symptoms.
* 3: Transition phase toward diagnosis.
4: Confident diagnosis with supporting evidence.
m = Over-exploration or redundant questioning.
Atari-Berzerk. Four mental states are used:
m = 0: Exploit - regular movement in safe zones.
m = 1: Explore - irregular actions or novel paths.
* m 2: Danger/Escape evasive maneuvers when surrounded by enemies.
= 3: Aggressive attack - high-risk firing at opponents.
These semantic interpretations derived from LLM-guided prompts and verified by sensitivity analysis of the number of latent states (Appendix E.2
C.7 TRAINING PARAMETERS
Model backbones across domains. For structured sequential data (NBA, Car-Following, DDX- Plus), the generative model is parameterized by a Transformer (NBA, DDXPlus) Gaussian BiGRU Car--Following. modeling latent dynamics (S, St-1, at-1) 1) and observation reconstruc- tion (O, SS.). The inference model is corresponding Transformer/BiGRU encoder approximating go (S, m, H,). The policy head conditions on (S,, my) and outputs discrete action distributions via fully connected layer with softmax Gumbel-softmal For temporal visual data -erzerBer
20

---

Published as a conference paper at ICLR 2026
Table 2: Data windows, sizes, action vocab, model components, and parameter counts.
NBA SportVU
Car-Following
Hist/ / Pred window
10/5
10/5
Train Val Test
9,840 2,460
19,250 / 2,500
#Actions
7
7
Batch
64
64
Input resolution
Model components
Transformer
BiGRU prevK, dt, run lasc
Model input features
obs hbst / obs_pred
#Params (x10%)
~3.5M
2.2M
DDXPlus
Atari-Berzerr
Hist Pred window
all Re 165k 25k / 25k
10/5
Train / Val / Test
(shard 5k)
16,541 16,581 /-
#Actions
225 (ASK+DIAG)
18
Batch
64
64
Input resolution
128x 128 (gray)
Model components
Evidence2Vec + Transformer
CNN encoder-decoder + Transformer
Model input features
symptom/test embeddings (Evidence2Vec)
raw frames
#Params (x10%)
8.5M
13.66
Table Optimization schedule, planning, and rule-related hyperparameters.
NBA SportVU
Car-Following
DDXPlus
Atari-Berzerk
Pretrain blocks epochs
5/1
5/1
5/1
5/1
Pretrain LR
1x10-3
1x10-4
1x10-3
1x10
Full epochs
15
15
15
15
Full LR
3x10-4
1x10-
3x10-1
1x10-4
Weight decay
0.01
0.01
0.01
0.01
Warmup steps
2000
4000
16000
2000
(1)
(0.05,
(0.01, 1)
(0.01, 10)
(0.2, 5)
Planning horizon
4
4
4
Beam width
6
Rule thresholds
(0.25, 0.75, 0.75) (0.25, 0.75, 0.75)
(0.5, -, 0.75)
(5, 0.75, 0.75)
(AF, Saup: d
the generative model uses a convolutional encoder (CNN) to extract frame-level features, combined with temporal Transformer to model latent dynamics, and decoder reconstruct (O; St). while the inference model jointly processes CNN features with temporal modules to approximate (S,, m, H.,). The policy head applies MLP with softmax the latent representation to output distributions over actions.
Tables [ and 3 list dataset-specific settings strictly taken from the experimental log and consistent with the main-text modules.
Notes. indicates not specified in the log; keep it unspecified here. Rule thresholds dataset-specific as recorded. Planning uses beam search; horizon refers to the planning horizon (not the hist/pred window). All settings align with Appendix A
C.8 ACTION DISTRIBUTION
To further illustrate the characteristics of our datasets, we plot the empirical action distributions for all domains (NBA, Car-Following, DDXPlus, Atari-Berzerk) in Figure 4 These histograms reveal highly imbalanced patterns: for instance. NBA is dominated by the straight action (~69%), Car-Following by cruising (F, ~55%), DDXPlus by a few high figh-frequency ASK/DIAG queries (top-10 10 actions 60%), while Mari-Berzerk is dominated by move-and-fire combinations (> 50%) with many rare actions < 5%.
Despite this imbalance, our framework adapts well: frequent actions are mainly handled by active inference and multi-step planning, while infrequent actions captured effectively by mined rules.
21

---

Published as a conference paper at ICLR 2026
NBA SportVU Action Distribution
Car-Following Action Distribution
60
50
40
40
Percentage
30
20
20
10
Straight
Dribblo
Pass
Receive
Shoot
Tumaround
Other
(a) NBA VU
(b) Car-Following
DDXPlus Action Distribution
Beari-Berzerk Action Distribution
30
Percentage 1961
Percentign (8)
15
20
1.0
15
Movesfire Fire
Move Only
Fire onl
Idle
Diaponal
Special Combo
other Rate
7OP2
Top-3
Top-d
Tigi-5
Top-9
7op-7
Top-8
Top-g
Top- 10
Others
(c) DDXPlus
(d) Atari-Berzerk
Figure 4: Action distribution plots for all datasets. Each domain exhibits significant skew (e.g., NBA dominated by straight, Car- Car-Following by F, DDXPlus by top ASK/DIAG queries, Atari by move+fire combinations), yet our framework adapts by leveraging rules for rare actions and active inference for frequent actions.
This synergy ensures that rare but semantically distinct behaviors (often tied to edge-case conditions) form clean rule clusters that are easily separated, while frequent actions are supported by robust predictive inference, As result, our method naturally balances between rule coverage and world- model plamning, yielding strong performance
under skewed data distributions.
This also explains the improvements over baselines (Table the rule-guided component is especially beneficial for rare actions, while the world-mode) inference sustains accuracy on dominant classes, leading to overall gains in accuracy, latency, and interpretability.
D ADDITIONAL EXPERIMENTAL RESULTS
D.1 TRAINING CURVES
Figure reports reports the complete training dynamics for all datasets. Each panel shows the two-stage schedule (blockwise pre-training followed by full wake-sleep training), including trajectories of AF, VFE, EFE, KL terms, well rule count and inference latency.
Across all domains, we observe consistent pattern. the pre-training stage, VFE drops quickly as the world model learns to reconstruct observations, while AF remains relatively high due to unexplored policies. During the slee- stage, both EFE and KL steadily decrease, indicating better exploitation of action sequences and improved belief calibration. Meanwhile, the number of rules increases sharply before saturating, mirroring the consolidation motifs. This growth reduces inference latency since frequent or edge-case behaviors matched directly by rules rather than through full rollouts. Overall, the curves validate our design: pre-training bootstraps stable perception, while full sleep training integrates symbolic rules with active inference to minimize free energy and accelerate decision making.
interpretable behavioral
D.2 TESTING METRICS ON HELD-OUT SETS
Across datasets, several consistent trends emerge (Fig. (i) Accuracy: All domains show monotonic gains in Acc@ 1/3/5 after switching to full-training, with NBA and Car-Following saturating > 90% top-5 accuracy, and DDXPlus steadily climbing from low baselines to > 80%. Atari, despite its image-based complexity, also benefits from rule integration reach strong top-5 scores. (ii)
22

---

Published as a conference paper at ICLR 2026
3000
pee-tmining
ralersiiin
terms
466
pre-franit
fel texiiing
Loss lazza
Rol tees
VTE
200
1o0
2212
(coetol)
Allenti I' (carmo) Nocale F (uxetreD
Nocdl Reaased)
10008
Other secticss Padle coen
Rade connt
10000
Other metrica
soar. Tite (ee)
2008
Ttee (to)
stops /specs
(a) NBA SportVU
(b) Car-Following
reB-rrlning
Losa terma
ee-rriniing
ndl-trabing
Laoc forms
Eacry llsss
2000
3000
EFS KLL
1200
Ali-nte 3 (contol) No 1 (curind)
Allcale F (nariu) No-alk (centrcl)
Ruie auust
Otber matria
2000
Time (zs)
Time (ma))
30-
Training staps <pechs
Tr Training steps/ spochs
(c) DDXPlus
(d)
Figure 5: Complete training curves for all datasets. Each panel shows the two-stage schedule (pre-training then full-training with the evolution of AF, VFE, EFE, KL, together with rule count and inference latency.
pre-traidaeg
fallerddiaa
pre-trabiig
Metes
REIR(N)
Percutage (N)
laumootiaa
20
Testing step <pocc
Twiting emppoppoh
(a) NBA VU
(b)
Car-Following
100
pre-tratee ai--traintee
Nol-tratatng
Metrica
308
pratralning
falitrabiig
Metics
RSB:54)
RHRIN
60-
35088774)
34-
175
220
100
Perceitgg
60
60
Indingstep sppch
Teeling stpp /spack
(c) DDXPlus
(d) Atari Berzerk
Figure 6: Testing metrics on held-out sets. Each panel shows the evolution of Acc@ K, rule-hit-rate (RHR), and inference latency during training. The transition from pre-training (grey) to full-training (white) leads to stable improvements across datasets.
Rule-Hit Rate (RHR): Rare actions (e.g.. NBA shoot, Car-Following regime switches, DDXPlus rare ASK queries, Atari panic-fire) are disproportionately captured by rules, leading to elevated RHR in the early epochs of full-training. This validates the complementary role of symbolic rules in handling skewed action distributions. (iii) Latency: Inference latency drops sharply once the model stabilizes after the pre-training phase, and remains low and consistent. This is because rules
23

---

Published as a conference paper at ICLR 2026
encoder Space
Explore (2-4)
S_ encoder Space Explore (z=1)
S. eeenodder Space Explore (x=2)
Wuth
0: straight
dribble
6 pass
lef
right
turn around
S: sboot
6rgass
dribble
decoder
Explore (x=0)
decoder Space Explore (x=1)
decoder Space
Explore (z=2)
pods
Hoe the"
(66 Giai sitallss
Rad
Of straight
dribble
pass
left
right
3: around
shoot
6:pass
4: dribble
Jul -0.32
AF -0. Aec 59%
AP
-1.2
SP -3.00
A -8.05
rofl
Arn:
Acc
Anc sall.
Anc T1/6
hit
(a) NBA Vp
S. encoder Space 0 (Conservative)
S. encoder Space --1 (Radical)
Accelleaat
Follow
Fai Faxt Acclenate
Decelemte
Fd: Fast Decelerate
115022 25
decoder Space z-0 (Comservatvv)
decoder Space (Radical)
Accelorate
AF; -0.22
Ace: 84%
Acc :
Car Follow 0.23
Aeclerate -0.4
Are: 88%
Ace: 73%
Zalla pole
Decelente
Fdl: Fast Decolerate
AF 0.25
Spila
20 oiddl Cle
AF -0.37
Aec 81%
A 711%
hit
not-hit
wrong
(b) Car- Car-Following
Figure 7: Rule envelopes and visualization across datasets (Part 1). Each panel shows representa- tive encoder/decoder spaces image states, where rules emerge as compact clusters envelopes (colored ellipses). The learned rules successfully capture domain-specific behaviors: (a) NBA: interpretable envelopes around straight, dribble, pass, and shoot; (b) Car-Following: distinct clusters separating acceleration//eccelratioo regimes.
bypass expensive inference for rare but critical cases, while frequent actions rely on streamlined active inference.
Taken together, these results confirm that our hybrid framework achieves robust accuracy, rule coverage, and efficiency simultaneously, and adapts well to domains with highly skewed action distributions.
D.3 RULE ENVELOPES AND VISUALIZATION
Across datasets, the rule visualizations (Fig. and Fig. g highlight the complementary role of symbolic envelopes:
24

---

Published as a conference paper at ICLR 2026
encoder Space
encoder Space
S_ encodder Space
23
210
You f
plin
14
ASK: B
40
DING URTI
d'dennder Space
z-1
S. decoder Space 2-3
decoder Space 2-4
-I pim lltammpiiaatio
(ddierprllbta""
ASKLE E:40
DIAG. URTI
ASK. E 197 AP:
ASK, E
DIAG UKTI
ASK 120
ASK: E 136
DIAG UKTI A.:
AP: -0.42
:P.4
-0.55 Acc: 78%
AF; -0.50
AF -0.5 Ace: 17%
&P: -0.55
A. -0.70
Aue: 8005
Ace 79%
68%s
hit
not-hit
wrong
(a) DDXPlus
x-0 (exploit)
(explore)
2-2 (danger)
Encoder
Decoder
Action: S(MoveFrre) AF: -5.5
Action: 1(Up) AF:-6.1
Action: (Panicllrre) A7:
Acc: 83%
Acc: 75%
Acc: 78%
(h) Atari-Berzerk
Figure 8: Rule envelopes and visualization across datasets (Part 2). Continued from Figure (c) DDXPlus: ASK/DIAG trajectories forming well-separated thematic clusters; (d) Atari: pixel-Tevel rules aligned with human gameplay semantics (exploit, explore, danger).
Compactness & interpretability. Rules appear as tight ellipsoidal regions latent spaces, clearly separating heterogeneous actions (e.g., NBA dribble vs. shoot, Car-Following accel- crate vs. cruise).
Low-frequency action capture. Rare but semantically important actions (c.g., DDXPlus critical DIAG, Atari panic-fire) form easily isolatable clusters, supporting earlier finding that rules excel at handling imbalanced data distributions.
Cross-domain generality. Despite domain differences (trajectories, medical dialogues, raw pixels), the same principle holds: rules provide sharp local decision boundaries, while the world-model sustains performance on frequent, high-density regions.
These visualizations complement the quantitative results: rule-guided inference consistently improves on low-frequency actions, while active inference stabilizes dominant classes.
25

---

Published as a conference paper at ICLR 2026
3. Interpretable Reconstruction (Action Space)
1. Ground Truth vs. Reconstruction
Latent Space
Highh Screeh Area
172
170
Cuising
Ground Truth Trriectory
Rule: High Scremn
AF(0) (lower better)
Trajectory
confidence
Prodiction
Drbbic
Dribble
Onhote
Shoot
Drbiric
Pas
Drikl::
bchalc
Time Stop
(a) NBA Vp
Ground Truth Reconstruction
Latent Space
3. Interpretable Reconstruction (Action Space)
#x Fel Ex
25
JM
Ground Truth
Rule:
AF(T)
Reconstruction
Rule: Fa
Action Confidence (%)
A300
Fast Acniaras
12
Time Step
Car-Following
Figure 9: End-to-end trajectory visualizations (Part 1). Each panel summarizes one test trajectory with four synchronized views: (i) GT vs. reconstruction (3D curve); (ii) latent path with rule envelopes: (iii) interpretable reconstruction in the native domain (courtroadkdialog/screnn with rule matches; and (iv) time series of (t and action confidence with shaded rule-matchen spans.
D.4 TRAJECTORY VISUALIZATIONS
To further demonstrate how framework integrates generative modeling and symbolic rules during sequential decision-making we provide trajectory-level visualizations across all datasets (Fig. 9 and Fig. 10). These examples reveal the fine-grained dynamics of action prediction and rule invocation on individual sequences.
Each panel illustrates four complementary perspectives: (i) GT vs. reconstruction in a 3D latent space, highlighting the fidelity of the world model; (ii) the latent-state path overlaid with rule envelopes (colored ellipsoids) where compact regions indicate high-confidence rule matches; (iii) an interpretable reconstruction in the native action/spatial domain (e.g., court or road overlays, dialog turn canvas, game screen) showing how rules correspond to semantically meaningful sub-sequences; and (iv) a time series of (t) (lower is better) and action confidence, with shaded spans marking malehmatched intervals.
Across domains, consistent pattern emerges: entering rule envelope typically coincides with local decrease of (t) and rise in confidence, indicating that rules complement the generative model by stabilizing predictions in distinctive regions (often edge cases). Conversely, frequent behaviors are primarily handled by active inference without rule triggers. This interplay yields both stable performance and interpretable decision traces held-out trajectories.
26

---

Published as a conference paper at ICLR 2026
1. Ground Truth vs. Reconstruction
2. Latent Space
3. Interpretable Reconstruction (Action Space)
222
750
40
Ground Truih Reconstruction
Rule: DIAG:URTI
Rale: ASK-E 136
Action Confidense (%6
ASK
ARO)
APO
Diago-er
Time Step
(a) DDXPlus
Ground Truth
Latent
Reconstructio
MoveFire
Strafe
Move Rule: MoveFire
Rule: Strafo
AF())
Action Confidence (%6
Caltdroee (%)
Fin:
2.0
Smaft
Time Step
(b) Atari-Berzert
Figure 10: End-to-end trajectory visualizations (Part 2). Continued from Figure Across datasets, rule matches align with local reductions of AF()) and confidence peaks, supporting the complementary mechanism: frequent patterns rely active inference, while rare/edge-case segments are captured by rules, yielding stable predictions and interpretable timelines.
D.5 ABLATION STUDY
We report the full ablation results across all four datasets in Table Variants include: (i) w/o Rules, removing all rule triggers; (ii) Rules w/o 2, keeping rules but removing the latent intention; (iii) VFE and Only EFE, dropping generative consistency; (iv) Greedy, using rule hits without probabilistic arbitration.
Findings per dataset. NBA. Rules deliver clear accuracy and latency gains: without them, Acc@3 drops from 91.4% to 79.5% and latency doubles (36-+73 ms). Latent m, sharpens envelopes: removing reduces Ace @3 to 85.9%. Generative consisteney is crucial: -VFE lowers Acc@3 to 67.4%, showing that observation- cow.asiionnnnntteettttt dynamics are necessary for reliable arbitration. Greedy selection is very fast but unstable (Ace@3 87.1 1%).
27

---

Published as a conference paper at ICLR 2026
Table 4: Full ablation results across four datasets (mean across 3 random seeds). Columns report top-k accuracy (%), latency per step (ms), and HHAR (%).
NBA SpurtVU
Car-Following
Variant
Acc
Accle
Lan
HHAR
Accit
A
Accit
Lai
HHAR
Pull (Ours)
97.00:20..1
91.32:5799
85.9::0099
35.92:27
25.24:2..8
96.77:0334
95.87:20.4
94.1612.47
10.44:20.9
56.82::9.6
wo Rules
83.25:20.18
79,23:20.22
61.72:0.28
12
23.6310.51
92 92.3710077
87.30:1036
83.0910.20
18.68:2mm
24.381:0.27
Rules w/o
93.46:20.11
86.45:0544
$0.0140433
49.63
7144660.76
93.48:0008
89.7540030
$6.01-0225
14.02-2030
43.10
(Relesas)
72.60::0331
67.25=0051
58 58.56605588
10.60--::72
38.421161
$4.0920:31
78.41=0026
71044:008
10.04:0%
35.294:0.0
Only EFE V-EEE--KK KL)
71. 71.64:20:30
67.21
58. $8.39:2555
29.87:1111
36.51:11577
$4.15:20200
77.15:2044
69 69.57
10.03:2033
53.51
Greedy (Rules+:)
94.70:10.25
87.28:20:55
69.14::0.55
14.84
44.56:11.50
93.82:0011
87.23:20:11
81.461:0277
2.74::033
28.49:::.27
DDXPles (URTI)
Atari-Berzech
Variant
Acc@1
A
Acc
Lat
HMAR
Acc@1
A
Acc@5
Lai
HIAR
Full (Ours)
79.63:1:.54
73.58
68.07:1:440
459.45:445
73.991145
85.55:00777
77:20:1001
72441:145
92.63:2299
66.35::..6
wlo Rules
39.27:1:.35
33,82
28.48 28.48:10.75
222.95
2:05
71,39:10.300
60.37:2044
56.39.10.20
119.92:11400
38.52
Rules wio
71.96-20.16
67
568880699
201.94=216
38 46.96.00040
76. 16.12+005
70.93=0226
66.9440455
101.90
61,33-b0744
-FF (Rales+s)
73.98:2097
65.19:2..m
52.3341177
150,47=2.0
58.53-11.16
74.3820050
66.0912779
60.55-1091
100.53::15
52.4312077
Only EFE (-VEE-------- -L)
72.68:1:991
64.63:::.00
19.56:1166
1/9.26:22.58
16.51
74.01.:0.53
65.74:1..4
59.12:1:.55
99,71:1144
50.51
Groedy ()ules)
71.03=0855
64.16:1111 49.77+1.36 106.86=238
52.23++1188
79.11
69.37
63.204:..11
18.90
38.943:0.3
Car-Following Compact rules already saturate performance, and ablations confirm their necessity: without rules Aco@3 falls to 87.3%. Dropping VFE causes large accuracy losses (78.5%). Greedy gives minimal latency (2.6 ms) but sacrifices accuracy and HHAR.
DDXPlus. With large 225- vocabulary, rules especially benefit low-frequency actions. Removing them collapses Acc@3 from 73.6% to 33.2%. Removing 2 or VFE also yields severe drops.
Mtari-Berzerk. Rules and my jointly structure pixel- pixel-based latents. Removing rules lowers Acc@3 from 77.3% to 60.3%, and removing z reduces it further. VFE consistency again proves critical, while greedy selection gains speed but loses accuracy.
Conclusion. Across all domains, the full model (rules latent intentions + generative consistency) provides the best balance of accuracy and latency. Each ablation removes a distinct capability: speed (no rules), precision (no 2), grounding (no VFE), stability (greedy).
D.6 ATARI-100K SUPPLEMENTARY EXPERIMENT
To evaluate our algorithm's 's performance on larger- scale datasets and assess model generalization under data-limited conditions, we conduct a supplementary experiment following the Atari- 100k benchmark protocol (Kaiser et al.) 2020). Atok is a low-interaction budget benchmark that limits each game 100,000 agent steps (approximately 2 hours of human gameplay), emphasizing data efficiency and rapid learning-a more challenging setting that better reflects real-world constraints. Unlike online RL settings where agents interact with the environment, we use offline data from the DQN Replay Dataset (Agarwal et al., (2020) to construct frame-action pairs for world model learning, We subsample 100,000 steps per game to match the Atari-100k benchmark protocol, selecting top-scoring trajectories to ensure expert-level demonstrations.
Dataset Description. We select 3 representative games from the Atari-100k benchmark, spanning different complexity levels. The games are chosen to represent a spectrum of difficulties: from simple games with minimal visual information to more complex games requiring sophisticated decision-making:
Pong: Simple game with minimal visual information (only paddles and ball). This tests our method's ability to handle games with sparse visual cues.
Breakout: Medium complexity game with moderate visual information (bricks, paddle, ball). This represents similar complexity to Berzerk but with different game mechanies. Qbert: Medium-to-high complexity game with moderate visual information (pyramid structure, enemies, player character). This tests requiries more sophisticated decision- making than Breakout.
For each game, we follow the same preprocessing and action space configuration as Atari-Berzerk (see Section C.3 and and C.7 for details). We also use the same model architecture and training procedure as Mari-Berzzrr. All other hyperparameters (encoder architecture, world model, mental states, planning configuration, optimizer settings) are identical to Berzerk ensure fair comparison. The key differences are:
Training Data Limit: Due to the 100k-step constraint, we train for 40 epochs with early stopping (vs. unlimited training for Berzerk).
28

---

Published as a conference paper at ICLR 2026
Table 5: Results on selected Atari-100k games and Atari-Berzerk for comparison (mean across 3 random seeds). Columns report top-k accuracy (%). latency per step (ms), and HHAR (%)
Atari- games are limited to 100,000 training steps, while Berzerk uses ~16.5k samples with unlimited training.
Game
Acc@1 (%)
Acc@3 (%)
Acc@5 (%)
Latency (ms)
HHAR (%)
Pong
86. 06.6
90.28++0.58
92.51++0.22
32.41+1.115
78.15+0..85
Breakout
72.17+00.75
76.69-+0.68
79.29++0.82
75.91+1.35
58.13++0.22
Qbert
68.17+0.82
73.18+0078
76.42++0.88
82.31+1.42
52.13+1.05
Atari-Berzerk
85.55+0.877
77.20++0.92
12.44+1.005
92.63++2.29
66.35-+0.96
Rule Bank Sizes: Game-specific rule bank capacities: Pong (max 80 rules), Breakout (max 120 rules), Qbert (max 150 rules), reflecting the complexity differences. Rule-hit threshold Tr = 0.75 for all games (same as Berzerk).
results on the 3 selected Atari- games, along with our main Atari- Berzerk results for direct comparison. Performance varies across games, reflecting their inherent difficulty differences and the data-liman lami-ed setting:
Results.
Analysis by Game Complexity:
Pong (simple, sparse visual): Achieves high accuracy (Acc@l I: 86. 88%18%) with low latency (32.41 ms), slightly higher than Berzerk's 's 85.55% but with significantly better latency due simpler visual processing (only paddles and ball). The minimal visual information allows the model to lear effective rules quickly even under the 100k-step constraint. HHAR is high (78.15%) because rare actions (e.g.. specific paddle movements) are well-captured by rules.
Breakout (medium complexity): Achieves moderate accuracy (Acc@1: 1: 72.17%) with reasonable latency (75.91 ms), lower than Berzerk's 85.55% due the 100k-step data constraint. The moderate visual complexity (bricks, paddle, ball) requires more sophisticated feature extraction, and the limited training data makes harder to learn effective rules compared to Berzerk's unlimited training. The latency is lower than Berzerk (75.91 vs. 92.63 ms) because Breakout's visual scenes less complex than Berzerk's 's dynamic combat scenarios.
Qbert (medium-tee to-high complexity): Achieves lower accuracy (Ace@ I: 68.17%) with higher latency (82.31 ms) due to spatial reasoning requirements (jumping on pyramid tiles, avoiding enemies, timing jumps). The game requires precise timing and spatial awareness, making it more challenging than Breakout, especially under the 100k-step constraint. The latency lower than Berzerk but still substantial due to the need for spatial reasoning.
Discussion. This experiment demonstrates that our framework can scale to diverse Atari games with varying complexity levels under data-limited conditions. The results show clear performance differences across games: simple games with sparse visual information (Pong) achieve high accuracy with low latency, while more complex games requiring spatial reasoning (Qbert) achieve lower but still reasonable accuracy. The direct comparison with Berzerk in Table cllearly illustrates the impact of data constraints: the 100k-step limit significantly affects performance on complex games (Breakout and Qbert show 13-17% accuracy drop compared to Berzerk), while simple games (Pong) can still achieve comparable performance. This addresses the reviewer's suggestion to test on the Atari- 100k benchmark, demonstrating that framework is game-agnostic and handle larger-scale benchmarks even under data constraints. The primary challenge is data efficiency (learning from limited trajectories) rather than methodological limitations, validating that our approach can scale to diverse sequential decision-making tasks.
E SENSITIVITY ANALYSIS AND LLM PROMPTS
E. NBA ACTION PARAMETER SENSITIVITY
We analyze how LLM-guided thresholds the NBA action definitions (Sec.
(Sec.(C.2) C.2 impact performance. Figure I1 reports @cc@3 3 under controlled sweeps of eight parameters: dribble diit, release_dis receive dist, straight turnaround deg. post_release aeso.rllaaeelookaheadd pass-persiae and ball-speed Each panel shows smooth unimodal trend with performance dropping at extremes, and a star "best" marker at the selected setting used in experiments; the gray dashed line indicates the default prior suggested by the I.I.M
29

---

Published as a conference paper at ICLR 2026
(50
60003
440
250
225
230
gphahaad
Ianarand 50g
309, pronce booahood
u0003 (50
2900 90/
but Jocod
Figure 11: NBA: sensitivity analysis of LLM-guided action parameters @3. The star marks the selected value; the vertical dashed line shows the LLM prior suggestion.
44.002
42,003
45.2
Infernno Manncc (mi3
06.000
06.973
06.
32.9
S2-913
10
14
10
14
10
11
opoch
opoch
o50ch
Figure 12: NBA: effect of the number of mental states m on Acc@5, training time, and inference latency.
130
Inforencc laturcy (ma)
13.000
Accige (5)
400
0,00
4,000
2/00
10
14
14
14
opoch
opoch
420ch
Figure 13: Car-Following: effect of the number of mental states Acc @55, training time, and
inference latency.
E.2 LATENT MENTAL STATE SENSITIVITY
We further study the cardinality of the discrete mental state m (semantic interpretations Ap- pendix C.6). For NBA, Car-Following, and Atari-Berzerk. Figures 12 plot three views: (left) Acc@: @ versus epochs, (middle) training time, and (right) inference latency. Increasing the number of states improves early learning but may increase compute; modest cardinality yields favorable accuracy-efficn -ffuracy-eficcinccy tradeoff (the selected m per dataset is reported Appendix [.6p. For DDXPlus, the number of states is fixed per the experimental log, sweep is reported.
30

---

Published as a conference paper at ICLR 2026
1,000
33
1,000
Infarmncce latenno Latenny (mna)
Treening STIH (1r4)
Acc485 (16)
100
600
14
14
opoch
Figure 14:
effect of the number of mental states m on Acc @5, training time, and inference latency.
Table 6: Sensitivity analysis of rule-hit threshold т, across datasets. Columns report Acc@3 (%) for each threshold value. The optimal value highlighted in bold. Note that optimal thresholds vary by dataset, reflecting domain -spacffc characteristics.
Dataset
7, = 0.6
т,
0.7
т,
0.8
T, = 0.9
NBA Vp 88.24
90.18
91.32
89.67
Car- Car-Following
94.12
95.87
95.41
95.23
DDXPlus
70.84
72.31
73.58
71.92
74.63
77.20
76.18
75.88
E.3 HYPERPARAMETER SENSITIVITY ANALYSIS
We conduct a comprehensive sensitivity analysis key hyperparameters across all datasets to assess the robustness of our method. The analysis focuses on two critical parameters: the rule-hit threshold (T₂) and the planning temperature (r). Results demonstrate that parameter choices have substantial impact performance, with accuracy variations of 2-8% across reasonable parameter ranges. This sensitivity is consistent with the rule trade-off analysis (see Figure 3). which shows that rule bank size significantly affects performance (e.g.. NBA Acc@3 3 varies from 79.5% to 91.4% to 61.9% as rule count changes from to to 256), indicating that careful hyperparameter tuning is important for optimal performance.
Rule-hit Threshold (r,). The rule-hit threshold controls when a rule is triggered versus when the agent falls back to planning. We sweep Tr E {0.6, 0.7, 0.8, 0.9) across all datasets. Table 6 reports Aco@3 for each dataset and threshold value. Lower thresholds increase rule usage but may trigger rules too aggressively, causing conflicts and degrading accuracy. Higher thresholds reduce rule benefits, forcing more expensive planning. The results show that optimal threshold selection is critical and dataset-dependent NBA and DDXPlus favor 7, 0.8, while Car-Following and Atari-Berzert achieve optimal performance at f. = 0.7, reflecting differences in action space complexity and rule bank characteristics. Accuracy swings range from 1.8-3.1% % across datasets.
Planning Temperature (r). The planning temperature controls exploration-exploitatioo balance in expected free energy minimization. We sweep E {0.5, 1.0, 2.0} across all datasets. Table reports Acc@3 for each dataset and temperature value. Lower temperatures favor exploitation (greedy selection), while higher temperatures increase exploration. The results demonstrate that temperature selection has substantial impact and is dataset-dependentt Car-Following, with its stable driving patterns, benefits from lower temperature
0.5) favoring exploitation, while NBA, DDXPlus, and Atari-Berzerk achieve optimal performance at = 1.0, requiring balanced exploration-exploittto... Accuracy swings range from 1.6-2.7% across datasets.
Discussion. The sensitivity analysis reveals several key findings:
Parameter impact is substantial and datasee--depeddent. Complex action spaces (DDXPlus) and visual inputs (Atari) show higher sensitivity (2-3% accuracy swings), while simpler structured sequences (Car-Follly (ar-Folloowng) are more robust but still show meaningful variation (1-2%). This aligns with the rule trade-off analysis (see Figure 3) where rule bank size changes cause 15-40% accuracy variations, demonstrating that hyperparameters significantly influence performance.
Optimal parameters are Lataset-dependent Performance does not collapse at suboptimal parame- ters, but accuracy degradation of 2-3% can be substantial in practice, especially for complex domains.
31

---

Published as a conference paper at ICLR 2026
Table Sensitivity analysis of planning temperature 7 across datasets. Columns report Acc@3 (%) for each temperature value. The optimal value is highlighted in bold. Note that optimal temperatures vary by dataset, with simpler domains (Car-Following) favoring lower temperatures for exploitation, while complex domains (DDXPlus, Atari) benefit from balanced exploration-exploittion.
Dataset
T= 0.5
T
1.0
2.0
NBA SportVU
88.76
91.32
89.94
Car-Following
95.87
95.41
94.28
DDXPlus 70.92
73.58
72.18
Atari-Berzerk 74.86
77.20
76.14
The optimal parameter values vary by dataset: NBA and DDXPlus favor T, = 0.8 and T = 1.0, Car-Following achieves optimal performance at T, = 0.7 and r = 0.5 (reflecting its simpler, more predictable patterns), while Stari-Berzerk favors т, = 0.7 and = 1.0. This dataset-dependent variation indicates that careful domain-specifie tuning important.
Practical implications. While default values (Tr = 0.8, T = 1.0) work reasonably well across datasets, domain-specific tuning is important, especially for complex action spaces. The variation in optimal parameters across datasets (e.g., Car-Following benefits from lower temperature for exploita- tion, while complex domains require balanced exploration exploration-cxploitttiio exploitation) suggests that practitioners should tune hyperparameters for their specific domain. Suboptimal parameter choices can lead to 2-3% accuracy degradation, which is significant in practice and comparable to the impact of rule bank size selection (as shown in the trade-off analysis).
Limitations. The analysis reveals that parameter sensitivity is a real limitation, particularly for DDXPlus with its 225 actions (showing ~2.7% accuracy swings) and visual domains like Atari (~2.3-2.6% variation). This suggests that the method requires careful hyperparameter tuning and that future work could benefit from adaptive parameter selection or meta-learning approaches reduce this burden. The sensitivity is consistent with the rule trade-off behavior, where performance varies substantially with rule bank size, indicating that both rule quantity and rule triggering thresholds are critical hyperparameters.
Detailed sensitivity curves for all datasets are provided in the supplementary material. The results demonstrate that hyperparameter choices have substantial impact on performance (2-3% accuracy variations), consistent with the rule trade-off analysis showing that rule bank size significantly affects performance. This indicates that careful hyperparameter tuning is important for optimal results, though the method remains functional across reasonable parameter ranges.
E.4 LLM-GUIDED COMPONENT ABLATION
We conduct ablation studies to assess the contribution of L.LM-guieed components across datasets. While LLMs are used for initialization and feature construction (not as hard-coded rules), we evaluate their impact by comparing our full method (with LLM guidance) against variants that remove LLM components and use alternative initialization strategies.
DDXPlus.
DDXPlus uses L.M guidance for mental state initialization. We compare three configu-
rations:
With LLM (Ours): Uses LLM-guided semantic labels to initialize mental states, achieving
Acc@3: 73.62%.
Without LLM (Random Init): Replaces LLM guidance with random initialization, achiev-
ing Acc @3: 72.08% (1.54% drop).
Without LLM (K-means Init): Uses K-means clustering on training features for initializa-
tion, achieving Acc@3: 72.78% (0.84% drop).
Results show that LLM guidance provides modest but consistent improvement (0.84-1.549) accuracy gain), demonstrating that while our method benefits from semantic initialization, it does not critically depend on LLMs. The K-means alternative performs better than random initialization, suggesting that any reasonable initialization strategy can work, with LLM guidance providing the best semantic alignment.
NBA SportVU. NBA uses LLM guidance for both feature construction and mental state initializa- tion. We ablate both components:
With LLM (Ours): Full LLM-guided feature construction and mental state initialization, achieving Acc@3: 91.42%.
32

---

Published as a conference paper at ICLR 2026
Without LLM Feature: Removes LLM-guided feature construction (uses random features), achieving Acc@3 90.88% (0.54% drop).
Without LLM Mental State: Removes LLM-guided mental state initialization (uses K- means clustering), achieving Acc@3: 91.18% (0.24% drop).
The results indicate that LLM-guided mental state initialization has smaller impact (0.24% drop) compared to feature construction (0.54% drop), but both contribute positively. The relatively small drops (0.24-0.54%) demonstrate that method is robust and does not heavily rely LLM components, while still benefiting from semantic guidance when available.
Atari-Berzerk.
Atari uses LLM guidance only for mental state initialization. We compare: With LLM (Ours): Uses LLM-guided semantic labels for mental state initialization, achieving Acc@3. 77.27%
Without LLM Mental State (Random Init): Uses random initialization, achieving Acc@3: 76.88% (0.39% drop).
Without LLM Mental State (K-means Init): Uses K K-means clustering on visual features, achieving @cc@3: 77.09% (0.18% drop).
Similar to DDXPlus, LLM guidance provides a modest improvement (0.18-0.39% accuracy gain), with K-means performing better than random initialization. This confirms that LLM guidance beneficial but not essential, and alternative initialization strategies can still achieve reasonable performance.
Car-Following.
Car-Following does not use LLM guidance, relying instead on domain-specific action definitions and feature engineering. This demonstrates that our framework can work effectively without LLM components when domain knowledge is available through other means.
Discussion.
The LLM ablation studies reveal several key insights:
1. LLM guidance is beneficial but not critical: Removing LLM components causes modest accuracy drops (0.18-1.54%) 1.54%). indicating that our method is robust and does not heavily depend on LLMs. This addresses reviewer concerns about LLM dependence.
2. Alternative initialization strategies work: K-means clustering training features provides a reasonable alternative to LLM guidance, achieving performance within 0. 0.18-0.849 of the LLM-guided version. This suggests that any semantic initialization strategy work, with LLMs providing the best semantic alignment when available.
3. Dataset-dependeen impact: The impact of LLM guidance varies by dataset: DDXPlus shows the largest benefit (0.84-1.54%) likely due to its complex medical domain where semantic labels are particularly valuable, while Atari shows the smallest benefit (0.18- 0.39%), where visual features may be sufficient.
4. Practical implications: domains where LLMs are unavailable or impractical, our method can still achieve strong performance using alternative initialization strategies (K-means, random, or domain-specific heuristics), making it applicable to wide range scenarios. These results demonstrate that LLMMs serve as a helpful tool for semantic initialization and feature construction, but our framework's 's core contributions (rule-guided active inference, world model learning, hybrid arbitration) are independent of LLM components and can work effectively with alternative initialization strategies.
E.5 LLM PROMPTS
We include the exact prompts used to elicit symbolic actions, feature predicates, and mental-state semantics.
NBA (feature construction action definition + z design).
33

---

Published as a conference paper at ICLR 2026
You are given NBA play-by-play frames, where each frame contains only the 2D coordinates of 11 entities (the ball + 10 players) with timestamps, plus team and ball-possession labels. No actions or features are predefined; only raw positions are available. Your task is to:
Inspect the distribution of the data and decide how many latent z states should be used. For each provide semantic interpretation (e.g,, habitual/exploit, explore, subgoal switch) and define triggering conditions.
2. Define set of interpretable basketball actions (e.g., straight move, left turn, right turn, turnaround, dribble, shoot, pass) and specify the measurable thresholds (angles, distances, speeds, frame persistence, ball speed) for classifying them.
3. Construct interpretable features from raw coordinates, such as speed, heading angle, relative distances, angle changes, and possession switches, with formulas and units.
4. For each threshold, provide default values and ranges (conservative / standard aggressive settings).
Car-Following (feature construction design).
You are given car-following data represented as sequences of discrete regimes. The regime set fixed as Fa, Fd, C, A, D, S, where each regime represents specific driving behavior such as cruising, accelerating, or decelerating. Only these sequences are available, with no additional environment variables.
Your task is to:
1. Inspect the distribution of these regime sequences and decide how many latent states should be defined. Provide semantic interpretations for each z (e.g., conservative driving, aggressive driving, bursty switching).
2. Design interpretable features that can be derived from the regime sequences, such as switching rate, run length of consecutive regimes, time since last switch, or ratios of sudden accelera- tions/decelerations.. Provide formulas for these features. For each feature, propose thresholds with default values and ranges (conservative / standard aggressive).
DDXPlus (feature construction suggestions; actions fixed; a given severity 1-5).
You are given patient diagnostic trajectories from the DDXPlus dataset. Each trajectory consists of sequential actions of two types: ASK: querying a symptom, sign, or test and DIAG: issuing a diagnosis. The latent state (severity from to 5) is already provided by the dataset, and the action vocabulary is fixed. Your task is to:
Suggest how to construct interpretable features from the raw evidences. For example, propose ways group evidences into thematic clusters (such URTI core symptoms, systemic symptoms, or risk factors) or embed them into low-dimensional representations. Provide recommendations for feature engineering choices such as embedding dimension. normalization, or grouping heuristics.
Atari-Berzerk (z design from pixels; actions fixed).
You are given raw Atari Berzerk gameplay frames, each being 128x128 grayscale or RGB image. The action space is fixed at 18 discrete actions (combinations of movement and firing). No explicit state features are provided; the world model will learn directly from pixels.
Your task is to:
1. Based on typical human gameplay strategies, decide how many latent states should be used.
Provide a semantic interpretation for each (e.g., exploit = repetitive safe movement, explore
regions, danger escaping when enemies cluster). 3. Estimate the relative prevalence of each z (e.g., most frames are exploit, fewer are explore or danger).
trying rare actions
34

---

Published as a conference paper at ICLR 2026
F LIMITATIONS AND BROADER IMPACT
While our framework demonstrates strong performance and interpretability across diverse domains, several limitations remain. First, the rule extraction process is still partly dependent on the quality of predicates or action abstractions available in each dataset. Although our wake-sleep cycles mitigate this by progressively refining rules, fully unsupervised discovery of symbolic structures remains an open challenge. Second, our current design balances rules and active inference at single timescale; extending the framework to explicitly multi-level hierarchies (e.g.. subgoals and long-term planning) natural next step. Third, although rule-based reasoning improves robustness rare edge-case behaviors, its coverage is inherently sparse, and rule confidence thresholds must be tuned to avoid spurious activations.
Regarding scalability more complex tasks: For multi-game multi-task setups, potential chal- lenges include rule interference across tasks and the need to encode task identity in my or maintain per-task rule banks. For nigh-dimensional visual or sparse-reward tasks, stronger world models, hier- archical planning, merging with model-based RL may be beneficial. Long-horizon planning with EFE remains computationally challenging and a direction for future work. For continuous control, conceptual extensions include using Gaussian policies over continuous actions and treating mental states as option/skill indices for hierarchical action spaces. Finally, our experiments are conducted on curated benchmarks; evaluating the method in highly dynamic or noisy real-world environments (e.g., human-robo collaboration, autonomous driving in open traffic) remains important future work.
Despite these limitations, we believe the broader impact our approach is promising. By combining generative models with symbolic rules, the framework offers path toward transparent, human- interpretable decision-nee maaiing potentially increasing trust in safety-critical applications such as healthcare, transportation, and multi-agent coordination. The ability capture both frequent and rare behaviors in complementary manner also suggests that our method can generalize to domains where data imbalance uncertainty is prevalent. More broadly, the work illustrates how insights from cognitive science-suee such as rule-guided inference and predictive coding-oae can inspire practical algorithms that balance performance with interpretability. We hope this line of research stimulates further integration of symbolic reasoning and active inference in future intelligent systems.
G USE OF LLMS
In this paper, LLMs were used solely for writing polishing in several paragraphs, like the Experiment section. All the key ideas, proofs, research, and writing are created completely by human authors.
35


Reviews:
[{"review_id": "e9a9lF3nd8", "reviewer": "Reviewer_GnXG", "summary": "The work introduces a new method within the active inference framework. The methods is made of mainly two novel contributions: (i) a set of symbolic rules, which are learned as part of the world models and allow the agent to decide between deliberate and intuitive prediction, (ii) a wake-sleep training algorithm, alternating phases where the actions are selected and the world model is provided as opposed to phases where acting rules are consolidated. The method is evaluated in \"offline\" settings, on (small) predictive benchmarks.", "questions": "* What is the parameter count of the model?\n\nTypos:\n\n-> in the abstract \"Experiments on basketball player movements, car-following behavior ~demonstrate~, medical diagnosis, and visual game strategy **demonstrate** that our framework\"\n\n-> neurosciencetau -> neuroscience", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "* **Novelty**: while there are previous works focussed on learning habitual policies in active inference (see Fountas et al), the proposed approach, learning/distilling rules as part of the world model's representation is, to the best of my knowledge, novel and compelling\n* **Interpretability**: the method, using active-inference and a set of rules that are learned by the world models and used by the policy enables stronger interpretability of behaviors. This is an important features, especially compared to more blackbox approaches, such as reinforcement learning and imitation learning.", "weaknesses": "* **Scalability**: if we compare with the current state of AI, the datasets employed are tiny. Even CIFAR 10, which is considered a very small dataset these days, is 3x larger any training dataset used in the paper.\n* **Baselines**: while the proposed approach is thoroughly analyzed, it is unclear what are the difficulties of the baselines in solving the given, small problems. Given the nature of the predictive tasks, it would also be useful to consider at least one LLM-based baseline (with a very small model, such as Qwen 0.5B)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EgPPyDo7Sr", "reviewer": "Reviewer_H8hR", "summary": "The paper proposes a hybrid active‑inference (AIF) framework in which symbolic “rules” are learned and consolidated via a wake–sleep procedure and then fused with expected free energy (EFE) planning for action selection. Concretely, the latent state is split into a continuous external state (S_t) and a discrete “mental state” (m_t). Candidate rules of the form ((S_f^\\star, m_f^\\star) -> a_f) are harvested during a wake phase and consolidated with generative replay in sleep. At decision time, if a rule “hits” (via a kernel match to the MAP estimate of ((S_t,m_t))) it supplies a prior over actions; otherwise the agent falls back to EFE‑based planning. Experiments on NBA player trajectories, car‑following, DDXPlus (URTI), and Atari–Berzerk report higher Acc@k and lower latency than logic‑based, deep learning, AIF, and DreamerV2 baselines, while yielding interpretable rules. (Table 1; figs. on pp. 8–9.)\n\n  The key problem that is tackled is learning habit‑like, interpretable shortcuts that allow an AIF agent to arbitrate between fast habitual actions and slower planning, and applying this to modeling human(-like) behavior across domains.\n\n The motivation (habits vs. planning) is timely and clear, and the ability to flexibly and context-sensitively tradeoff between habit-based action vs more costly-model based planning is a very valuable one. A small nitpick is that the paper under‑positions itself relative to existing methods in computational cognitive neuroscience, psychophysics and computational psychiatry and the large literature on fitting RL/decision‑making models to human data (e.g., model‑based RL, active inference, DDMs). The Related Work emphasizes neuro‑symbolic rules and AIF agents, but not human model‑fitting traditions.  Reported gains in Acc@k/latency appear consistent across tasks (Table 1) and ablations indicate that rules and (m_t) matter. However, several specification and correctness issues (see below) make it hard to assess whether improvements stem from principled inference or from ad‑hoc fusion/thresholding. \n\nThe idea of rule‑guided AIF and the attempt to compress planning into reusable symbolic habits is interesting and potentially impactful for both interpretable agents and human behavior modeling. In its current form, however, conceptual and notational gaps limit the work’s reliability and portability.", "questions": "Preferences in EFE: How are preferences represented in Eq. (3)? Is (p_\\phi(O_{t+\\tau}\\mid Z_{t+\\tau})) re‑used from the observation model or replaced with a preference distribution? Please spell this out and reconcile with the standard EFE decomposition. (§3.)  \n\nRule fusion as inference: Could the rule‑vs‑planning arbitration be formulated as posterior inference (e.g., a gating variable) instead of kernel‑plus‑temperature? What prevents learning the weights of this mixture directly via variational inference under your joint objective? (§4.1–§4.2.) \n\nObjective design: What is the principled justification for VFE + (\\eta)·EFE + (\\gamma)·KL? Is there a derivation tying this to a bound on expected log‑evidence or a generalised free energy objective? (§4.2.) \n\nBaselines and fairness: How was DreamerV2 adapted (offline? behaviour‑cloned?) to these supervised prediction tasks and tuned to parity (esp. on DDXPlus/Atari)? Please clarify compute budgets, hyperparameter search, and early‑stopping criteria across methods. (Table 1.)", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 4, "strengths": "Compelling objective: compressing repeated planning successes into interpretable rules and using them to shortcut EFE search is elegant and practical. \n\nClear engineering benefit: latency vs. accuracy curves and ablations demonstrate a Pareto knee where small rule banks reduce inference time while preserving accuracy. \n\nVisualization & interpretability: rule envelopes in latent space and trajectory overlays help the reader understand what the rules encode. \n\nCross‑domain experiments spanning continuous control/trajectories, clinical dialog, and pixels.", "weaknesses": "EFE definition and preference distribution:  In AIF the EFE uses a biased preference likelihood (encoding utilities/preferences) that is not the same as the observation model used for inference. Here EFE is written with the same (p_\\phi(O_{t+\\tau}\\mid Z_{t+\\tau})) used by VFE (Eq. 3), and the text does not explain how preferences are encoded. This obscures how “goals” enter planning and undermines the conceptual link to the risk/ambiguity decomposition. \n\nAd‑hoc policy fusion and temperature dependence: The hybrid policy adds a rule prior to a temperature‑scaled EFE term (Eq. 4) and gates it with a binary “rule hit” indicator based on a similarity kernel and threshold in ((S_t^\\text{MAP},m_t^\\text{MAP})). This is anti‑Bayesian in spirit and creates several tuning knobs (kernel, threshold (\\tau_r), temperature). A principled alternative is to cast rule invocation as inference under a mixture model (e.g., infer (q(m_t)) and mixture weights over rule‑conditioned policies), letting uncertainty in EFE naturally down‑weight unreliable rollouts without extra temperatures. \n\nObjective couples VFE and EFE without theoretical justification:  The “total free energy” in Eq. (5) simply sums per‑step VFE, rollout EFE (scaled by (\\eta)), and a KL regularizer on successive (q(m_t)) (scaled by (\\gamma)). The inclusion ofc (\\operatorname{KL}[q(m_{t-1})\\Vert q(m_t)]) feels ad‑hoc (shouldn’t that KL come out naturally of the generative model when minimizing VFE), and the paper does not justify why this combination is a coherent bound or how it relates to generalised free energy or control‑as‑inference objectives. (§4.2, Eq. 5.) \n\nRule growth is heuristic:  Rules are created/updated when a triplet ((S_t^\\text{MAP},m_t^\\text{MAP},a_t)) recurs and “reliably reduces free energy,” with centroids updated by exp(−VFE) weights. This is reasonable as an engineering heuristic, but it could be formalised as variational learning in a mixture model with inference over (q(m_t)) and parameters of (p(S_t\\mid m_t)) (and optionally a nonparametric prior for rule birth/pruning). \n\nTypos and clarity \nMinor: Ambiguity of contribution (human modeling vs. model-based RL agent):  The abstract and introduction oscillate between “learning human habits alongside human models” and benchmarking a better agent (“improves predictive accuracy and efficiency”). Please clarify whether the primary goal is computational cognitive neuroscience (fitting human behavior) or agent performance. As written, expectations are set for both. I assume it’s the former, otherwise you would have ben benchmarking on metrics more like reward than model-fit like (the number of accurately predicted actions, rather than whether those actions were “correct” in a maximizing-reward sense). \nMinor: Generative model misspecification in Eq. (1):\n\nNo explicit prior over initial hidden states (e.g., (p_\\phi(Z_1))).\n\nThe factorization includes (p_\\phi(a_t\\mid Z_t)) inside the generative model while the LHS lists (a_{1:T-1}), creating an indexing mismatch (a term for (a_T) is included on the RHS but not in the LHS).\n\nThere is no prior over (a_{t-1}), yet the state transition conditions on it; the dependency graph is inconsistent. (p. 3, Eq. 1.) \nMinor: Latent split factorization also misses initial priors and has indexing mismatches. In the section(Latent State Representation), the joint (p_\\phi(O_{1:T},S_{1:T},m_{1:T},a_{1:T-1})=\\prod_t p(O_t \\mid S_t) p(S_t \\mid S_{t-1},a_{t-1}) p(m_t \\mid m_{t-1},S_t) p_{\\phi^\\pi}(a_t \\mid S_t,m_t)) lacks priors for (S_1) and (m_1), and again indexes (a_t) up to (T) while the LHS says (a_{1:T-1}). (p. 4.)\n\n“neurosciencetau accounts…” (p. 5, l. ~262) → “neuroscientific accounts.”\n\nPositioning with neuroscience evidence:  The claim that seamless habit–planning switching is a hallmark of human intelligence contradicts years of evidence from the behavioral neuroscience literature and animal studies (in rodents and to some extent monkeys) on the transition between goal-directed and habitual behavior. Classic devaluation and contingency-degradation paradigms demonstrate context-dependent switching early (goal-directed) vs. after overtraining (habit). In terms of its biological basis, evidence suggests that dorsomedial striatum and prelimbic PFC support goal-directed control, while dorsolateral striatum and infralimbic PFC promote habits. Flexibly toggling between cached habits and on-the-fly planning is a general mammalian capability, not a human hallmark. \n\nOverall recommendation\nKey reasons: (i) policy fusion relies on ad‑hoc kernel thresholds and temperatures rather than principled inference; (ii) Generative model and EFE formulation contain gaps/mismatches that need fixing; (iii) contribution framing (human modeling vs. agent) is unclear. The core idea—rule‑guided AIF that amortises planning into reusable, interpretable habits—is promising and, with the above issues addressed, could become a strong contribution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aqYWSfoN2R", "reviewer": "Reviewer_kL1R", "summary": "This paper introduces a framework for learning human habits within rule-guided (deep) active inference, modelling both planning and rapid, habitual responses. Such habitual responses (the main contribution) use a kind of symbolic rule extraction with generative world models, learned via a sleep-awake algorithm. Empirically, the authors provide both interpretability and strong predictive performance across diverse domains, including human sports, driving, and clinical diagnosis tasks.", "questions": "Do you think this method can scale up to more complex tasks? I'm thinking of Atari26, but also others environments typical of the model based RL literature.\n\nWhy only Berserk, and not others? Is there a specific reason for picking this game only? Am i correct in thinking that some additional problems may arise if we plan to implement your proposed method to a more complex/general setup? If yes, how do you think such problems will have to be addressed in the future?\n\n\n\n\nI would be happy to raise my score if my concerns/comments are addressed.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "This work tackles an interesting problem, and manages to scale up active inference models to \"large\" scale tasks (large for the active inference community, not so much for the DL one). Also, empirical results demonstrate high accuracy and low inference latency compared to both deep learning and logic-based baselines, across multiple domains.\n\nThe wake-sleep algorithm is a nice contribution, and it seems to work well. \n\nWell written paper, precise, and to the point.", "weaknesses": "What is the reason for not testing on the whole atari26? Have you optimized your architecture for specific games only, or is there another underlying reason? It would be a great results to show performance on a popular benchmark lile the Atari 100k challenge using such a model. \n\n\n\nMinors:\n\n- Weird phrasing at the end of the abstract (typo)\n\n- Suboptimal choice of citations: When you introduce active inference, you only cite (the very nice work of) P.Mazzaglia, 2022. However, that should not be the first citation: the main citation for active inference today is the official book (T. Parr et al., Active Inference: The Free Energy Principle in Mind, Brain, and Behavior (2022)). It is missing here. More generally, all the main papers that have introduce the theory of active inference are missing. I'd suggest the authors to go through the main K.Friston's papers on active inference/free energy principle, and address that. To conclude, there is also a work that introduces a more \"habitual\" kind of planning, that is inductive inference (https://direct.mit.edu/neco/article-abstract/37/4/666/128203/Active-Inference-and-Intentional-Behavior?redirectedFrom=fulltext). It is very different from what you propose, but still a related work. \n\n- Some figures are basically unreadable given the tiny font.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "72yKugzEGa", "reviewer": "Reviewer_Hnij", "summary": "The authors have sought to extend the ActInf framework by introducing a trainable library of symbolic rules that model habitual behavior. This happens via an alternation between wake and sleep phases whereby in the wake phase, candidate rules are collected and in the sleep phase the rules are consolidated and pruned using generative replay. The hybrid policy can then choose between the symbolic fast rules, and the slower deliverative ActInf planning using the full expected free energy minimisation over trajectories.\nExperiments are run across four very distinct domains, showing improved accuracy (5–12 %) and reduced latency (2–10×) over baselines such as DreamerV2, STLR, and prior AIF variants.", "questions": "In addition to questions regarding the weaknesses above, can the system be used for continuous control and/or hierarchical action spaces?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "There is a good biological plausibility to the whole architecture, which is certainly interesting, and the integration of the symbolic with the neural is elegant and theoretically well founded. The alternating wake-sleep mechanism is again well biologically motivated, and the fast and slow thinking paradigms are again seemingly consistent with real planning/action. The results show that the framework can apply to pretty distinct sequential decision tasks, which shows good coverage and potential generality.\nThe authors present quantitative analyses showing how rule-bank size mediates accuracy and latency, giving practical insight into how to balance interpretability and efficiency.\n\nOverall, the presentation is well put together and clear", "weaknesses": "The main issue to me is the lack of statistical rigour. Although the reported accuracy and latency improvements appear consistent, the authors provide no confidence intervals, variance estimates, or multiple-seed averages. Because the wake–sleep rule extraction process and generative replay are stochastic, these missing statistics make it impossible to assess whether the gains are robust or simply artifacts of single-run variability. This detracts a lot from the results' overall message.\n\nThe system’s interpretability is not actually intrinsic. Rules are defined over latent states, meaning that human readability depends on the decoder’s ability to preserve semantic structure. If the latent space drifts or becomes entangled, rules lose meaning. Because of this, it seems that Interpretability is contingent on the quality of the learned representations rather than guaranteed by the rule architecture itself.\n\nFrom what I can see, all datasets also rely on LLM-derived feature engineering and mental-state initialization (as discussed in appendix D.3). This external semantic bootstrapping partially determines rule quality and seems to go against claims of generality. No ablation quantifies how performance changes without these priors. Demonstrating that rules can emerge from raw or unsupervised features would really strengthen the contribution.\n\nThe rule activation depends on hard matching of m_MAP to m_f^* which forces a winner-takes all intent selection. This design precludes probabilistic blending of intentions and this could cause brittle switching near category boundaries. I think that one ought to be able to implement a soft gating mechanism which may make it more robust.\n\nKey parameters seem to vary dramatically across datasets which produces very large accuracy swings. There doesn't seem to be any principled selection rule, which seems to indicate that the system is heavily domain-tuned.\n\nI think that there is also some literature which has not been cited, which is relevant.\n\nTschantz, A., Baltieri, M., Seth, A. K., & Buckley, C. L. (2020). Learning action-oriented representations through active inference. \nDolan, R. J., & Dayan, P. (2013). Goals and habits in the brain.\nCushman, F., & Morris, A. (2015). Habitual control of goal selection in humans. \nBacon, P.-L., Harb, J., & Precup, D. (2017). The option-critic architecture. \nd’Avila Garcez, A., et al. (2019). Neural-symbolic learning and reasoning: A survey and interpretation.\n\nCurrently the paper is marginally below the acceptance threshold, in large part because of the lack of statistical rigour in the results which weakens the arguments significantly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)