Learning Nonlinear Causal Reductions to Explain Reinforcement Learning Policies
Authors: Armin Kekić
Jan Schneider
Dieter Büchler
Bernhard Schölkopf
Michel Besserve
Venue: ICLR 2026 Poster (2026)
Keywords: Causal Abstraction
Causal Representation Learning
Reinforcement Learning
Explainable AI
Primary area: causal reasoning
TL;DR: We use a causal perspective to learn policy-level explanations of the global behavior of trained RL agents.

Abstract: Why do reinforcement learning (RL) policies fail or succeed? 
This is a challenging question due to the complex, high-dimensional nature of agent-environment interactions. 
We take a causal perspective on explaining the global behavior of RL policies by viewing the states, actions, and rewards as variables in a low-level causal model. 
We introduce random perturbations to policy actions during execution and observe their effects on the cumulative reward, learning a simplified high-level causal model that explains these relationships.
To this end, we develop a nonlinear Causal Model Reduction framework that ensures approximate interventional consistency, i.e., the simplified high-level model responds to interventions in a way consistent with the original complex system. 
We prove that for a class of nonlinear causal models, there exists a unique solution that achieves exact interventional consistency, ensuring learned explanations reflect meaningful causal patterns.
Experiments on both synthetic causal models and practical RL tasks—including pendulum control and robot table tennis—demonstrate that our approach can uncover important behavioral patterns, biases, and failure modes in trained RL policies.

Published as a conference paper at ICLR 2026
LEARNING NONLINEAR CAUSAL REDUCTIONS TO EXPLAIN REINFORCEMENT LEARNING POLICIES
Armin Kekić
Jan Schneider
Dieter Büchler 1,2,3,4
Bernhard Schölkopf
Michel Besserve
1 Max Planck Institute for Intelligent Systems, Tübingen, Germany
2 University of Alberta, Canada
3 Alberta Machine Intelligence Institute (Amii)
CIFAR AI Chair
5 Tübingen AI Center, Tübingen, Germany
6 ELLIS Institute, Tübingen, Germany
7 Technische Universität Braunschweig, Germany
a armin.keii o, jan schne i der, buecher. (ttee de (besserve, bs @tue. mpg. de
ABSTRACT
Why do reinforcement learning (RL) policies fail or succeed? This is challenging question due the complex, high-dimensiona: nature of agent-environment interactions. We take causal perspective on explaining the global behavior of RL policies by viewing the states, actions, and rewards as variables in low-level causal model. We introduce random perturbations to policy actions during execution and observe their effects on the cumulative reward, learning a simplified high-level causal model that explains these relationships. To this end, we develop a nonlinear Causal Model Reduction framework that ensures approximate interventional consistency, i.e., the simplified high-level model responds to interventions in way consistent with the original complex system. We prove that for a class of nonlinear causal models, there exists a unique solution that achieves exact interventional consistency, ensuring learned explanations reflect meaningful causal patterns. Experiments on both synthetic causal models and practical RI. tasks including pendulum control and robot table tennis demonstrate that our approach can uncover important behavioral patterns, biases, and failure modes in trained RL policies."
INTRODUCTION
In recent years, reinforcement learning (RL) has demonstrated remarkable successes in diverse domains, from achieving superhuman performance in games like Go [1] and Atari [2], to enabling sophisticated control in robotics [3]. The field has also seen significant adoption in real-world applica- tions, including autonomous systems [4], recommendation engines [5], and process optimization [6]. As RL systems continue to be deployed in increasingly consequential settings, understanding the behavior and decision decision-making processes of trained policies becomes a practical necessity for ensuring reliability, safety, and trust. Hence, natural and critical question arises: "Why did a policy fail or succeed?" These so-called policy-level explanations (PLE) [7] allow to assess the overall behavior and competency of an RI. agent. Notably, pointed out by Milani al. [7, Section 8],
"When RL is applied to real-world domains, training directly in that domain may be prohibitively expensive or potentially unsafe. As a result, the agent is trained on a simulated domain, which designed be similar to the real-world domain but may differ in subtle ways. [...] In this case, it is critical that the human operator agrees with
"oint supervision.
Code is available http://gghhac.ooomeeeccccooocoe.

---

Published as a conference paper at ICLR 2026
the overall approach of the agent, [...] which requires understanding the entire policy. PLEs are best suited to this case [compared to other RL explainability approaches]."
Trustworthy PLEs are challenging to obtain, as RL often uses parameter-rich neural networks to map from complex observation spaces to actions at each time step. with indirect consequences on the overall behavior that are not easily accessible to human intuition. Moreover, the credit assignment problem determiinnn which actions contributed most significantly eventual outcomes is a fundamental obstacle to developing policy explanations. In particular, the dynamic dependencies between states and actions induce spurious correlations with the outcome, meaning that those features can be predictive of the outcome without causing it [9].
Px(X)
P.0) (z,) (H,
Agent
w(1)
S
R
Intervention
P2 (X)
(Z.) ~ platt) (Z,Y)
Env
Axt
8A
(b) Approximately Commutative Diagram.
Interventions
State acctinn
wh (La(1)
8Ao
84
6Az
(i))
J
X
time
Ao
A
A3
(1
state Laction
= X x1(1
Z
Sr
((.)
Actaons/States
To ( Σ, R
time
state / /action
Ro
R
Y
S (X))
Rewards
Low-level
High-level
time
(a) From RL Policies Causal Explanations.
(c) Interpretable Reduction Function Class.
Figure 1: Learning Causal Explanations of RL Policies. (a) shows how RL policies are translated Causal Model Reduction problem. We sample episodes from the interactions between a trained agent and its environment, where the sampled actions are perturbed by random shifts 8A,, We treat the episode variables as nodes a low-level causal graph, with 5'A, acting as shift interventions on the ac- tions A, Map 7 condenses the low-level causal variables to simpler high-level model with two vari- ables: the target Y. summarizing the rewards, and the cause Z, summarizing the subset π (1) of low- level nodes. Similarly, W1 maps the vector of low-level interventions I to its high-level counterpart J. (b) shows how the maps 7 and w are learned by making the diagram approximately commutative, minimizing the divergence between the distributions
each side of the sign. (c) shows nonlinear interpretable function class that can be used to learn the reduction maps TI and w1 A state/action vari- able's trajectory X is encoded through equally spaced Gaussian kernels to feature vector 4 (X)).
In this work, we take a causal perspective on PLE summarized in Fig. 1, focused on policies learned on simulations of real-wol real-world physical systems, with continuous state and action spaces. We formulate the problem as Causal Model Reduction (CMR), where we treat the joint system of actions, environment variables, and rewards as a complex low-level causal model. To causally probe the low-level model, we introduce random perturbations to policy actions as interventions and observe their impact on cu- mulative rewards. We learn mapping simplified high-level causal model with few "macroscopic" variables, summarizing the most important factors influencing the expected reward. As our main learning signal, interventions in similar fashion. The map from the original causal model to the reduced one can thus be used to explain the behaviors that are most influential the success failure of the RL policy.
interventional consistency: the low- and high-level models should respond to
2

---

Published as a conference paper at ICLR 2026
Our contributions are: (i) formulating the problem of explaining RL policy behavior as Causal Model Reduction and developing a nonlinear extension of Targeted Causal Reduction (TCR) [10] that learn high-level causes target phenomenon in complex systems, (īi) providing theoretical guarantees of solution uniqueness for a broad class of nonlinear models, ensuring unambiguous explanations despite the identifiability challenges of nonlinear systems, (iii) introducing class of interpretable nonlinear reduction functions that help explain the behavior captured by the learned reductions, (iv) demonstrat- ing experimenaally that our approach uncovers behavioral patterns and biases in two trained RL tasks.
BACKGROUND
Notation. We use lowercase letters for deterministic variables and capital letters for random variables. X ~ P means has distribution P. We use boldface for column vectors, and Xg for the subvector of X restricted to the components in set S. The number of elements in set S is #S.
2.1 STRUCTURAL CAUSAL MODELS (SCMs)
SCMs are mathematical framework for representing cause-effect relationships in complex systems. Definition 2.1 (Structural Causal Model [11, 12]). An n-dimensional SCM is a triplet M- Pv) consisting of: (i) directed acyelic graph 9 with vertices, (ii) joint distribution Po over exogenous n of structural equations, where Pay are the variables indexed by the set of parents of vertex j in G. This induces joint
or noise variables (U, Iin (iii) a set (=(X, == fy (Paj. :
distribution PM over the endogenous variables X = [XI, [X.]......]]?
The endogenous variables X encode the system's observables, where each variable Xj is determined through the deterministic causal mechanism fj. its parent variables Paj, and the exogenous noise variable Uj- One often encountered class of SCMs are additive noise models where structural equations take the simplified form X, == (Pa,) Uy.
Interventions are encoded SCMs by replacing one or several structural equations. An intervention transforms the original model M= S, Pu) into an intervened model = _ pl)) where is the vector parameterizing the intervention. The base probability distribution of the unintervened model is denoted PM or simply PM and the interventional distribution associated with
( denoted p(() In this work, we focus on shift interventions, which modify the structural equation of variable X₁ by shifting it by scalar it
(Xi== = S===1(aa (Par,,
(Xi = fi (P U:) + +
(2.1)
These can combined to form multi-node interventions with vector parameter i. Shift interventions are particularly well suited to the case of simulators of physical systems with continuous state and action space, which is the main focus of the present work. In particular, they can be interpreted as an additional force or momentum in discreiicet equations of motion.
2.2 CAUSAL MODEL REDUCTIONS (CMRs)
Causal models with a large number of variables can be difficult to interpret and work with. Causal Model Reductions (CMRs) [10] are dimensionality reduction approaches that map such detailed low-level causal models to approximate high-level descriptions with fewer variables while preserving the essential causal properties. They closely relate to several notions of causal abstraction [13-19].
Low- and High-level SCMs. We consider two causal models:
low-level SCM L with endogenous variables X Pc (on range X') with exogenous variables
U ~ PP, and set of possible interventions I, leading to interventional distributions (P()) JieT. A lower-dimensional high-level SCM H with endogenous variables Z P#t (on range 2),
exogenous variables W Pw, set of interventions J, and distributions (P (P(5)) )jEJ.
"This SCM definition allows for confounding between variables through the potential lack of independence
between the exogenous variables (.,). Although Def. 2.1 uses the common assumption of acyelic graphs for simplicity, our approach is also compatible with some families of causal graphs with eycles. See App. A.
3

---

Published as a conference paper at ICLR 2026
X is then reduced into Z-valued high-level random variable using deterministic map : - 2.
Interventional Consistency. The key criterion for good reduction is interventional consistency: the high-level model should respond to interventions ways that correspond to the original model's behavior under analogous interventions (ef. [13, 18-20).. To formalize this, we consider *# [Pc]. the so-called push-forward distribution by т of the low-level model distribution, such that
T# PP2]-
(2.2)
Interventional distributions are pushed forward in the same way from the low-level to the high-level, allowing to define the notion an exact transformation:
Definition 2.2 (Exact transformation [19]). A map T X + 2 is an exact transformation from L to H. if surjective, and there exists a surjective intervention map w - J such that for all i I
p(w())
T#
P
(2.3)
That is, regardless of whether we first intervene through i in the low-level model L and then map to the high-level H. or first map to H and intervene with w(i), we arrive at the same distribution.
2.3 TARGETED CAUSAL REDUCTION (TCR)
While exact transformations are theoretically elegant, they present two practical challenges: (i) achiev- ing perfect equality in Eq. (2.3) is often infeasible, so we need systematic method to learn approxi- mately consistent reductions from data, and (ii) there are many possible consistent reductions and finding a meaningful one needs additional constraints. Targeted Causal Reduction (TCR) [10] ad- dresses these challenges by focusing explaining specific target variable of interest and providing a concrete learning objective. Below, we describe the case of a single high-level cause, the general case is described in App. A.2.
Definition 2.3 (General TCR Framework). In TCR, we learn transformations (T, w) between models C and H. under the following assumptions:
Target: H contains only two nodes: a predefined scalar target variable = To (X that quantifies a phenomenon of interest, and high-level cause Z ti (X) that needs to be learned.
2. Parameterized High-level Models: The high-level SCM is constrained to a class of linear additive Gaussian noise models (Hy)ver with parameters 7 be learned, implying that (i) the mechanism Z - Y is linear, with (linear) causal coefficient a and additive noise Wo and (ii) the exogenous variables (Wo, W1 2} have factorized Gaussian distribution Pw Pw.Pw. Pw.-
3. Constructive Transformations The respective components of the maps and w associated to the ligel cause and effect depend on disjoint subsets (1) and (0) of low-level variables:
T (xx) (Fo (0)), (I(((()))
w (0, w ((T()))
(2.4)
the first component of w is set to zero to prevent direct high-level interventions on Y, forcing changes Y to be explained through high-level interventions on
4. Approximate Consistency Objective: Rather than requiring exactness of Eq. (2.3), TCR minimizes the consistency loss, quantifying an average discrepancy D between the push-forward interventional distributions and their corresponding high-level model distributions:
Coons(T,w, = LEiP, [ (X)) p)[ (w)()) (Y, (1) (x.
with p(
T
((()
(2.5)
The relationship between those distributions and the learned reduction maps is shown in Fig. lb. The expectation taken over prior distribution of interventions P,.
RReaarks: In practice. (0) and x can be fixed based on domain knowledge, with x(1) typically encompassing all variables potentially influencing the target that are not in T(0). The intervention prior Pr can he uninformative (e.g. i.i.d. Gaussian) informed by domain expertise.
4

---

Published as a conference paper at ICLR 2026
Intuition behind TCR. TCR provides form of etiological explanation for the target phenomenon Y. identifying the origins of its variation. It accomplishes this by creating an interpretable formu- lation where changes in Y due to low-level interventions are explained through high-level causal mechanism. TCR forces explanations to flow through the learned high-level cause Z. When asked "What causes changes in Y?", TCR answers with specific causal statement: "Y is changed by acting on the high-level cause Z = (Xx(1)) through intervention ((+(1))) ". The interpretability of the maps and w provides valuable insights: T reveals which properties of the system influence Y, while shows which types of interventions most effectively change these properties.
Linear TCR. The TCR formulation by Kekić et al. [10] imposes some significant constraints: (i) Linear transformations for T and w, simplifying analysis and facilitating interpretability but limiting expressivity; and (ii) Gaussian KL-divergence approximation for the discrepancy D in Eq. (2.5), where distributions are approximated as Gaussians with matched means and variances (see App. A.3). This has the benefits of taking a differentiable closed-form and allowing easy optimization.
FROM RL POLICIES TO CAUSAL MODEL REDUCTION
RL episodes inherently form causal chains where the agent's observations lead to actions, which cause environmental changes resulting in new states and rewards. These interactions naturally map structural causal models. To identify the causal impact of specific actions on overall performance, we need to observe counterfactual outcomes- wwhhtt would have happened if the agent had acted differently. We achieve this by introducing controlled perturbations to the policy's chosen actions and observing their effects. Specifically, we construct episodes where each action A, selected by the policy is perturbed by a small random shift d.A, ~ N(0, o1) before being executed in the environment:
(Su, A5 R1, S1, A1 5511 R2, ..., Sr-1, - Rr, Sr)
(3.1)
These perturbations serve as shift interventions in our causal framework and allow us to observe how deviations from the policy's intended actions propagate to the cumulative reward.
To formalize this as a TCR problem, we define the low-level endogenous variables X consisting of Su...., S and rewards X (= -R-(RR.... ) as shown in Fig. 1a. The interventions are ( 8Ar-1. 0, ... 0)T, where we intervene only on actions and not on states, and the target variable the cumulative reward Y= -- R.
states and actions X (1)==AAu Ar......,A-. Ar
With this formulation,
now apply the TCR framework learn simplified high-level model with one high-level cause that best explains variations the target variable Y. The reduction maps Ti and wi identify which aspects of the states and actions most significantly influence the overall performance, thereby providing an interpretable causal explanation of policy behavior.
4 NONLINEAR TARGETED CAUSAL REDUCTION (NTCR)
While the linear TCR framework introduced in Sec. 2.3 provides a principled approach to learning simplified causal models, many real-world phenomena particularly in reinforcement learning involve inherently nonlinear relationships that cannot be adequately captured by linear maps. In this section, we extend TCR to nonlinear settings while preserving its key benefits of interpretability and causal consistency. This extension, which we call nonlinear TCR (nTCR), allows us to discover and represent causal patterns in complex interactive systems such as those controlled by RL policies. Despite the added flexibility of nonlinear functions, we show that our approach maintains strong theoretical guarantees. Specifically, we demonstrate both the existence and uniqueness of reductions with exact interventional consistency for a broad class of nonlinear causal models.
4.1 NORMALITY REGULARIZATION
The original discrepancy of linear TCR in Eq. (2.5) does not enforce distributions to fully match (see App. A.3), and the nonlinear reduction can therefore lead to a highly non-Gaussian high-level cause. We therefore introduce a normality regularization encouraging गi (X) ) to be Gaussian. In addition to being more faithful the initial idea of Def. 2.2, this allows for easier interpretation of
5

---

Published as a conference paper at ICLR 2026
the learned causes (as it enforces simple unimodal cause distribution). The theory behind the choice of consistency objective is further discussed in App. D.
Given the push-forward distribution ((() (Z) (with CDF (2)) (z)) of the high-level cause under
intervention we first standardize this distribution to zero mean and unit variance to obtain P ( (Z). We then measure the deviation from normality as the 1-Wasserstein distance Wi between this standardized distribution and the standard normal distribution (0,0 1) (with CDF ((x)))
nnrm = -(e) (i( ((() T,sid (Z), 1D] = PLo U/ |Fpc (2)
(2)()) (2) (((x)) (4.1)
The total optimization objective for nTCR then becomes Lioaal = Loons + Thorm Cnorm. Here, 1/oorm is a hyperparameter that controls the strength of the normality regularization, balaneing the trade-off between achieving interventional consistency and maintaining Gaussian-like distributions for the high-level cause. In practice, we compute from using differentiable approximation of the CDF and by evaluating the Lı distance at finite set of points, presented in App. C.I. This approach allows for efficient optimization while maintaining the desired distributional properties of the high-level causes.
4.2 EXACT SOLUTIONS
Nonlinear TCR allows much larger space of functions to fit the learning objective. This can lead to overfitting and even potential non-identifiabiiity issues where multiple reductions could exist, which would strongly limit the interpretability of the approach. The following statement provides identifiability guarantees even in the nonlinear case, as long as we obtain exact solutions.
Proposition 4.1 (Uniqueness). Assume (i) the low-level model an additive noise SCM the form
X (1)) (X)( + U2(1) + (x(1) X (0) fo(X X xx1)) U =(0))
Uz(1) P1 indep. of Un(0) Po. (4.2)
(ii) P1 admits probability density with non-vanishing Fourier transform, (iii) The high-level model has non-zero causal effect (a + 0). Then, if there exists a constructive transformation following Def. 2.3 that is exact for all in)) ER #()) is also unique up to multiplicative and additive constants.
See App. B.1 for the proof. The above result does not guarantee the existence of such exact transformations. The following proposition shows we can construct families of low-level additive noise models that admit nonlinear exact transformation 7₁ 72(X (X)x(1)) + To (xxx))))-
Proposition 4.2 (Existence). Assume low-level additive SCM following Eq. (4.2). Assume addition- ally that (i) U is jointly Gaussian, (ii) fo(X (Xx(0)= X x = ho (x((x(())) x<())) + B ((xx(1) - fx((x()))
for some functions ha, SI, some matrix B E [(**(0)***00
and (ii) Y = To (x
a (X+(0) (x)))(())) for arbitrary non-vanishing a E R# then the transformation de- fined by the following maps is exact
7: (X a B(X+(1) fi fX()())) (1((m1))) a Biz
(4.3)
See App. B.2 for the proof. We do not expect to cover all possible nonlinear exact transformations with this class of models. Neither do we expect that simulations of real-world systems considered in applications satisfy the above conditions and therefore admit an exact transformation." However, the above results can be used to validate our algorithms, as discussed in Sec. 5.1.
4.3 INTERPRETABLE NONLINEAR FUNCTION CLASS
When the reduction allowed be nonlinear, it can be challenging to interpret what the high-level cause and intervention represent in the system without constraints on the mappings Ti and w1- For example, RL episodes possess inherent temporal structure that we can exploit build more interpretable function classes for these mappings. We can express X' as a Cartesian product across
We thus use approximate consistency objective find explanation that "as consistent as possible".

---

Published as a conference paper at ICLR 2026
d features, with each feature represented as time series X=X' x where X represents the space of possible trajectories for the j-th feature over time. Each feature's 's trajectory space can be further decomposed as =²= x x... x X Leveraging this structure, we define interpretable nonlinear function class for TI as:
The intervention space I can be decomposed similarly.
T W/e exp (-(x= 1- (15)) (4.4)
(X)
Σ
W (x,(
Where Wjat are learned weights, and X² is the value of feature j at time t. For our implementation, we use Gaussian kernels Φ1,5 as the basis functions, where ( are fixed parameters, set to span the range of values typically encountered, see Fig. Ic. The kernel width jja and the number of kernels control the vias-variance trade-off: wider kernels and fewer kernels both increase smoothing (higher bias, lower variance). This can be viewed continuous one-hot encoding of variable- value pairs across the episode, allowing for smooth interpolation between similar states. This approach ensures that nonlinear reduction remains interpretable, we identify which features, at which time steps, contribute most significantly to the high-level causal explanation by examining the learned weights Wj.s-
CASE STUDIES
5.1
EXPERIMENTS ON SYNTHETIC DATA
Train Consistency
Tau Identification
Omega Idantification
5.0
nTCR
Consistency
linear TCR
ld. Loss
Id. Loss
0.5
2.5
0.0
0,0
7500 10000
5000
10000
2500
10000
Training Steps
Training Steps
Training Steps
Figure 2: Identification of Ground-Truth Solutions for Synthetic Low-level Models. Consistency loss (left) and the identification losses measuring agreement with the ground-truth solutions (definition in App. E. for the 7- and w-functions (middle and right) over the reduction training run.
Before applying our approach to RL problems, we first validate its theoretical properties on synthetic data generated from known low-level causal models. This controlled setting serves two important purposes: (i) it demonstrates that for this model class, nTCR can indeed find the unique solution that perfectly minimizes the causal consistency loss, confirming our theoretical guarantees; and (ii) validates that our training methodology successfully converges to the correct solution.
We generate synthetic data from 10 randomly sampled low-level causal models following the structure described in Prop. 4.2 with dim (X.) = 2 and dim(Xi) 9. Each entry in and B are sampled from a standard normal distribution. For these experiments, we use neural networks as generic function approximators for both Tı and W1. For more details the experimental setting, please refer to App. E. Fig. 2 shows the consistency loss and an identification measure that quantifies how closely the learned and w-maps match the ground truth solutions." We observe that nTCR successfully finds reductions with near-zero consistency loss and converges to the theoretical solution outlined in Prop. 4.2, validating both our theory and implementation.
5.2 PENDULUM
Pendulum [21] is classic swing-up control task where pendulum must be brought to the upright position. With limited torque, the agent must develop momentum through strategic oscillations rather than directly moving to the goal position. The state space comprises three variables: x-y coordinates (0)s sin(O) (f) and angular velocity, while the action is the applied torque. The reward
*The function W1 is defined analogously, operating on the intervention space I=(1)- "A normalized 1.2 loss between the learned and ground-truth high-level cause; see App. E.1 for details.
7

---

Published as a conference paper at ICLR 2026
Palicy A
Policy B
Ang.laa Volcely
Theta
T'iiular Ve oaly
Thura
3.00
3.14
8.00
5.14
1.57
0.00
0.0
0.00
-02
0.5
corque
lorque
lorque
2.00
7.51
2.00
7.61
0.1
1.00
3.72
1.00
8.72
0.00
0.0
3,007
80
0.00
44.07
000
-1 10
-1.66
1.00
-0.06
-05
2.00
-7.6
2.00
-7.65
23
Time
Time
Time
50
Time
50
Pendulum System
Mean Rewards by Inital Angio
X
20
28 01.2.04
Moon
-00
Mear
Torque
30 3..3 04l
00 %
Condition
50 100 125
180
200 175
to 3ro4) (:3rld -r84]
Figure 3: Pendulum Task. The top two rows show the learned nTCR and w-maps for two policies A and B. The heatmaps show the learned reductions 7/ (x*) and wf (+'), where indexes the state/action variables (angular velocity, theta, and torque). Since we only intervene on torque, it is the only variable with nonzero w-map. The bottom left plot shows the pendulum system setting. The middle plot on the bottom row shows the mean reward for Policy A for pendulums starting in the left quadrant (Condition 1) and those starting in the right quadrant (Condition 2). The standard error of the mean is shown (error bars are smaller than the data point in the plot). The bottom right plot shows the mean x z-ppoiition of the pendulum for episodes under Policy and the 90% confidence interval.
function penalizes deviation from the upright position, angular velocity, and action magnitude. For the reductions, we choose the more natural feature 0 instead of the x- and y-positions. More details on the experiment setting, and the agent and nTCR training are given in App. E.2.
Fig. 3 (left, Policy A) shows that nTCR identifies two trajectory groups with significant reward varia- tions: pendulums starting the bottom-righh right cornrr swinging clockwise (red, higher rewards) versus those starting bottom-left swinging counterclockwise (blue, lower rewards). The reduction shows that the agent performs better clockwise than counterclockwise a surprising bias given that initial condi- tions are uniformly sampled and the environment has mirror symmetry along the axis of gravitation.
For Policy B (Fig. 3, right), the learned omega map reveals that shifting torque more negative values would improve performance, particularly towards the end of the episode. Analysis confirms this (Fig. 3, bottom-right); the policy fails to consistently stabilize the pendulum upright, allowing to tip over in the positive 0 direction before applying positive torque for full rotation recovery. nTCR correctly identifies that applying more negative torque at the top position would prevent this instability.
5.3 TABLE TENNIS
We evaluate approach robot table tennis simulation based on the real-world setup developed in [22, 23]. The environment features a four degrecs-of freedom artificial muscles, positioned task to return incoming balls launched by a ball gun to a desired landing point on the opponent's side of the table. The state space includes the robot state (joint angles, joint velocities, and air pressures in each muscle) and the ball state (position and velocity). The action space consists of the changes in desired pressures for each of the eight pneumatic muscles. The task requires precise timing and positioning to successfully intercept the ball and direct it toward the target. More details on the environment, the RL- and nTCR-traat ITC--rrainig are given in App. E.
of-eeedoom robot arm actuated by pneumatic side of table tennis table, shown in Fig. 4 (a,b). The robot's s
"Example episodes for the two policies are shown in the supplementary video files. "The behavior of the trained policy can he in the supplementary video files.
8

---

Published as a conference paper at ICLR 2026
Joint 3
Joint
Joint
T.boen
T T.lomm
T-leint Prx.
2:16
0.001
1.93
1:50
Id]
[0]
[3]
0.001
0.000
0.000
0,000
00001
-0.002
-0.001
00001
Ball Fos. X
T Ball Pos.
T Boll Pox. Z
1s1
0.0085
0.002
0.01
2.2
0.19
-2.0
-0.13
-1.54
(2)
-3.07
-2.55
20.002
2.10
0.01
80
Time
100
80
Time
100
80
Time
100
Figure 4: Table Tennis Task. A robot arm is trained to return incoming balls to target location on the opponent's 's side of the table (a). The robot arm and its rotational axes are shown in (b). (c) shows the positions of 400 balls that the robot missed relative to the racket. (d-i) show the learned 7 reduction maps for the joint positions 0-2 and the ball position. The reduction maps for the other variables are shown in App. E.3. The dotted vertical lines show the time range where the robot typically hits the ball.
The learned maps for the joint positions 0-2 and the ball positions shown in Fig. 4. We make two key observations: one regarding the movements of the robot arm and another with respect to performance differences across ball trajectories. Joint O is the main axis used swing the robot arm to accelerate the ball (see Fig. 4 (b)). The 7-map for this variable is shown in Fig. 4 (d). The map highlights two critical state regions for joint position 0 during the period when the arm swinging towards the ball (regions A and B in Fig. 4 (d)): (A) the arm swings back (negative angle values) before the ball arrives and forward when the ball arrives, leading to high rewards, and (B) the arm swings forward too early, leading to low rewards.
The TCR reductions for the ball position reveal two clear trends: (i) balls that travel further toward the
outside edge of the table are more difficult to hit (Fig. 4 (g)), and (ii) balls that bounce further from the robot, or closer to the net, present greater challenges (Fig. 4 (h. i)). We validate these findings with an analysis of the missed balls, shown in Fig. 4(c). We observe that more balls miss the racket toward the top compared to the bottom, and more to the left (corresponding to the direction toward the outside of the table) than to the right. These observations are consistent with the reduction values for joint positions 1 and 2, shown in Fig. 4 (h, i). Around the time of ball contact, they have their largest positive contributions near π //4--corresponding to relatively high and further out racket- and their largest negative contributions for larger angles- conleponinr to a lower racket closer to its base.
6 RELATED WORK
Explainable reinforcement learning (XRL) focuses on understanding the decision-making processes of learning RL agents. Milani et al. [7] propose taxonomy for XRL approaches into three categories: feature importance methods explaining the immediate context for single actions (e.g. (24-26]), learn- ing process methods that reveal influential experiences from training (e.g. [27-29]), and PLEs [30-32]. which is the category our approach belongs to. Specifically. nTCR belongs to the subcategory of techniques that extract abstract states, as we reduce the dimensionality of the states, actions, and reward spaces into high-level variables. A notable causal approach to XRL is Madumal et al. [33], who generates contrastive explanations for agent behavior through counterfactual analysis, though their approach differs from ours in that they require predefined causal graph structure and focus on explaining individual actions rather than identifying high-level causal patterns across episodes. Also related to nCCR is [34] who rely perturbations of the agent's 's policy to identify the time steps where the agent's 's control is crucial. Our approach goes further to extract a representation of what the policy actually does, with description of the states and actions that benefit hinder the global performance.

---

Published as a conference paper at ICLR 2026
Our approach is an extension of linear TCR [10] which is itself grounded in theoretical work on causal abstraction [13-16, 19]. While these works establish formal conditions for valid causal abstractions, few have addressed the challenge of learning such abstractions from low-level systems. Geiger et al. [17, 35] focus on language models, where high-level variables and interpretations are already available and used to constrain neural network's structure. Our approach, in line with Kekić et al. [10], takes the opposite direction: high-level abstractions are built from the ground up. This conceptually related causal feature learning [18] but differs in two key aspects: the use of shift interventions, and building continuous rather than strictly discrete high-level representations. Linear TCR was previously applied by Kekić et al. [10] only to scientific simulations. The similarities between simulating physical systems and RL environments motivated us to study and adapt this framework to the context of policy explanation.
Another related field is the study of Markov Decision Process (MDP), which model the systems we want to control with RL.-agents. Several works [36-38] have taken a causal perspective to study MDPs and related models in order to analyze policy optimization, and notably the relevance of specific state variables for those policies. These methods can be leveraged to learn causal representations that are suitable for training RL-agents, which contrasts with goal of interpreting already trained policies.
7 DISCUSSION
We have introduced nTCR, a nonlinear dimensionality reduction framework for causal models, and used it to generate PLEs for trained policies. We have experimentally demonstrated that nTCR (1)
can learn causally consistent explanations when they exist (as predicted by our theoretical results), (2) can identify biases and failure modes in RI. policies Pendulum and robot table tennis simulation.
Our framework is subject to number of constraints: (1) it makes linear and Gaussian high-level model assumptions that may need be dropped in some applications, although it poses further interpretabil- ity challenges; (2) it requires to sample many perturbed episodes of interactions with the RL agent, which makes it mainly suited to scenarios where the agent is trained in simulation before real-world transfer, (3) it uses shift interventions, suited to continuous action spaces of the real-world applications we focus on, and adaptation discrete action spaces could be considered, egg. applying shift interven- tions to the logits of the network implementing the categorical policy used to sample discrete actions.
While nTCR overcomes the limitations of linear TCR by allowing for arbitrary functions as reduction maps, one of the central challenges for methods generating explanations is balancing expressivity with interpretability.. Our Gaussian kernel-based function class (Sec. 4.3) represents a middle ground, offering more expressivity than linear maps while maintaining interpretability. More complex or less structured state/action data, such as for policies leveraging image inputs, may require additional steps for representing them appropriately, e.g. segmentation methods [39] or saliency maps [24, 40].
The applicability of our method primarily to simulated environments makes sense because it is designed for scenarios where policies can be safely probed through interventions, primarily in simulators before real-world deployment (e.g. in robotics or autonomous systems). While we focus on simulation due to safety and cost, the framework could, in principle, be applied to real systems where controlled interventions are feasible, and frameworks that combine simulated and real data [41].
Barring these limitations, nTCR provides model-agnostic causal PLEs that can notably be used to assess the behavior of RI. agents in simulations before their potential transfer to the real world.
ACKNOWLEDGMENTS
Jan Schneider is supported by the Konrad Zuse School of Excellence in Learning and Intelligent Systems (ELIIA)), sponsored by the Federal Ministry of Education and Research.
REFERENCES
[1] David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche. Julian Schrittwieser, Toannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. Mastering the game of Go with deep neural networks and tree search. Nature, 2016. [Cited on page 1.]
10

---

Published as a conference paper at ICLR 2026
[2] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, loannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing Atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602, 2013. [Cited on page 1.]
[3] Jens Kober, J. Andrew Bagnell, and Jan Peters. Reinforcement learning in robotics: A survey, The International Journal of Robotics Research, 2013. [Cited page
[4] Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski. Bernhard Firner, Beat Flepp. Prasoon Goyal. Lawrence D. Jackel, Mathew Monfort, Urs Muller, Jiakai Zhang, et al. End end learning for self-driving cars. arXiv preprint arXiv: 1604 2016. [Cited on page 1.]
[5] M. Mchdi Afsar, Trafford Crump, and Behrouz Far. Reinforcement learning based recommender systems: survey, ACM Computing Surveys, 2022. [Cited on page
[6] Rui Nian, Jinfeng Liu, and Biao Huang. A review on reinforcement learning: Introduction and applications in industrial process control. Computers & Chemical Engineering, 2020. [Cited on page 1.]
[7] Stephanic Milani, Nicholay Topin, Manuela Veloso, and Fei Fang. Explainable reinforcement learning: A survey and comparative review. ACM Computing Surveys, 2024. [Cited on pages and 9.1
[8] Marvin Minsky, Steps toward artificial intelligence. Proceedings of the IRE, 2007. [Cited on page 2.1
[9] Pim de Haan, Dinesh Jayaraman, and Sergey Levine. Causal confusion in imitation learning. NeurIPS, 2019. [Cited on page 2.]
[10] Armin Kekić, Bernhard Schölkopf, and Michel Besserve. Targeted reduction of causal models. UAI, 2024. [Cited on pages 3, 4, 5, 10, 15, 16, 19, 26, and 27.]
[11] Judea Pearl. Causality. Cambridge University Press, 2009. [Cited on page 3.]
[12] Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements of Causad Inference. The MIT Press, 2017. [Cited on page 3.J
[13] Sander Beckers and Joseph Y. Halpem. Abstracting causal models. AAAI, 2019. [Cited on pages 3, 4, 10, and 29.]
[14] Sander Beckers, Frederick Eberhardt, and Joseph Y. Halpemn Approximate causal abstractions. UAI, 2020.
[15] Riccardo Massidda, Atticus Geiger, Thomas Ieard, and Davide Bacciu. Causal abstraction with soft interventions. CLeaR, 2023.
[16] Eigil F. Rischel and Sebastian Weichwald. Compositional abstraction error and category of causal models. UAI, 2021. [Cited on pages 10 and 29.]
[17] Atticus Geiger, Christopher Potts, and Thomas Icard. Causal abstraction for faithful model interpretation. arXiv preprint arXiv:2301.0477 04709, 2023. [Cited on pages 10 and 29.]
[18] Krzysztof Chalupka, Pietro Perona, and Frederick Eberhardt. Visual causal feature learning. UAI, 2015. (Cited on pages and 10.]
[19] Paul K. Rubenstein, Sebastian Weichwald, Stephan Bongers, Joris M. Mooij, Dominik Janzing, Moritz Grosse-Wentrup, and Bernhard Schölkopf. Causal consistency of structural equation models. UAI, 2017. [Cited pages 3, 4, 10, and 29.]
[20] Hamza Keurti, Hsiao-Ru Pan, Michel Besserve, Benjamin Grewe, and Bernhard Schölkopf. Homomor- phism autoencoder learning group structured representations from observed transitions. ICML, 2023. [Cited on page 4.1
[21] Mark Towers, Ariel Kwiatkowski, Jordan Terry, John U Balis, Gianluca De Cola, Tristan Deleu, Manuel Goulão, Andreas Kallinteris, Markus Krimmel, Arjun KG, et al. Gymnasium: Standard Interface for Reinforcement Learning Environments. arXiv:2407. 17032, 2024, [Cited on pages and 22.]
[22] Dieter Büchler, Simon Guist, Roberto Calandra, Vincent Berenz, Bernhard Schölkopf. and Jan Peters. Learning to play table tennis from scratch using muscular robots. IEEE Transactions Robotics, 2022. [Cited pages& and 23.]
11

---

Published as a conference paper at ICLR 2026
[23] Simon Guist, Jan Schneider, Hao Ma, Le Chen, Vincent Berenz, Julian Martus, Heiko Ott, Felix Grüninger. Michael Muchlebach, Jonathan Fiene, Bernhard Schölkopf, and Dieter Büchler. Safe & accurate at speed with tendons: A robot am for exploring dynamic motion. In Robotics: Science and Systems, 2024, [Cited on pages and 23.]
[24] Samuel Greydanus, Anurag Koul, Jonathan Dodge, and Alan Fern. Visualizing and understanding Atari agents. ICML, 2018. [Cited on pages 9 and 10.]
[25] Rahul Iyer, Yuezhang Li, Huao Li, Michael Lewis, Ramitha Sundar, and Katia Sycara. Transparency and explanation in deep reinforcement learning neural networks. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, 2018
[26] Alexander Mott, Daniel Zoran, Mike Chrzanowski, Daan Wierstra, and Danilo Jimenez Rezende. Towards interpretable reinforcement learning using attention augmented agents. NewrIPS, 2019. [Cited on page 9.]
[27] Francisco Cruz, Richard Dazeley, Peter Vamplew, and Ithan Moreira. Explainable robotic systems: Under- standing goal-driven actions in reinforcement learning scenario. Neural Computing and Applications, 2023. [Cited on page 9.]
[28] Andrew Anderson, Jonathan Dodge, Amrita Sadarangani, Zoc Juozapaitis, Evan Newman, Jed Irvine, Souti Chattoppadhaay, Alan Fern, and Margaret Burnett. Explaining reinforcement learning to mere mortals: An empirical study. IJCAI, 2019.
[29] Ioana Bica, Daniel Jarrett, Alihan Hüyük, and Mihaela van der Schaar. Learning "what-if" explanations for sequential decision decision-making. ICLR, 2021. [Cited on pages and 30.J
[30] Tom Zahavy, Nir Ben- Bei-Zrihem, and Shie Mannor. Graying the black box: Understanding DQNs. ICML, 2016. [Cited on page 9.1
[31] Nicholay Topin and Manuela Veloso. Generation of policy-level explanations for reinforcement learning. AAAI, 2019. [Cited on page 30.]
[32] Sarath Sreedharan, Siddharth Srivastava, and Subbarao Kambhampati. TLdR: Policy summarization for factored SSP problems using temporal abstractions. ICAPS, 2020. [Cited on page 9.]
[33] Prashan Madumal, Tim Miller, Liz Sonenberg, and Frank Vetere. Explainable reinforcement learning through causal lens. AAAI, 2020. [Cited on page 9.]
[34] Zelei Cheng, Xian Wa, Jiahao Yu, Wenhai Sun, Wenbo Guo, and Xinyu Xing. StateMask: Explaining deep reinforcement learning through state mask. NeurlPS, 2023. [Cited on page 9.]
[35] Atticus Geiger, Hanson Lu, Thomas leard, and Christopher Potts. Causal abstractions of neural networks. NeurlPS, 2021. [Cited on pages 10 and 29.]
[36] Biwei Huang, Chaochao Lu, Liu Leqi, José Miguel Hermández-Lobato, Clark Glymour, Bernhard Schölkopf. and Kun Zhang. Action-sufficient state representation learning for control with structural constraints. ICML, 2022. [Cited on page 10.]
[37] Yudi Zhang, Yali Du, Biwei Huang. Ziyan Wang, Jun Wang, Meng Fang, and Mykola Pechenizkiy. Interpretable reward redistribution in reinforcement learning: A causal approach. NewrIPS, 2023.
[38] Yuren Liu, Biwei Huang, Zhengmao Zhu, Honglong Tian, Mingming Gong, Yang Yu, and Kun Zhang. Learning world models with identifiable factorization. NearlPS, 2023. [Cited on page 10.]
[39] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson. Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. ICCV, 2023. [Cited on page 10.]
[40] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-CAM: Visual explanations from deep networks via gradient-based localization. ICCV, 2017. [Cited on page 10.]
[41] Katie Kang, Suneel Belkhale, Gregory Kahn, Pieter Abbeel, and Sergey Levine. Generalization through simulation: Integrating simulated and real data into deep reinforcement learning for vision-based au- tonomous flight. In ICRA, 2019. [Cited on page 10.]
[42] Stephan Bongers, Patrick Forré, Jonas Peters, and Joris M Mooij. Foundations of structural causal models with cycles and latent variables. The Annals of Statistics, 2021, [Cited pages 15 and 17.]
12

---

Published as a conference paper at ICLR 2026
[43] Jean-François Cardoso. Dependence, correlation and gaussianity in independent component analysis. JMLR, 2003. [Cited on page 16.]
[44] Antonin Raftin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernestus, and Noah Dormann. Stable-Baselines Reliable reinforcement learning implementations. JMLR, 2021. [Cited on pages 22 and 23.]
[45] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov. Proximal policy optimization algorithms. 1644007707.06377, 2017. [Cited on pages 22 and 23.J
[46] Therdora-Mara Pislar. Sara Magliacane, and Atticus Geiger. Combining causal models for more accurate abstractions of neural networks. CLeaR, 2025. [Cited on page 29.1
[47] Fabio Massimo Zennaro, Nicholas Bishop, Joel Dyer, Yorgos Felekis, Anisoara Calinescu. Michael Wooldridge, and Theodoros Damoulas. Causally abstracted multi-armed bandits. UAI, 2024. [Cited on page 29.]
[48] Doina Precup, Richard S Sutton, and Satinder Singh. Thcorctical results on reinforcement learning with temporally abstract options. In Machine Learning: ECML-98: 10th European Conference on Machine Learning Chemnitz, Germany, April 21 23, 1998 Proceedings 10, pages 382 393. Springer, 1998. [Cited page 30.]
[49] Thomas Dietterich. State abstraction in MAXQ hierarchical reinforcement leamiing NewrIPS, 1999. [Cited on page 30.]
[50] Stratis Tsirtsis, Abir De, and Manuel Rodriguez. Counterfactual explanations in sequential decision making under uncertainty. NeurlPS, 2021. [Cited page 30.]
13

---

Published as a conference paper at ICLR 2026
Appendix
CONTENTS
A Additional Background
14
A.I Cyclic Structural Causal Models
14
A.2 Targeted Causal Reduction (Multiple Causes)
15
A.3 KI. Divergence and Information Geometry
15
Proofs of Main Text Results
16
B.1 Uniqueness of Exact Transformation
16
B.2 Analytical Solution
17
C Additional Method Details
18
C.1 Differentiable Normality Regularization
18
Additional Theoretical Results
19
E Experimental Details
20
E.1 Synthetic Experiments
20
E.2 Pendulum
22
E.3 Table Tennis
Additional Experimental Results
25
F.1
Effect of Normality Regularization
25
F2
Linear TCR for Pendulum Policies
26
F3
Linear TCR for Table Tennis Policies
27
F.4
Comparison of Consistency Loss between Linear and Nonlinear TCR
28
F5
Pendulum Policy Reductions under Different Intervention Strengths
28
Additional Related Work
29
H
Use of Large Language Models (LLMs)
30
A ADDITIONAL BACKGROUND
A.
CYCLIC STRUCTURAL CAUSAL MODELS
Definition A.1 (SCM (with cycles allowed)). An n-dimensional SCM is a triplet M = (9, S, Pv)
consisting of:
a joint distribution Py over exogenous variables {Usbssm
directed graph & with vertices.
a set S [X, := (Paj, Uy).j = 1,...,ni n) of structural equations, where Pay are the variables indexed by the set of parents of vertex j in G,
14

---

Published as a conference paper at ICLR 2026
such that for almost every u, the system (z, == {pa Ug)} has unique solution g()), with measurable.
The unique solvability condition allows to consider very general class of SCMs by allowing cycles (g may not be DAG, while DAG would imply unique solvability). Moreover, we allow hidden confounding through the potential lack of independence between the exogenous variables {U,}- See Bongers et al. [42] for a thorough study of these models. Under these conditions, the distribution Po entails well-defined joint distribution over the endogenous variables P(X). .. By unique solvability, we define the mapping from exogenous to endogenous variables of this model under interventions
as (+) >> a such that ( (U) PM (X).
A.2 TARGETED CAUSAL REDUCTION (MULTIPLE CAUSES)
In this paper, focus single high-level cause for Targeted Causal Reduction, this is used to develop our RL explainability method. Here, we present the more general TCR formulation with multiple high-level causes.
TCR Framework with Multiple Causes. extends the single-cause version as follows:
The general TCR framework with multiple causes
1. Target-oriented Structure:
We designate a scalar target variable Y = (X) that quantifies a phenomenon of interest. The high-level model has (n++1) endogenous variables {Y, Zi...., Zn. where Zi...., Zn are the learned high-level causes of Y.
Parameterized High-level Models: The high-level SCM is constrained a class of linear additive Gaussian noise models h with parameters to be learned. The exogenous variables (W, ...,. W, have factorized Gaussian distribution Pw = II Pw:
3. Constructive Transformations The linear maps and w are constructive, meaning each dimension depends only on a designated subset low-level variables:
(To T1 - Tn) with Th (z)) (wo, W1, ,,..,..,,,, wi) with We Wy. (+x
(A.1)
(A.2)
Here, the alignment function partitions the N low-level variables into non-overlapping subsets. Specifically, (0) identifies indices of low-level variables that determine the target Y. while x (1), (n) identify indices for each of the high-level causes. These subsets satisfy x x<k) n
(1) = for all & + ensuring each low-level variable contributes to at most one high-level variable.
Our non-linear TCR framework may in principle be extended this multiple cause setting, but would require the introduction of extra regularization term in the optimized objective in order to constrain the method separate distinct causes, as elaborated in [10].
A.3 KL DIVERGENCE AND INFORMATION GEOMETRY
The Kullback-Leite Leibler divergence be used as a pseudo-metric in the space of probability densities with common support over a given space that we take to be R[d] for simplicity. By definition, for two densities p, q over the same support
DKL()llg) = -
p(x) log
p(z) ((x)
dr.
turns out this can be used to define projections on specific manifolds of probability densities. In particular, can define the projection of density the manifold of Gaussian densities Mg.
(roje()) = N (u(p), Var(p)
where plp) and Var(p) are the mean and covariance matrix associated density p.
It is easy to check that Proje (p) is the closest point to p the manifold of Gaussian densities in the sense that
DRL(p, Proje(p)) min DKL (p. ().g)
M
15

---

Published as a conference paper at ICLR 2026
This moreover leads to property analogous the Pythagorean theorem in Euclidean spaces: for any Gaussian density q and any (possibly non- Gaussaan density we have the decomposition (see [43] for details)
(RI g) = DRL (ro)c(p)) + (Poo( (p),
While the second term DKL. (Projg (p), q) has classic analytic expression that is optimized in linear TCR [10), the first term can be written
(P Proig(p)) = H(Proic(p) H(p)
where (qq) - p(z) log(p(x)) dæ denotes the entropy associated density q. Using invariance of the KL divergence by parameter transformation, we also have
(KL(P, (p)) = (Pnorm: ((0, La) (NV La)) - H (Pnorm),
where Pnorm denotes the density obtain by normalizing the first and second order statisties of p (by removing the mean and dividing by the standard deviation), and Ia denotes the identity matrix for the dimension d of the considered space. This term is called negentropy, as is up to an additive constant the opposite of the entropy, with the additional specificity that it vanishes when the density Gaussian (hence with maximum entropy).
The entropy is very challenging to estimate for arbitrary distributions. In contrast, the KL. divergence between two (possibly multivariate) Gaussians has an analytic expression.
DKL Ei)l Σ2))
1 (og(det(E2)/dette)) d tr(2 (#2 11) 2 (n2 - mi) (A.3)
2
Hence, [10] use a Gaussian Kl.-divergence approximation to optimize linear TCR, defined as
Da(p, g) DKL (P)ojc(P))
(A.4)
where q is the density of the high-level model, and p is the push-forward density of the low-level model. Because is assumed Gaussian, this discrepancy has an analytic expression which allows optimizing the loss of Eq. (2.5). From the above development we immediately get that
Dc(p.0 9) S (KL(P. = DKL Proje + DKL (Projo(()) g).
(A.5)
and in particular, the inequality is strict if is not Gaussian. As consequence, minimizing De does not guarantee priori that the KI. divergence is minimized.
The resulting expression of the consistency loss for linear TCR used by Kekić et al. [10] is thus
Loons (T, (.7) 7) Ei-P, Da (p! (Y, (1(()) (X, (X, 2 with p(t) T# [ (A.6)
with the above defined DG-
B PROOFS OF MAIN TEXT RESULTS
B. UNIQUENESS EXACT TRANSFORMATION
Proposition 4.1 (Uniqueness). Assume (i) the low-level model is an additive noise SCM of the form
X x<(1)) [X (()) + UR(1) + &x(1) ( X xx
Uz(1) P1 indep. of Uz(0) Po. (4.2)
X x
x(0)
(ii) P1 admits probability density with non-vanishing Fourier transform, (ii) The high-level model has non-zero causal effect (ar + 0). Then, if there exists a constructive transformation following Def. 2.3 that is exact for all im E mm)) is also unique up to mullipicaative and additive constants.
Proof. If there is an exact transformation, marginal probability densities of the target and its cor- responding push-forward are matched for all interventions. So the expectations of the target with respect to these densities are matched across interventions, such that
E Eco Y = E (0 [Y],
16

---

Published as a conference paper at ICLR 2026
which implies (using linearity of the high-level cause-effect mechanism)
E pp Too Xx)] = ((pct)) [Z]] + B[Wo]-
Moreover, the exact transformation assumption also guaranties the match between expectations for the high-level cause, such that the previous equality can turned into
ppt ( (To (Xo)] a.Epto [T (X1)) B[Wo]
Assuming there exist solution achieving consistency for all i, it must therefore satisfy, up to an additive and multiplicative constant
E poo 7 (X1)) Ecs [To(X))]
Consider alternative solution, it satisfies well, up additive and multiplicative constant
Epto [ (Xx)) Epco (To)))
Then for all (T(1) (I))())
aEp (1 n. (X1) = 0.
Assume non-trivial model, then a o O. Using y (id fi) the mapping from exogenous
endogenous variables (restricted to nodes in π (1)), we can rewrite the condition as
pı(u - 1)ōn (g(()) du = p1 * 8T1 o gg()) 0,
where "*" denotes the convolution operation. Applying the Fourier transform we get
Flp * δτ₁ o g] (E) FF)))(() F(ór, gE( = 0, for all & (T())())
we get that if the Fourier transform of P1 does not vanish, then the Fourier transform of 8T o g must vanish everywhere, leading to g = 0. Assuming g invertible, we get 87₃ = 0 so the reductions are identical up to an additive and multiplicative constant.
B.2 ANALYTICAL SOLUTION
Proposition 4.2 (Existence). Assume a low-level additive SCM following Eq. (4.2). Assume addition-
ally that (i) U is jointly Gaussian, (īi) fol (X a(0)= X Xx(1)) ho X x<0))) + B X x(1) (x(x((1))
for some functions hgs S1, some matrix B E ((*= and (ii) Y = To(Xxt)
a (Xx(0) ha(X X xo))) for arbitrary non-vanishing a E ##) then the transformation de- fined by the the following maps exact
71 (X)(()) = a B(X+() f((x)())), @1(1x()) Biz())
(4.3)
Proof. Because of the choice To. we have
Y= T B (X, fi())) a Un T (Ur (Ui+ + t To.
Such that for some constant "Cst." independent of ix(11
E|y] C'st. a Bia(1)
Moreover, the high-level model's interventional marginal distribution of Y is given by
= aZ + Wo = 7I (x ((1)) + awi (4x(1) + Wo = aT ((id - fi) fz)=1(()) + own own ($x(1)) + Wo.
with Wy an exogenous variable independent of T. (X1). where we use the map "(id - fi ) 11 from exogenous to endogenous variables of the low-level model restricted nodes in 7 Note that "(id - f1'" is invertible by directed acyclic assumption on the graph. If we would allow cycles the graph, this condition would be obtained by the unique solvability assumption of [42]. By matching
17

---

Published as a conference paper at ICLR 2026
the expectation of Y, this implies that up to arbitrary additive constant / (that depends the choice of T1), we have
owi (f1) = 3 (ix
and we fix = 1 to remove one degree of freedom (because and therefore Z can be rescaled accordingly). Now, by cause consistency, get that the pushforward distribution associated to
(X (2) ((1)
( ((((ddf)-+(0 (U1 + ix(1)))
matches the shifted high-level distribution of
Wi 3 a (1a
we reparametrize using TI(.) = (id f1), we get
71 (U>(1)) + (1) = W1 + + Biz(1)
A Gaussian high-level cause satisfying consistency can thus be achieved by using " + 8+a Bu (because linear combination of jointly Gaussian random variables is Gaussian, and distribution shifts due to interventions are consistent). Such that we obtain
x + B + a B(xx(1) - fi (1))
This is also a sufficient condition have a valid exact transformation given the Gaussian assumptions.
C ADDITIONAL METHOD DETAILS
C.I DIFFERENTIABLE NORMALITY REGULARIZATION
Sec. 4.1, we introduced a normality regularization term that encourages the high-level cause distri- bution to be Gaussian by measuring the Wasserstein distance between the standardized pushforward distribution P(4)
r.sud (Z) and the standard normal distribution (0, 1). To implement this regularization way that allows end-to-end gradient- based optimization, we need differentiable approximation
of both the empirical cumulative distribution function (CDF) and the distance measure.
Our approach consists of three main steps:
Standardization.
Given mini-batches of samples from the pushforward distribution of the high- level cause, we first standardize these samples to have zero mean and unit variance:
2 U2
7
(C.I)
where μl, and a the empirical mean and standard deviation of the samples.
Differentiable CDF Approximation. For a differentiable approximation of the empirical CDF, we add smooth step functions of the samples:
F Σ 0(xx - 2;)
(C.2)
where (1) - is the sigmoid function serving smooth version of the step function, 2; are the standardized samples, is the number of samples, and is smoothing factor that controls the sharpness of the transitions in the CDF approximation. Higher values of s create sharper transitions that better approximate the true step function but may lead to sharper gradients.
18

---

Published as a conference paper at ICLR 2026
Wasserstein Distance Approximation. We approximate the 1-Wasserstein distance between distributions by computing the L1 distance between their CDFs at a finite set of evaluation points {21, {21, ,.,...., uniformly spaced in the interval |-4, 4] (covering most of the probability mass of a standard normal distribution):
W1
Σ ( (xy))
(C.3)
m
where 4 is the CDF of the standard normal distribution, computed using the error function: ((z)
[1 + erf(z/v2))
This approximation gives us a fully differentiable loss function that effectively measures how closely the distribution of our high-level cause follows Gaussian distribution. In practice, we use the same number evaluation points as we have samples our mini-batch, and we set the smoothing factor 10.0, which provides a good balance between approximation accuracy and gradient stability.
D ADDITIONAL THEORETICAL RESULTS
As explained in App. A.3, the consistency loss Leons used in linear TCR [10] is only KL divergence between the Gaussian approximation the push-forward distribution of the low-level model and the (Gaussian) high-level model. Therefore, does not explicitly enforce perfect match between distributions, which is the theoretical requirement of an exact transformation according to Eq. (2.3).
In principle, however, the setting of Prop. 4.1 does provide additional constraints that theoretically allow identifiability when Lcons vanishes, as shown below.
Proposition D.1 (Identifiability using DG). We assume that the prior Pı over interventions has a strictly positive density with respect to the Lebesgue measure. Additionally, we assume the setting of Prop. 4.1: (i) the low-level model is an additive noise SCM of the form
<<()>> X x
[f((* + +U + (=(1)) fo() x()) +U(0))
=
Un(1) ~ P1 indep. of Uz(O) ~ Po: (D.I)
(ii) P1 admits a probability density with non-vanishing Fourier transform, (ii) The high-level model has non-zero causal effect (a + 0). Then, there exists a constructive transformation that makes Leons vanish, is also unique up to multiplicative and additive constants.
Proof. We notice that the proof of Prop. 4.1 only exploits equality between the expectations of Z or Y obtained under, on the hand, the high-level model distribution and, on the other hand, the push-forward low-level distribution (for all possible interventions). Since we assume that Loons vanishes, then the discrepancy De must vanish for all i and therefore there is a match between the interventional expectations of Tı (X) for all i, and therefore T₁ is identified following the same argumentation.
Potential Limitations Identifiability based on Leons- However, the above theoretical identifia- bility guarantee requires optimizing with respect to an expectation that integrates over all possible values of the intervention vector, which cannot be evaluated in practice, as we necessarily sample from a finite number of interventions. In contrast, linear TCR has theoretical identifiability guarantees under a finite number of interventions [10]. While we could not derive an equivalent result for nonlinear TCR, conjecture instead negative result for the loss Lcons of linear TCR that illustrates the practical difficulties of the optimization of Lrons-
Conjecture D.2. We assume that the prior Pi has a density that vanishes an open subset of its domain. Additionally, we assume the setting of Prop. 4.1: (i) the low-level model is additive noise SCM of the form
X ( X (0)
fX U + +i fo(X X xx)1) +
Uz(1) P1 indep. of Un(o) Po. (D.2)
by continuity in i and because its probability density is strictly positive, if the discrepancy non-zero any point it would lead to strictly positive expectation, thus strictly positive Lcons-
19

---

Published as a conference paper at ICLR 2026
(ii) Pi admits probability density with non-vanishing Fourier transform, (iii) The high-level model has a non-zero causal effect (a + 0). Then, there exists constructive transformation that makes Leons vanish, there are infinitely many other constructive transformations that also make Lcons vanish.
Justificattin for the Conjecture.
Having Lcons vanish only entails equality between first- and second-order moments of the low-level and high-level push-forward distributions lead constraints on the convolutions of T1 with the exogenous density p1. As exemplified in the proof of Prop. 4.1, using (id fi) leads to
conditions of the form
8)ōTI (g(u)) * 8n g(t) 0
on the support of Pt- So we can choose any non-vanishing function 87, such that p1 * dTi o vanishes outside the support of Pt to obtain different F. satisfying this constraint. This can be done by choosing a function in the Fourier domain of the form h(E) = (()))) with k non-zero but vanishing on the support of P1, such that h k deconvolved by P1- Then obtain that h = k. such that ô7 = k g Given the set of such k is infinite, we conjecture that it is possible to find class of function satisfying both the constraints on first and second order moments at the same time.
Practical Issues with the Optimization of Lrons and Enforcing Non-Gaussianity. Following the reasoning of the above conjecture, we speculate that optimizing Leons with a highly expressive function class may lead to overfitting, in the sense that many highly nonlinear functions can be found that satisfy the first and second order moment constraints of the Gaussian approximation of the KL divergence. Enforcing an exact match between distributions instead of only first and second order moments may help avoid this behavior, e.g. by avoiding the nonlinear function to generate multimodal distributions.
While this can be done by adding to Leons an addition negentropy term enforcing Gaussianity (see App. A.3), entropy is very challenging estimate from finite samples and use differentiable objective. Instead, we resort to the differentiable measure -on-Gaassianitty introduced in Sec. 4.1 based on Wasserstein distance (see App. C.I) that proved stable in our experiments, and is known to have numerical benefits over the KL divergence.
Experiments using this measure as regularizer are shown in App. F.1 and are compatible with our theoretical insights, in the sense that it improve performance when the sampling of the intervention does not cover broad enough range of the exogenous distribution.
E EXPERIMENTAL DETAILS
Data. For each experiment we sample Ni.t distinct interventions and for each distinct intervention, we generate Nep RL episodes that apply that intervention. For the synthetic experiments, we equivalently sample Nop SCM samples that apply the same intervention. That is, we keep the intervention fixed but samsapple the exogenous noise. For all datasets, we use 80% for training and 10% for validation and test, respectively.
E SYNTHETIC EXPERIMENTS
Nonlinear Causal Model for Synthetic Data. For our synthetic experiments, we implement a class of nonlinear causal models that follow the structure described in Prop. 4.2. That is, we sample SCMs with the structure given in Eq. (4.2), where, | (0) and [ = 9, creating an 1 11-dimensional system (9 low-level cause variables plus 2 target variables). The vector E R and the matrix B E x9 have entries sampled from a standard normal distribution, providing the linear mapping from causes to effect and from the low-level variables in T(0) to the target, respectively, by Prop. 4.2. The exogenous noise variables Un(1) and Un(0) are sampled from a normal distribution with mean and standard deviation 1.0. The nonlinear functions fi and ho consist of component- wise nonlinear mechanisms represented by randomly initialized two-layer neural networks with 10 hidden units and tanh activation functions.
required
20

---

Published as a conference paper at ICLR 2026
Average Reward vs. Intervention Strength (Pendulum)
Average Reward Intervention Strength (Table Tennis)
Average Reward
Avarage Reward
Noward Sea Dey
250
0.5
-500
0.0
0.0
0.5
1.0
1.5
2.0
0.00
0.25
0.50
0.75
1.00
1.25
1.50
1.75
2.00
Std. of intervention Distribution
Std. of Intervention Distribution
(a) Pendulum Task
(b) Table Tennis Task
Figure A: Relationship Between Intervention Strength and Task Performance. These plots illustrate how the average episode reward varies as a function of the intervention strength a (standard deviation of the Gaussian perturbations 8A,, (0,00²)) for both RL tasks. The shaded regions represent +1 standard deviation of the reward distribution across episodes. The star markers indicate our selected intervention strengths, which were chosen at the threshold where increased perturbation begins to significantly degrade policy performance. This selection balances between introducing sufficient causal signal for the reduction algorithm and preserving the policy's 's core behavior patterns.
Intervention Generation. For each experiment, we generate interventions by sampling from standard normal distribution, creating shift interventions (x()) NO 1). We generate Nint 10" distinct interventions, and for each intervention, we sample Nep = 1024 data points from the causal model.
Neural Network Architecture for nTCR. For these synthetic data experiments, the reduction maps and W1 are parameterized using neural networks with residual connections. Each network consists of a first linear layer that maps the input dimension hidden dimension of 256, followed by 8 hidden layers with Softplus activation functions and residual connections. The final layer maps the hidden representation the high-level cause/intervention. This architecture with residual connections helps in learning complex functions while maintaining gradient flow during training. This amounts to approximately 1.2 10" parameters for 7ı, w1 and the high-level causal model combined. Unlike our RL experiments, we do not use the Gaussian kernel representation described in Sec. 4.3 since we are only interested verifying the identification properties of our approach, not interpretability.
nTCR Training. We train our nTCR model using the Adam optimizer with an initial learning rate of 5 10-4 and cosine annealing learning rate scheduler that gradually reduces the learning rate to zero over the course of training. For the normality regularization, we set 7/orm = 1.0 to encourage the learned high-level cause follow Gaussian distribution. We batch size of 512, meaning that each training step processes 512 distinct interventions sampled from our prior distribution P(t).
Identification Metric. To quantify how well our learned reduction maps align with the ground truth solutions from Prop. 4.2, we employ a normalized, debiased L2 loss. Let 2/eamed T -leamee (X;) be the high-level cause computed using our learned reduction, and = (X,) be the corresponding ground truth value for sample i. We first standardize both sets of samples:
leamed glearedd
gleamed
(E.1)
Σ (2) leamed learned
2
Ett
(E.2)
E, (2f)) 20r
where gleamed and 28 are the respective sample means. We then find the optimal rescaling coefficient that minimizes the L2 distance:
Σ, gleamedd 5gt
arg min
Σ
(c- glearned
(E.3)
Σ,
(2)earned)
21

---

Published as a conference paper at ICLR 2026
The final identification loss is the normalized minimum L2 distance:
Lia(T)=
1 Vri
E(e (c glearned
2
(E.4)
where n is the number of samples. The same procedure is applied to evaluate the wi map. This metric accounts for the fact that the theoretical solutions are only identified up to multiplicative and additive constants, while measuring how well the learned reductions capture the structural properties of the ground truth solutions.
Compute Resources. All synthetic experiments were conducted on NVIDIA Quadro RTX 6000 GPUs with CPU cores allocated per run. Each complete training required approximately 48 hours of computation time. We performed 10 different runs, each with a different randomly generated causal model.
E.2 PENDULUM
Trajectories with Highest and Lowest Predicted Reward
Highest Y_hat
Lowest Y hat
Angular Velocity
2
Thota
Torque
0-
-2
40
20
60
Time
Time
Time
Figure B: Episodes with Highest and Lowest Predicted Reward. The plot shows episodes from
the test dataset with the highest and lowest cumulative reward predicted by nTCR for Policy A. The red lines display the 10 episodes with highest predicted rewards, while the blue lines show the 10 episodes with lowest predicted rewards.
Each episode in the Pendulum task [21] has length of T 200 steps. We apply interventions to the actions for the first 75 steps, and define the target variable as the cumulative reward for the remainder of the episode Y= >>29 Rg. The reward of the Pendulum task is defined as Re -0? 0.1 0?_ 0.001 A3, which penalizes deviations from the upright position, large angular velocities, and large torque applied by the policy.
RL Agent Training. We train the policies for Sec. 5.2 with the Stable-Baselines [44] implementa- tion of Proximal Policy Optimization (PPO) [45]. For the Pendulum experiments, we use the default hyperparameters of Stable-Basse Baselines 3. The RL agent collects million environment transitions, and the total wall-clock training time is 50 minutes on Intel Xeon W-2245 CPU. The training requires less than 1 GB of memory.
Policy Data. For the analysis of the Pendulum policy, we collect dataset of Nint 100,000 interventions with Nop 100 episodes per intervention. Collecting the datasets of Policy and B in Sec. 5.2 takes about 150 CPU core-hours each on compute cluster, with 4GB of memory core.
We train nTCR model using the Adam optimizer with initial learning rate of 0.01 and a cosine annealing learning rate scheduler that reduces the learning rate to zero over the course of training. We batch size of 64 and train for 90 epochs with weight decay of 0.01. For the normality regularization, we set Thorm = 10.
TCR.
For interpretable nonlinear function class (Sec. 4.3), variable at each time step. Before training, determine the minimum and maximum values xf
128 Gaussian kemelss for each
22

---

Published as a conference paper at ICLR 2026
and niaxx for each variable across all episodes. The kernel centers (,g, (yy, 1 over the range [miinn Fmna) and the kernel widths are as:
equally spaced
2 min
jet =
(E.5)
128
where the constant == 8 acts smoothing parameter- higher values lead to smoother learned reduction functions.
For computational convenience, we pre-sample all episodes under interventions and store the data rather than simulating episodes during training, though the latter approach would also be feasible in principle.
Episodes with High and Low Predicted Reward. Fig. B displays episodes from the set with the highest and lowest predicted cumulative rewards according to the learned nTTR model for Policy A. These episodes correspond to the characteristic trajectory patterns identified in Fig. 3(left): clockwise swinging motions starting from the right quadrant tend to have higher rewards than episodes with counterclockwise motion originating from the left quadrant.
Compute Resources for nTCR. All nTCR training experiments for the Pendulum task were conducted on NVIDIA Quadro RTX 6000 GPUs with CPU cores allocated per run. Each complete training run required approximately 2 hours of computation time and used roughly 4 GB of GPU memory.
E.3 TABLE TENNIS
In the simulated table tennis task [22, 23), episodes terminate when the racket hits the ball, which typically happens between steps 90 and 100. The environment then continues simulating the ball to determine its landing position. If the agent does not hit the ball, the episode terminates
as the ball falls below the height the table, which typically happens after 125 135 steps. Since our method requires episodes of fixed length, pad the data T = 150 steps by appending zeros after episode termination.
The agent receives non-zero reward only at the end of the episode, which depends whether the agent managed to hit the ball. This reward is given by
R
racket touches the ball
R2
Rhit
ball below the table
(E.6)
otherwise,
where R rewards the agent for hitting the ball close to the desired position on the table and Rhit penalizes missing the ball depending on the minimum distance between the ball and the racket. These reward terms are defined
[p land
drs
Rd max
-022
(E.7)
3
Rhit = min
ball
racket
12,
(E.8)
where pandd is the landing point of the ball on the table, des is the desired landing point at the center of the opponent's side, and phall and packet are the 3D positions of ball and racket at timestep t, respectively.
RL Agent Training. Similar to the Pendulum task, we train the table tennis policies in Sec. 5.3 with the Stable-Baselines 3 [44] implementation of PPO [45]. We use the algorithm hyperparameters tuned by Guist et al. [23]. which are displayed in Tab. A. The RL agent collects 3 million environment transitions, and the total wall-clock training time is roughly 10 hours on an Intel Xeon W-2245 CPU. Running the simulation and the RL training requires GB of memory.
23

---

Published as a conference paper at ICLR 2026
2.16
1.06
poe
0.001
0.80
1.59
TR pus
0.78
1.88
T
0.001
1.29
0.54
T/401) piu.
0.001
2:000
0.000
-.00
0.000
-8.00
-0.001
-1.80
0.0055
-1.51
-0.001
-1.29
48 88
0.0005
18:35
80.35
0.0025
6.12
0:24
9:80
5.88
P.28
-7.38
-12.20
-241
-54 62
-0.0085-15
15.90
-0..25
-0.0005
-6.25
muusse
prosslepesss.
musole
0.000
0.89
0.89
0.99
0:001
0.001
0.001
30.40
0.000
0.49
0.000
0.49
0,000
0.40
0.000
70.55 25
0.25
.0001
0.25
.0.001
0.25
0.00
-0.002
0.00
0.00
0.00
mmuade
Emaccppeu.s 5
mmaach press.6
musil press.
0.74
0.001
0.74
0.002
0.74
0.0025
00:00
0.0000
0.49
0.000
0.49
0,000
0.60
0.0000
0.25
-0.0005
0.25
0.00
0.011
0.25
0.00
-0.002
-0.00
0.25
0000
bbll pcs.
T bell pou.
b buliposs
1.67
3.88
0.0005
4.70
7.12
0.76
1.72
2:024
000000
2:25
0.0000
-0.19
0.00
P.2.0
-0:0005
000026
-0.01
Thel vel
Thel vol
That
$1.75
4.79
001
12.05
7.40
2.15
5:04
0.00
2.41
0.000
-1.54
0.000
0.055
-0.025
7.50
-7.76
9.23
precs
prwce
1.00
1.00
1.00
0.02
1.00
0.055
0.80
0.50
0.50
$50.50 50
-1.00
-1.00
-0005
-1.00
0.50
-0.02
-0.50
-1.20
1.00
0.0085
1.00
1.00
0.025
2 8.00
0.000
000000
0.00
0.00
0.000
-.0.5
-1.00
-0:085
0.80
1.00
-00025
-1.00
0.50
.0.1
-1.00
00.00
-0 02025
Des. Muscla
Das Musda Press.
Das. Musdo Presa
Des Mazcle Prese 3
0.001
1:29
0.0005
0 0.0005
0.0006
3.000
0.0000
0.11
0.0000
0.05
0.000
0.15
0.0000
5.1.7
0.0005
1.48
-00005
1.25
-124
-2:70
-2.50
s Des Muccle Prese
Des Mudde Press 5
M Des Made Precx
Dec Musci Prece
0.0025
0.001
4
2.000
1.27
0.0000
0.16
1.25
0.000
0.01
1.34
0.000
0.06
1.40
0.0000
-1.32
-1.28
0,0025
-0501
100
60
100
Figure C: nTCR Reduction Maps for Table Tennis Task. Linear parameter in high-level model 0.0013 (bias 0.5964).
Policy Data. For the table tennis analysis, collecting the dataset of Nin: 100,000 interventions with Nep = 100 episodes per intervention takes about 3400 CPU core-hours, with 8GB of memory per core.
TCR. We use similar nTCR training settings as for the Pendulum task, with the following modifications: learning rate of 0.0001 and weight decay of 0.1 (manually tuned for optimization stability), 32 Gaussian kernels per variable-time pair (chosen to fit GPU memory constraints), and smoothing parameter = (smaller than Pendulum to provide necessary resolution and avoid oversmoothing with fewer kernels). Fig. shows the full set of nTCR maps for the table tennis task.
Compute Resources for nTCR. All nTCR training experiments for the table tennis task were conducted on NVIDIA A A100-SXM4-40GB GPUs with 8 CPU cores allocated per run. Each complete training run required approximately 5 hours of computation time and used around 20 GB of GPU memory.
24

---

Published as a conference paper at ICLR 2026
Table A:
3 PPO Hyperparameters used for Training the Table Tennis Policy.
Parameter name
Value
batch_sizz
310
clip_range
0.4
clip_range_vs
0.3
ent_coef
7 x 10
gae_lambdda
ganma
0.9999
learning_rate
0.00015
max max_graa__na norm
0.1
n_epochs
34
n_steps
5000
num_hiddee
380
num_layers
1
vf_coef
0.56
F ADDITIONAL EXPERIMENTAL RESULTS
EFFECT OF NORMALITY REGULARIZATION
To better understand the role of our normality regularization introduced in Sec. 4.1, we conduct an ablation study on synthetic data across different noise regimes. We vary the variance of the intervention distribution while keeping the exogenous noise levels in the low-level SCM constant, effectively creating different signal-to-noise ratios.
nTCR w/ Reg. (high noise)
nTCR w/ Reg. (med. noise) nTCR w/o Reg. (med. noise)
nTCR w/ Reg. (low naise) nTCR Reg. (low noise)
nTCR w/o Reg. (high noise)
Trsin Consistency
Tau identification
Omega Identification
0.3
10
Consistoncy
Loss
0.2
0.2
10-1
0.1
0.0
2000
4000
0
2000
4000
2000
4000
Training Steps
Training Steps
Training Steps
Figure D: Effect of Normality Regularization on Achieved Consistency and Identification of Ground-Truth Solution. The figure shows the consistency loss (left) and the identification losses measuring agreement with the ground-truth solutions (definition in App. E) for the and w-functions (middle and right) over the reduction training run for synthetic low-level SCMs with low-level variables. The interventions for the high-, medium- and www-exogenous-nood noise settings, respectively. Note that, since we keep the exogenous noise levels in the low-level SCM constant, low variance of the intervention distribution corresponds to high exogenous noise relative to the interventions. The lines for normality regularization have Thrrm 1; to turn off regularization we Mmorm 0. We show average values over 10 sampled SCMs.
sampled as (1) NN h 1), with
(0.1, 10.0}
Fig. shows several important insights about the effect of normality regularization across different noise regimes. First, the identifiability metrics demonstrate that learning ground truth reduction maps becomes progressively more challenging as we move from the low-noise setting (high intervention variance) the high-noise setting (low intervention variance). This reflects the fundamental signal- to-noise ratio: when interventions have small variance relative to the exogenous noise, the signal about the most influential factors for the target becomes harder detect and extract.
Most significantly, we observe that normality regularization has the strongest beneficial effect in the high-noise regime, where it substantially improves the identification of ground truth solutions for
25

---

Published as a conference paper at ICLR 2026
both and w maps. In contrast, for the low- and medium- moise settings, the regularization provides minimal additional benefit, suggesting that when the signal socullto-noiss ratio is favorable, the consistency loss alone is sufficient to guide the learning toward the correct solution. To understand why normality
100%
Cunulalive Probability
Empirical CDF
Gaussian CDF (1a=0.26, c=2.42)
60%
40%
20%
0%
Z
Figure E: Non-Gaussianity of the High-level Cause. The figure shows the empirical CDF (blue) for the high-level cause for SCM sampled in Fig. E (high noise setting) for an nTCR with Thorm The yellow dashed line shows the CDF for a Gaussian distribution with the same mean and variance.
regularization particularly crucial the high-noise regime, Fig. shows the empirical CDF of the learned high-level cause distribution in the high-noise setting without normality regularization. The distribution shows highly non-Gaussian behavior with multiple peaks, indicating that the т reduction is exploiting the flexibility of the nonlinear function class minimize the consistency loss by artificially shaping the high-level pushforward distribution. Such multi-modal distributions are undesirable for interpretability, as they introduce additional complexity that obscures the underlying causal structure (see also our discussion of Gaussianity in App. D).
These results demonstrate that normality regularization serves an important inductive bias that prevents overfitting to the training data while maintaining the interpretability of the learned high-level causes. The regularization ensures that interventions must be sufficiently strong relative to the exogenous noise to be effectively learned a principle that aligns with the fundamental requirement that causal interventions should produce detectable changes in the target phenomenon (see also Fig. A).
F.2
LINEAR TCR FOR PENDULUM POLICIES
Linear Weight
r Angular Velocity
Lineer Weight
Theta
Linear Weight Linear Weight
T Angular Velocity
Linear Weight
r Theta
0.00
-0.25
0.00
-0.25
Linear Weight
0.25
Torque
Linear Weight
0.2
Torque
T Torque
Linear Weight
0.0
Torque
0.00
0.00
0.0
-0.25
50
Time
Time
Policy A
(b) Policy B
Figure F: Linear TCR [10] for Pendulum Task. Linear parameters in high-level model: 5.6821 (bias -2852299 for Policy A and 94.0380 (bias -115.0152) for Policy B.
The reduction maps for linear TCR [10] policies and are shown in Figs. Fa and Fb, respectively. These provide baseline for comparison with our nonlinear approach (nTCR).
Policy A: Limitations of Linear TCR. For Policy A, linear TCR identifies predominantly negative weights in the second half of the episode for the 7-maps for Theta and Torque. While this correctly captures that deviations from the upright position (nonzero Theta) and large corrective actions (high Torque) late in episodes correlate with lower rewards, the linear approach fails to capture the
26

---

Published as a conference paper at ICLR 2026
more nuanced pattern revealed by nTCR. For the first half of the episode, the linear weights show inconsistent, unstable patterns without a clear interpretable structure. The r-map for Angular Velocity appears largely neutral except for unexplained positive peak at the final time step.
In contrast, nTCR (as shown in Fig. 3, left) clearly distinguishes between two distinct trajectory classes: clockwise swinging (from the right quadrant) versus counterclockwise swinging (from the left quadrant). This critical asymmetry in policy performance is entirely missed by the linear model, which can only capture monotonic relationships between state variables and expected reward.
Policy B: Observations Consistent with nTCR. For Policy B, linear TCR shows a strongly negative 7-map for Torque at the start of episodes and a relatively neutral pattern in the second half. This suggests episodes starting with positive torque tend to have worse outcomes. The w-map indicates shifting torque toward more negative values would be beneficial, which aligns with nTCR's s findings. These two observations are consistent with Policy B's failure to tip over in the positive 0 direction before applying corrective torque- can be captured by linear relationships. this case, both linear TCR and nTCR correctly identify the key intervention (applying more negative torque) that would improve policy performance.
the pendulum
Comparative Advantages of nTCR. Overall, we observe that linear TCR has fundamental limita- tions in capturing complex behavioral patterns in RL policies:
Non-monotonic relationships: Linear TCR cannot capture U-shaped or other non-monotonic relationships between states and expected rewards.
Trajectory classes: Linear TCR fails to distinguish between qualitatively different trajectory
classes (like clockwise vs. counterclockwise motion) that are not linearly separable.
These results empirically validate the need for nonlinear extension to TCR, demonstrating that nTCR can uncover important qualitative patterns in policy behavior that remain hidden to linear approaches.
F.3 LINEAR TCR FOR TABLE TENNIS POLICIES
The reduction maps for linear TCR [10] for the table tennis task are shown in Fig. G. While linear TCR provides a computationally simpler baseline, it reveals fundamental limitations when applied to complex robotic control tasks.
Temporal Importance Identification. Linear TCR successfully identifies that the most crucial periods for policy success or failure occur just before and during ball contact. This can be observed from the largest magnitude contributions in the linear reduction maps, which appear predominantly around the time window when the robot typically hits the ball (indicated by the dotted vertical lines). This temporal insight aligns with the physical intuition that precise timing and positioning during ball contact critical for successful returns.
Limited State Representation. However, the resolution of linear TCR is fundamentally constrained: it can only indicate whether a particular variable at a specific time contributes positively or negatively the expected reward average over the entire state and action distribution observed in the dataset. Linear reductions cannot distinguish between different values that a variable might take at any given time step. For instance, while nTCR can identify that balls bouncing closer to the net present greater challenges for the robot (Fig. 4(h,i)). it would be hard to deduce such value-dependent relationships from linear TCR. The linear approach averages over all ball positions at each time step, obscuring the specific spatial patterns that influence task difficulty.
Expressivity vs. Simplicity Trade-off. While linear TCR offers benefits in terms of training simplicity and straightforward interpretation of the learned reductions, its limited capacity to represent state-dependent relationships severely constrains its applicability to complex control scenarios. The table tennis task exemplifies this limitation: successful policy explanation requires understanding not just when certain variables matter, but also which specific values of those variables lead success or failure. This distinction between temporal importance and state-value dependencies shows the necessity of nonlinear extension for capturing meaningful causal patterns in more complex tasks.
27

---

Published as a conference paper at ICLR 2026
Tjoind pow.
T: pos.
Jann para
T 1o08 pos.
0.001
0.006
0.005
30,000
0.000
0.000
Y join vel
giit vol
vjciavel
Y pénnt vol
0.005
0.0025
0.000
0.0030
-0.03
0.005
0,0025
-0.005
musoc
muude pIoss.
muusol
0.005
musclo press. 3
50.01
0.000
P,:00
0.0000
0.000
rrseel
Trtuck
0,0025
0.00
0.0000
-0,0025
0.000
-0.01
0,0025
0.00
-0.0025
0.0000
0.01
0.000
vol.x
Vbllvvol V
0.025
0.00
0.000
-0.01
0.025
0.025
Teeae musdl prwar
des mudde powes.
T des muide prees.
muadl preex.
0.00
0.00
0,0025
-0.01
-0.01
Tee. muscle creas 4
des. madde presx.
des. muade prees.
deec musdle press.
0.005
0.005
0.005
-0.00
-0.01
0.005
Des. Midde Press
Des. Musele Press
Des. Muscle Press.
Des Muscle Press.
0.02
0.00
0.00
-0.01
Des.
Musele Fress.
Des. Press.
Muscle
0.01
0.01
0.000
0.00
0.01
10 0.025
50
100
150
90
100
150
50
100
150
50
100
150
Figure G: Linear Reduction Maps for Table Tennis Task. Linear parameter in high-level model 0.6933 (bias 0.6780).
F.4 COMPARISON OF CONSISTENCY LOss BETWEEN LINEAR AND NONLINEAR TCR
F.5 PENDULUM POLICY REDUCTIONS UNDER DIFFERENT INTERVENTION STRENGTHS
Fig. I shown our nTCR approach across different intervention strengths for Policy A of the Pendulum task presented in Sec. 5.2. We vary the standard deviation of the Gaussian perturbations applied to the policy's 's actions from very weak (a = 0.1) to very strong (a 3.0). The intervention strength in the main experiments in Fig. were chosen heuristically: the interventions were chosen be strong enough to induce a slight but measurable deterioration in the average reward.
For weak interventions (a 0.1), the reduction maps are hard learn, as the perturbations are insufficient to generate measurable variations in cumulative rewards. While there is resemblance to the patterns Fig. 3, the low signal-to-noiss ratio prevents reliable identification of the behavioral structure. As increase the intervention strength moderate levels (o 1.2 to 1.6), clear and consistent patterns emerge. The distinctive clockwise versus counterclockwise trajectory separation that characterizes Policy A's bias becomes clearly visible across this range. The w maps also remain qualitatively stable in this regime, with the magnitude of the map increasing with increasing range of observed interventions.
28

---

Published as a conference paper at ICLR 2026
10P
TCR
nTCR
0,407
0.364
Consistency
-0
0.098
0.095
Table Tennis
Pendulum Policy
Pendulum
Policy
Environment
Figure H: Consistency Loss Comparison. The figure shows the interventional consistency loss (2.5) achieved by linear TCR (blue) and nonlinear TCR (orange) validation data for the three policies shown in Sec. 5.
Angular
Theta
Torque
Torou
8:00
3.14
025
2.00
0.55
7.51
0,25
0 0.1
0.00
06
0.00
000
0.00
0.00
-0.07
0,000
1.57
4.00
3.14
4:25
-2.00
-7.45
-0023
3.14
2.00
7.51
O= 1.2
0.00
0.00
4.11
-0.1
4.00
1.57
1.00
0.25
3.79
0.1
0= 1.4
4.00
-157
+1.00
-3.05
0.1
3.72
0,029
1.6
-0.95
4.00
0.14
-2.00
-7.85
8:00
2.14
2.00
7.21
0= 3.0
0.00
0.00
0.00
0.00
0.0
-007
0.00
0.00
-3.14
2.00
7.89
1im
Time
Time
Figure Pendulum nCCR under increasing Intervention Strength. The figure shows nTCR maps for Policy A shown in Sec. 5.2 for different intervention strengths 1.4 is also shown in Fig. 3.
When interventions become too strong (a = 3.0), the learned patterns begin to diffuse and lose sharpness. This occurs because large perturbations push the policy far outside its normal regime, generating trajectories that no longer reflect the policy's intended behavior. Despite this degradation extreme values, our key findings about Policy A's rotational bias remain stable across practical range of intervention strengths (a E [1.2, 1.6)), demonstrating the robustness of our approach to this hyperparameter.
G ADDITIONAL RELATED WORK
While extensive theoretical work has established the foundations of causal abstractions [13-16, 19], practical applications of these frameworks have only recently begun to emerge. Notable applications include understanding the behavior of language models [17, 35] and neural networks more gener- ally [46), and developing causally-ifoormed approaches to multi-armed bandits [47]. These emerging
29

---

Published as a conference paper at ICLR 2026
applications demonstrate the growing practical relevance of causal abstraction theory across many domains.
Notions of abstraction, with the idea of reducing the dimensionality of system, have also been explored in RL. Temporal abstraction [48] reduces the time dimension, while state abstraction [49] reduces the set of states of a Markov Decision Process (MDP). Both types of RL abstraction share the characteristic of preserving an MDP-like structure by only exchanging fine-gres fineggrined states or actions for coarse grained-ones. Both approaches have been primarily used learn more efficient and robust RL policies by reducing the dimensionality of the problem, but have also been explored to provide explanations. A notable examples is [31] which learns graph between abstract states to explain how a policy works.
In contrast, TCR eliminates the MDP structure focus on explaining single target variable (a final/cumulative reward in RL) and few direct causes and interventions on them. The explanation provided is then of different nature, focusing on "why" a policy succeeds or fails (explaining outcomes) in relation to high-level description of the relevant aspects ("what"" of the system it acting on, instead of explaining its decisions and therefore "how" operates. Moreover, TCR is applicable any learned policies (it does not put constrained on the training) and allows for comparison between policies based on fundamentally different approaches.
Another type of explanation in RL are counterfactuals "what-if" scenarios. For instance, [29] builds interpretable parametrizations of sequential decision-making by explaining policy in terms of preferences over "what-if" outcomes. In [50] search algorithm for optimal counterfactual explanations is presented. The counterfactual explanation specifies an alternative sequence of actions differing in at most k actions from the observed sequence that could have led the observed process realization to a better outcome. In contrast to counterfactual approaches, and more generally to explanation of a single trajectory/data donit,, TCR provides coarse-graine gearse-grained overview of the key factors that alter the outcomes, average across all scenarios.
H
USE OF LARGE LANGUAGE MODELS (LLMs)
LLMs were used to polish the writing of the paper.
30


Reviews:
[{"review_id": "Ev7bxAccFf", "reviewer": "Reviewer_EFs4", "summary": "This paper presents Nonlinear Targeted Causal Reduction (nTCR), a framework for generating policy-level explanations (PLEs) for reinforcement learning (RL) agents through the lens of causal model reduction (CMR). Building on prior work in Targeted Causal Reduction (TCR) [Kekić et al., UAI 2024], the authors generalize it to nonlinear settings to better capture complex relationships in RL environments.\nThe core idea is to treat the RL system as a low-level structural causal model (SCM) over states, actions, and rewards, introduce random perturbations to actions as interventions, and learn a high-level, reduced causal model that remains approximately interventional-consistent with the original one.\n\nThe paper proves conditions for existence and uniqueness of exact nonlinear causal reductions under additive noise SCMs and introduces normality regularization via Wasserstein distance to maintain interpretability and stability. Experiments on synthetic data, the classic Pendulum control task, and a robotic table tennis simulation show that nTCR can uncover interpretable behavioral patterns and failure modes in RL policies—e.g., biases toward specific trajectories or torque directions.", "questions": "1. The paper focuses on continuous action spaces where \"shift interventions\" are naturally defined. You briefly mention extending the framework to discrete actions by perturbing the underlying policy parameters. Could you elaborate on the challenges this might pose? Specifically, how would you ensure the resulting interventions are both meaningful for causal discovery and don't simply push the policy into chaotic, out-of-distribution behavior, especially in high-dimensional state spaces like Atari where the input is an image? \n---\n\n2. You correctly identify that the high sample complexity of requiring many perturbed episodes makes nTCR primarily suited for simulated environments. Looking forward, what do you see as the most promising path to applying this powerful explanatory framework in real-world, non-simulated settings where extensive intervention is infeasible? For instance, could it be combined with a learned world model, or could a limited set of real-world interventions be used to fine-tune an explanation initially learned in a less-perfect simulator?\n\n---\n\n3. How sensitive are the learned causal reductions to the choice of intervention strength (σ) and the number of perturbed episodes? Appendix Figure A shows how σ was selected based on this trade-off. How robust are the final learned explanations (i.e., the τ and ω heatmaps) to this choice? For example, if you were to select a slightly different σ, would the identified causal patterns remain stable, or are the explanations highly sensitive to this hyperparameter?\n\n---\n\n4. The paper focuses on a single high-level cause. Could you elaborate on the challenges of extending the nTCR framework to discover multiple, distinct high-level causes? Would the normality regularization need to be adapted, for instance, to a multi-modal prior to prevent the causes from collapsing into (a single, less-interpretable composite cause) one?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. **Bridging causality and RL interpretability**:\nIt connects causal abstraction theory with reinforcement learning policy analysis—a growing but underexplored intersection—and provides a unified, formal framework to derive explanations that are causally meaningful rather than purely correlational.\n\n2. **Conceptually grounded contribution**:\nThe paper builds upon rigorous causal inference foundations (structural causal models, causal abstraction, targeted reductions) and extends them into nonlinear domains relevant for RL explainability.\n\n3. **Theoretical depth**:\nThe authors derive uniqueness and existence guarantees for exact transformations under nonlinear SCMs—addressing identifiability, a critical issue in causal representation learning.\n\n4. **Experiments**:\nThe combination of synthetic validation (matching theory) and applied tasks (Pendulum, robotic table tennis) is well executed. The qualitative analyses show clear, human-understandable insights into policy behaviors and biases.\n\n5. **Interpretable nonlinear design**:\nThe use of Gaussian-kernel–based function classes for τ (state/action to high-level cause) and ω (intervention mappings) is interesting and helpful, allowing visualization of how specific features and time steps influence outcomes. \n\n6. **Model-Agnosticism**: The approach treats the RL policy as a black box, which means it can be applied to explain any policy, regardless of its architecture, as long as it can be executed in a simulator.", "weaknesses": "1. **Dependence on interventional assumptions**:\nThe method relies on shift interventions on continuous actions and Gaussianity assumptions at the high level, limiting its applicability to discrete or stochastic policy settings. Future work directions mention this but downplay its importance.\n\n2a. **Interpretability trade-off**:\nAlthough Gaussian kernel maps improve interpretability, the framework still produces dense, high-dimensional weight maps that may be difficult to interpret without significant post-hoc visualization. Human usability of these explanations is not evaluated.\n\n2b. _Interpretability of ω Maps_: The paper provides excellent interpretation for the learned τ maps (how states/actions map to the high-level cause). However, the interpretation of the ω maps (how low-level interventions map to high-level ones) is less explored. For the Pendulum, it is simple (\"more negative torque is better\"), but for more complex actions, interpreting the learned ω map could be challenging.\n\n3. **Limited discussion on uncertainty or robustness**:\nThe causal reductions’ stability under noise or alternative intervention magnitudes is briefly shown in ablations but not rigorously analyzed.\n\n4. **Single High-Level Cause**: The current formulation focuses on explaining a target variable via a single learned high-level cause Z. While powerful, many complex policy failures might result from the interaction of multiple distinct factors. The paper mentions this as a possible extension but does not elaborate on the challenges, such as ensuring the learned causes are genuinely distinct (disentangled).\n\n---\n\n**Minor Typos**\n- Ln 126-127: \"whih\" $\\rightarrow$ \"which\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4KCPWFSZNd", "reviewer": "Reviewer_pKWp", "summary": "This paper studies the explainability of RL policies at the policy level. \nIt proposes a nonlinear Causal Model Reduction framework that abstracts low-level environment dynamics into a high-level causal model, guided by intervention consistency. \nExperiments on synthetic causal systems demonstrate the utility of the approach, and results on a complex control task validate its practical application.", "questions": "Can you explain Eq. 2.4 more? What do you mean by Eq. 2.4? If I understand correctly, \\pi_0, \\pi_1 are subsets of the actual state variables. Is it correct?\n\nDo the learned state variables retain their significance across time—i.e., if a variable is important at step t, is it still important at t+1 or t+n?  In stationary MDPs, a common assumption is: the state dimension should be time-invariant [1], and the long-term return is additive[2]. Additionally, there is also a discussion about what kind of state variables could be significant for policy learning [3]. How do these conclusions influence the high-level abstracted causal model learning? \n\n[1] Huang, B., Lu, C., Leqi, L., Hernández-Lobato, J. M., Glymour, C., Schölkopf, B., & Zhang, K. (2022, June). Action-sufficient state representation learning for control with structural constraints. In International Conference on Machine Learning (pp. 9260-9279). PMLR.\n\n[2] Zhang, Y., Du, Y., Huang, B., Wang, Z., Wang, J., Fang, M., & Pechenizkiy, M. (2023). Interpretable reward redistribution in reinforcement learning: A causal approach. Advances in Neural Information Processing Systems, 36, 20208-20229.\n\n[3] Liu, Y., Huang, B., Zhu, Z., Tian, H., Gong, M., Yu, Y., & Zhang, K. (2023). Learning world models with identifiable factorization. Advances in Neural Information Processing Systems, 36, 31831-31864.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The problem addressed is highly important for RL. Framing it from the perspective of policy-level decision-making explainability is both interesting and technically challenging, as it requires linking internal policy mechanisms to long-term performance. Advancing this line of work could meaningfully improve the interpretability, debugging, and reliability of RL systems in complex settings.\n\nThe paper is well written and easy to follow, and the theoretical part appears strong.\n\nThe core idea is clear: use an intervention-consistency high-level model to represent how the long-term returns are influenced.", "weaknesses": "The application domain of the method is not fully clear. Please articulate the types of scenarios where it is intended to be used and detail the constraints introduced by the surjectivity assumption. As a concrete edge case: if the long-term (episodic) return is binary—i.e., a single scalar only at the final step—does the method still apply, and under what conditions could the method be applied? \n\n\nAdditionally, for readability, it would be helpful to include a worked example in the introduction to demonstrate how the learned high-level model explains the impact of a specific action on long-term returns.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qWyrsR8iwT", "reviewer": "Reviewer_heb9", "summary": "The paper presents an extension of the causal model reduction framework to better address the problem of explaining RL policies. Previous work in this area only used linear models. The authors extended this by introducing a class of non-linear reduction functions that remain interpretable. This can better capture behaviors in the learned reductions and help explain policies. The paper provides theoretical guarantees of the correctness of their approach, along with an experimental evaluation that demonstrates that the approach uncovers patterns leading to target behaviors in two RL tasks.", "questions": "At this stage, my initial recommendation is to reject the paper, although this is not a strong or definitive rejection. My main concern with the submission is the evaluation, which I find insufficient to convincingly support the paper's claims. There is no sufficient evaluation comparing previous approaches and the suggested one. The only comparison available is on synthetic data with specific parameters, which provides limited evidence for the paper's contribution. Also, the writing is somewhat dense in several sections of the paper, particularly in sections 4 and 5, but this part can be improved with a small revision.\n\nDetailed questions and comments to the authors:\n\n- Is the use of linear targeted causal reduction for explaining RL agents discussed in previous work? This part of the paper should be clearer.\n\n- In line 261, what is $\\eta_{norm}$? I could not find an explanation in the text.\n\n- Could you provide some intuition about why the assumptions made in Propositions 4.1 and 4.2 are reasonable within the task you aim to address? I did not find a discussion about this in the paper.\n\n- It would be great if the authors could provide more details about the choice of function described in Section 4.3. Specifically, are these functions used in previous work for similar cases? Can they capture linear behavior patterns? How are the fixed parameters set?\n\n- Section 5.1: Could you comment on the choice of configuration you made for this validation? Most importantly, setting the h function to zero. I like the idea of using synthetic data for validation, but the evaluation seems limited to very specific parameters, which narrows the authors' claim about validation. This is especially unusual, given that it is synthetic data, which can be relatively easy to generate and experiment with.\n\n- sections 5.2-5.3: The purpose of this evaluation is somewhat unclear. I was expecting a comparison between TCR and nTCR, focusing on accuracy (I guess sampled from the environment since we lack ground truth) and the computational overhead each one requires. It would help if the authors could clarify what they aimed to evaluate.\n\n- Although the pendulum and table tennis are interesting and well known use cases, they contain a low number of variables (especially the pendulum). This makes me think, can you comment on the limitations of the current reduction approaches? Is a reduction necessary in cases like explaining a pendulum agent?\n\nAdditional feedback:\n\nI found sections 4 and 5 to be a bit dense. It involves a lot of technical details without enough explanation or context. I recommend adding explanations or interpretations of the main points for the terms and choices discussed in Section 4, as well as the details of the experiments in Section 5.You can allocate space for this by shortening the beginning of Section 2 or using the extra page provided if the paper is accepted.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The problem is well motivated and relevant.\n- The theoretical contribution seems rigorous.", "weaknesses": "- The writing in the paper is quite dense in several areas, specifically in sections 4 and 5.\n- Lack of comparison with similar approaches for causal reduction.\n- Evaluation is limited to specific use cases/configurations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)