Title: Reward-oriented Causal Representation Learning
Venue: NeurIPS 2025
Accepted: yes
Keywords: causal representation learning

causal bandit

reward-oriented
Abstract: Causal representation learning (CRL) is the process of disentangling the *latent* low-dimensional causally-related generating factors underlying high-dimensional observable data. Extensive recent studies have characterized CRL identifiability and *perfect* recovery of the latent variables and their attendant causal graph. This paper introduces the notion of *reward-oriented* CRL, the purpose of which is to move away from perfectly learning the latent representation and instead learning it to the extent needed for optimizing a desired downstream task (reward). In reward-oriented CRL, perfectly learning the latent representation can be excessive; instead, it must be learned at the *coarsest* level sufficient for optimizing the desired task. Reward-oriented CRL is formalized as the optimization of a desired function of the observable data over the space of all possible interventions and focuses on linear causal and transformation models.  To sequentially identify the optimal subset of interventions, an adaptive exploration algorithm is designed that learns the latent causal graph and the variables needed to identify the best intervention. It is shown that for an $n$-dimensional latent space and a $d$-dimensional observation space, over a horizon $T$ the algorithm's regret scales as $\tilde O(d^{\frac{1}{3}}n^{\frac{1}{3}}u^{\frac{2}{3}}T^{\frac{2}{3}} + u\sqrt{T})$, where $u$ measures total uncertainty in the graph estimates. Furthermore, an almost-matching lower bound is shown to scale as $\Omega(d^{\frac{1}{3}}n^{\frac{1}{3}}p^{\frac{2}{3}}T^{\frac{2}{3}} + p\sqrt{T})$, in which $u$ is replaced by $p$ that counts the number of causal paths in the graph.

Reward-oriented Causal Representation Learning

Abstract

Causal representation learning (CRL) is the process of disentangling the latent low-dimensional causally-related generating factors underlying high-dimensional observable data. Extensive recent studies have characterized CRL identifiability and perfect recovery of the latent variables and their attendant causal graph. This paper introduces the notion of reward-oriented CRL, the purpose of which is to move away from perfectly learning the latent representation and instead learning it to the extent needed for optimizing a desired downstream task (reward). In reward-oriented CRL, perfectly learning the latent representation can be excessive; instead, it must be learned at the coarsest level sufficient for optimizing the desired task. Reward-oriented CRL is formalized as the optimization of a desired function of the observable data over the space of all possible interventions and focuses on linear causal and transformation models. To sequentially identify the optimal subset of interventions, an adaptive exploration algorithm is designed that learns the latent causal graph and the variables needed to identify the best intervention. It is shown that for an n -dimensional latent space and a d -dimensional observation space, over a horizon T the algorithm's regret scales as ˜ O ( d 1 3 n 1 3 u 2 3 T 2 3 + u √ T ) , where u measures total uncertainty in the graph estimates. Furthermore, an almostmatching lower bound is shown to scale as Ω( d 1 3 n 1 3 p 2 3 T 2 3 + p √ T ) , in which u is replaced by p that counts the number of causal paths in the graph.

1 Introduction

Consider a data-generating process in which latent low-dimensional causally-related variables are mapped to observational high-dimensional data through an unknown transformation. Causal representation learning (CRL) is the process of using observational data to learn the latent, unobserved generating factors, i.e., the latent variables and the latent causal graph that specifies their causal interactions. CRL is considered a significant step toward understanding the world by learning appropriate representations that support causal interventions, reasoning, and planning [1].

CRL literature. There exists rich recent literature on CRL identifiability analysis - the objective of which is establishing conditions under which the latent space can be recovered uniquely - across various models for the latent causal model (e.g., linear, parametric, and non-parametric) and the unknown transformation (e.g., linear, parametric, and non-parametric). Some representative studies include [2-13]. Aiming to establish possibility/impossibility results, the existing literature is primarily focused on the asymptotic setting of access to an infinite number of observations, with limited studies on finite-sample guarantees [4]. Specifically, we refer to [2-5] for the most closely related works in terms of setting and methodology.

Learning a unique, causal representation underlying the observed data is hypothesized to enable more improved and robust reasoning for downstream tasks [1]. This is often encountered in technological, social, and biological domains where the observed data often lack straightforward interpretations and are generated by an unobserved data-generating mechanism with interpretable semantics.

Reward-oriented CRL. To understand the recovery limits in CRL, the existing literature has so far decoupled the CRL from the downstream objectives. This decoupling means that CRL may expend extra effort learning fine-grained details that do not contribute to the downstream objective, or conversely, lack sufficient accuracy to be useful. To address this gap, we introduce the notion of reward-oriented CRL , which directly integrates the downstream goal into the representation learning pipeline. In this paper, we consider rewards that are functions of the latent variables. Since the latent variables are not directly observable, efficient optimization of the reward may include, as a sub-task, recovering these latent representations. In this case, one would only need to learn the latent variables and graph to the coarsest level to optimize the downstream objectives. To formalize the objectives, we define a utility function that maps the latent causal system to a downstream utility. This utility will be subsequently optimized over the space of possible interventions in the causal system. For example, consider a robotic arm with causally related joint variables. The arm's movements are monitored from camera images, and our downstream objective is to use these images to optimize the arm's movements for a specific task. In such a task, achieving high placement accuracy does not require perfect recovery of all latent variables; instead, it suffices to capture the critical joint relationships or to bound estimation errors below a level that leaves final task performance unaffected. Sending commands to the arm to adjust its actions is the intervention model on the underlying causal system.

Connection to causal bandits. Optimizing utility in the latent space faces uncertainties in several ways, e.g., causal model, transformation, and probability models. Inevitably, an algorithm optimizing utility needs to explore the system to resolve these uncertainties before committing to a decision. A data-adaptive exploration of interventions is intimately related to the literature on causal bandit (CB), albeit with two significant differences. First, in CB, the learner directly interacts with the causal system and observes the causal variables, a premise that does not hold in reward-oriented CRL. Second, in the CB literature, it is often assumed that the causal graph's topology is known, which is not the case in reward-oriented CRL. Some representative studies on various causal models (e.g., linear and non-linear) and intervention models ( do , stochastic hard, and stochastic soft) include [14-26].

Due to these two significant differences with CB, designing reward-oriented algorithms will differ significantly from the CB algorithms by including a process that can perform CRL. This process has to perform an accurate-enough CRL that facilitates identifying the best intervention. Hence, the CRL process we need to design differs from the existing ones that aim for perfect latent recovery.

We focus on reward-oriented CRL with (i) linear structural equation models over an n -dimensional latent causal system, (ii) linear transformations mapping to d -dimensional observations, and (iii) linear utility functions. Table 1 summarizes the main guarantees on latent space recovery and regret bounds until time T , where u measures total uncertainty and p counts the number of causal paths. We defer the discussion of do intervention to Appendix I. Our key observations include the following.

Finite-sample identifiability. We provide finite-sample identifiability for linear SEMs, accommodating both stochastic soft and hard interventions. This is a critical step, since during exploration, we have access to only a finite number of samples.

Almost matching regret bounds. We establish upper and lower regret bounds for reward-oriented CRL. These bounds are specified in terms of the topology of the causal graph. Under soft interventions, these bounds match in their dependence on graph parameters and the time horizon T .

Refined bounds for causal bandits. The reward-oriented CRL framework subsumes CB by setting the latent to observation transformation to the identity function. We show that our algorithm also improves the state-of-the-art regret bounds for the relevant CB settings.

Notations. For n ∈ N , we define [ n ] ≜ { 1 , . . . , n } . Vectors are represented by lowercase bold letters, and element i of vector v is denoted by v [ i ] . Matrices are represented by uppercase bold letters, and we denote row i and element ( i, j ) of matrix A by [ A ] i and by A [ i, j ] , respectively. Moore-Penrose pseudoinverse of a matrix A is denoted by A † . For any matrix A we denote the rank, column and null spaces of A by rank ( A ) , col ( A ) and null ( A ) , respectively. 1 denotes the indicator function. Sets and events are denoted by calligraphic letters. The cardinality of the set A is denoted by |A| . For a vector x and positive semidefinite matrix A , we define ∥ x ∥ A = √ x ⊤ Ax as the weighted ℓ 2 norm. The ℓ 2 -norms of vector x ∈ R d and matrix A are denoted by ∥ x ∥ 2 and ∥ A ∥ 2 , respectively. Finally, ˜ O is an order notation that ignores constant and poly-logarithmic factors.

2 Reward-oriented CRL Framework

Data-generating process. Consider a data-generating process that transforms high-level, lowdimensional latent variables into low-level, high-dimensional observable data. Formally, consider a causal system represented by an unknown directed acyclic graph (DAG) G with n nodes generating causally-related latent random variables Z ≜ [ Z [1] , . . . , Z [ n ]] ⊤ . These latent variables are mapped to a higher-dimensional observed data X ∈ R d by an unknown linear transformation G ∈ R d × n according to X = G · Z , where d ≥ n and G is full column rank. The set of parents and ancestors of node i ∈ [ n ] are denoted by pa ( i ) and an ( i ) , respectively. We denote the probability density function (pdf) of Z by p and denote the conditional pdf of Z [ i ] given its parent variables by p i ( z [ i ] | z [ pa ( i )]) . Wecall a permutation π = ( π 1 , . . . , π n ) of [ n ] a valid causal order if for all i, j ∈ [ n ] , the membership i ∈ pa ( j ) implies π i < π j . We denote the maximum in-degree of G by d G and the length of its longest causal path by L . The latent causal variables are assumed to be related through a linear structural equation model (SEM) specified by Z = B · Z + ε , where B ∈ R n × n is the edge weight matrix and ε ∈ R n accounts for the exogenous noise, whose expected value is denoted by ν ≜ E [ ε ] . The weight matrix B directly models the causal relations in the sense that the element B [ i, j ] is non-zero if and only if j ∈ pa ( i ) .

Interventions. We consider two types of intervention mechanisms: stochastic hard and soft , the distinction of which is how they impact the marginal distributions of the latent variables.

Soft intervention: As the least restrictive form of intervention, when applied to node i ∈ [ n ] , a soft intervention changes the conditional distribution p i ( z [ i ] | z [ pa ( i )]) to a distinct one. Equivalently, this alters the observational linear causal mechanism to an alternative one as follows:

Hard intervention: As a special case of soft interventions, a hard intervention on node i removes its ancestral statistical dependence and changes p i ( z [ i ] | z [ pa ( i )]) to a marginal distribution that is only a function of z [ i ] . This mechanism is equivalent to setting B ∗ = 0 .

where [ B ∗ ] i denotes the vector of post-intervention edge weights, and ε ∗ [ i ] denotes the postintervention noise with mean ν ∗ [ i ] ≜ E [ ε ∗ [ i ]] . Consequently, we define the interventional weight matrix B ∗ as the composition of rows { [ B ∗ ] i : i ∈ [ n ] } and the mean vector as ν ∗ .

For simplicity of notations, throughout the analysis, we assume only one intervention mechanism per node. However, this can be readily generalized to an arbitrary number as discussed in Appendix H. We allow multiple nodes to be intervened on simultaneously and denote the space of possible interventions by A ≜ 2 [ n ] . We denote the probability measure induced by the set of interventions a ∈ A and the associated expectation by P a and E a , respectively.

Reward-oriented CRL. In reward-oriented CRL, the objective is to identify the set of interventions in A that maximizes an expected reward defined as a function of Z . In this paper, we focus on linear reward functions specified by U ( Z ) ≜ θ ⊤ Z + ε U , where θ ∈ R n is an unknown reward parameter and ε U represents a utility noise term with mean 0 . We denote the expected value of the utility U under intervention a ∈ A by

A learner's objective is to identify the optimal intervention a ∗ , denoted by

All aspects of probability distributions, i.e., the topology of G , pre- and post-intervention matrices, and noise distributions, are unknown. To identify a ∗ , the learner performs a sequence of interventions

and receives feedback consisting of the observed data X and reward U ( Z ) . The objective is to identify a ∗ with the fewest number of interventions, which are selected adaptively based on the data and the collected rewards. The sequence of interventions over time is denoted by { a t ∈ A : t ∈ N } . Accordingly, we denote the set of the latent variables, data, and reward collected up to t by

To assess the efficiency of the learner in identifying a ∗ , we define the cumulative utility regret that it incurs relative to an oracle with access to the best intervention a ∗ as

We remark that using such a regret-based approach to identify a ∗ can be naturally posed as a multiarmed bandit problem in which each of the 2 n possible interventions can be represented by one arm. Applying a vanilla bandit algorithm results in a regret that scales exponentially in n , rendering a regret that, even for a moderate-sized latent structure, can be highly inefficient. The objective in this paper is to design algorithms that can break the exponential dependence on n by properly leveraging the intricate causal structures among the latent variables.

Identifiability metrics. Circumventing the exponential dependence of the regret on dimension n , and efficiently identifying a ∗ hinge on properly recovering the latent variables Z and the underlying causal graph G , both unobserved. These recoveries are the core objective of CRL. Recent studies provide extensive identifiability guarantees for CRL - which specify conditions under which Z and G can be uniquely recovered (up to some uncertainty) from the observed data X . These guarantees are asymptotic , assuming access to an infinite number of samples of X . In the reward-oriented CRL, however, the decisions at each time t must be based solely on t samples. To formalize the types of identifiability guarantees needed in our framework, we first specify the known infinite-sample identifiability guarantees that apply to the setting of this paper (linear SEMs and transformation), followed by their finite-sample counterparts. For this purpose, given X t , we define ˆ G t , ˆ Z t , and H t as estimates of G , Z t , and G † , respectively. We also denote the transitive closure of G by G tc .

Theorem 1. Under linear SEM, linear transformation, and one intervention per node, CRL is endowed with the following infinite-sample identifiability guarantees.

(i) Hard intervention ( [2, Theorem 2] ): It is possible to perfectly recover G and recover Z up to scaling, i.e., ˆ G ∞ = G and ˆ Z ∞ ≜ H ∞ X ∞ = C H Z ∞ for some diagonal matrix C H ∈ R n × n .

(ii) Soft intervention: It is possible to perfectly recover a transitive closure of G [2, Theorem 1] and recover Z up to scaling and mixing with parents, i.e., ˆ G ∞ = G tc and ˆ Z ∞ = H ∞ X ∞ = C S Z ∞ for some matrix C S ∈ R n × n such that for any given j , C S [ i, j ] = 0 for all i / ∈ { j } ∪ pa ( j ) .

See Appendix E.1 for details. Next, we specify finite-sample counterparts of these identifiability statements in a probably approximately correct (PAC) sense similar to [4].

Definition 1 ( ( ϵ, δ ) -PAC recovery) . For a given t ∈ N , the finite-sample estimates ˆ G t , ˆ Z t , and H t are said to achieve ( ϵ, δ ) -PAC recovery if the following statements hold with probability at least 1 -δ

(ii) Soft intervention: ˆ G t = G tc and ˆ Z t = H t X t = ( C S + E t ) · Z t , where C S is a full rank matrix, for any given j , C [ i, j ] = 0 a for all i / ∈ { j } ∪ pa ( j ) , and ∥ E t ∥ 2 ≤ ϵ .

(i) Hard intervention: ˆ G t = G and ˆ Z t = H t X t = ( C H + E t ) · Z t , where C H is a full-rank diagonal matrix, and we have ∥ E t ∥ 2 ≤ ϵ .

Assumptions. Next, we outline the assumptions adopted for the reward-oriented CRL framework. First, we adopt the following CRL assumption (see, e.g., [2, Assumption 1(b)] and [3, Assumption 1]), which ensures that the effects of an intervention can always be traced from the changes in the precision matrix (inverse covariance) of latent variables Z .

Assumption 1. An intervention on any non-root node i changes row i of the weight matrix, i.e.,

̸

̸

This assumption automatically holds for hard and do interventions and is mild for soft interventions, requiring that the effect of an intervention is not limited to the exogenous noise distributions. Similar assumptions are common in the CRL literature and can effectively be interpreted as requiring that the statistics we use be non-degenerate. Next, we provide SEM-related assumptions that are standard in the causal bandit literature [21].

Assumption 2 (Weight matrices) . Matrices B and B ∗ and the utility parameter θ are unknown but have finite entries with known ranges. We denote the range of these entries by m B ∈ R + , i.e., | B [ i, j ] | ≤ m B , | B ∗ [ i, j ] | ≤ m B , and | θ [ i ] | ≤ m B for all i, j ∈ [ n ] .

Assumption 3 (Noise model) . We assume that the statistical models for the noise ν and ν ∗ are unknown and bounded. We define the m ε ∈ R + to specify the range of noise terms, i.e., | ε t [ i ] | ≤ m ε for all i ∈ [ n ] and t ∈ [ T ] . Finally, we assume the utility noise ε U is 1 -sub-Gaussian.

For simplicity in the presentation and without loss of generality, we set m B = m ε = 1 . An immediate conclusion of Assumptions 2 and 3 is that Z is bounded, i.e., ∥ Z ∥ ≤ m for some m ∈ R + .

H

t

latent variable

Z

t

latent estimates

ˆ

Z

t

observation

X

t

{

a

s

:

s

∈

[

t

]

}

a

t

+1

weight matrices

B

,

B

∗

DAG

G

DAG estimate

ˆ

G

t

weight matrices estimates

A

t

,

A

∗

t

transformation

G

estimate of

G

†

CRL:

precision matrices

CB:

UCB-based selection

3 Reward-oriented CRL Algorithm

Now we introduce the R ewardo riented C ausal R epresentation L earning algorithm (RO-CRL) for hard and soft intervention. The pseudocode is provided in Algorithm 1 in Appendix A.

3.1 Algorithmic Overview & Key Properties

Identifying the best intervention a ∗ defined in (3) requires estimating the expected utility under 2 n distinct statistical models associated with 2 n possible intervention combinations a ∈ A . The key to breaking the exponential dependence of the number of estimation routines we have to perform on n is leveraging the intricate connection among all the 2 n statistical models. Specifically, all the models inherit their randomness from two sources of noise models, ν and ν ∗ . The algebraic forms of the statistical models are shaped by the weight matrices B and B ∗ and the utility parameters θ . Hence, learning 2 n distributions can be reduced to estimating two noise models and estimating the entries of B , B ∗ , θ . Estimating these parameters, however, depends on access to the causal graph G (informative about the sparsity structures of B and B ∗ ) and the latent variables Z t , both of which are not directly observable.

To address this, RO-CRL properly explores various intervention combinations to form reliable enough estimates of G and Z t from observations X t . Achieving the right level of exploration is critical to avoid collecting excessive data X t , which will compromise the regret. Attaining the right exploration, therefore, depends on determining what the coarsest possible estimates of G and Z t are that ensure reliable identification of the optimal intervention a ∗ . Hence, RO-CRL balances a trade-off that involves an adaptive exploration schedule that focuses on recovering G and Z t to the extent needed to find a ∗ . At each time t , RO-CRL constructs estimates for the graph ˆ G t and inverse transform H t using differences between sample precision matrices with samples from the observational and interventional data. Since CRL only recovers variables up to scaling factors, we compute robust estimates of appropriately scaled B and B ∗ . After sufficient exploration of interventions, they are subsequently selected according to the upper confidence bound (UCB) principle. The overall pipeline of this process is illustrated in Figure 1.

Precision matrix differences. The main statistic we use to estimate the latent variables and graph is the differences in precision matrices under various interventions. Denote the precision matrices of Z and X under distribution P a by Θ Z a and Θ a , respectively. Accordingly, define R Z i and R i as the associated precision differences between observational ( a = ∅ ) and single-node interventional ( a = { i } ) data, i.e., R Z i ≜ Θ Z { i } -Θ Z ∅ and R i ≜ Θ { i } -Θ ∅ . To specify the empirical counterparts of these precision differences, we denote the number of times that a ∈ A is selected until time t by

<latexi sh

1_b

64="G0

wQYvz

JNOqm

k

n

FB

u

>A

CUH

c

V

T9

E

2

L

3

P

+

p

oj dy

8

W

X

/S

75r

Z

D

R

g

M

f

I

K

<latexi sh

1_b

64="I

dZ

mqR3

f8

D

nQ7F

X

>A

CY

c

VH

Tg

E

r

G+

jW

N

w/

BJ

0y

Sz

O

o

P

k

U

v

K

5

2

p

9

M

<latexi sh

1_b

64="orzAmXk

D5+O9

M0

Tn

p

cH

>

CW

V

B

N

EF

K

Ij

Z

2

R

d

y

/

S

8

G

LY

f

3

q

w

g

Ju

Q

7

U

<latexi sh

1_b

64="5OASj

G

BR

/y pE

F

c

>

C

H

VDLTgI

3

un

m

0Q

X

d

U9

q

7

w

2M

v

kY

r

Z

Wz

N

+

K

J

8

<latexi sh

1_b

64="Mj7uE

H

095/

UL

Iv

WN

zo

>A

CVX

c

2

3p

q

+

B

d

w

GJ

K

f

r

F

8

Dmn

P

O

T

yR

Y

Z

k

Q

<latexi sh

1_b

64="

o

0Hf3SNrvRq

7ImLn2Bwy

G

>A

C

X

c

VD

Tg

FO

8Q

K

E

J

u

5

UY

d

k

/

j

W

9

Z

+

M

z

P

p

<latexi sh

1_b

64="VmM7Zo

q

pD

QGn gTkO/3I

>A

C

c

HL

F

2

8

X

NK

j

E

R

0

B

zWr dY

J

f

v

S

y

U

+

5

u

9

P

w

<latexi sh

1_b

64="AE

oVy

+9USnYuO

0

5qNQ

>

C

X

c

FL

wM

J

d3/

H

vBg

T

8

R7

G

r

k

K

mj

D

Z

z

P

p

W

2

f

I

<latexi sh

1_b

64="c8WoF

gz2

H

k

Sq

5/

CD

>A

X

V

NTwI

EO

u

Qd jG

R

M

P

3

K

nZ

0

p

f

r

7

y

B

+

L

Y

J

m

v

U

9

<latexi sh

1_b

64="9oSHyKcWn wB

g3

dX

0

A

>

FNT

I

EO2uq

f

5

MP

/

j

Rz

k

+

v

Y

L

Z

r

Gm

8

Qp

U

D

J

7

<latexi sh

1_b

64="q

8I9S

yYFr

2f uKXkMNV

ncR

>A

CU

7T

w

H

5Q

B

OWL

D

PE

g

+p

J

d

G

Z

z

m

3

v

o

0

j

/

<latexi sh

1_b

64="

zA

KWFvMG9o/3

7

SY

g

>

CVn

c

N

w

E

2u

r

H

X

I

U

j

BZL

k

Pfp

O

0T8

q

y

R

+d

5

J

Qm

D

<latexi sh

1_b

64="T/7d vnz

gfc

Gp

m

YwXrFE

>A

CQ

VDL

I

3

j

MK

okuJ

S

BO

U

N

8

ZH

+

5

9y

q

P

W

2

R

0

<latexi sh

1_b

64="UO37yNm/VzQ

HTP

f

Rg

>A

C

X

c

J

wEB2p

d

0

S

I

k

W

o

Z

KY

+j

9

v

8

M

n

L

5

r

F

u

q

<latexi sh

1_b

64="M7qcAdy3g jWQU

G

RDr

Y

>

C

X

VFLS8N

E

2v

k

P

B

5

9

K

wn

T

m

+

u

/

z

f

p

0

H

o

ZJ

I

O

Hence, the empirical sample mean and covariance of X for a ∈ A are given by

Accordingly, the empirical precision and precision differences are denoted by Θ a ,t ≜ (Σ a ,t ) † and R i,t ≜ Θ { i } ,t -Θ ∅ ,t , respectively. We use the following properties of R i,t to estimate G and Z .

Lemma 1. The non-zero rows of latent precision difference R Z i describe the latent graph:

̸

Furthermore, the observable precision difference R i = ( G † ) ⊤ R Z i G † is subspace-constrained:

3.2 Key Processes in RO-CRL

RO-CRL consists of two main stages. It begins with a forced exploration phase , in which we apply node-level atomic interventions on all nodes a fixed number of times. The purpose of forced exploration is to establish initial estimates for the relevant statistics. This is followed by an adaptive exploration stage , in which we dynamically select a sequence of interventions adaptively to the observations and utilities. This stage itself consists of several inference processes, whose collective objective is to specify a rule for translating observations into an intervention selection.

Stage 1 - Forced exploration. To construct initial estimates of R i , the algorithm explores the observational model along with each of the single-node atomic intervention models for T 0 times. To formalize this, the set of interventions in this forced exploration phase is denoted by A 0 ≜ {∅ , { 1 } , { 2 } , . . . , { n }} . The constant T 0 is chosen such that the initial R i are sufficiently accurate to produce a reliable enough graph estimate for the latent causal graph G .

Stage 2 - Adaptive exploration. After forced exploration, at every t ≥ ( N +1) T 0 , we perform three inference and decision procedures to identify the next intervention a t +1 .

CRL: First, we recover the latent variables and graph. This is initiated by first finding the estimate H t for the inverse of G , followed by using that to find estimates ˆ G t .

UCB-based intervention selection: Subsequently, we introduce a UCB-based decision rule that leverages the above estimates in conjunction with the reward U t to specify a t +1 .

Under-sampling rule: Finally, we implement a process that ensures that single-node interventions are explored sufficiently over time. When an intervention is deemed under-sampled, it will be prioritized for sampling over the intervention. This process ensures that the estimates H t and ˆ G t are updated incrementally, preventing total estimation error in Z t from growing linearly with t .

3.3 Inference & Decision Rules

The quality of the inference and decision rules in Stage 2 is critical for identifying the sequence of interventions. In this subsection, we specify these rules.

1 - CRL rules. The CRL rules are implemented in two stages: baseline recovery and refined recovery . The baseline recovery process serves two key roles: (i) it estimates the causal graph G and latent variables Z t under soft interventions, and (ii) it acts as an intermediate step for estimating G and Z t under hard interventions. The refined recovery stage is applied only when hard interventions are present, with the specific goal of improving estimates produced by the baseline process. As expected, hard interventions yield stronger recovery guarantees, with their advantage stemming from this additional refinement step.

1.a - Baseline recovery steps. The baseline recovery of the latent space is based on the shared properties of soft and hard interventions, such as Lemma 1, and it has three inference routines.

(i) Inverse transform H t . We construct the baseline estimate H t ∈ R n × d row by row by assigning [ H ] the principal eigenvector of R , (11)

where principal eigenvector denotes the eigenvector associated with the largest absolute eigenvalue of a symmetric matrix. The intuition here is that, due to Lemma 1, with high probability, the following property holds with a low error level.

(ii) Variables ˆ Z t . The estimated latent variables are subsequently estimated via

and are approximately equal to the true latent variables Z t up to mixing with parent variables.

(iii) Graph ˆ G . To form a graph estimate, we first compute the estimated latent precision differences

and then assign the non-zero rows of these matrices as edges according to:

̸

Next, we find the transitive closure of ˆ G t as the estimated graph ˆ G t . This procedure directly mirrors the graph property in Lemma 1, and recovers the transitive closure of G with high probability. We denote the parent set of node i in ˆ G t by pa t ( i ) . In this rule, γ > 0 is a threshold for determining if a row of the estimated latent precision differences is zero. To ensure that this threshold can differentiate between true edges and non-edges, we need to select it carefully. We show that choosing it in instance-dependent interval γ ∈ (0 , γ ∗ ) is necessary, where

̸

1.b - Refined recovery under hard interventions. When using hard interventions, the intervention target becomes independent of its non-descendants in the latent space. We impose this independence condition on the estimated latent variables by constructing minimum mean-squared-error (MMSE) estimates of a node's non-descendants in the estimated graph, which is equal to the transitive closure of the true graph with high probability. Specifically, we do the following.

(i) Refined inverse transform H t . Using the sample covariance matrices defined in (8), we first compute the estimated latent sample covariance matrices via

We collect the MMSE estimates of node i on pa t ( i ) in environment { i } in matrix Ξ t ∈ R n × n as

Using these MMSE estimates, we update the estimate H t as

(ii) Variables refinement. We re-apply the relation ˆ Z t = H t X t . We show that this procedure refines the estimates ˆ Z t to be approximately equal to the true variables Z t up to scaling.

2 - UCB-based selection rule. After estimating the latent variables and the graph, we specify our selection rule. Formalizing this rule involves characterizing confidence ellipsoids of the relevant parameters under different interventions. To this end, we first estimate the graph parameters and subsequently use the confidence ellipsoids of these estimates to construct the confidence ellipsoid of the interventions, enabling a full description of the UCB-based selection of interventions.

(iii) Graph refinement. We refine the graph estimate ˆ G t by applying (15) using the updated H t . Since the variables are now recovered up to scaling by Lemma 1, this step returns the latent graph G exactly with high probability. Therefore, we do not take the transitive closure of ˆ G t .

2.a - Parameter estimation. We first estimate the SEM parameters B , B ∗ , and utility parameters θ using the estimated variables ˆ Z t and the estimated graph ˆ G t . To keep the notation compact, we present the following algorithm and regret analysis as if the noise means after recovery are known, i.e., ˆ ν a for a ∈ A are known. This can be readily relaxed to accommodate unknown mean setting by using the same reparameterization technique as in [21], which involves adding a dummy node 1 to the graph estimates ˆ G t , along with the associated modifications to ˆ Z t and pa t ( i ) .

As we use finite samples X t to estimate Z t , there will inevitably be an estimation error. Therefore, at each time t ∈ N , we design the weighted ridge regression estimators A t and A ∗ t at time t for B and B ∗ , respectively. To estimate the observational weights [ A ] i , we only use the samples in which node i was not intervened. Conversely, to estimate the interventional weights [ A ∗ ] i , we use the samples in

which node i was intervened. We encode these selection rules via diagonal weight matrices W i,t and W ∗ i,t , which we will specify later. The i -th rows of the estimates A t and A ∗ t are specified as follows.

where we have defined weighted and doubly weighted Gram matrices as

where Z i,t,s ≜ Z t [ pa t ( i ) , : s ] and I n is the identity matrix. An important consideration in the above estimates is the design of the weight matrices. At each time t , we construct diagonal matrices { W i,t , W ∗ i,t : i ∈ [ n ] } to softly filter out the outlier samples that are likely to have higher estimation error. The diagonal elements for s ∈ [ t ] are defined as

The designs of the diagonal weights are inversely proportional to ζ t , which is a bound on the cumulative estimation error. This design uses smaller weights when ζ t is higher. The weights are also inversely proportional to a weighted ℓ 2 norm, often referred to as the weighted exploration bonus . A higher exploration bonus means lower confidence in the sample and hence, lower weights.

Similarly, with Z θ,t,s ≜ Z t [: , : s ] , we define the estimate for θ at time t ∈ N as follows.

2.b - Confidence ellipsoids and decision rule. After estimating the SEM and utility parameters, we use a UCB-based rule for sequential selection of interventions. The UCB under intervention a ∈ A is defined as the maximum value of expected utility when the weights are in the confidence ellipsoids of a , denoted by C a ,t . In order to construct the confidence ellipsoid for a , we first form the following confidence ellipsoids for the estimated parameters , i.e., A t , A ∗ t and θ t :

where { β i,t ( δ t ) ∈ R + , t ∈ N , i ∈ [ n ] } and { β t ( δ t ) ∈ R + , t ∈ N } are sequences of confidence radii that control the size of confidence ellipsoids and δ t is the tolerance of wrong estimates at time t . Accordingly, we define the relevant confidence ellipsoid for node i under intervention a ∈ A as

Based on these, at time t , our algorithm selects the intervention that maximizes the UCB. Let L t be the length of the longest causal path of ˆ G t . Due to the linear structure in SEMs, it is defined as

3 - Under-sampling rule. RO-CRL iteratively update various estimates. The performance of the UCBstep is contingent on the performance of the CRL step producing increasingly accurate estimates ˆ G t and H t . To ensure such accuracy, we impose a rule requiring that single-node interventions be sufficiently explored over time. Such interventions are needed to construct the necessary statistics for the CRL step. To formalize this, we define the set of under-explored interventions as

̸

where the function f t ( ˆ G t ) is a non-decreasing term that controls the adaptive exploration to collect observations X t when necessary. If A UE t = ∅ , the algorithm is forced to random sample from A UE t .

4 Regret Analysis for RO-CRL

In this section, we present the node-level instance-dependent regret results for RO-CRL. We also present algorithm-independent regret lower bounds to complement RO-CRL's achievable regret.

As an intermediate step toward regret analysis, we first provide PAC identifiability guarantees for recovering G and Z t . These results depend on the least frequently selected single-node intervention up to time t , denoted by s t ≜ min a ∈A 0 N a ,t .

Theorem 2 (Sample complexity) . For any instant t of RO-CRL that satisfies s t ≥ N ( ϵ, δ ) , where under Assumption 1, the estimate H t constructed in (11) (or (19) ) and estimate ˆ G t constructed in (15) ensure ( ϵ, δ ) -PAC recovery of Z t and G under soft (or hard) interventions specified in Definition 1. This implies that with probability at least 1 -δ , the error term E t specified in Definition 1 under both hard and soft interventions satisfies

The regret bounds have delicate differences under hard and soft interventions captured by the constants u ∈ { u S , u H } . Let m ∈ { S , H } indicate the intervention type, and H m ( i ) ∈ {H S ( i ) , H H ( i ) } such that H H ( i ) = pa ( i ) and H S ( i ) = an ( i ) . Then u is defined as

By setting δ t = 6 δ π 2 t 2 for confidence radii and controlling T 0 to satisfy T 0 ≥ N ( ϵ max , δ nT 0 ) , we ensure that with probability at least 1 -4 δ , the choice of f t defined in (32), ζ t for weights design and β 's for confidence radii will be equal to or in the order of

where under soft intervention u = u S and under soft intervention u = u H .

Theorem 3 (Regret upper bound) . Under Assumptions 1-3, with probability at least 1 -4 δ , the average cumulative regret of RO-CRL is upper bounded by

When the noise means ˆ ν a for a ∈ A are unknown, the impact on the regret upper bound order is reflected in the parameter u , where the value will be set to 1 if i is a root node instead of 0 .

Next, we establish a lower bound on the regret in the unknown mean setting. As no finite-sample lower bound for CRL is known in the literature, we provide the lower bound by fixing the estimation error as in Theorem 2. As the reward depends on how noise terms cumulatively contribute to the utility, the lower bound depends on the number of paths in the G . Denote m i,j as the number of causal paths from node i to node j in G tc , and denote the number of paths p as p ≜ n +1+ ∑ n i =1 ∑ n j =1 m i,j .

Theorem 4 (Lower bound under soft intervention) . For any CRL-based algorithm that satisfies Theorem 2, there exist instances of a causal model on G tc and estimation error E t such that the expected cumulative regret of any algorithm is at least

Remark 1 (Tightness or the bounds) . When comparing the upper bound in Theorem 3 and the lower bound in Theorem 4, we observe that both bounds show similar behavior with respect to graph-dependent parameters and the time horizon T . The only discrepancy comes from u S and p , where u S has an extra factor to count the dimension of graph connectivity in transitive closure.

Remark 2 (Causal bandits) . Our reward-oriented CRL reduces to the standard causal bandit setting if we set G = I n and the utility function to U ( Z ) = Z [ n ] . By these choices, our regret bounds immediately provide regret bounds for causal bandits. Specifically, our upper and lower bounds simplify to ˜ O ( u H √ T ) and Ω( p √ T ) , respectively, where p is defined on G instead of G tc . These bounds match at ˜ Θ( d L G √ T ) for graph-independent regret bounds, eliminating the gap d in previous results [24].

0

10000

20000

30000

40000

T

0

2000

4000

6000

8000

Cumulative regret

RO-CRL

0

10000

20000

30000

40000

T

0

2000

4000

6000

8000

Cumulative regret

RO-CRL

5 Experiments

In this section, we evaluate the empirical performance of RO-CRL. We report the regret of ROCRL under both soft and hard interventions. Additional experiments, including CRL recovery, scaling behavior, comparison to baselines, and assumption violations, are deferred to Appendix C. 2

Latent graph. We generate a random acyclic graph on n nodes by enforcing a strictly lowertriangular weight matrix. To ensure that the UCB in RO-CRL is computable for any randomly generated graph, we force all weights and noises to be positive (See Appendix B). Specifically, observational weights are drawn from [0 . 25 , 1] and interventional weights are set 0 . 1 times the corresponding observational weights for soft interventions and to 0 for hard interventions. Noise terms are sampled i.i.d. from the uniform distribution U [0 , 1] . We set n = 5 for these experiments and repeat the experiments 50 times.

Figure 2 illustrates the cumulative regret of RO-CRL under two types of interventions: soft interventions (left) and hard interventions (right). In both cases, we observe an initial phase of forced exploration (highlighted in yellow), during which the algorithm collects sufficient interventional data to estimate the underlying causal structure and variables. After this phase, the regret curves begin to flatten, indicating that RO-CRL effectively identifies near-optimal interventions and achieves sublinear regret.

6 Conclusion

In this paper, we have introduced the framework for formalizing and analyzing reward-oriented causal representation learning, the objective of which is to optimize a downstream task over the space of possible interventions in the causal system. The key difference between reward-oriented and conventional CRL is that CRL's objective of perfectly recovering the latent graph and variables can be excessive for optimizing a downstream task. Specifically, for reward-oriented CRL, one needs to learn the latent graph and variables at the coarsest level that enable the identification of optimal interventions. To resolve uncertainties associated with the latent causal graph and the latent probability distributions, we have adopted a sequential framework to explore different interventions and identify the optimal one with as few time instances as possible. We have designed an adaptive exploration algorithm that learns only the coarsest representation necessary to optimize the downstream reward. We have provided finite-sample latent recovery guarantees for the causal graph and its variables, and regret bounds for different types of interventions. In the standard causal bandit setting, these bounds simplify and match, even improve upon previous results.

Acknowledgments and Disclosure of Funding

This work was supported in part by the Rensselaer-IBM Future of Computing Research Collaboration (FCRC).

References

Bernhard Schölkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. Toward causal representation learning. Proceedings of the IEEE , 109(5):612-634, May 2021.

Chandler Squires, Anna Seigal, Salil S. Bhate, and Caroline Uhler. Linear causal disentanglement via interventions. In Proc. International Conference on Machine Learning , Honolulu, HI, July 2023.

Burak Varıcı, Emre Acartürk, Karthikeyan Shanmugam, Abhishek Kumar, and Ali Tajer. Scorebased causal representation learning: Linear and general transformations. Journal of Machine Learning Research , 26(112):1-90, 2025.

Emre Acartürk, Burak Varıcı, Karthikeyan Shanmugam, and Ali Tajer. Sample complexity of interventional causal representation learning. In Proc. Advances in Neural Information Processing Systems , Vancouver, Canada, December 2024.

Tianyu Chen, Kevin Bello, Francesco Locatello, Bryon Aragam, and Pradeep Ravikumar. Identifying general mechanism shifts in linear causal representations. In Proc. Advances in Neural Information Processing Systems , Vancouver, Canada, December 2024.

Kartik Ahuja, Divyat Mahajan, Yixin Wang, and Yoshua Bengio. Interventional causal representation learning. In Proc. International Conference on Machine Learning , Honolulu, HI, July 2023.

Jikai Jin and Vasilis Syrgkanis. Learning linear causal representations from general environments: Identifiability and intrinsic ambiguity. In Proc. Advances in Neural Processing Systems , Vancouver, Canada, December 2024.

Jiaqi Zhang, Chandler Squires, Kristjan Greenewald, Akash Srivastava, Karthikeyan Shanmugam, and Caroline Uhler. Identifiability guarantees for causal disentanglement from soft interventions. In Proc. Advances in Neural Information Processing Systems , New Orleans, LA, December 2023.

Julius von Kügelgen, Michel Besserve, Wendong Liang, Luigi Gresele, Armin Keki´ c, Elias Bareinboim, David M Blei, and Bernhard Schölkopf. Nonparametric identifiability of causal representations from unknown interventions. In Proc. Advances in Neural Information Processing Systems , New Orleans, LA, December 2023.

Simon Buchholz, Goutham Rajendran, Elan Rosenfeld, Bryon Aragam, Bernhard Schölkopf, and Pradeep Ravikumar. Learning linear causal representations from interventions under general nonlinear mixing. In Proc. Advances in Neural Information Processing Systems , New Orleans, LA, December 2023.

Burak Varıcı, Emre Acartürk, Karthikeyan Shanmugam, and Ali Tajer. General identifiability and achievability for causal representation learning. In Proc. International Conference on Artificial Intelligence and Statistics , Valencia, Spain, May 2024.

Wendong Liang, Armin Keki´ c, Julius von Kügelgen, Simon Buchholz, Michel Besserve, Luigi Gresele, and Bernhard Schölkopf. Causal component analysis. In Proc. Advances in Neural Information Processing Systems , New Orleans, LA, December 2023.

Burak Varıcı, Emre Acartürk, Karthikeyan Shanmugam, and Ali Tajer. Linear causal representation learning from unknown multi-node interventions. In Proc. Advances in Neural Information Processing Systems , Vancouver, Canada, December 2024.

Finnian Lattimore, Tor Lattimore, and Mark D Reid. Causal bandits: Learning good interventions via causal inference. In Proc. Advances in Neural Information Processing Systems , Barcelona, Spain, December 2016.

Elias Bareinboim, Andrew Forney, and Judea Pearl. Bandits with unobserved confounders: A causal approach. In Proc. Advances in Neural Information Processing Systems , Montréal, Canada, December 2015.

Sanghack Lee and Elias Bareinboim. Structural causal bandits: Where to intervene? In Proc. Advances in Neural Information Processing Systems , Montréal, Canada, December 2018.

Yangyi Lu, Amirhossein Meisami, Ambuj Tewari, and William Yan. Regret analysis of bandit problems with causal background knowledge. In Proc. Conference on Uncertainty in Artificial Intelligence , virtual, August 2020.

Vineet Nair, Vishakha Patil, and Gaurav Sinha. Budgeted and non-budgeted causal bandits. In Proc. International Conference on Artificial Intelligence and Statistics , virtual, April 2021.

Yangyi Lu, Amirhossein Meisami, and Ambuj Tewari. Causal bandits with unknown graph structure. In Proc. Advances in Neural Information Processing Systems , virtual, December 2021.

Scott Sussex, Anastasiia Makarova, and Andreas Krause. Model-based causal bayesian optimization. In Proc. International Conference on Learning Representations , Kigali, Rwanda, May 2023.

Burak Varıcı, Karthikeyan Shanmugam, Prasanna Sattigeri, and Ali Tajer. Causal bandits for linear structural equation models. Journal of Machine Learning Research , 24(297):1-59, 2023.

Shi Feng, Nuoya Xiong, and Wei Chen. Combinatorial causal bandits without graph skeleton. In Proc. Asian Conference on Machine Learning , Hanoi, Vietnam, December 2023.

Zirui Yan, Arpan Mukherjee, Burak Varıcı, and Ali Tajer. Improved bound for robust causal bandits with linear models. In Proc. IEEE International Symposium on Information Theory , Athens, Greece, July 2024.

Zirui Yan, Dennis Wei, Dmitriy Katz-Rogozhnikov, Prasanna Sattigeri, and Ali Tajer. Causal bandits with general causal models and interventions. In Proc. International Conference on Artificial Intelligence and Statistics , Valencia, Spain, May 2024.

Muhammad Qasim Elahi, Mahsa Ghasemi, and Murat Kocaoglu. Partial structure discovery is sufficient for no-regret learning in causal bandits. In Proc. International Conference on Machine Learning Workshop on Foundations of Reinforcement Learning and Control-Connections and Perspectives , Vienna, Austria, July 2024.

Zirui Yan and Ali Tajer. Linear causal bandits: Unknown graph and soft interventions. In Proc. Advances in Neural Information Processing Systems , Vancouver, Canada, December 2024.

Roman Vershynin. High-dimensional probability: An introduction with applications in data science , volume 47. Cambridge University Press, Cambridge, UK, 2018.

Chandler Davis and W. M. Kahan. The rotation of eigenvectors by a perturbation. III. SIAM Journal on Numerical Analysis , 7(1):1-46, 1970.

Jiafan He, Dongruo Zhou, Tong Zhang, and Quanquan Gu. Nearly optimal algorithms for linear contextual bandits with adversarial corruptions. In Proc. Advances in Neural Information Processing Systems , New Orleans, LA, December 2022.

Tor Lattimore and Csaba Szepesvári. Bandit Algorithms . Cambridge University Press, Cambridge, UK, 2020.

Yoan Russac, Claire Vernade, and Olivier Cappé. Weighted linear bandits for non-stationary environments. In Proc. Advances in Neural Information Processing Systems , Vancouver, Canada, December 2019.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Sections 4 establish the results claimed in the abstract and introduction.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The major limitations of the work is the linear assumption on SEMs, transformations and utility functions. We discuss these in detail in Appendix.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We provide a full set of assumptions at the end of Section 2 and referenced in the theorem statemnts. All proofs are included in the Appendix.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: See the details in the experiments section in the Appendix C.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We release the code with instructions.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: The algorithm details are described in Section 3 with pseudocode provided in Algorithm 1. All the parameter setting are discussed. The experimental details are provided in the experiments section (Section 5) and additional experiments (Appendix C).

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [No]

Justification: The theoretical results are based on high probability finite sample guarantee and average regret. We do not include the variance and error bars as a metric in this paper; therefore, it is not defined and applicable.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: The experiments only need CPUs.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research focus on theory only include the synthetic dataset and hence, do not have potential potential harmfulness.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: This paper is mostly theoretical and does not pose potential negative societal impacts.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: This paper is mostly theoretical and does not pose such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: Portions of the publicly available code of [3, 4], available under Apache 2.0 license, is adopted in the code of our experiments.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: We released code for the experiments.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: The core method development in this research does not involve LLMs

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Reward-oriented Causal Representation Learning Supplementary Materials

Table of Contents

A Pseudocode

Algorithm 1 Reward-oriented CRL (RO-CRL)

̸

B Computational complexity of RO-CRL

The computational cost of RO-CRL per step can be broken down into two parts:

CRL: CRL routine depends on matrix inversions ( O ( d 3 ) ), which can be expensive for large d . However, since the transformation is linear, we can detect the supporting subspace and project the samples to it, effectively reducing the observation dimension d to n , and yielding an overall per-step complexity of O ( n 4 ) .

UCB-based selection: The computational bottleneck is the intervention-selection step: the UCB is intractable on general causal graphs [26, Section 3.3]. To make the algorithm practical, [26] proposes techniques that efficiently compute an upper bound on the UCB, which we can incorporate into our approach. However, the upper bound is loose, and its robustness to imperfect latent recovery is unknown.

In the experiments, we use the following modification to ensure efficiency in the intervention selection step for RO-CRL.

Non-negative edge weights and noise (shifted system): We investigate the setting where all weights and noises are positive. A general system can be linearly transformed into such a non-negative system. A key property of such systems is that all variables are monotone in their parents. To calculate the UCB, we can therefore maximize nodes sequentially in causal order and use the closed form in linear bandits. The resulting complexity is O ( ∑ i ∈ [ n ] | pa t ( i ) | 3 + n 3 ) for UCB selection in (31), where n is the number of latent dimensions.

C Additional Experiments

Latent Recovery Figure 3 shows the variations of the graph recovery rate versus the sample size s t . As expected, it is observed that the recovery improves with more samples s t . Furthermore, the hard interventions, the stronger special case, consistently yield higher recovery rates than soft

0

20000

40000

s

0.00

0.25

0.50

0.75

1.00

Graph recovery rate

Hard intervention

Soft intervention

0

20000

40000

s

0.00

0.25

0.50

0.75

1.00

Graph recovery rate

Hard intervention

Soft intervention

t

t

interventions. Figures 4 and 5 show the norm of the error term E t and the average estimation error on latent variables Z for varying s t . We can see both terms decay, which conforms to the theoretical decay √ 1 /s t rate established in Theorem 2.

0

20000

40000

s

t

0

1

2

3

4

Avg. of

E

t

Hard intervention

Soft intervention

0

20000

40000

s

t

0

1

2

3

Avg. of

E

t

Hard intervention

Soft intervention

0

20000

40000

s

t

0

1

2

Avg. error on Z

Hard intervention

Soft intervention

0

20000

40000

s

t

0

1

2

Avg. error on Z

Hard intervention

Soft intervention

Performance of CRL when varying n and d . In Table 2, we provide additional experiments to evaluate the performance of CRL across different n , d , and intervention types, plotted against sample size s t . These results show that the CRL performance of our RO-CRL algorithm is mostly consistent across variations in system parameters: Variable recovery errors decay with increasing number of samples s t consistently with the theoretical rate of 1 /s t (Theorem 2) across all settings. Observation dimension d does not affect results thanks to a dimensionality reduction step. Finally, increasing n from 5 to 8 does not lead to a significant drop in variable recovery performance. However, graph recovery performance significantly degrades for n = 8 under soft interventions, which can be explained by noting that the 'true' graph under soft interventions is the transitive closure, which includes a high number of indirect edges. Such indirect effects are hard to track from the precision matrices, which makes the graph recovery for soft interventions more difficult.

(a) Latent recovery (MSE).

Varying observation dimension d : Thanks to dimensionality reduction techniques and the forcedexploration schedule, CRL's regret remains essentially unchanged as we increase the feature dimension d from 5 to 100 while keeping the latent dimension at n = 5 . This observation is consistent with some previous findings in CRL under linear transformations [3] that the performance of CRL is not that sensitive to the feature dimension d , and this behavior does not contradict our lower bound analysis, which is derived under the error bounds from CRL. Table 3 reports the corresponding runtime for different d at a fixed time horizon T , and the additional computational cost introduced by higher d remains well within practical limits.

Varying latent dimension n : (which also affects the u t ) for fixed d = 10 . From Table 4, we observe that the cumulative regret grows by increasing n . We note the increase stems from two factors: (i) exploration cost for CRL varies with n , and (ii) the reward range becomes larger as n increases. These trends will be more clearly visualized in the figures to be included in the final version.

Baselines. We compare RO-CRL with two baselines: vanilla UCB and a modified version of RO-CRL, where CRL is performed for a fixed fraction of the sample budget, after which the bandit proceeds without accounting for latent recovery error. Table 5 reports the cumulative regret at the given final horizon T for different algorithms. As shown, both baselines perform significantly worse than RO-CRL. The performance gap between RO-CRL and Modified RO-CRL will continue to increase with longer time horizons.

Assumption Violations.

Assumption 1: We run experiments where Assumption 1 is violated and provide the regret and variable recovery results in Tables 6-7. First, we note that graph recovery consistently fails (the graph recovery rate is exactly 0) in this setting. However, Table 6 shows that latent variable recovery still works: Indeed, its analysis is independent of Assumption 1. In Table 7, we evaluate cumulative regret versus time, and observe that the bandit algorithm does not achieve a sublinear regret after the forced exploration phase ( t ≥ 18000 in this case), meaning that the RO-CRL algorithm requires Assumption 1 to appropriately capture the true uncertainties and make good decisions.

Boundedness violations We conduct experiments that deliberately violate two key assumptions: bounded noise and known parameter bounds. The specific violations are as follows: Noise violation: We replace the bounded noise with unbounded Gaussian noise. - Parameter violation: Instead of properly rescaling the system, we multiply all parameters by a factor of 10 to simulate the violation. Note that such a scaling can be offset by properly scaling the latent variables. Therefore, in these experiments, we explicitly set a fixed scale of the latent variables.

As shown in Table 8, both violations result in linearly growing regret after the forced exploration phase ( t ≥ 18000 in this case). This result aligns with theoretical expectations: violating the noise boundedness or known-parameter assumptions leads to confidence radii that no longer appropriately capture the true uncertainty, thereby causing increasingly poor decisions.

D Additional Notations

In this section, we present the notations that will be useful in our analyses.

Given a symmetric matrix A ∈ R d × d , we denote the vector of eigenvalues of A ordered in descending order by λ ( A ) ∈ R d and the matrix of eigenvectors by Q ( A ) ∈ R d × d such that A = Q ( A ) · diag ( λ ( A )) · Q ( A ) ⊤ .

D.1 Precision Matrices

In this paper, we will consider precision matrix differences as our learning signal for CRL, similarly to [2]. The precision matrix of a distribution is the inverse of its covariance matrix. In case of the latent variables, Z , let us denote the pre-intervention precision matrix by Θ . Note that the implicit linear SEM can also be written explicitly as

Let us denote the vector v as the variances of entries of ε , and v ∗ as its interventional counterpart ε ∗ . Using this formulation, the precision matrix is given by

Let us construct two matrices K , K ∗ ∈ R N × N row by row as such that term i in (41) is equal to ([ K i ]) ⊤ [ K ] i . Since this is only a function of the generation mechanism of node i , the precision matrix of the latent variables under action a ∈ A , denoted by Θ Z a , is given by

Therefore, K , K ∗ fully parameterize the precision matrices of the latent variables under any action a ∈ A . Finally, note that the precision matrices of observed variables, denoted by Θ a for action a ∈ A , are given by

Thus, defining Q = KG † and Q ∗ = K ∗ G † , both in R N × D , the observed precision matrices are given by

Similarly to the latent case, Q and Q ∗ fully parameterize the observed precision matrices { Θ a : a ∈ A} . In this paper, the CRL algorithms we design solely depend on estimates of Q and Q ∗ .

D.2 Causal Bandit Notations

We define the row of weights under intervention as

First, we provide notations that are useful in our analyses. We denote the singular values of a matrix A ∈ R M × N , where M ≥ N , by

In the proofs, we often work with zero-padded vectors and corresponding matrices. As a result, the matrices that contain these vectors have non-trivial null space leading to zero singular values. In such cases, we use the effective smallest singular value that is non-zero. We denote the effective largest and smallest eigenvalues that correspond to effective dimensions of a positive semidefinite matrix A with rank k by

For a square matrix U = AA ⊤ ∈ R N × N , we denote the effective largest and smallest eigenvalues by

Then we construct data matrices that are closely related to Gram matrices. At time t ∈ N and for any node i ∈ [ n ] , the data matrices U i,t ∈ R n × t and U ∗ i,t ∈ R n × t consist of the weighted observational and interventional data, respectively. Specifically, we define

where we denote √ A as the square root of the matrix A .

We denote the relevant data matrices for node i ∈ [ n ] under intervention a ∈ A by

Combining (51) and (52), we have

Similarly we define the data matrices that are related to ˜ V i, a ,t as

The relevant data matrices for node i ∈ [ n ] under intervention a ∈ A are

Define N ∗ i,t as the number of times that node i ∈ [ n ] is intervened, and N i,t as its complement, i.e.,

Accordingly, for any i ∈ [ n ] , t ∈ N and a ∈ A , define

To proceed, we define the second-moment matrices and their effective largest and smallest eigenvalues as

where κ min > 0 is guaranteed since there is no deterministic relation between nodes and their patients. These variables are inherent to the system and remain unknown to the learner.

Lastly, we define ˜ A a ,t and ˜ θ a ,t as the weights that attains UCB a ,t , i.e.,

Accordingly, we define the auxiliary variable ˜ Z t generated according to the following SEM

with the exact same realization of the noise ε t and the UCB is calculated as

E Analysis of CRL Steps

In this section, we first prove the high probability error bounds for recovering the latent variables and graph using finite samples. For notational clarity, we drop ∞ from infinite-sample estimators and use these forms without a t subscript as infinite sample limits.

E.1 Proof of Infinite-sample Guarantees

In the infinite sample limit s t →∞ , covariance and precision matrix estimates converge to their true values. This enables us to derive identifiability guarantees without additional error terms. First, let us start by proving the main motivating property of precision differences.

Proof of Lemma 1. Using the expansion (43), we note that the precision difference between null and single-node interventional distributions is very structured.

̸

Using the definition of K , K ∗ in (42), row i of either of these matrices have non-zero entries only in coordinates i or pa ( i ) . Therefore, only the principal submatrix at coordinates ( { i }∪ pa ( i ) , { i }∪ pa ( i )) can be non-zero in R Z i , therefore, for any row j , the norm ∥ [ R Z i ] j ∥ 2 = 0 implies j ∈ { i } ∪ pa ( i ) .

Using this fact, we can prove the second part of the lemma. Since R i = ( G † ) ⊤ R Z i ( G † ) , row sparsity of R Z i immediately implies that

Rank-2 assumption. Next, let us investigate R Z i and R i under Assumption 1. This assumption simply ensures that [ K ] i and [ K ∗ ] i have different directions for all i ∈ [ n ] that is not the root node, which implies the following.

Lemma 2. Under Assumption 1, the precision matrix difference R Z i has rank 1 if and only if i is a root node, and has rank 2 otherwise. The rank of R i is equal to the rank of R Z i .

̸

Proof: Assumption 1, that is, [ B ] i = [ B ∗ ] i is used directly with the definitions of [ K ] i = ( v [ i ]) -1 / 2 [ I -B ] i , and [ K ∗ ] i = ( v ∗ [ i ]) -1 / 2 [ I -B ∗ ] i . If i is a root node, both [ B ] i = [ B ∗ ] i = 0 , but we must change the noise variance, which yields a rank-1 update to the matrix, with only the ( i, i ) -th element being non-zero. If i is not a root node, then Assumption 1 ensures that e i -[ B ] i and e i -[ B ∗ ] i have different directions, thus the overall difference of outer products yields a rank-2 matrix.

Soft interventions. Proof of the soft intervention inverse transform estimation results are identical to [3, Lemma 4]. For graph estimation, we follow closely to proof of [3, Lemma 5] as follows.

In the infinite sample regime, using soft interventions, we recover the latent variables up to mixing with parent nodes, i.e.,

where C S = HG with C S [ i, j ] = 0 only if j ∈ { i } ∪ pa ( i ) . The 'estimated latent' precision matrix difference is computed via

̸

̸

Using [3, Lemma 13], the up-to-parents mixing in C S results in a up-to-descendants mixing in [ C S ] -⊤ , that is, [ C S ] -⊤ [ i, j ] = 0 only if j ∈ { i } ∪ de ( i ) . On the other hand, using Lemma 1, only non-zero rows of R Z i are j ∈ { i } ∪ pa ( i ) . This implies that all rows and columns j ̸∈ { i } ∪ an ( i ) will be zero in ˆ R Z i . Therefore, the estimated graph ˆ G is at worst a supergraph of the transitive closure of the true graph G .

Hard interventions. Proof of the hard intervention post-processing steps in the infinite sample regime is identical to [3, Proof of Theorem 3].

E.2 Proof of Finite-sample Guarantees

In this section, we start by defining the error bounds for sample covariance and precision matrix estimation [27].

Lemma 3. With probability 1 -δ , the maximum error term in sample covariance matrices Σ a ,t for a ∈ A 0 is bounded by

Similarly, for a bounded condition number for covariance matrices, the maximum error in the sample precision difference matrices R i for i ∈ [ n ] is upper bounded by

The proof strategy for finite-sample guarantees follows closely to [4]: We first show that infinitesample guarantees can be recovered with low error if the estimation errors are low enough. Next, we use the error bounds of the specific precision difference estimator to derive overall error bounds for the overall CRL procedure.

Inverse transform estimation. In inverse transform estimation, the estimate H t is constructed row by row as the principal eigenvectors from finite-sample precision differences R i,t . We have the following result on the stability of this estimation procedure.

Lemma 4. Denote the principal eigenvector of R i by H i and that of R i,t as H i,t . Denote the minimum separation of the top two eigenvalues of R i by η ∗ , that is,

Using Davis-Kahan symmetric sin θ theorem [28], when ∥ R i -R i,t ∥ ≤ η ∗ / 2 , we have

Following [4, Lemma 17], the inverse transform estimation is upper bounded by

and the overall transformation error E t in Definition 1 is upper bounded by

Graph estimation. The graph estimation procedure consists of two steps: Computing the estimated latent precision difference matrices and thresholding them to recover edges. Let's first focus on the error bounds on the computation side, which then yield error upper bounds in order to achieve perfect graph recovery.

The equation for computing the estimated latent precision differences is

and similarly for finite-sample counterparts. A perturbative error bound on the pseudoinverse term is given by, when ∥ H t -H ∥ ≤ 1 / (2 ∥ H † ∥ ) ,

Therefore, a bound on the estimation error in ˆ R Z i,t is given by, when ∥ H † t -H † ∥ ≤ ∥ H † ∥ ,

The spectral norm bound is a natural upper bound on the ℓ 2 norm of any row of a matrix. In other words, under the same circumstances, we have, for any j ∈ [ n ] ,

Since the minimum nonzero entry of ˆ R Z i is γ ∗ , it suffices for the error to be below

to estimate all the edges in ˆ G correctly, i.e., to ensure ˆ G t = ˆ G . By shifting constants, this means

is sufficient for (i) correct (soft) graph recovery, and (ii) for the error bound in (74) to hold.

Hard interventions. For the post-processing in hard interventions, we use the linear minimum mean square error estimator, which is defined via a linear algebraic equation

This part has three possible sources of error: Estimation of H t / ˆ Z t , finite sample estimation of covariance matrices, and incorrect graph estimation. Since with bounded error the graph will be correctly estimated, we focus on the other two. Specifically, ˆ Σ Z { i } ,t is actually defined via

which, whenever ∥ Σ { i } ,t -Σ { i } ∥ ≤ ∥ Σ { i } ∥ , has an error upper bound

Given that ( ˆ Σ Z { i } [ ˆ pa ( i ) , ˆ pa ( i )]) has bounded condition number, collecting constants, we get

and therefore, after update H t ← ( I n -Ξ t ) H t , the inverse transform estimate error becomes

If we use the error bound in Lemma 3, with probability 1 -δ , we have

Since the order of the error did not change, the required error level for graph recovery also remains in the same order.

In summary, ensuring that the precision difference estimation error is upper-bounded by

ensures that (i) the graph can be correctly identified, and (ii) the variables to be recovered with error term in the same order as the precision difference errors, under both hard and soft interventions. This means that we can simply unify the constants by choosing the worst case, and use Lemma 3 to prove Theorem 2, which is restated here for the sake of completeness.

Theorem 5 (Sample complexity) . For any instant t of RO-CRL that satisfies s t ≥ N ( ϵ, δ ) , where under Assumption 1, the estimate H t constructed in (11) (or (19) ) and estimate ˆ G t constructed in (15) ensure ( ϵ, δ ) -PAC recovery of Z t and G under soft (or hard) interventions specified in Definition 1. This implies that with probability at least 1 -δ , the error term E t specified in Definition 1 under both hard and soft interventions satisfies

Proof: Note that many intermediary results require the precision difference estimation errors to be upper-bounded. As such, we provide error bounds where the error is required to be below a certain threshold. Then, the error bounds in Lemma 3 can be mapped to sample complexity statements to be used in our setting: With probability 1 -δ , the maximum error term in the sample covariance and precision matrices is bounded by ϵ if the instance satisfies

That is, if s t satisfies the above, for any ( ϵ, δ ) , the following error bounds hold with probability at least 1 -δ .

Then, given that ϵ max corresponds to the maximum tolerable error level of analysis provided in this section, we get the error bounds and sample complexity statements for the CRL objectives provided in the theorem statement.

F Proof of Regret Upper Bound (Theorem 3)

In this section, we prove the regret upper bound in Theorem 3 in three parts. For simplicity in notations, we provide the analysis for the setting in which the noise mean is known. The regret bounds for the more realistic setting in which the noise mean is unknown follow the same steps in a straightforward way by padding a dummy node to the graph and latent variables and noises, introducing one extra degree of freedom.

Part 1. we provide a decomposition of node-level utilities in Section F.2, which provides the intuition behind our UCB construction in (31).

Part 2. Then we establish a more general bound than Theorem 3 as Theorem 6, which holds under mild under-sampling conditions. Its proof consists of four steps:

We show the system and its estimation error remain bounded in Section F.1, and there is a transformed linear SEM that ˆ Z ∞ satisfies.

We show high probability ellipsoidal confidence sets for the parameter estimates A t , A ∗ t and θ t in Section F.3.

We quantify how these uncertainties propagate along the causal paths in Section F.4.

Combining the above, we provide the high probability regret bounds in Section F.5.

Part 3. Finally, leveraging Theorem 6 in Part 2 , we complete the proof of Theorem 3 by choosing an appropriate f t ( ˆ G t ) to balance exploration and exploitation.

We will first prove all parts under known transformed mean setting , that is ˆ ν a for a ∈ A , and then discuss how it generalizes to unknown transformed mean setting. We emphasize that, although our proof follows the high-level structure of [23], it departs crucially in how we define the utility function and account for uncertainty in both Z [ pa [ i ]] and Z [ i ] , a direct use of their formulation leads to suboptimal bounds. Moreover, we derive instance-dependent regret upper bounds that generalize those in [23]. Such refined bounds are essential in the transitive closure setting, where the maximum in-degree can be of the same order as n .

F.1 Bounded System and Error Bound

F.1.1 Known Transformed Mean

Transformed SEM. As CRL, even in the infinite sample regime, only recovers the variables up to scaling, the estimates ˆ Z ∞ do not obey the original SEM Z = B Z + ε . Instead, we obtain the following system.

Lemma 5. As a result of Theorem 1, the estimated latent variables ˆ Z ∞ are Markov with respect to the estimated graph ˆ G ∞ . Specifically, there exist weight matrices A , A ∗ ∈ R n × n such that where Λ is a diagonal matrix chosen so that A and A ∗ share the same support as B and B ∗ . As a result, A [ i, j ] and A ∗ [ i, j ] are non-zero only if j ∈ pa ∞ ( i ) . Finally, there exist exogenous-noise vectors ˆ ε ∞ , ˆ ε ∗ ∞ (with independent entries) such that ˆ Z ∞ follows a linear SEM. For any time t ∈ [ N ] and i ̸∈ a t , the estimated variables follows the following linear SEM

Similarly, i ∈ a t alters the generating mechanism for node i to

where we have at each time t ∈ N , the mean of the noises satisfies E [ˆ ε ∞ [: , t ]] = ˆ ν t under observation and E [ˆ ε ∞ [: , t ]] = ˆ ν ∗ t under intervention. And we use the term

Proof: Similar to (40), we know SEM follows

So under the infinite sample estimate and under t ∈ N with i ̸∈ a t , we have the following relations

=

∞

CZ

[:

, t

]

(98)

So that we have the relation

Rearrange (101) that we get

And we know that ˆ ε = Λ ε . Similarly, the result holds for A ∗ under i ∈ a t .

Similarly, the utility can be calculated as

As A and A ∗ are problem dependent constant matrices with finite element, there exists m A ∈ R + such that | A [ i, j ] | ≤ m A and | A ∗ [ i, j ] | ≤ m A and | ˆ θ [ i ] | ≤ m A for i, j ∈ [ n ] .

To simplify notation, we omit redefining other quantities (e.g. the intervention mean µ a ), which remain almost unchanged.

Cumulative estimation error. In Theorem 2 we characterized high-probability bounds on E t , which captures the error in estimating Z t . Recall that we have scheduled δ t = 6 δ π 2 t 2 as in Section 4 and impose the following condition on forced exploration T 0 and f t ( ˆ G t ) for general regret upper bound.

We note that the setting we adopt for Theorem 3 later satisfies the above condition.

We know that at time t , Algorithm 1 uses the estimates H t to estimate ˆ Z t only when there are no under-explored interventions. That is, we have for a ∈ A 0 the following condition holds

Then according to Theorem 2, we have with probability at least 1 -6 δ π 2 t 2 , the following error bound

Define the error of estimating ˆ Z t , ∆ t ∈ R n × t by

For each s ∈ [ t ] , we have the following 2-norm bound for the error bound for the estimates for Z s at time t as follows.

where the first inequality holds is due to the triangle inequality for norm, and the second inequality holds is due to (106) and the boundedness of latent variables Z .

We define the estimation error of sample s ∈ [ t ] at time t ∈ N as

Plugging (109) into the SEM in (93) and (94), We obtain at time s ∈ [ t ]

Hence the error term e t ∈ R n × t in the finite sample SEM is defined as

We can bound the above error for i ∈ [ n ] and s ∈ [ t ] as

where we set C ′ = C · max a ∈A max i ∈ [ n ] ∥ [ I n +1 -A a s ] i ∥ 2 , which is an instant-dependent constant. We define the cumulative estimation error bound at time as

ζ t t ≥ nT 0

where C ′ is an instance-dependent constant. This choice allows the following condition to hold.

Remark 3. For the efficient RO-CRL , we need to set the cumulative estimation error bound ζ t as

Or one can do more adaptive based on when you reset the whole estimates. The regret order of efficient RO-CRL will be the same as RO-CRL but with a larger constant multiplier.

Bounded variables. We have the following for the estimates for t ∈ N

where the last inequality is due to the bounded variable ∥ Z ∥ ≤ m

Similarly, for all s ∈ [ t ] , we have the following bound for ˆ Z t [: , s ] as

where we have defined

Recall from (104) that

Hence, we have

F.1.2 Generalization to Unknown Mean Setting

To handle an unknown post-transform noise mean, we augment the graph ˆ G t with a dummy node, prepend a 1 to each variable and noise vector, and adjust the weight matrices accordingly. In such a case, we define

Subsequently, the estimate at time t is given by

Analogous to Lemma 5, for all t ∈ N with a t = ∅ , the padded variables satisfy

where we have defined A p ∈ R ( n +1) × ( n +1)

Similarly, we can define the weights and noises under intervention as

where the estimated mean values are

The weights for the utility parameter can be appended as

Since the dummy node requires no estimation, the choice of ζ t and the cumulative estimation error bounds remain valid. Consequently, identical order bounds apply to both ˆ Z ∞ and ˆ Z t , as only an extra dimension of 1 has been added.

F.2 Decomposition of Node-level Utility

Similar to [21, Lemma 1], we present the following decomposition for the expected utility value. Our design of the mean value estimator in the UCB definition in (31) is based on this lemma.

Corollary 1. When ˆ ν a is known, for intervention a ∈ A , the expected utility is related to the noise vector ε via

where A ℓ denotes the ℓ -th power of matrix A .

Proof. We first show the results for the expected value of Z under intervention a ∈ A as follows. We know from the linear SEMs that

So, in linear SEMs, each latent random variable Z i can be specified as a linear function of the exogenous noise variables ε via recursive substitution of the structural equations. And the inverse has a simple expansion since A a is strictly lower triangular. Specifically,

where (133) holds due to L is the maximum path length and A ℓ becomes a zero matrix for ℓ ≥ L +1 . Define the variable ˆ Z ≜ H ∞ X = A a ˆ Z + ε Hence, we obtain

Since ε and a are independent, the expectation of each ε i is 0 for i ∈ [ n ] , and dummy noise. Then, we obtain the following results for the mean value

Then, by the definition of expected mean of utility in (2), we have

Under unknown transformed mean. Under unknown transformed mean, we are working with A p and A ∗ p , we define the transformed weight matrix with padding as

For the expected utility value under an unknown transformed mean, we have the following lemma. Lemma 6. Given intervention a ∈ A , the expected utility value depends to the noise vector ε via where A ℓ denotes the ℓ -th power of matrix A .

Proof: The first few steps for the proof of Lemma 1 still hold, with A p a , and maximum length L +1 instead of L due to the dummy node, the changes in the proof start from (137), where we have

Then, as the expected mean of utility defined in (2), we have

F.3 Proof of Concentration Inequality

In this section, we provide the concentration inequality and then note how to extend it to the unknown noise mean setting. We notice that certain parts of the proof can be replaced by a non-time-uniform argument (which does not improve the rate), so we keep the same flow as previous results while focusing on the essential difference. We note we always work on the SEM associated with (93) and (94).

As the estimation quality is related to the degrees of freedom, we begin by introducing the in-degree of our graph estimates. We denote d i,t as the number of parents of node i in ˆ G t . Then, by Theorem 2 together with our choice of the confidence levels { δ t } , it holds with probability at least 1 -δ , d i,t = | pa ( i ) | under hard intervention and d i,t = | an ( i ) | under soft intervention with all i ∈ [ n ] and t ∈ [ N ] .

Lemma 7 (Confidence ellipsoids) . With probability at least 1 -3 δ , for any node i ∈ [ n ] and t ≥ 1 , we have

where ˜ m = ˜ O ( (1 + √ d/f t ( ˆ G t )) m ) and

Proof: We observe that regressing [ ˆ Z t ] i -[ e t ] i on [ ˆ Z t ] pa t ( i ) , restricted to time indices under the observational mechanism, yields an unbiased estimator of A . This connects the proof of our concentration lemma to prior results in robust causal bandits [23]. However, a key distinction lies in the fact that both random variables on the node i and its parent nodes pa t ( i ) have estimation errors. This introduces two sources of noise, which, if treated using previous techniques that assume noiseless anchors using regression [ Z ] i on [ Z ] pa t ( i ) , would result in a multiplicative increase in the error. To avoid this, we use the regression of [ ˆ Z t ] i -[ e t ] i on [ ˆ Z t ] pa t ( i ) as our anchor instead of relying on the true variables. A technic based on [29]

We will provide the proof corresponding to the observational weights [ A t ] i , while the proof for the interventional weights [ A ∗ t ] i and the utility parameters θ follows similarly.

We prove it by first establishing it for a given time t ∈ Z . For any node i ∈ [ n ] , we decompose the error in estimation ∥ [ A t ] i -A i ∥ V i,t [ ˜ V i,t,t ] -1 V i,t for s ∈ [ t ] as follows.

where ˆ A t refers to the auxiliary estimators which correspond to the ridge regression estimator when knowing the estimation error e t on ˆ Z t , i.e.,

Next, we bound the two error terms I 1 and I 2 .

Bounding I 1 . The stochastic and regularization errors can be bounded by the following lemma. We notice we do not need the time uniform bounds, but non-time uniform bounds will not improve the regret order (See [30, Section 20] for a related example).

Lemma 8. For all node i ∈ [ n ] , for given t ∈ [ T ] with probability at least 1 -δ t , we have

Proof: This lemma follows the result from [23], which is based on [31, Theorem 1].

Based on Lemma 8 and our scheduling of δ t such that ∑ t ∈ N δ t = δ , we immediately have the following lemma

Lemma 9. For all node i ∈ [ n ] , with probability at least 1 -δ , for all t ∈ N , we have

Bounding I 2 . Now we need to bound the fluctuation error I 2 , which can be decomposed as

where (164) and (165) follow the definition of weighted norm and weight matrix, (166) and (167) hold due to the triangle inequality and weights are non-negative, (169) holds due to ∥ x ∥ [ ˜ V i,t ] -1 ≤ ∥ x ∥ [ ˜ V i,s ] -1 , and (170) is obtained using the definition of the weights and the property ζ t ≥ ∑ t s =1 | e t [ i, s ] | in (116).

Finally, substituting the results of Lemma 9 and (170), with probability at least 1 -δ , for all t ≥ 0 , we have

Similarly, for the estimators for interventional weights, with probability at least 1 -δ , for all t ≥ 0 , we have

Asimilar proof can be get for the utility U where the in-degree is n and only one concentration bound instead of n is needed, hence, with probability at least 1 -δ , for all t ≥ 0 , we have

Combining the results in (171), (172) and (173) we complete the proof.

Finally, in the unknown noise mean setting one simply replaces each d i by d i +1 (to account for the dummy node) and adjusts ˜ m accordingly.

F.4 Cumulative Estimation Error

Lemma 7 in the previous section provides high-probability error bounds on our estimators. Due to the causal structure, these errors accumulate and propagate along the causal paths, leading to the estimation error in the utility. So we provide the following optimistic cumulative estimation error for utility U . To start with, we define the detailed cumulative uncertainty u β,t ∈ { u β, S , u β, H } .

Lemma 10. If ˆ G t = G or ˆ G t = G tc for all t ∈ [ N ] , and [ A ] i ∈ C i,t and [ A ∗ ] i ∈ C ∗ i,t for all t ∈ N and i ∈ [ n ] and θ t ∈ C θ,t for all t ∈ N , then we have

where u = u H if ˆ G t = G and u = u S if ˆ G t = G tc , and we define the term

where τ = α 2 ˆ m 6 κ 2 min , α = √ 16 3 log((max i ∈ [ n ] d i +1) T 5 / 2 ( T +1)) and

Proof: This proof is a cumulative estimation error at the node level of [23]. We first prove that when the conditions of the lemma hold, the latent variables i ∈ [ n ] have a cumulative estimation error as follows.

Lemma 11. If ˆ G t = G or ˆ G t = G tc for all t ∈ [ N ] , and [ A ] i ∈ C i,t and [ A ∗ ] i ∈ C ∗ i,t for all t ∈ N and i ∈ [ n ] and θ t ∈ C θ,t for all t ∈ N , then for i ∈ [ n ] we have

where u i = u i, H if ˆ G t = G and u i = u i, S if ˆ G t = G tc that is defined in (35) .

Proof: We establish (182) via induction on the causal depth. We define the causal depth of node i as the length of the longest directed causal path that ends at node i ∈ [ n ] in ˆ G t and denote it by L i .

Induction Step. Assume that the property holds for causal depths up to L i = k . We show that it will also be satisfied for L i = k +1 .

Base step: L i = 1 . This is according to the same proof in [23] with a change on ˆ m , and using the bounds for cumulative error that ζ nT 0 ≤ ζ t ≤ ζ T for nT 0 ≤ t ≤ T .

For this purpose, we start with the following expansion and apply the triangular inequality to find an upper bound for it.

where the transition to (185) holds due to the triangular inequality via adding and subtracting terms [ ˜ A a t ,t ] ˆ Z t [ pa t ( i ) , t ] ; and (186) holds since the triangle inequality of L 2 norm and | ˜ A a t ,t [ i, j ] | ≤ m A and similar proof as in the base step.

Next, we find an upper bound on the first summand in (186). We notice that the summation is taken over all parents of node i . Thus, we aim to find an upper bound for the error bound for each parent. Based on the induction assumption, for each node j ∈ pa t ( i ) , we have

Subsequently, plugging (187) into (186), we obtain

Combining the results in (186) and (188), we conclude

which proved the desired results in (182).

Now we are ready to prove the final result for the utility function.

Here we conclude the proof.

F.5 Proof of General Regret Bounds

Theorem 6 (General Regret upper bound) . By setting confidence radius β i,t ( δ t ) and β t ( δ t ) according to Lemma 7, f t ( ˆ G t ) and T 0 satisfy the condition in (104) . If we define f T = max t ∈ [ T ] f t ( ˆ G t ) , then with probability at least 1 -4 δ , the average cumulative regret of RO-CRL is upper bounded by

Proof: We start by defining the event in which, over T rounds, all confidence sets C i,t contain the ground truth parameters f i . Specifically, Now define the error events E i and E ∗ i for i ∈ [ n ] for each estimator

where the β i,t ( δ t ) and β t ( δ t ) is chosen as in (155) and (156). Accordingly, define the event that at least one of the confidence ellipsoids of estimators does not contain the true parameters at at least one time index

By invoking the union bound on probability and Lemma 7, we have

Next, we decompose the regret defined in (5) under the events E .

̸

where we have used the set of inequalities

Then using the lemma 10 and cumulative estimation ζ t defined in (115), we have

We note that the same results can be obtained for the unknown transformed mean setting.

F.6 Proof of Trade-off Upper Bounds (Theorem 3)

Proof: Based on the Theorem 6, the remaining is to balancing the two terms

while satisfying the condition

Hence, by setting where u t is defined as

which establishes the desired result.

We note that the same proof steps work for the unknown transformed mean setting with the change in u .

F.7 Refined Upper Bound for Causal Bandit

Graph-dependent Bound Under the causal bandit setting, since U ( Z ) = Z [ n ] , we eliminate all uncertainty in estimating the n -dimensional parameter θ . In fact, we have ζ t = 0 for all t ∈ [ N ] , so no forced exploration is needed. If we set ζ t = 1 (or set weight matrices to be I n ), by applying Lemma 11 in Section F.4 together with the concentration inequality in Lemma 7, we immediately obtain our regret upper bound with high probability. From (209), we have with probability at least 1 -2 δ , the regret bound of modified RO-CRL for CB is

Graph-independent Bound To get a graph-independent bound that corresponds to the maximum in-degree d G and the maximum length of a causal path L . To match the corresponding lower bound that is in the unknown transformed mean setting, we show the bound for u H ,N in this region. In particular, we have

As we have the fact

So we obtain

Hence, the regret bound for the modified RO-CRL for CB is

G Proofs of Lower Bounds

Equivalent definition of p . An alternative definition of p is p = 1 + ∑ n i =1 ( p i +1) and p i is the number of causal path from noises to node i , which is defined recursively as follows (on G tr ).

On G :

On G tr :

G.1 Graph-dependent Lower Bound for Causal Bandit

Theorem 7. For any given graph G there exists a causal bandit instance on G such that the expected regret of any causal bandit algorithm is at least

where p n is defined in (219) on G .

Let Π be the set of all policies on the set of stochastic bandit environments I , which contains all the possible bandit instances sharing the same DAG G and satisfying the conditions. The minimax regret is defined as

where [ R T | π, I 0 ] denotes the expected regret of policy π on the bandit instance I 0 . We will consider a set ˜ I , instead of I , that contains two bandit instances. By definition of minimax regret, a lower bound for the regret of any policy on ˜ I is also a lower bound for the minimax regret since

Following this property, the central idea of the proof is as follows. Consider two linear SEM causal bandit instances that differ by a small fraction and are hard to distinguish. At the same time, we can construct them to have different optimal interventions, indicating that a selection policy cannot incur small regret for both at the same time under the same data realization. Note that the difference of the rewards, or equivalently the regrets, observed by these two bandit instances under the same intervention can be computed by tracing the effect of the differing edge parameter over all the paths that end at the reward node.

We consider two linear SEM causal bandit instances I, ¯ I ∈ I 0 that is parameterized by I ≜ { B , B ∗ , ε } and by ¯ I ≜ { B , B ∗ , ¯ ε } . We note here that we assume the mean of the noise is conditionally independent, that is, it can be dependent on the intervention. We note that the algorithm discussed previously can work under this setting without any modifications. For each node i ∈ [ n ] in I , we have where Bern( q ) denotes the Bernoulli random distribution with probability q . The noise is reversed in the second bandit instance ¯ I , which is

This is where the difference between the two bandit instances lies: if ( i → j ) is an edge in the graph, we have

Except for this, all the rest are the same. Next, consider a fixed bandit policy π that generates the following filtration over time

The decision of π at time t is F t -1 -measurable. Accordingly, define P t and ¯ P t as the probability measures induced by F t by t rounds of interaction between π and the two bandit instances I and ˜ I . When it is clear from context, we use the shorthand terms P and ¯ P for P T and ¯ P T , respectively. We will show that π cannot suffer small regret in both instances at the same time and under the same filtration F T .

By Lemma 1, since all the elements of observational and interventional weights are non-negative, the optimal intervention is the one that maximizes the expected value of each noise. The optimal action between two bandit instances only differs. This means optimal intervention for I is a = ∅ and that for ¯ I is a = [ n ] . Define E i lb as the event in which the decision on node i is sup-optimal at least T 2 times after T rounds on bandit instance I , i.e.,

We note that the event E i lb is defined on the σ -algebra defined by the filtration F t , that induces both P t and ¯ P t . We compute the expected instantaneous regret when node i is chosen sub-optimal in the first bandit instance, and the total regret is the summation over these nodes. Note that each path passes a node that node i contributes to the expected regret. Furthermore, since every weight is positive, in I , when a suboptimal action is chosen, the impact on average regret is determined by the number of paths that start from the node i and end at the reward node n . Then, by the definition of E lb , we have

where (230) holds as we break down the regret and (231) holds due to the definition of E j lb in (228). Similarly, for ¯ I , each node i that is not intervened, it will occur at least δ ( m i,n +1) regret. Applying the same steps as in (229),-(231), we obtain

By combining (231) and (234) we have

Next, we characterize a lower bound on P ( E i lb ) + ¯ P ( E i, c lb ) for i ∈ [ n ] , which involves the KullbackLeibler (KL) divergence between P and ¯ P , denoted by D KL ( P ∥ ¯ P ) . For this purpose, we leverage the following theorem.

Theorem 8 (Bretagnolle-Huber inequality) . Let P and ¯ P be probability measures on the same measurable space (Ω , F ) and let A ∈ F be an arbitrary event. Then,

By invoking Theorem 8, from (235) we obtain

It remains to compute exp( -D KL ( P ∥ ¯ P )) to conclude our proof, for which we leverage the following result.

Lemma 12. The KL divergence between P and ¯ P , the probability measures induced by F t on I and ˜ I , is equal to

Proof: Note that a Bayesian network factorizes as

Additionally, the two bandit instances differ only in the mechanism of the first layer. Then, D KL ( P ∥ ¯ P ) can be decomposed as

By noting that the KL-divergence between two Bernoulli random variables with probabilities p and q is given by

Since give Z [ pa ( i )] , the Z [ i ] under P and ¯ P are both shifted Bernoulli random variables. From the above, we obtain for node i ∈ [ n ]

where the last inequality holds since ( 1 2 + δ ) log(1+2 δ )+( 1 2 -δ ) log(1 -2 δ ) < 1 2 log ( 1 (1+2 δ )(1 -2 δ ) ) for 0 < δ < 1 / 2 . And hence we have

If we choose δ = 1 √ T to balance the terms in the lower bound, we obtain

for T ≥ 5 , the term ( 1 -4 T ) Tn/ 2 is an increasing function of T . Hence, for T ≥ 5 lower bound ( 1 -4 T ) Tn/ 2 ≥ 0 . 2 2 . 5 . By setting c = 1 8 × 0 . 2 2 . 5 , we have

, we have the

G.2 Graph-independent Lower Bound for Causal Bandit

To get a graph-independent bound that corresponds to the maximum in-degree d G and the maximum length of a causal path L . When L = 0 and d > 1 , we have

̸

As we have the fact that

So we obtain

Hence, the regret bound for the modified RO-CRL for CB is

G.3 Proof of Reward-oriented CRL Lower Bound (Theorem 4)

In Section G.1, we have shown a lower bound of Ω( p n √ T ) for the causal bandit. A similar result can be obtained for reward-oriented CRL as follows.

Corollary 2. When having the knowledge of G and H ∞ , there exists a causal model instance on G tc such that the expected regret of any algorithm is at least

where now p is defined on transitive closure G tc .

Proof: We set θ = 1 n for both instances I and ¯ I . The proof then proceeds almost the same as for Theorem 7, with the only modification being the way interventions affect the utility. In this setting, each path terminating at node j ∈ [ n ] contributes to the overall utility. Hence, selecting the suboptimal intervention at node ∑ j ∈ [ n ] ( m i,j +1) . As a counterpart of (237), we have

where the last formulation provides the p in the lower bound

Now we have shown the regret lower bound under the perfect scenario on the error E t = 0 . Now we construct two more instances of causal models with E t occurring adversarially. We construct two instances of the causal model on G tc and demonstrate that under specific deviations, no algorithm can distinguish between them and the initial stage.

Let us examine the parameterization of the two causal models, referred to as I ′ = { B , B ∗ , ε } and ¯ I ′ = { B , B ∗ , ¯ ε } . For the existing edges in graph G tc , ( i, j ) for i < j and i, j ∈ [ n ] , we define

For the noises, for I ′ and ¯ I ′ we define

where ε 0 is a constant that is defined later. And we define the parameters θ = 1 n for both. Thus, the only difference between the two bandit instances lies in the mean of the noises. In the first causal model, the optimal action is when all the nodes are intervened . In contrast, in the second causal graph model, the best action is associated with all the nodes being not intervened .

We first consider any algorithm with forced exploration (or an under-sampling rule). Consider for time horizon T that the algorithm forced each intervention in A 0 to perform N T times, the Theorem 2 provides the upper bound for the estimation error

We notice ε 0 controls the scaling of the system, and hence, controls the scaling of estimates and constants. By setting ε 0 such that ∥ C ∥ 2 = ∥ E t ∥ = O (√ d N T ) , we have ε 0 = O (√ d N T ) . We can set E t adversarially as

In such cases H t = 0 , so that all estimates ˆ Z t = 0 , no information is posted to the learner, and the learner cannot distinguish between these two instances. Consequently, there must exist a bandit instance at which the algorithm plays the sub-optimal choice on each node i at least T/ 2 times. We note the sub-optimal intervention blocks the causal flow from an ( i ) to de ( i ) ∪ { i } to the reward. Hence, we have the following reward decomposition

where de ( i ) is the descendants set of node i and we note the term 1 { k ̸∈ a t } is used to avoid counting any path more than once. By dropping the non-negative term associated with the 1 { k ̸∈ a t } , we can lower bound the regret by

Similarly, we have

By combining (268) and (269), we obtain

Hence, among these two causal graph instances, there will be at least one instance that incurs a regret of

At the same time, the forced exploration period will incur a regret of order nN T . Combining the two we have

Lastly, we need a trade-off between the two. By setting N T to be on the order

This balances the two terms and provides the result.

We note that the reward-oriented CRL method without forced exploration cannot leverage Theorem 2 to obtain meaningful regret bounds, as at all times we can have H t = 0 and the regret scales linearly with T under the setting that ε 0 = 1 .

Finally, combining the results in (259) and (274), we know that there exist a causal graph instance in I , ¯ I , I ′ and ¯ I ′ such that

H Multiple Interventions

In this section, we state what changes are needed to adapt RO-CRL for multiple interventions per node. We focus on soft interventions , where hard interventions can be viewed as a special case of this. To maintain consistency, we redefine several key terms.

H.1 Latent Data-generating Process and Intervention

Consider the case where there are s possible distinct intervention mechanisms on each node i ∈ [ n ] . We denote [ B k ] i as the weight for intervention k at node i , and define B k as { B k = { [ B k ] i | i ∈ [ n ] } , the full set of such weights. We reserve B 0 = B for observational weights. Similarly, we can define ν k for k ∈ { 0 }∪ [ s ] . And we define the intervention as a vector a ∈ R [ n ] in this case as a [ i ] ∈ { 0 }∪ [ s ] . The intervention space has cardinality of |A| = ( s +1) n instead of 2 n . Under intervention a ∈ A , the Z follows the SEM

where we have

H.2 Algorithm Modification

Theorem 2 requires only one intervention per node, and its sample-complexity and error bounds remain unchanged. Hence, the CRL component of the algorithm requires no modification.

For UCB selection,*we let** [ A k t ] i **denote the robust estimate for each intervention k ∈ { 0 } ∪ [ s ] as

where we have defined the weighted and doubly weighted Gram matrices as

Weight designs. The diagonal elements for weight matrices are defined as

Confidence ellipsoids. After performing estimation in each round, we construct the following confidence ellipsoids for k ∈ [ s ]

H.3 Changes in the Regret Bounds

Theorem 9 (Regret upper bound) . Under Assumptions 1-3, with probability at least 1 -4 δ , the average cumulative regret of RO-CRL is upper bounded by

where we set u = u S for soft interventions and u = u H for hard interventions..

The estimates θ t for θ and the confidence ellipsoids C θ,i remain the same. We will skip some unimportant parts and focus on the changes under this setting. We refer the reader to [23] for detailed steps. We notice the change mainly due to the changes in the following lemma and how we choose the confidence levels. We note that when m = 1 , the theorem reduces to the setting discussed in the main paper.

Now we discuss the changes needed for the proof steps discussed in Section F.

First, Lemma 6 and the discussion in Section F.1 still hold. Second, Lemma 7 in Section F.3 holds for all k ∈ [ s ] with mild change of

Now, we modify Lemma 10 to the following lemma.

̸

Lemma 13. If ˆ G t = G or ˆ G t = G tc for all t ∈ [ N ] , and [ A ] i ∈ C i,t and [ A ∗ ] i ∈ C ∗ i,t for all t ∈ N and i ∈ [ n ] and θ t ∈ C θ,t for all t ∈ N , then we have

Proof: The changes to the proof are mainly in the Base step , where we aim to prove the following. For node i ∈ [ n ] with causal depth L i = 1 , we show that

where B ′ is defined as and E 1 is defined as

where we define

When the causal path of a node is L i = 1 , according to SEM defined in (93), we have the following expansion:

Then, the weights we defined are bounded in the range [ 1 ζ t ˆ m , 1 ζ t ] for t ∈ N . Leverage these bounds if we define the following constants. Note, we drop the index t for these constants, but they are a variable of time.

In order to proceed, we need upper and lower bounds for the maximum and minimum singular values of U i, a t ,t . These bounds depend on the number of non-zero rows of U i, a t ,t matrices, which equals the values of the random variable N i, a t ,t . Let us define the constant

Then for every t ∈ [ T ] , and n ∈ [ t ] , we define the error events corresponding to the maximum and minimum singular values of U i,t and ˜ U i,t as

Similarly, we can define E ∗ i,n,t and ˜ E ∗ i,n,t by replacing N i,t and U i,t (or ˜ U i,t ) by N ∗ i,t and ˜ U i,t (or ˜ U ∗ i,t ), respectively.

As a result of [23, Lemma 8], if we define the union error event E i, ∪ as

Bounding term E [ 1 {E i, ∪ } ∑ T t =1 λ i,t ] . Same as [23], we use the fact that ζ t increase with time t and obtain

Bounding E [ 1 {E c i, ∪ } ∑ T t =1 λ i,t ] . We define the function h ( x ) as

And we define the g function when x > τ as follows.

We have the following theorem to show the monotonicity and relation of h ( x ) and g ( x ) .

Lemma 14. [23, Lemma 10] h ( x, ζ t ) and g ( x, ζ t ) are both decreasing functions of x when x > τ and h ( x ) < g ( x ) , where τ is defined as α 2 ˆ m 6 κ 2 min .

Now we are ready to bound the last term

We define the set of time indices at which the chosen actions are under-explored as

It can be readily verified that |H i | ≤ 8 τ . Furthermore, when x ∈ H i , we have

Then we can bound the summation when H i occurs as follows.

Now we only need to bound the remaining part when t ̸∈ H i

Note that when t ∈ H c i , we have N i, a t ,t > τ and

We have

Now we discuss the major changes. We define the number of times that node i is under mechanism k ∈ { 0 } ∪ [ s ] at time t ∈ [ N ] as

Using the above results and noting that H c i excludes those samples from initial rounds, we obtain

We bound the discrete sums through integrals and define

Since g ( x ) is a positive, decreasing function, for any k ∈ N , k ≥ 4 τ +1 we have

Then, the summation in (318) is upper bounded by

Since g ( x ) is positive and decreasing, and G ( y ) is defined as an integral of the g function with a positive first derivative and negative second derivative, it can be deduced that G is a concave function. Thus, we have

Next, we proceed to establish an upper bound for the function G , which can be upper bounded as

where (324) is due to the inequality √ x + y ≤ √ x + √ y and √ x -y ≥ √ x -y √ x when x ≥ y , and we use closed-form integral and discard positive terms in (325). Combining the results in (307), (313), and (325), let E 1 denote the accumulation of terms that exhibit at most logarithmic growth rates with respect to T and ζ T .

Therefore, the final result for the bound is

And the rest of the proof remains almost the same, with only a difference in confidence constants.

I Remark on do interventions

In this section, we discuss the results for do interventions, a further restricted type of intervention. It is a subclass of hard interventions in which intervention on node i removes both ancestral connections and the randomness of Z [ i ] and sets the noise variable ε ∗ [ i ] to a fixed known value.

Remark 4 ( do interventions) . Under do interventions, there exists a set of n +1 interventions such that getting one sample from each suffices to construct H ∞ for which H ∞ G is a full-rank diagonal matrix. The resulting regret bounds depend on the causal model assumptions. For instance, in the standard setting where each latent variable Z [ i ] ∈ [ k ] for i ∈ [ k ] with intervention space A = [ k ] n and the conditional distributions p i ( Z [ i ] | Z [ pa [ i ]]) are arbitrary, all latent variables contribute to the utility node. In this case, the problem reduces to a multi-armed bandit, with a matching regret bounds scale as Θ( √ k n T ) even without the latent recovery.

J Broader impacts, Limitations and Further Discussions

This paper is purely theoretical, and the experiments only use synthetic data, so we are not aware of any negative societal impacts. Our principal limitation is the assumption of linearity. Specifically, we assume linear structural-equation models, linear transformation models, and linear utility functions. However, these assumptions are standard in both causal bandit and causal representation learning work under soft interventions, with the non-linear setting remaining largely unexplored. Extending reward-oriented CRL beyond linear models, therefore, remains an important avenue for future research.

We note that even in the simpler contexts (e.g., CRL or CB), the linear versus nonlinear models are investigated extensively. Here, we outline how each component can be individually generalized. To provide a concrete methodology for extensions, we first note that our current analysis consists of two parts: (i) analyzing finite-sample guarantees of a CRL algorithm, and (ii) analyzing how the CRL guarantees translate to reward guarantees. We discuss how the same decomposition can address nonlinear models as well:

Latent SEM: It is possible to generalize the algorithms and performance guarantees to nonlinear SEMs. While the overall RO-CRL framework and pipeline remain intact, such a generalization has two distinct implications: one for the CRL guarantees and another for the CB guarantees. On the CRL side, the subroutine can be replaced with methods designed for general SEMs that also provide finite-sample guarantees (e.g., [4]). As a result, the performance guarantees stated in Theorem 2 would need to be adjusted accordingly. On the bandit side, the UCB rule and reward estimators can similarly be replaced with their counterparts developed for nonlinear causal bandits (e.g., [20, 24]). Such generalization, however, is highly non-trivial and even in the simpler contexts (e.g., standalone CRL and CB), the linear and nonlinear settings are investigated separately, each with its exclusive technical challenges.

Transformation: There are existing results for CRL under nonlinear transformations [1012]. However, finite-sample analysis of CRL under general transformations is still an open problem. Nevertheless, we note that any future finite-sample result can be integrated into our RO-CRL framework to extend it to nonlinear transformations: Given that the estimator provides a graph and variables recovery error bound similar to Theorem 2, these estimates and error bounds can be used by the robust causal bandit algorithm. Similar to SEM generalization, as long as the setting does not violate the CB assumptions, the overall pipeline would work.

Utility: Our utility-function assumption is a direct extension of the latent SEM one. Specifically, the framework can be modularly generalized to accommodate non-linear utilities when adopting the robust non-linear estimates in bandit settings. The key open question, therefore, is the same one that arises for latent-SEM causal-bandit methods: How robust are these algorithms to violations in the underlying variable?

In all three cases, the structure of the analysis, error control in CRL, followed by robust bandit optimization, remains valid. The main requirement is that the substituted components provide compatible error guarantees.

Potential use cases for RO-CRL . Finally, we list some domains where latent interventions are plausible and RO-CRL could be applied.

Robotics: Consider a robot operating in an environment and observed through highdimensional sensory data such as images. Interventional CRL enables recovery of interpretable latent factors (e.g., joint angles) that causally generate visual data. CRL is particularly suitable for robotics in vision-based contexts because high-dimensional observations (images) obscure the causal variables and robots have known or assumed causal structure (e.g., a joint causes an end-effector position). In robotics, interventions on the latent variables can be achieved physically or via simulation. Physical interventions include, for instance, directly changing a joint angle or velocity (e.g., move joint 3 by +5 ◦ ) and moving the robot gripper to a fixed position while leaving other joints free. In this context, an example downstream objective could be moving the robot arm to a specific configuration using only the utility values and learned latent factors as control input.

Genomics: In gene regulatory networks, we observe high-dimensional expression data, typically measured from bulk or single-cell RNA sequencing. These arise from lowerdimensional latent variables representing unobserved biological drivers such as transcription factor activities, regulatory protein states, or signaling pathway activations. CRL is a natural fit for gene regulatory analysis because gene expression is influenced by a structured, latent causal process involving transcriptional and post-transcriptional regulation. Standard representation learning techniques fail to recover biologically meaningful variables, often mixing causal and non-causal factors. Node-level interventions in the CRL framework correspond to perturbing individual latent regulatory variables. In gene networks, such interventions are biologically realizable via gene knockouts or CRISPR interference to silence specific genes or transcription factors and overexpression systems to activate regulators, to name a few. In this context, an example downstream objective could be to optimize a certain type of biological response over the set of some possible genetic modifications.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a new reward-oriented causal representation learning framework where the goal is to learn a latent representation well enough for some downstream task. The framework is formally described for soft and hard interventions, with corresponding identifiability guarantees under a linear SEM. \n\nThe RO-CRL algorithm is then presented as a method for recovering latent variables and discovering the optimal interventions. The algorithm first tests each atomic intervention a few times to get initial estimates of the inverse of $\\mathbf{G}$ and $\\mathcal{G}$. Using the estimates and reward, the next intervention is selected according to a UCB-based rule. The authors then provide regret upper bounds for their algorithm and a general lower bound on CRL algorithms, showing that their method is close to optimal.\n\nSTRENGTHS AND WEAKNESSES:\nWith the caveat that it is possible I misunderstood some parts of the submission as it was quite technical:\n\n**Strengths**\n- The paper is well-structured, though could benefit from additional diagrams to better explain the algorithm/idea.\n- The idea is novel and interesting, potentially sparking a new direction for future CRL research.\n- The paper contains significant theoretical contributions consisting of both novel identifiability results and new bounds.\n- Even without the representation learning part, the work contains state-of-the-art regret bounds for the causal bandit setting.\n\n**Weaknesses**\n- The motivation behind the idea could be clearer. While the notion of concentrating on the downstream task instead of perfect identifiability makes sense, it is unclear in what scenarios you'd be able to intervene on every single node in the causal SEM but not have access to the latent variables. In a sense, being able to intervene in this manner implies knowledge of at least what the latent variables are.\n- The algorithm could use further experimental evidence.\n- The restriction to linear SEM, transformation models and utility functions is somewhat restrictive.\n\nThere is a large number of typos/grammatical mistakes (e.g. just from the first two pages):\n- \"to the extended needed\" -> \"to the extent needed\"\n- \"Aiming at establish the possibility/impossibility results\" -> \"Aimed at establishing possibility/impossibility results\"\n- \"enable more improved\" -> \"enable improved\"\n\nQUESTIONS:\n- Given the above, what would be an example of a real world scenario where we can intervene on each of the nodes in the causal graph but we still need to learn the latent variables occurs?\n  - In the robot example given in the introduction, it feels like if we are able to intervene on each of the joint angles, then we'd have access to their values.\n- How does the recovery of latent factors compare with the normal CRL setting?\n- How computationally expensive is the algorithm?\n\nIt is clear that considerable theoretical work was done. If the motivation and the setup were made clearer, I would be willing to increase my score.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-20"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel framework called Reward-oriented Causal Representation Learning (RO-CRL), which rethinks the goal of causal representation learning (CRL). Rather than aiming for perfect identification of latent variables and their causal relations, the authors propose to learn the minimal representation sufficient to optimize a downstream reward function. The paper formalizes this notion in the context of linear structural equation models (SEMs) and linear transformations from latent to observed space, with linear utility functions. A key contribution is a regret-minimizing algorithm that adaptively explores interventions to identify the optimal intervention with respect to the unknown reward. The algorithm is analyzed under both soft and hard interventions, achieving nearly tight regret bounds. The paper also presents finite-sample identifiability results and shows that the proposed framework generalizes and improves upon causal bandit methods.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper demonstrates high quality in its theoretical development, offering a detailed analysis of the RO-CRL algorithm. The manuscript is overall well-written. However, the structure of the paper is somewhat unusual and a bit confusing. For instance, the notation is explained at the end of the introduction.  Regarding significance, the shift from complete identifiability to task-oriented sufficiency is relevant, addressing common limitations of traditional CRL approaches when deployed in real-world decision-making systems. The originality of the work lies in formalizing the objective of utility-maximizing representation learning within a causal inference framework and in developing a regret-minimizing algorithm that explicitly avoids unnecessary modeling detail.\n\nHowever, there are some limitations. The main restriction is the reliance on linearity assumptions across the SEM, transformation, and utility function, which may reduce applicability to real-world domains. While these assumptions simplify the analysis, a discussion on potential extensions to more general settings would be relevant. The empirical evaluation is limited and relegated to the appendix, which makes it harder to assess the practical effectiveness of the method. Overall, there is a gap between theory and empirical interpretation.\n\nQUESTIONS:\nHave the authors considered how the RO-CRL framework could be extended to non-linear SEMs or transformations? Even a qualitative discussion on potential challenges or future directions would strengthen the broader relevance of the work.\n\nThe paper would benefit from a more visible main-text discussion of empirical results, perhaps with illustrative toy examples that demonstrate key phenomena (e.g., how coarse representations suffice for specific rewards). Are the empirical claims consistent across different intervention types and graph structures?\n\nThe algorithm relies on known bounds for parameter magnitudes and noise levels. How sensitive is RO-CRL to violations or inaccuracies in these bounds?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles the problem of learning causal representations (causal graphs and structural equations) in a setting where the goal is not to learn a representation that inverts the data generating process, but that can accurately model which interventions maximize a reward. The analysis is restricted to the setting where both the causal and reward models are linear. An algorithm for inference of the latent parameters in the limited time horizon is derived, as well as associated regret bounds. Some preliminary experimental results are obtained to validate the method.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n- This paper formulates a novel and interesting problem of learning causal representations with the goal of maximizing a reward instead of inverting the data generating process. This could have important theoretical and practical applications\n- The theoretical analysis seems impressive and generalizes some existing results. It importantly operates in the finite sample setting\n\n## Weaknesses\n- I would like to see some experimental validation of the regret bounds. Figure 4 does suggest sublinear regret, but it would be great to have a more detailed analysis\n- It would also be interesting to have experimental comparison with other appropriate bandit algorithms\n- The paper is clear, but a bit heavy to read. I suggest trying to include a figure that explains the main algorithm. This could greatly help understanding\n- There is also little discussion of practical examples of applications of this framework. I suggest trying to include some to ground the discussion and highlight possible uses.\n\nQUESTIONS:\n- In line 31, a large number of papers from the same authors are cited. Please be careful about self-citation and try including citations from a more diverse set of authors\n- As a suggestion for readability, I strongly suggest not using bold typeface to put emphasis, it is distracting for the reader. I suggest using italics if necessary\n\nLIMITATIONS:\n- Limitations should be discussed in the main text and not in the appendix", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work considers the problem of causal representation learning from a reward-oriented perspective, where, contrary to prior works which optimize for perfect identifiability, they optimize for the coarsest level of identifiability for a specific downstream task. Focuses on the specific task of identifying an optimal set of interventions as specified by a linear utility function in terms of the latent variables. Considers linear SEMs with linear transformations to the latent space. Provides an algorithm, RO-CRL, for the reward-oriented CRL task under hard and soft interventions and specifies regret bounds for both settings.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. This work addresses a key gap in CRL literature by developing methods that achieve identifiability only to the extent necessary for maximizing performance on downstream tasks. Current works focus primarily on perfect identifiability, requiring an excessive and arguably unrealistic number of interventions, thus making this work potentially more applicable.\n2. All proofs and theoretical findings seem correct.\n3. Paper was generally well written and enjoyable to read.\n\nWeaknesses:\n1. My primary concern is that the paper claims that prior works focused on perfect identifiability employ an ‘excessive’ number of interventions to achieve perfect identifiability; however [3] (as cited in the manuscript) indicates that a single atomic intervention on each node is sufficient for identifiability, where RO-CRL enforces more interventions than that in just stage 1of the algorithm. This seems misleading and questions whether this work is really capable or discovering meaningful causal representations more efficiently.\n2. Stage 1a and 1b of RO-CRL seems to indicate the ability to take both hard and soft interventions as desired; however, given the interventions are on unknown latent variables, it seems restrictive to think that you could know entirely whether an intervention is hard or soft.\n3. The work includes very limited synthetic experimental results and no real-world experimental results. It would be very interesting to see a real world example of how your approach could learn representations geared for a specific downstream task.\n\nQUESTIONS:\n1. The SEM, transformation and utility function all being linear seems restrictive. Could these results be extended to any nonlinear setting?\n2. Is the dimension of the latent space assumed to be known a priori? If the causal variables are inherently latent, how could you know how many there are?\n3. Similarly, why can we assume that interventions on single latent variables aren’t actually affecting many latent variables?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces reward-oriented CRL, which shifts the focus from perfectly recovering latent causal representations to learning them only to the extent needed for optimizing a downstream reward function. The authors consider linear structural equation models (SEMs) with linear transformations mapping latent variables to observations, and linear utility functions. They develop the RO-CRL algorithm that adaptively explores interventions while maintaining confidence bounds, providing finite-sample identifiability guarantees and regret bounds of $\\tilde{O}(d^{1/3}n^{1/3}u^{2/3}T^{2/3} + u\\sqrt{T})$ for soft interventions, with nearly matching lower bounds.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n**Quality:**\n- The paper provides rigorous theoretical analysis with finite-sample PAC identifiability guarantees (Theorem 2) that extend beyond typical asymptotic results in CRL\n- The regret bounds are nearly tight, with upper bound $\\tilde{O}(d^{1/3}n^{1/3}u^{2/3}T^{2/3} + u\\sqrt{T})$ and lower bound $\\Omega(d^{1/3}n^{1/3}p^{2/3}T^{2/3} + p\\sqrt{T})$ differing mainly in graph-dependent constants\n- The algorithm design cleverly balances CRL objectives with bandit optimization through weighted ridge regression and UCB-based selection\n- As a special case, the framework improves existing causal bandit bounds from $\\tilde{O}(d^{3/2}L\\sqrt{T})$ to $\\tilde{O}(d_G^L\\sqrt{T})$\n\n**Clarity:**\n- The paper is well-structured with clear motivation for task-oriented CRL\n- The algorithmic components (forced exploration, adaptive exploration with CRL rules, UCB selection, and under-sampling) are logically presented\n- Mathematical notation is consistent and appropriately defined\n\n**Significance:**\n- Addresses a fundamental gap between CRL's goal of perfect recovery and practical needs where coarse representations suffice\n- The framework naturally connects CRL and causal bandits, two previously disparate areas\n- The finite-sample analysis is particularly valuable for practical applications\n\n**Originality:**\n- The reward-oriented perspective on CRL is novel and well-motivated\n- The technical approach of using precision matrix differences for simultaneous graph learning and intervention optimization is creative\n\n## Weaknesses\n\n**Quality:**\n- The restriction to linear SEMs and linear transformations is certainly limiting\n- The assumptions (particularly Assumptions 1-3) are somewhat strong: bounded noise, known ranges for parameters, and intervention effects on weight matrices.\n- The dependence on the uncertainty parameter $u$ in the bounds is not well-characterized in terms of problem instances\n\n**Clarity:**\n- The relationship between $u_S$, $u_H$, and $p$ in the bounds could be explained more intuitively\n- The weight matrix design in equations (24)-(26) appears ad-hoc and could benefit from clearer justification\n- The transitive closure recovery under soft interventions vs. exact graph recovery under hard interventions needs better exposition\n\n**Significance:**\n- The improvement from $u$ to $p$ in bounds may not be substantial in many graphs where these quantities are similar\n\n**Originality:**\n- While the reward-oriented perspective is novel, the technical tools (precision matrices, ridge regression, UCB) are somewhat standard\n\nQUESTIONS:\n1. Could the precision matrix approach generalize using kernel methods or neural approximations for the nonlinear setting?\n\n3. How does $u$ scale with graph properties in typical instances? Can you provide examples where the gap between $u_S$ and $p$ is large vs. small? This would help understand when the bounds are truly tight.\n\n4. What happens when Assumption 1 is violated (i.e., interventions only affect noise)? Could the algorithm be modified to handle or detect such cases?\n\n5. What is the per-round computational complexity of RO-CRL? The eigendecomposition in equation (11) and matrix inversions throughout seem expensive for large $d$?\n\nNot a question but the work of Chen et al. NeurIPS'24. \"Identifying General Mechanism Shifts in Linear Causal Representations\" seems relevant as additional CRL literature. There, the authors indentify general interventions under a similar linear CRL setting.\n\nLIMITATIONS:\nI have only skimmed over the appendix so far. The authors could discuss further:\n- Computational scalability concerns for high-dimensional observations\n- Sensitivity to assumption violations (especially bounded noise and known parameter ranges)", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-12"}