Variance-Reduced Long-Term Rehearsal Learning with Quadratic Programming Reformulation
Authors: Wen-Bo Du
Tian Qin
Tian-Zuo Wang
Zhi-Hua Zhou
Venue: NeurIPS 2025 poster ()
Keywords: decision-making
probabilistic optimization
structural model
Primary area: probabilistic_methods
TL;DR: We present the first long-term rehearsal learning approach, which demonstrates favorable properties such as variance reduction and optimality.

Abstract: In machine learning, a critical class of decision-making problems involves *Avoiding Undesired Future* (AUF): given a predicted undesired outcome, how can one make decision about actions to prevent it? Recently, the *rehearsal learning* framework has been proposed to address AUF problem. While existing methods offer reliable decisions for single-round success, this paper considers long-term settings that involve coordinating multiple future outcomes, which is often required in real-world tasks. Specifically, we generalize the AUF objective to characterize a long-term decision target that incorporates cross-temporal relations among variables. As directly optimizing the *AUF probability* $\mathbb{P}_{\operatorname{AUF}}$ over this objective remains challenging, we derive an explicit expression for the objective and further propose a quadratic programming (QP) reformulation that transforms the intractable probabilistic AUF optimization into a tractable one. Under mild assumptions, we show that solutions to the QP reformulation are equivalent to those of the original AUF optimization, based on which we develop two novel rehearsal learning methods for long-term decision-making:
    (i) a *greedy* method that maximizes the single-round $\mathbb{P}_{\operatorname{AUF}}$ at each step, and 
    (ii) a *far-sighted* method that accounts for future consequences in each decision, yielding a higher overall $\mathbb{P}_{\operatorname{AUF}}$ through an $L/(L+1)$ variance reduction in the AUF objective. We further establish an $\mathcal{O}(1/\sqrt{N})$ excess risk bound for decisions based on estimated parameters, ensuring reliable practical applicability with finite data. Experiments validate the effectiveness of our approach.

Variance-Reduced Long-Term Rehearsal Learning with Quadratic Programming Reformulation
Wen-Bo Du, Tian Qin, Tian-Zuo Wang, Zhi-Hua Zhou
National Key Laboratory for Novel Software Technology. Nanjing University, China School Artificial Intelligence, Nanjing University, China {duwb, qint, wangtz, chouzh]a@lmnnn .nju. maj..edu.com edu. cn
Abstract
In machine learning, a critical class of decision makiing problems involves Avoiding Undesired Future (AUF): given predicted undesired outcome, how can one make decision about actions to prevent it? Recently, the rehearsal learning framework has been proposed address AUF problem. While existing methods offer reliable decisions for single-round success, this paper considers long-term settings that in- volve coordinating multiple future outcomes, which is often required in real-world tasks. Specifically, we generalize the AUF objective to characterize long-term de- cision target that incorporates tross-temporal relations among variables. As directly optimizing the AUF probability P. PAUF over this objective remains challenging, we derive an explicit expression for the objective and further propose a quadratic programming (QP) reformulation that transforms the intractable probabilistic AUF optimization into tractable one. Under mild assumptions, we show that solutions to the QP reformulation are equivalent to those of the original AUF optimization, based on which we develop two novel rehearsal learning methods for long-ter term decision-making: making: (i) greedy method that maximizes the single-round IP PAUF each step, and (ii) a
method that accounts for future consequences each decision, yielding higher overall P PAUP through an (.( variance reduc- tion in the AUF objective. We further establish an O(1 / N) excess risk bound for decisions based on estimated parameters, ensuring reliable practical applicability with finite data. Experiments validate the effectiveness of our approach.
Introduction
Machine learning (ML) methods have demonstrated remarkable success in diverse real-world predic- tion tasks [1]. Instead of solely focusing on the prediction, Zhou [2] emphasizes another important issue, i.e., if an ML model predicts that something undesired is going to occur, how to find effective actions prevent it from happening. This problem is known as avoiding undesired future (AUF) [2].
Consider portfolio management system an example, which employs MI. model using economic indicators (X) to predict monthly portfolio returns (Y). If a predicted return is undesirably low, an ideal system would have the ability of suggesting alter allocation weights (Z) across asset classes (c.g., stocks, bonds) to enhance profitability. Due to expensive transaction costs, alterations must be carefully justified. Let denote the desired region specified by Dicision-makens AUF can be framed as finding alterations z that maximize the probability of Y S. In real-world tasks, decisions are often temporally coupled, e.g., aggressive risk- risk-seeking might boost short-term returns while increasing future vulnerability. This motivates more practical AUF goal: finding a sequence of alterations (z, ...., 2f41.) to maximize the probability that the aggregated outcome Y e T Σ, Y; falls within S. This long-term formulation captures cross-period trade-offs, promotes sustainable performance, and encompasses the single-round setting as a special case (L = 0).
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Generally, the AUF probability PAUr(-) mea- sures the likelihood that- falls into the desired region& S conditioned on the selected alterations. However, the dependence of PAUF on alter- ations is typically inaccessible and highly com- plex, particularly in long-term settings, which makes the identification of effective decisions challenging. In such cases, leveraging the struc- tural relations among variables {X, Z, Y be- comes essential. While correlation is often sufficient for prediction, it is generally inadequate for guiding alterations [2]. Causal relations, in contrast, would be more informative, but identifying them usually requires restrictive and untestable assumptions and may involve factors that are not actionable [3, 4]. Recognizing that correlation is insufficient and causation is somewhat too luxu- rious to decision-making Zhou [5] propose the irfluence relation as the foundational concept for decision-makinn making, which different from correlation and causation, depicted in Fig. I; and propose to study rehearsal learning approaches to identify and exploit influence relation.
Correlation
Prediction
Influence
Decision
Actionable
Causation
Discovery
Relations between correlation, causation, and influence (reproduced from Zhou [2]).
Figure I:
Building on this, rehearsal learning approaches I6. 7] have been developed and been shown effective in single-round AUF, where the objective is to ensure Y E S for immediate outcomes. These methods typically leverage distributional properties of Y to reformulate the problem identifying alterations that satisfy a constraint P PAUF 2 т, thereby circumventing the intractable probabilistic optimization. Extending this idea into long-term settings, however, introduces several fundamental challenges: (i) Temporal dependencies among outcomes Y; across different rounds complicate the analysis of PAUF, as one must now account for the aggregation of correlated Y, rather than single outcome; (ii) Long-term decision-makinn round, since even small deviations can accumulate into substantial performance degradation over time; (iii) The dimensionality of alteration sequence grows with the horizon length, substantially increasing computational complexity. These challenges lie beyond the capabilities of existing approaches.
demands alterations that closely approximate the optimal IP PU at each
To enable the rehearsal learning framework in long-term AUF scenarios, new methods are demanded to address the aforementioned challenges. In this paper, we first generalize the decision objective to a long-term aggregated form, which subsumes cxisting formulations as special cases. Under this generalized formulation, we show that the aggregated objective can be explicitly decomposed into: (i) the cumulative effect of selected alterations and (ii) the aggregated noise unrelated to alterations, making it feasible to design decision-making strategies that bypass the explicit modeling of temporal dependencies among outcomes Y, Leveraging this decomposition, we propose a quadratic programming (QP) reformulation of the original intractable optimization. This reformulation yields a tractable optimization target, and scalable to long time horizons. We further establish theoretical guarantees for the optimality of the QP solution under mild assumptions, which effectively mitigates the error accumulation issues inherent in the aforementioned constraint- formulations. Notably, we make an independent contribution that our result applies to the class of log-concave noise distributions, which generalizes beyond the Gaussian noise assumption [6-8]. Based on the results above, we develop two new rehearsal-basee -based methods focused on long-ter outcomes: (i) a greedy approach that maximizes the single-round IP PAU at each step, outperforming existing rehearsal-based methods, and (ii) far-sighted approach that anticipates the future consequences of current decisions, consistently achieving a higher overall P PAUF by attaining an L/(L [ 1) variance reduction rate in the AUF objective. Finally, recognizing that true structural parameters are often unknown in practice, we establish an excess risk bound for decisions made using parameters estimated from observational data. Experiments validate our theory and demonstrate the effectiveness and efficiency of our approach.
Our contributions can be summarized as follows:
1. We generalize the AUF problem to a long- terg-termpp perspective and derive an explicit expression for the objective, offering greater practical flexibility while subsuming existing formulations.
2. We introduce QP reformulation, making the AUF optimization tractable and efficiently solvable. The optimality of the QP solution is then established under mild assumptions.
3. We develop two new rehearsal- algorithms based on the QP reformulation, and the far-sighted one can achieve better performance in long-term scenarios due to its variance-reduction property.
4. We establish an (1/vNN bound on the excess risk incurred when using estimated parameters instead of the true ones, guaranteeing reliable performance in practical scenarios.

---

2 Preliminaries
probabilistic graphical model, termed the structural rehearsal model (SRM), was proposed by Qin et al. [6] to characterize influence relations among variables in the AUF problem. The SRM comprises a set of rehearsal graphs and corresponding structural equations, denoted as {(G,, 0t)), where represents the decision round. Each graph G; (V E,) encodes the variables V. and induced edges E, at round I, while the parameter set 0, includes both the generating parameters associated with E, and the noise parameters related to V,. The formal definition of SRM is provided in Appx. B. Notably, existing research on rehearsal learning considers only the graph structure and influence relations within a single decision round, neglecting cross-round influences. That is, they assume no edges exist between Vi and V22 for 1 + tz.
In this work, we extend the SRM to multivariate time series setting {V vejo, that captures both lagged and contemporaneous influences within a finite time window K. Specifically, let E, denote all cross-tine time edges pointing into variables at time t, i.e. edges from V to V?² for all 0 S t' < and S i.j S [V]. The complete graph is then defined as GK (Ulg(o.x V., Uee (E, U Egross), as illustrated in Fig. 2. Gx models the qualitative influence relations among variables across different time steps, where V, represents the variable set the AUF problem and E, U Eggros captures the influence relations among variables at time t. There are two types of edges: (i) a directed edge Vi V2 (i + j < t) indicates that V unilaterally influences Vi'; whereas (ii) a bidirectional edge V + V2 (i + j) signifies mutual influence between V and Vi² For instance, birth rates unilaterally influence demographic structures, whereas supply and demand in an idealized market exhibit mutual dependence, as changes in one directly affect the other.
To quantitatively describe these influences, let θ = 0g U 0 denote the extened parameter set of G in time f, where gs corresponds to the parameters associated with Egross The structural equations governing the variables Vi can then be parameterized by O₁;
V = fl e/ for 0 s S K and 1 SSiSs S
(1)
where PAo A {u u - Vz in GK) represents parents of V/', and ey denote the noise. The structural function H. (-) and the probability density function (PDF) of ef are parameterized by O² In this work, we make the first-order Markov assumption of the time series for simplified expression (can be straightforwardly extend to general cases), under which we focus on a basic yet essential class of the AUF problem, characterized by stationary linear structural equations fl in Eq, (1), iee.
- AV, + BV. + Ec: for 1 K,
(2)
where A and B are determined by θ, and satisfy A) < 1 (p(.) denotes the maximal absolute eigenvalue) to ensure stationarity [9]; and where e, follows a white noise process with E(e,] A 0, and Ei A 0 for any t * s, otherwise an invariant Σ. this work, the noise is assumed to follow symmetric log-concave distribution, i.e., the PDF fv( is log-concave and satisfies fe(v) = f-vv for all v. This family includes many common zero-mean distributions, such as Gaussian, uniform, and Laplace. Finally, the decision-making task focuses on identifying suitable alterations R2)))), which disrupts existing influence links, as detailed in Appx. B.
Zx
2
2
Z
Zi
XL_
Y
X7
Y!
Z
Z
Z
Zt
x2_
Y2
X?
Y?
Z
ZZ
fine
Figure 2: An example illustrating variables and edges at rounds and t. Red edges indicate cross-round influence relations. The variable sets X, Z, Y are enclosed by dashed lines.
3

---

3 The proposed approach
In this section, we first present a generalized formulation of the AUF problem in Sec. 3.1, incorpo- rating long-term aggregation over target variables. In Sec. 3.2, we derive an explicit expression for the aggregated AUF objective w.r.t. the alteration sequence and propose a QP reformulation with theoretical guarantees of optimality, offering a new perspective for rehearsal learning. Building on this theoretical foundation, in Sec. 3.3, we propose two novel rehearsal based algorithms for solving the formulated problem. Finally, in Sec. 3.4, we establishe theoretical guarantees both the variance reduction and the excess risk. All proofs of the theoretical results are provided in Appx. C.
3.1 Formulation
We treat the AUF problem as a sequence of immediate decision decision-making processes during a certain time period t.... T' In each decision round, the decision maker should make timely decisions based on all available information; while the final decision target is to maximize the probability of Y E 8 within this time period. Each decision round involves two critical time points: the moment when the ML prediction made after observing of the concerned outcome Y, as illustrated in Fig. 2. Specifically, in given decision round f, the decision maker first observes X, x, and incorporates it into the historical dataset, defined
and the moment just before the generation
as D, A {xe} U {x,, 2f.yi yike.. The ML model then predicts the values of the target variables Y, Y YIT over a future horizon. Let S S R denote the predefined desired region for the
target variables Y, if the predicted values fail to satisfy the predefined criterion, Le.. Y & S, corrective decisions should be made to prevent undesirable outcomes. This pacision-making problem in time t can be formally expressed as the following probabilistic optimization problem:
HT
Y, E Dr, Rh(z,, ... Eirz)
arg max (m
P
EY,
(3)
s.t.
2 (Z) for i=t, ,
where T + 1 is the length of the remaining target time horizon specified by the decision-maker, A()) represents the feasible domain of each alteration 2 and the rehearsal operator Rh(-)) denotes the execution of the alteration sequence Z; - z for i = .....,ttt T. This operator modifies the local graph structure by disrupting specific influence links, as detailed in Appx. B.
Previous studies primarily focused distribution on the target probability [6, 10]. In this paper, we consider a more general formulation where the AUF objective is defined over aggregated target variables across time period, which can reduce to the prior setting. This generalization is commonly encountered in practical scenarios. For instance, when Y, denotes the annual GDP growth rate, the objective might be to maintain a high average rate over several consecutive years. Additionally, we generalize the noise distribution to symmetric log-concave family, which includes Gaussian, uniform, and Laplace distributions as special cases, thereby expanding the practical applicability.
simplified setting where T 0, and assumed Gaussian
The main challenges of the formulated problem in Eq. (3) are twofold: (i) The target variables Y, across time are generally dependent, making it substantially more difficult to characterize the distribution and probabilistic properties of their aggregation compared to the single-time case, particularly under non-Gaussian noise. (ii) The dimensionality of decision variables grows with the horizon length T, introducing additional computational overhead and rendering direct extensions of prior rehearsal learning methods infeasible. For instance, the asymptotic solution by sampling methods in Qin al. [6] becomes intractable in polynomial time unless the decision variable is singleton (i.e., | = 1), complicating its application to the aggregated setting.
3.2 QP reformulation of the AUF problem
In this subsection, we first derive a functional expression for the aggregated target variables w.r.t. the selected alteration sequence and the noise sequence. Based on this expression, we propose a QP reformulation of AUF problem Eq. (3), which is computationally efficient to solve. Furthermore, establish theoretical guarantee for the optimality of the QP solution under mild assumptions.

---

To maximize PAUF, the first step is to derive the functional expression for the AUF objective Y w.r.t. the selected alteration sequence. Recall from Sec. 3.1 that decisions are made after observing X, x, and just before Ze occurs. In this time point, the aggregation of Y can be derived as follows:
Proposition 3.1. Let o and ég denote ( ,2, - ... :
and E _eT]
Given + (2 A, in Eq. (2)), Dr (including V and xx) and alteration sequence éi, it holds that:
1
ST
EY
Y Mx. Nv + HEē Fé,,
(4)
1=t
where M E RRIXIIII, N E R H E ( and F E (I are constant matrices based on parameters O and time period T, while Ex, ExTT are i..dd noise vectors.
Although the target variables Y; are dependent across i E (0, T). Prop. 3.1 shows that their aggregation can be explicitly decomposed into: (i) mean component, determined by the value of alteration variables zI given the observed x, and vi-1; and (ii) a stochastic component, governed by the aggregated temporal noise, with dependence captured by Fē,. Moreover, Prop. 3.2 shows that the distribution family can be preserved under symmetric log- noise.
Proposition 3.2. Let e A Fē, denote the aggregated noise term as defined in Eg. (4). If e follows a symmetric log-concavely distribution, it always holds that e is also log-concavely distributed and symmetric about the origin, for any finite time point E "t, and finite time window T E Ez..
As established in Prop. 3.2, the distribution family is preserved under aggregation of the target variables Y, thereby retaining desirable properties of log-concave distributions, such as the unimodal- ity [11]. Nevertheless, directly solving the probabilistic optimization in Eq. (3) remains challenging. as the resulting PDF of the aggregated outcome can still exhibit considerable complexity.
To avoid directly solving the probabilistic optimization, existing rehearsal-basee methods typically reformulate it by imposing a constraint of the form PAUF 2 т. and then search for plausible alterations that satisfy this constraint. Specifically, Qin et al. [6] propose a Monte Carlo (MC) method that samples from the distribution P(Y | x, Rh(2)) )). and approximates the solution by requiring that a proportion greater than T of the samples satisfy Y, E S. In contrast, Du et al. [7] search for probabilistic region P such that P(Y E P x; Rh(xf) = т, and ensure that all points within P fall into the desired region S after performing the alteration. These methods are theoretically grounded under Gaussian noise assumption to ensure P. PAUP 2 r, but the optimality can not be guaranteed.
Instead, we leverage the unimodality property of log-concave distributions to guide the decision- making process. Recall that the goal is to shift as much probability mass as possible into the desired region S. As shown in Eq. (4), the alterations affect only the mean of the target variables, while the variance is determined by Fē,, Therefore, if S is a closed region in R then adjusting z to move the peak of the unimodal distribution toward the center of mass of S can effectively increase the likelihood of favorable outcomes. Let M, N. H be defined as in Prop. 3.1, and let s denote the center of mass of S. This heuristic motivates a deterministic QP reformulation that minimizes the distance between the shifted mean and the region S, formulated as:
arg min 2
Mx, + Nv H
S
(5)
While this heuristic idea is simple to implement, there remains risk that the resulting solution may deviate significantly from the true maximizer of the P PAUF. even in simple scenarios (see Appx. A for discussion). To clarify the conditions under which the QP reformulation is effective, introduce the following assumptions and establish the optimality of the reformulation under these conditions.
Assumption 3.3. The following assumptions constrain the system and the problem formulation. 1. (Linear system) The structural functions fr in Eq. (1) are linear with additive noise terms e/ where E, follows a symmetric log-concave distribution, as defined in Eq. (2).
2. (Unique targets) Let μt E[L((T + 1) E Y, D:, Rh(25)) denote the mean vector of the
AUF target as defined in Eq. (4), which is an affine function of Z Given and T, vector 0jt/0z cannot be expressed as a linear combination of the set {0pl 0225 lizi for any 1 1 i (Y).
3. (Symmetric S) The desired region S in Eq. (3) is centrally symmetric (about s) convex region.
5

---

Remark. Note that the first two assumptions pertain to the underlying system, while the last one concerns the AUF problem itself. Specifically,, the linear system assumption permits a broader class of symmetric log-concave noise distributions, generalizing prior works that often assume Gaussian noise for theoretical tractability. The unique targets assumption also reasonable: if it were violated, i.e., 3i such that 0ui//žz can be expressed as a linear combination of (apz /0/4/02z 1944, then the variable y would be redundant in the seeiiioonmmainng process, as its constraints would be fully captured by those the other target variables, and Y could therefore be omitted. Finally, since the desired region& is defined by the Aecision-maker it is typically specified manually. While previous studies primarily consider convex polytopes as the region shape, our symmetric S assumption accommodates additional shapes such as circular and elliptical regions, though non-symmetric ones excluded. This assumption is practical in many settings. For instance, interval constraints each dimension of Y (i.e., O₁ S Y < by for each i) naturally satisfy it and are common in real-world applications. Building these assumptions, we establish theoretical guarantees for the QP.
Theorem 3.4. Let A(Z) I | and 2 denote the solution the QP defined in Eq. (5). Under Ass. 3.3 with any finite 1, T E Zt,, the following inequality holds for any alternative zi:
T
t+T N -.EESE Dr, Rh(())
Dr, Dr, RM(E))
Thm. 3.4 providev an optimality guarantee for decisions made (by Eq. (5)) immediately after observing X, X, in round t. This solution is not necessarily globally optimal from the future perspective at time t ++T T, since less information is available at time t compared to t + T. Nevertheless, Thm. 3.4 exhibits more favorable applicability in prior AUF settings that consider only the target outcome Y of the current decision round (i.e., == 0), because the solution in this case coincides with the global optimum and thus outperforms existing methods. Moreover, even when T * 0, the solution can also be viewed as the best possible decision at time t given all available information. This naturally motivates iterative method that solves for ž at each round and exccutes only the current zf. In what follows, we formally propose such two new rehearsal- learnnnn methods based on Thm. 3.4.
3.3 Rehearsal learning methods based on the QP reformulation
In this subsection, we utilize aforementioned results to develop two new rehearsal learning algorithms for solving the AUF problem defined in Eq. (3), including greedy algorithm and a far-sighted one.
greedy approach. Recall that the goal of AUF is to shift the probabil- ity mass of the average target vari- ables Y over a time period into the desired region S as much as possi- ble. A straightforward approach is to greedily maximize the probability of Y, E S at each round i, which intuitively leads to a relatively high probability of Y E S as well. This procedure is summarized in Alg. 1. Since focus on single-round opti- mality in each decision round, we set T = 0 and compute the invariant ma- trices M, N, H using the formula detined in Eq. (9), Appx. C. Then, in each decision round from time to to le, the alteration 2 is chosen by selecting solution of the QP reformulation defined in Eq. (5). Note that in this case, the greedy nature of the approach implies 2f & ž because T = 0.
Algorithm 1 GMuR (Greedy Multi-round Rehearsal)
Input: start/end time totte, SRM para. + desired region S
1: Determine the symmetric center s of desired region S
2: Compute M, N, H in Eq. (4) by T 0 and +
3:
> Computation formula in Eq. (9), Appx. C
4: Let Hhs denote the matrix H H (HHT)-
5: Compute H₁ Hha M, H2 HhaN, SH Haas
6: for t = to to te do
7: Observing Vi-1 and X₁
8: Execute z = Sy H H2V
9: Receive y: and set D,+1 D, U (xe, y
Output: suggested alterations {zí {
By using the GMuR approach described in Alg. 1, Thm. 3.4 guarantees that each alteration of maximizes the probability of Y, E S. This further ensures that the GMuR approach outperforms prior rehearsal-based methods under Ass. 3.3, since those methods single-round AUF setting, which also aims to optimize the probability of Y, E S. Moreover, this method is computationally efficient as the matrices M, N, H remain invariant across rounds due to the fixed horizon T = O, resulting in an overall time complexity of O| ly 1 (t. - (o)\/v[zz) which outperforms existing rehearsal thased methods [6, 7] time complexity as discussed in Appx. 4.
specifically designed for the

---

A far-sighted approach. Although the GMuR approach maximizes the probability that the current target Y fall into S, it overlooks the sequential nature of the AUF objective, which concerns the aggregate Y over a hori- zon. To mitigate potential informa-
Algorithm FarMuR (Far-sighted Multi-round Rehearsal)
Input: start/end time tofte, SRM para. 6 desired region 1: Determine symmetric center of S, Set L = te - to
for = to to Le do
3:
Observing V;-1 and x,, update T te -t
4: Compute M, N, H in Eq. (4) by T and
tion loss, we propose a far-sighted
5:
Computation formula in Eq. (9), Appx. C Reweight (M, N, H) = I+T
strategy in Alg. 2, which selects alter- ations by considering not only current targets but also future ones. Specifi- cally, the method iteratively updates and reweights the matrices M, N, H according to the remaining horizon T and total horizon L., repeatedly solves for z/. and executes only the current zf.
6:
T+1 [M,N,H) H)
7: Compute 2, HT (HH") (s Mx, MM, Nv
8:
Exccute z _/2 2 |z|] and drop others
9: Receive and set Di+1 Dr U (x,, yr
10: Update = /L/(L
Output: suggested alterations {e It=to
After each decision round, also adjusts the region center
The FarMuR approach accounts for the aggregation of Y over the remaining time horizon at each decision round, resulting in time-varying matrices M, N, H. As consequence, the time complexity of Alg. 2 increases to ((te - (o) / [v]). higher than that of Alg. 1. However, this increased computational cost leads improved performance, discussed in the following subsection.
3.4 Theoretical analysis
Recall that Alg. I and Alg. 2 shift E| toward As Y follows unimodal distribution as shown in Prop. 3.2, the probability mass falling within S is affected not only by E but also by Ya. Theorem 3.5. When Y is singleton (i.e., |Y| 1), let Varo Var[Y, Dr, Rh(zz)), and let A denote the rehearsal learning process as in Alg. 1 or Alg. 2 that iteratively selects and performs allerations from round I. to round L+ L.. Define Var and Varz as Var I+I Eitt Y; A under A0tL corresponding to Alg. 1 and Alg. 2, respectively. Then the following holds:
Var
1
and Varz
1
Varo
L+1
Varo
(I+
Thm. 3.5 shows that the variance of the decision objective reduces when L increases, leading to improved PAUF expectations are the same (= as seen in Appx. C.4). Moreover, the alteration sequence selected by Alg. 2 yields a more substantial variance reduction than that of Alg. 1, resulting in higher PAUF of same L. This phenomenon is demonstrated in Fig. 4, and the intuition is that the variance of Yea+L= T EE Y; under the rehearsal learning process can be decomposed as:
L+
IL j-1 Cov (Y, Y,
Σ Var Y/
Amtte] +
ΣΣ
Σ
A
(6)
(L+1)
1)
Intuitively, the variance of the aggregation is determined by the two components above. Since the rehearsal learning method in Alg. 2 subtracts a weighted Y, from region center at each round, introduces additional negative correlations between consecutive outcomes Y++1 and Y.. As result, an additional negative term appears in the second component of Eq. (6) for the aggregation A generated by Alg. 2, further reducing the overall variance compared to the GMuR approach.
Another important consideration how decision decision-making is affected when the true parameter set θ is unknown, and an estimate Ô used to approximate matrices M, N, in QP reformulation Eq. (5). Theorem 3.6. When N samples are used to estimate Ô as in Appx. C.5, let t denote the alteration selected by solving Eq. (5) with θ and ūة denote the one selected with true A. Under additional assumptions that θ is bounded and noise is Gaussian, it holds that || 12 $ ((L/VN).
Thm. 3.6 ensures that reliable alterations can be identified by our approach without requiring prior knowledge of the true system parameters O. Importantly, the samples used for parameter estimation need not be collected through interactions, but can instead be obtained from historical observations. This contrasts with classical reinforcement learning (RI.) settings [12), including both online and offline RL as discussed in Appx. 4. Further discussion of our approach is provided in Appx. A.
7

---

4 Related work
In this section, we review related work on rehearsal learning and two other related directions:
Rehearsal Learning. Rehearsal learning aims to uncover the influence relations among variables in AUF and leverage these relations for decision-making [6. 7. 10, 8. 13]. Specifically, prior rehearsal learning methods typically reformulate the intractable probabilistic AUF optimization into constraint of the form IA AAF 2 т, with theoretical guarantees established under linear structural equations with additive Gaussian noise. Qin et al. [6] propose a mrixed-intege linear programming (MILP) approach based on sampling techniques to solve the reformulated AUF problem. Du et al. [7] consider decision costs and potential variations in influence relations, and derive an explicit computational formula for contour lines P(Y E S x. R)(25)) = T for any т. leading to more efficient algorithm. These two approaches could not theoretically maximize PAUF- Qin et al. [10] then propose heuristic method to address AUF problem under potentially nonlinear and non-Gaussian settings, though there is no theoretical guarantee and it underperforms prior rehearsal learning approaches when the underlying system is linear. Du et al. [8] present an aptimization-basee approach to directly optimize the AUF probability, which achieves the optimal AUF probability under the CARE condition though it only considers single-round AUF and assumes Gaussian noise, a setting to which our method naturally reduces. Tao et al. [13] establish a sequential decision-making approach that formulates the single-round AUF into sequence of sub-problems and solves them sequentially, considering different setting from ours. Hence, we do not include the latter three methods in our experimental comparison. Our approaches
theoretically established to achieve higher AUF probabilities than prior methods, even under non- aanGasssian noise, while also being more efficient w.r.t. [V].
Reinforcement Learning (RL). RL has achieved notable suc- cess across a wide range of decision-making tasks [12]. Classical RL methods [14-16] rely on extensive interactions with the en- vironment, which are infeasible in AUF where interactions are rare 12. 6]. While recent advances in offline and hybrid offline- online RL [17-20] aim to reduce the reliance on online interaction by utilizing pre-collected datasets, they do not match the AUF setting because the reward function could diverge between obser- vational and online data. In contrast, rehearsal learning leverages fine fine-grained structural information among variables, enabling effective decision-making from observational data without requiring interactions. To illustrate the challenge faced by RL in this context, consider the example in Fig. 3. Let X, represent the state, Z (an actionable variable) the action, and [[YY, E S) the reward. Due to the presence of unactionable variable Z,", the conditional distribution P(Y, x4, 2f) differs from P(Y. Te, Rf(=f)) )), where Rh( 2f") denotes the result of applying an action based solely on the action- able component. As result, the reward function inferred from observational data, 1(Y, 8 x 2 can deviate significantly from the one with actions, II(Y; E S It, (h((2?)) This discrepancy renders the standard (s, a, r) tuples extracted from observational data ineffective for learning reliable online policies. underscoring the necessity of exploiting structural information in AUF. In addition, class of model- RL, namely causal RL methods [21-23] incorporates structural information to handle certain types of confounding during offline policy evaluation. However, unlike our approach that explicitly exploits graphical structure (including all ancestral relations), these methods use structure more narrowly and further difficulty in AUF scenarios where the reward (Y, E S) may vary across rounds. Moreover, rehearsal learning emphasizes identifying the underlying influence relations, which is crucial for human-in-tien -n-ohh- decision-making The learned influence relations serves as interpretable guidance for human decision-makers, and since rehearsal itself does not alter the environment, humans can safely choose whether to adopt the suggested decisions; this property lies beyond the scope of RL, which directly interacts with and potentially alters the environment.
X
ZH
Y
Z
Figure 3: An example illustrating the potential distribution shift.
Causality, Extensive research has explored the use of structural models to support decision decision-making, most of which is grounded in the framework of SCMs [3]. Some approaches aim at system identifi- cation through active interventions or incorporating expert knowledge [24-31], but typically do not consider downstream objectives such as estimating or optimizing causal effects. Additionally, causal bandit methods have been developed to address optimal arm identification problems [32-40]. These approaches generally aim to identify a universally optimal action that maximizes the expected reward. In contrast, rehearsal learning supports mutually influenced relations, and aims to maximize the AUF probability by identifying context-specific optimal actions tailored to different observed contexts x, and allows for more flexible specification of desired region S beyond simple maximization.
8

---

5 Experiments
Sire Y. Tiih
Ler + Y, with sight alteration desired rugion
selected alteration dedired region
Shart aigtitt
desired region
Varince: 3.30
Varlamoe: 9:12 Pluu
Varianee: 4,59
Var unce: 2,25 Prag 12
Prag 21.1%
Peg 3%
(a) No action
(b) Action for single-round (c) GMuR approach
(d) FarMuR approach
Figure 4: Results on the toy example. and Y+ denote values of the target variable in adjacent decision rounds. The results illustrate that variance reduction leads significantly higher PAUF-
We visualize our approaches using a toy example modeling a simplified Texas Hold' em game, fol- lowed by evaluations on synthetic and real-world datasets. Our methods are compared against baseline approaches and established rehearsal learning methods, including QWZ23 [6] and MICNS [7]. We do not include RL methods in our comparisons because: (i) they are unsuitable for the AUF setting due to limited interaction opportunities and discrepancies between reward functions in interactional versus observational data, as discussed Appx. 4; and (ii) previous studies [6. 7] have demonstrated their inadequate performance for AUF tasks. Experimental details are provided in Appx. D.
Toy example for visualization. Consider an example involving a simplified Texas Hold" em game, where the variables X, Z, Y are all singletons, and their graphical relations are illustrated in Fig. 5. Let X, denote the average skill level of the opponents in hand t, Ze the action you take in that hand, and Y, the yield from hand Due to overlapping participation across nearby hands, there exist cross-round influences, represented by red edges. Specifically, X, -> X++ captures the dependence of future opponent skill levels on current ones, and Z, + Y++ reflects the impact of your current action on future yields via revealed playing style. The AUF target is to maintain moderate average yield over the course of the game. Specifically,
X
V
X
Ze
Y
V
X
Z++
Y
Figure 5: The underlying graphical model
suppose we plan to play I. + hands the table; let I+I Et Y; denote the average yield, and we aim to keep it within safe interval S S b. This is because financial loss, while too high a yield may cause potential disputes. Assume that the variables X, and Z, are fully observable and rationally controllable, and the noise term is Gaussian.
low a yield leads to
As illustrated in Fig. 4, no policy is applied, the actions are passively influenced by X,, resulting in an AUF probability IP(a S w S b) of only 21% as shown in Fig. 4(a). When we actively perform alterations on Zr the AUF probability increases to 51% for a single-hand outcome, as demonstrated in Fig. 4(b), representing the optimal performance guaranteed by Thm. 3.4. For sequential play (with L = 1), the results obtained using the short-sighted and far-sighted policies are shown in Fig. 4(c) and 4(d), respectively. These results demonstrate that proposed far-sighted FarMuR approach outperforms the short-sighted GMuR approach by better controlling the variance. It is worth noting that the variance in this example is deliberately chosen for clear demonstration purposes. In real games, the variance should not be so small, which would otherwise lead to a significantly lower AUF probability under distributions of Y or Y, even within a short sequence of rounds under any policy. In what follows, we first briefly introduce the datasets, with details/graph structures listed in Appx. D.
Synthetic Data. We construct synthetic dataset with dimensions of X,, Z, and Y, set to 2, 4, and 2, respectively, to evaluate cases involving miltti-dimensionall outcomes Y. Variables z and Z designed to exhibit mutual influence, such that changes in either affect the other. The desired region for target Y is defined as a circular region S = {y ly - s|l2 I 0.83. The parameters of the structural equations, as well as the variances of the additive noise terms, are manually specified.
Bermuda Data. The Bermuda dataset records environmental variables in the Bermuda area and has been widely used in prior research [6, 7, 41, 42]. The generation order of variables in this dataset recorded [43]. The dimensions of X,, Z,, and Y, are 3, 7, and 1, respectively, and the desired region& S for Y is defined as S (NEC E [1.9. 2.0] J. The parameters of the structural equations are estimated by fitting least-squares linear models to the normalized data, and the additive noise terms modeled by residuals using either Gaussian or Laplace distributions.

---

Table 1: AUF probability P(Y S Att-t) (%) evaluated on two datasets, where L denotes the horizon length fe to. Each value is estimated using 1000 Monte Carlo samples, averaged over 5 random seeds. Results are reported as mean standard deviation, best results are highlighted in bold.
Datasct
Win. Len.
No action
QWZ23 [6]
MICNS [7]
Ours (GMuR)
Ours (FarMuR)
L 0
0.9 = 0.19
4.5 0.46
4.9 0.46
4,9 0.32
4,9 = 0.32
Synthetic
(Gaussian)
L 4
3.7 = 0.82
20.3 = 0.64
20.1 0.97
20.7 0.72
65.1 + 0.94
I. 8
3.5 + 0.72
32.2 4: 1.12
31.3 0.75
32.4 +: 0.33
95.0 + 0.42
L O
1.0 + 0.19
8.4 + 0.72
9.9 0.82
10.2 0.65
10.2 + 0,65
Synthetic (Laplace)
L 4
3,9 + 0.57
21.3 + 0.87
21.5 1.10
21.9 + 0.31
72.8 + 1.51
L 8
3.8 + 0.37
33.4 0.72
33.8 1.60
34,6 + 1.35
93.2 = 0.31
L ==0 O L = 4
1.8 1: 0.26
7.3 1: 1.19
7.2 0.47
10.3 1: 0.85
10.3 1: 0.85
Bermuda (Gaussian)
0.9 1 0.29
15.5 1: 1.34
15.9 1.19
21.9 1 0.26
47,9 1 1.29
L = 8
0.4 + 0.13
20.8 + 1.75
20,2 2.78
28.2 + 1.37
74.3 + 1.77
= O
1.6 + 0.15
7.9 + 0.87
7.5 = 0.67
14.1 1.29
14.1 + 1.29
Bermuda (Laplace)
L = 4
+ 0.27
17.6 + 1.31
15.1 1.64
23.8 + 1.19
59.4 = 0.94
L =
1: 0.12
24.1 1 0.77
21.5 1.61
30.1 1.71
80.3 1 1.03
Time window length (O)
Time window length (L)
Number of observational samples
Varience the target
GW/Z25
Single found varbacc Varif [Ail
MIChS
gr 21 of Gready Methed
-f- VarlP Aaoual Gready Method
Greedy fansighted
21 Tan-ighted Welhed
Vart? [Aueal Sgnttd
G00 12
Figure 6: Results the Bermuda dataset, illustrating the variance reduction w.r.t. the time horizon L, average executing time w.r.t. L, and gap | 2 l2 w.r.t. the number of observational samples N.
The full results on the two datasets are presented in Tab. 1, demonstrating that our methods outperform existing rehearsal learning approaches as well as the no-action baseline (reflecting the distribution without alteration), under both Gaussian and Laplace noise. Specifically, the GMuR approach Alg. dominates existing methods, as guaranteed by Thm. 3.4, while the FarMuR approach Alg. 2 further improves the AUF probability by leveraging stronger variance reduction, as shown in Thm. 3.5.
Fig. 6 presents additional results on the Bermuda dataset, confirming that the FarMuR method achieves higher variance reduction rate (left panel, cf. Thm. 3.5) and demonstrates the convergence of excess risk for both our approaches (right panel, of. Thm. 3.6). The execution time is plotted against L (middle panel), thus the slopes of the lines represent the time complexity w.r.t. other factors beyond L, such as VI, and our methods are significantly more efficient than existing approaches. Although the difference slopes between our two methods is not prominent Fig. (as the scale dominated by the much larger execution times of prior methods), Fig. 10 clarifies that the FarMuR method exhibits a steeper slope than the GMuR method. This highlights a trade-off: as increases, the FarMuR method offers superior AUF performance but at the cost of increased execution time compared to the GMuR approach. Additional results are provided in Appx. due to space constraints.
Conclusion
In this work, we consider an essential class of decision-making tasks termed AUF. Recognizing that practical AUF often hinges on outcomes over extended time horizons, we generalize the rehearsal learning framework and propose an generalized AUF problem formulation to better accommodate long-term scenarios. To address the inherent challenges stemming from the probabilistic nature of the formulated problem, we introduce QP reformulation and establish its optimality under mild conditions, even with non-Gaussian noise, thereby enhancing the framework's practical utility. Building on the QP reformulation, we develop two novel rehearsal- outperform existing methods. Furthermore, we provide theoretical guarantees for our approach, including variance reduction properties and an excess risk bound when using estimated structural parameters. Experimental results validate both the effectiveness and efficiency of our methods.
algorithms that significantly
10

---

Acknowledgements
This research was supported by NSFC (62495092,62406113)), Jiangsu Science Foundation Leading- edge Technology Program (BK20232003), and Collaborative Innovation Center of Novel Software Technology and Industrialization.
References
[1] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature, (21(1233)((77338) 444, 2015.
[2] Zhi-Hua Zhou. Rehearsal: Learning from prediction to decision. Frontiers of Computer Science, 16(4):
1-3, 2022.
Peter Spirtes, Clark Glymour, and Richard Scheines. Causation, Prediction, and Search. MIT Press, 2000.
Bernhard Schölkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. Toward causal representation learning. Proceedings of the IEEE, 109(5); 612-634, 2021.
[5] Zhi-Hua Zhou. Rehearsal: Learning from prediction to decision. Keynote at the CCF Conference on AI, Urumqi, China, 2023.
[6] Tian Qin, Tian-Zuo Wang, and Zhi-Hua Zhou. Rehearsal learning for avoiding undesired future. In Advances in Neural Information Processing Systems 36, pages 80517-80542, 2023.
Wen-Bo Du, Tian Qin, Tian-Zur Wang, and Zhi-Hur Zhou Avoiding undesired future with minimal cost non-stationary environments. In Advances in Neural Information Processing Systems 37, pages 115741-135700 2024.
[8] Wen-Bo Du Hao- Yi Lei, Lue Tao. Zion-Zoo Wang, and Zhi-Hua Zhou Enabling optimal decisions in rehearsal learning under CARE condition. In Proceedings of the 42nd International Conference on Machine Learning, pages 14536-14561, 2025.
James D Hamilton. Time Series Analysis. Princcton University Press, 1994.
[10] Tian Qin, Tian-Zu Wang, and Zhi-Hua Zhou. Gradient- based nonlinear rehearsal learning with multivariate alterations. In Proceedings of the 39th AAAI Conference on Artificial Intelligence, pages 26859-26866 2025.
[11] Sudhakar Dharmadhikari and Kumar Joag-Dev. Unimodality, Convexity, and Applications. Elsevier, 1988.
[12] Richard S. Sutton and Andrew G. Barto. Reinforcement learning: An introduction, 2nd Edition. MIT Press, 2018.
[13] Lue Tao, Tian-Zuo Wang, Yuan Jiang, and Zhi-Hua Zhou. Avoiding undesired future with sequential decisions. Proceedings of the 34th International Joint Conference on Artificial Intelligence, pages 6245-6253, 2025.
[14] Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning. In Proceedings of the 4th International Conference on Learning Representations, 2016.
[15] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. CoRR, abs/1 0es//177.06633 06346663777 2017.
[16] Tuomas Haarooja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic; Off- Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In Proceedings of the 35th International Conference on Machine Learning, pages 1861-1870, 2018.
[17] Yuda Song. Yifei Zhou, Ayush Sekhari, Andrew Bagnell, Akshay Krishnamurthy, and Wen Sun. Hybrid RL: Using both offline and ouline data can make RL efficient. In Proceedings of the 11th International Conference on Learning Representations, 2023.
[18] Vitchyr H. Pong. Ashvin Nair, Laura Smith, Catherine Huang, and Sergey Levine. Offline meta- reinforcement learning with online superviinv. In Proceedings of the 39th International Conference Machine Learning, pages 17811-17825 2022.

---

[19] Jiachen Li, Edwin Zhang, Ming Yin. Qinxun Bai, Yu-Xiang Wang, and William Yang Wang. Offline reinforcement learning with closed-form policy improvement operators. In Proceedings of the 40th International Conference on Machine Learning, pages 20485-205228, 2023.
[20] Dan Qiao and Yu-Xiang Wang. Offline reinforcement learning with differential privacy. In Advances in Neural Information Processing Systems 36, pages 61 695--6144,, 2023.
[21] Nathan Kallus and Angela Zhou. Confounding-robuss, policy evaluation in infinite-horizon reinforcement learning. In Advances Neural Information Processing Systems 33, pages 22293-22304, 2020.
[22] David A Bruns- Smith. Model-free and model-based policy evaluation when causality is uncertain. In Proceedings of the 38th International Conference Machine Learning, pages 1116-1126, 2021.
[23] Chinmaya Kausik, Yangyi La, Kevin Tan, Maggie Makar, Yixin Wang, and Ambuj Tewari, Offline policy evaluation and optimization under confounding. In Proceedings of the 27th International Conference on Artificial Intelligence and Statistics, pages 1459-1467, 2024.
[24] Yangbo He and Zhi Geng, Active learning of causal networks with intervention experiments and optimal designs. Journal of Machine Learning Research, 9(84):2523 2547, 2008.
[25] Murat Kocaoglu, Alex Dimakis, and Sriram Vishwanath. Cost-optimal learning of causal graphs. In Proceedings of the 34th International Conference on Machine Learning, pages 1875-1884, 2017.
[26] Tian Qin, Tian-Zuo Wang, and Zhi-Hua Zhou. Budgeted heterogeneous treatment effect estimation. In P Proceedings of the 38th International Conference on Machine Learning, pages 8693-8702, 2021.
[27] Jiaqi Zhang, Louis Cammarata, Chandler Squires, Themistoklis P Sapsis, and Caroline Uhler. Active learning for optimal intervention design in causal models. Nature Machine Intelligence, pages 1-10, 2023.
[28] Jiaqi Zhang, Kristjan Greenewald, Chandler Squires, Akash Srivastava, Karthikeyan Shanmugam, and Caroline Uhler. Identifiability guarantees for causal disentanglement from soft interventions. In Advances in Neural Information Processing Systems 36, pages 50254-50292, 2023.
[29] Tian-Zuo Wang and Zhi Hua Zhou. Active causal effect identification with expert knowledge. SCIENTIA SINICA Informationis, 53(15)(333)))) 21:12412222355555 2023.
[30] Tia Wang, Wen-Bo Du, and 7hi-Hua Zhou, An efficient maximal ancestral graph listing algorithm. In Proceedings of the 41st International Conference on Machine Learning, pages 50353-50378, 2024.
[31] Tian-Zuo Wang, Wen-Bo Du, and Zhi- hua-Hua Zhou saanoomial-dllay mag listing with novel locally complete orientation rules. In Proceedings of the 42nd International Conference on Machine Learning, pages 62804-62824, 2025.
[32] Elias Bareinboim, Andrew Forney, and Judea Pearl. Bandits with unobserved confounders: A causal approach. In Advances in Neural Information Processing Systems 28, pages 1342-1350, 2015.
[33] Finnian Lattimore, Tor Lattimore, and Mark Reid. Causal bandits: Learning good interventions via causal inference. In Advances in Neural Information Processing Systems 29, pages 1181-1189, 2016.
[34] Rajat Sen, Karthikeyan Shanmugam, Alexandros G Dimakis, and Sanjay Shakkottai. Identifying best interventions through online importance sampling. Procedinggs of the 34th International Conference on Machine Learning, pages 3057-3066, 2017.
[35] Sanghack Lee and Elias Bareinboim. Structural causal bandits: Where intervene? In Advances in Neural Information Processing Systems 31, pages 3-733, 2018
[36] Junzhe Zhang and Elias Bareinboim. Ncar-optimal reinforcement learning in dynamic treatment regimes. In Advances in Neural Information Processing Systems 32, pages 13401-13411, 2019.
[37] Sanghack Lee and Elias Bareinboim. Characterizing optimal mixed policies: Where to intervene and what to observe. In Advances in Neural Information Processing Systems 33, pages 8565-88 (676,8 2020.
[38] Yangyi Lu, Amirhossein Meisami, and Ambuj Tewari. Causal bandits with unknown graph structure. In Advances in Neural Information Processing Systems 34, pages 14817-24828, 2021.
[39] Min Woo Park, Andy Arditi, Elias Bareinboim, and Sanghack Lee. Structural causal bandits under markov equivalence. In Advances in Neural Information Processing Systems 38, page appear, 2025.
[40] Min Woo Park and Sanghack Lee. On transportability for structural causal bandits. CoRR, abs/2511. 179553, 2025.
12

---

[41] Travis A Courtney. Mario Lebrato, Nicholas R Bates, Andrew Collins. Samantha De Putron, Rebecca Garley, Rod Johnson, Juan-Carlos Molinero, Timothy Noyes, Christopher L Sabine, et al. Envirommental controls on modern scleractinian coral and reef-scale calcification. Science Advances, 3701)5 2017.
[42] Virginia Aglietti, Xiaoyu Andrei Paleyes, and Javier González. Causal Bayesian optimization. Proceedings of the 23rd International Conference on Artificial Intelligence and Statisticx, pages 3155-3164, 2020.
[43] Andreas Andersson and Nicholas Bates. In situ measurements used for coral and reef-scale calcification structural equation modeling including environmental and chemical measurements, and coral calcification rates in bermuda from 2010 to 2012 (BEACON project). 2018. http://lod.bo boo dooooooooooooooooooooooo www.dddooooooooooooooo wttp.oorrri//ddase//20000/
[44] András Prékopa. On logarithmic concave measures and functions. Acta Scientiarum Mathematicarum, 34: 335 343, 1973.
[45] Theodore W Anderson. The integral of a symmetric unimodal function over symmetric convex set and some probability inequalities. In Proceedings of the American Mathematical Society, pages 170-176, 1955.
13

---

A Discussion of our approach
In this section, we discuss the proposed approaches in this work, including the QP reformulation and the two proposed algorithms. First, although the heuristic idea defined in Eq. (5) yields tractable QP reformulation, it is not always optimal. When assumptions as defined in Ass. 3.3 do not hold, the QP reformulation can be far from optimal. In what follows, we present counterexample in Fig. 7 that violates the unique target assumption in Ass. 3.3 thus not optimal.
A counterexample Consider simple example where T = 0. Y 2. and | = 1, with the underlying structural equations defined in Eq. (1) are linear Gaussian. In this case, let I = E| D, Rh(22)) }]; it has been proven that a by + c1, by2 + c2] as shown in Eq. (4), where oj and b, are constants [6]. Hence, there always exist fixed constants a, B and such that [a, p1pT & c, indicating that the trajectory of sloped line according
Desined nigiom 8 fiX RRM Bi
Conooss Fur
POP
ful I RNN by Big 1
Laate of
I can be represented to different values of 25, as illustrated in Fig. Let the diamond region shaded in red denote the desired region S, and draw the vertical line V from center s of S; the intersection point between v and the trajectory of ja corresponds the solution Eq. (5), and the probability P(YY & x, Rh(zE)) equals the overlapping area of its probability density function (PDF) and the region S. is evident that this approach does maximize the AUF probability, as the intersection point is far from the solution to the original Eq. (3), which is the center of the ellipses shaded in blue. In this case, the center My (zS) of the target variables Y can only moving within a sub-dimentional space of the target space IR xl, thus, even when the domin of z can take the whole space, A(Z) = R 1 it not necessarily contains a point z such that (z) matches s.
Figure 7: A counterexample of QP reformulation.
Furthermore, we emphasize that our proposed methods primarily address scenarios with fixed start time to and end time te. Nevertheless, Thm. 3.4 guarantees that the QP reformulation remains existing information) in any current decision round for arbitrary t and T, Consequently, our approaches can be readily extended to scenarios with varying time horizons, such as when te is not fixed but instead maintains a constant distance L from the current time t.
optimal (conditioned
B Definition
In this section, we provide comprehensive definitions and discussions of the Structural Rehearsal Model (SRM), probabilistic graphical model introduced by [6] to represent influence relations among variables in AUF problem. The SRM comprises set of rehearsal graphs and associated parameters (for the structural equations) { ((G,, 0.)}. The original definitions and discussions of the SRM can be found in Qin et al. [6].
The rehearsal graph G. models the qualitative influence relations among variables, which is denoted by G, = (V,, E.). Specifically, Ve represents the variable set of the AUF problem and E, represents the edges expressing influence relations among variables in round f. There two types of edges in Ge, a directional edge X + Y means that influences Y, and a bi-directional edge ++ Y means that X and Y are mutually influenced. For example, sunlight unilaterally influences the plant growth, whereas rainfall and river flow are mutually influenced, as changes in either one affect the other. The definition of the rehearsal graph follows.
Definition B.1 (Mixed graph, [6]). Let G = (V, E) be a graph, where V denotes the vertices and E the edges. G is a mixed graph if for any distinct vertices u, v E V, there is at most one edge connecting them, and the edge either directional ( uu v or u + v or bi-directional (u + u).
Definition B.2 (Bi-directional clique, [6)). A bi-directional clique C -= (Ve, E*) of a mixed graph G = (V, E) is a complete subgraph induced by V C V such that any edge E E is bi-directional. C' is maximal if adding any other vertex does not induce bi-directional clique.
Definition B.3 (Rehearsal graph, [6]). Let G (V, E) be a mixed graph. Let ((.)) denote all maximal bi-directional cliques of G, where C. = (V;, E,). G is a rehearsal graph if and only if:
14

---

Z
Z3
X
Z
X
Z
X
7
X
Z
(a) rehearsal graph G
(b) GRMCza 22)
(c) G GRicZ m2))
(d) G RRilla 22
Figure 8: example reproduced from Du al. [7]. Fig. 8(a) is a rehearsal graph, Fig. 8(b)~8(d) show alteration graphs under different alterations. When an alteration is applied to certain variables, all incoming arrows to these variables are removed, while the rest structure remains unchanged.
Vi nin V5 = for any + j.
2. Vi [[], if there is any edge pointing from some u E V\ to some v E V then Vo Vi,
u + v.
3. There exists topological ordering for (C:) following the directions of directional edges between C,s.
The topological order of bi-directional cliques (Ci) reflects the generation order. We generalize the definition time series cases (Gx), with the definition of structural equations as in Sec.
Besides, the decision-making process focuses on identifying suitable alterations, as defined in Eq. (3) and Eq. (5). An alteration & refers a decision action specified by human decision-maker represented as set of vertex-value pairs (e.g.. < + {21 = ž1 } in Fig. 8(b)). Meanwhile, the rehearsal executing a given alteration, thereby modifying the
operation, denoted by Rh(-), corresponds original graph structure as illustrated in Fig. 8(b) 8(d). Specifically, the rehearsal operation removes all original influence links that point into vertices involved in E, and fixes the values of these vertices according to E; meanwhile, it preserves the influence relations among the remaining vertices in the resulting graph GRth(E) In the time series setting where alterations are performed over the current and the following T time steps, the operation is denoted by Rh(z,. ...1..... 2(4r) to represent this sequential process, the same as Eq. (3) and Eq. (5).
C Proofs
C.1 Proof of Prop. 3.1
2 Proposition 3.1. Let o and o, denote i -14T-11 ... 24 TT and [< EL--1-' _TT].
T
Given θ (2 A, in Eq. (2)), De (including Vz-1 and xx) and alteration sequence 2i, it holds that:
1 HT
Σ Mx: + Nv. + HH + Fër,
where M E RI E R| H E (X(THI] and F E R|MX((++) VI are constant matrices based on parameters O and time period T, while E., EE4T are i.i.d. noise vectors.
Proof. Let Eg 4 (Laxz, Oxxz: Oxxy) Eg A (0.xz: Ixxs, 0xxy) T. Ey A (0yxz Opxz: Tuxy) and let V, = X,, Z. Y,) denote the variable list. Under natural process, it holds that:
V, AV, + BV.-1 Es
Then under control sequence without observing x, it holds that:
V, Rh(z)) Ezzz
E + Ey E,) (AV. + BV.-1 +
If we can observe X, X2 under control sequence, then it holds that:
V. X₁, Rh Rz(2)) == Exx; + Ezzz + E,,E,) (AV. + BV + EE),
Starting from time point just observed X Xs, it holds that (omiting Dr, Rh(z
,
E k+i + EE + E.) (AV + BV + Exti) i2 1
V k+i
(7)
Exx; + Ezz EE,E,)) (AV₁ + BVk-I+E;
i= 0.
15

---

Considering that only Va-1 and X2 are observed, we want to express Vi+t by Va-1, Xs., and {zf, ).
Let U = (ExEE + E A) Ez. C (I- + E, A) A) Ex + Ey
and r CB, it can be derived that (it can be verified that E V kti #
Vi+i U2b4; + +VV PV + CENtG. i21. 1.
Meanwhile, let Ü denote (I Ey ETA) Ez, Č denote (I Ey EyEJA) Ey EJ. I denote CB, and E denote (I By EJA) Ex can be verified that V A z and E - xx)
Vx = Ex, Üz + Tva-1 + Cε
(8)
Iteratively using the equations above, it can be derived for 2 1 that:
i-1 Σr (UZf+te, + CEXti-S)
Va+i V +
i=0
i-1
-I Ex, Un . + CEE)
+ EEr
r
Uz UZ + CExti-s)
5=0
r [xxx + (v=-1) +
r'U
Erunh j=0
+
r'Cex +
Σ r riCe ==
constant
control variables
Clearly, the value of Y ie.. V 4 is affected by the sequence of control variable {Ze+<)_= and the more closer, the influence will be more big, because the invertible series limits that [ 1. Meanwhile, noticing that ΣL Y
Ey
Ei
Veti, thus Σ Y: can be expressed
T r E
T i-1
T
Exi + (vv-1)
+ E,
Σ r
(Uziti-j + CELLES) + E
r
Uz
+ CEL
i=0
i=1 ==
j=0
1=0
2 ++
T Σ r
T-1
T
E
(Ex,
(Ex,+
E
IU, (I+
Σ rr, 1=0
20
1=0
ELT
T-1
T
114-11
+ E
IC, [I,C,+)
Σ
r'c.
1=0
E141
Et
T-1 Σ
In this case, let M T E
Σ
r' H
Ey
IU, (U,
==0
1=0
=
T-1
T
N P
F
E.
IC,
IC,
Σ
r'c.
Σ r
Prop. 3.1 is proven.
16

---

C.2 Proof of Prop. 3.2
Lemma C.1 (Prékopa, 1973, Theorem 7). Let f.9 be logarithmic concave functions defined in the space RP, Then the convolution of these functions, i.e.,
J f(x-yigly)dy.
also logarithmic concave in the entire space RP.
Lemma C.2 (Dharmadhikari and Deag 1988, Lemma 2.1). Suppose p-dimentional random
vector v is log-concavely distributed, and let M denote a constant matrix with shapes RRmpp, y S Then Mv is also log-concavely distributed.
Lemma C.3. Suppose p-dimentional random vectors V1, V2 are independent and are log-concavely distributed. Then V1 + V2 is also log-concavely distributed.
Proof. Let f1, f2 denote the PDF of random vectors V1, V2, and let f denote the PDF of random vector v = V1 + V2. By definition, can be derived that:
[
f(v)
-
JRp
f2(v v1)-vv)
By Lemma C.I, the PDF f logarithmic concave, thus vı + V2 is also log-concavely distributed.
Proposition 3.2. Let e A Fe, denote the aggregated noise term as defined in Eq. (4). If e; follows symmetric log-concavely distribution, it always holds that is also log- eege-oncavely distributed and symmetric about the origin, for any finite time point E Z, and finite time window T Ze..
Proof. For the assert log- log-concave distribution, it can be proved by induction that:
For l: = 2, by Lemma C.2 and Lemma C.3, it can be proven that F1 + F25; also obeys log-concave distribution, where F, is the i-th part of F;
Assuming that for (2 2), it also holds that E F obeys log-concave distribu-
tion;
For k = k/' + 1, it holds that ΣK' F,E1444113 FITIE, + Σ = FEEETI- By
Lemma C.2, Lemma C.3 and the inductive hypothesis, Σ F,5 also obeys a log- concave distribution.
For symmetric property, can be proven that -e =a FF, =a F(-è.) =a Fé; =a e, where =d means obeying the same distribution. The third = holds because the noise is assumed to be symmetric in Ass. 3.3.
C.3 Proof of Thm. 3.4
Lemma C.4 (Anderson, 1955). Lei x E Rd be a random vector with probability density function [wx such that (i)
and (ii) (y f(y) 2 u) is convex for every u(O oe). IfP convex set RRd symunetric about the origin, then let denote arbitrary constant vector Rd it holds that:
Theorem 3.4. Let ((Z) Ig and i denote the solution the QP defined in Eq. (5). Under Ass. 3.3 with any finite 1, T E Z,, the following inequality holds for any alternative zi:
T
tT
Σ
YYSs
Dr, RM(E))
P
Σx,
YYES
D., Rh(z))
T
T+1
17

---

Proof. Recognizing that when we simultaneously shift the distribution and the desired region, the probability mass will not change, i.e., given V' E R let S' denote the region of the same shape as while centered at s'
the symmetric center of region S), then it always holds that:
L+ Y, ES
L+7 Σ Y, -E Dr. RM(E))
Σ
Dr, Rb(E))
Meanwhile, recall from Prop. 3.1 that after alteration 2/, 7+1 iist Y, can be expressed as:
Σ
Y,
Mx. Nv H Fé,.
T
i=t
Hence, let v(2)) Mx, + Nv Hž, and let denote Fē,, it always holds that:
HT
P
Σ
Y.ES Dr, RM(2S)
- Ap(ee AP(EES e' Dr, n(M(2))
-
P p ( e- s' E eSo Dr. 12((2)))
where S' is the region of the same shape as S while centered at s' = (Mx, + Nv. + Hzf). and So the region of the same shape as S while centered at the origin
If we can prove that: (a) e is a random vector always satisfying the constraints (i) and (ii) in z, then Thm. 3,4 can be straightforwardly proven according to Lemma C.4; and (b) s' = 0 if 2 Lemma C.4.
We first prove (a) that e is random vector always satisfying the constraints (i) and (ii) in Lemma C.4.
Since it is guaranteed by Prop. 3.2 that is always symmetric about the origin for V finite time point
t E E+ and finite time window T E Z.,. thus the symmetric property holds for (i) in Lemma C.4. For (ii) in Lemma C.4, let fo(-) denote the PDF of random vector e, it can be derived that for Vu 2 0:
{ fe( 2 u} + ty log fo (y) log u).
Meanwhile, following from Prop. 3.2 that is always log-concavely distributed for finite time point t E 24 and finite time window T E Z+, by definition of log-concave distribution, the logarithmic PDF log f-(-) concave. Hence, for Vy 92 {y log foly) 2 log u} and X E [0. 1], it can be
derived that:
log Jog (1 \) 2 > log fc(y)) (1 A) log fa(U2) zAlog + (1 - A) log u log u,
ie., Ay + (1 1)92 E ly log Je(y) > log u) always holds, illustrating that the set {y
log fe(y) 2 log u is always concave. Because log u is a reversible function and takses value in (-00, 2 [0, oo). thus we have proven that (ii) in Lemma C.4 always satisfies for
Then, we prove (b) that 5 ă can lead to s' Recall from Eq. (5) that the alteration 2 is selected by:
+ = arg min
Mx, + Nv. + H
Consider linear equations define as:
H =s Mx,
Nv.-1
(10)
Since H is row full rank otherwise conflict the unique target assumption defined in Ass. 3.3 (and thus
the row number is greater or equal than the column number), the augmented matrix [H E Mx,
Nv,-1] must have the same rank as matrix H. This to say, there least one solution exist for Eq, (10),
and choosing ūt as the solution to Eq. (10), it can lead to = s (Mx; + Nv + HzŠ) 2 0.
In summary, we have proven that (a) e is random vector always satisfying the constraints (i) and (ii)
in Lemma C.4; and (b) s' 0 if z i hence, the proof of Thm. 3.4 is established.
18

---

C.4 Proof of Thm. 3,5
Theorem 3.5. When Y is singleton (i.e., |Y| = 1), let Vary Var[Y; | D2, Rh(zi)) and let A denote the rehearsal learning process. Define Var, and Varz as Varl 1 Ei=t Y; A under AutLL corresponding to Alg. I and Alg. 2, respectively. Then the following holds:
Var
1
Var2
1
and
Varo
L+1
Varo
(L+
Proof. Recall from Eq. (8), conditioned on Rh(z)) and D, (including X Vt-1), Yt can be expressed as:
Y Dr, RM(2)) = o VV, q +
where ey By since Ey is a zero vector with only its last element equal to 1; is constant depending on zi, X,, and vt-1; and Ea denotes the additive noise associated with the variable Y,, i.e. the last element of E; in Eq. (2). Hence, it follows that:
Varg = (c+ + e c + o)) (2)
(11)
To compute Var,, review the rehearsal learning process A from the GMuR method Alg. 1. Since Alg. 1 only focuses on the current Y; in each decision round i, T & 0 and be derived from Eq. (9) that:
In Alg. 1: M ej, e H J
Note that H is row vector in this case, thus can be derived that:
H'HHH)
and from line 8 of Alg. 1, it follows that (since H is row vector):
H s - cf=x; cfTvv-)).
(12)
Starting from time t, it can be derived for & sis t + that:
Y; c Ex; + Uz Tv,-1 Ce,)
=c ==X=x H + cgy - + o
(e a H)
=e =x + (s e. - efTvi-1) + +efiv +
=s + e
The 3rd equality holds from Eq. (12). Hence, it can be computed (under A r. from Alg, 1) that:
1
Var
L
1=1
t+L
1+L j-1
(Y,, +
2
Σ
(Y,
(L+ + 1)²
(L+1)²
!=!
j=(+1 i=t
(13)
H+
1+L j-1
+ 1) Σ
(s 6/ +
2
Σ
Σ
(s (x +5y)
(L+ 1)2
j=t+1
Σ
(L 1)
L+1
To compute Varz, we review the rehearsal learning process A from the FarMuR method Alg. 2.
It can be derived from Eq. (9) that in the last round (i.e., i t + I. or i = 1.)) of Alg. 2:
1 o/= = 1 Jr. = 1
In Alg. 2, decision round tz: M =
Jú.
L+1
1+1
L+1
19

---

Note that in round = te. the expression of M. N, H are similar to those in Alg. because it is the last decision round. Thus, can be derived (similar with Eq. (12)) that:
1 e ofT
H
S1LL L
Ex1+
(14)
L.
Hence, it follows that:
+ e (ExI+L + Ú2i + Tv++1-1 + CE(+L)
= EX (L + 1)H2 + eef +
(e a (L 1)H)
= ([++))544 Ex TT + Tv + ELL
=(I+ + 1)sc+L + cttL
H441
= + (L+1)
8 1-
Σ
Y
The 3rd equality holds from Eq. (14) and the 5th equality holds from the updating operation (line of Alg, 2). It can be further derived that:
1
Ev
1
L+I
L
L+
1=1
Hence, it can be computed (under A from Alg. 2) that:
H+
4+
1
Vare
Y
L+1
imt
Lmt
8
L+1 ItL:
L L+ -
(15)
( (CH+))E(+1)
(L + 1)2
1
(L + 1)2 (2)
Combining Eq. (11), Eq. (13), and Eq. (15) completes the proof of Thm. 3.5.
C.5 Proof of Thm. 3.6
Theorem 3.6. When N samples used to estimate Ô as in Appx. C.5, let 2. denote the alteration selected by solving Eq. (5) with θ, and i denote the one selected with true @. Under additional assumption that O is bounded and noise is Gaussian, it holds that | 22 < O(I/VN).
We first present the estimation of θ (including À and B) follows. In practical scenarios, the true parameter values are typically unknown. In such cases, we aim to estimate the parameters A and B from the historically collected data. Let P
PAN] denote the matrix of parent values for the j-th variable, and v Vv ...., vz the corresponding observed values of the variable itself. Then, the parameters associated with the generation of the j-th variable can be estimated via the following least squares estimation (LSE):
[PA]
1 2
arg min 3
v
Note that the parameter vector 3 represents the j-th row of the concatenated coefficient matrices A and B. Then the proof of Thm. 3.6 is detailed follows.
20

---

Proof. Define 5 AA BJ. According to Lemma C.3 of Du et al. [7], under the assumptions that ② is bounded and the noise follows Gaussian distribution, the parameter estimation error satisfies:
n - - A A
1
B -
N
We first analyze the Lipschitz continuity of the matrix functions M(2), N(2), and ((2)) defined in Eq- (9). Note that (I A) is invertible (i.e., full rank), which Lipschitz continuous w.r.t A as proven as: Let A1, 1, A R V be matrices such that A and A are invertible. Let II S denote the upper bound of the F norm. It can be A
the constant > O, 11
derived that:
|| A1) A2)
111 =I --) (A, A2)(T 12)
$ | (1- A
||A A2l1
r ||(I (11-A)) 1F
<> |A A2lFF.
In this case the resolvent (I- A) inherits the above Lipschitz property. Besides, the Lipschitz continuity of matrix multiplications (bounded matrices) can also proven as: For VS2, S22 and bounded matrices P(2), Q(2)) that are Lipschitz continuous w.r.t. Ω can be multiplicated together, it follows that:
P(R)) P(2 Q((2) P())) [Q(), Q( (P(). P(L2)] Q(2)).
Applying the triangle inequality for the Frobenius norm, follows that:
[P(2))Q(())) P(22))Q(22)l)
= P(),)) (Q()) - Q(R2)) + | [P()) - P(22) Q(
P(), Q( (0()(()))0(())lll----))) P P(R))]
- P(222)
S(UpLQ + +UDL)) -
Up and U2 are finite upper bounds because P, Q are bounded matrices.
Since M(2). N(2), and H(22) are constructed via matrix multiplications and such resolvents, they all have Lipschitz continuity w.r.t. Ω. Finally, because H is row full rank otherwise conflict the unique target assumption defined in Ass. 3.3, it can be verified that HH is full rank.
Mx, Nv,-1) Lipschitz contin- uous w.r.t. Ω because it has been proven that matrix multiplication and full-rank inverse (similar to the proof of (I- A) preserve Lipschitz continuity. Combining this with the parameter estimation error bound, we conclude:
Hence, the pseudo-inverse solution 2, (S2) H H (HHT)
(11) |
(M) L 1 ael so (N)
z
(52)
Taking square roots completes the proof: || -2; SO(1/VN).
D Experimental details
We provide the detailed information of Sec. and additional experiments this section. First, all experiments were run Nvidia Tesla A100 GPU and two Intel Xeon Platinum 8358 CPUs. Then we provide true parameters the synthetic dataset, with variables in the dataset illustrated Fig.
The synthetic dataset includes V | x² Zt, Z2 Z Zt, y y1,y,, and it holds that V, = AV, + BV + Et. where the instantaneous influence relations are recorded in:
0
0
0
0
0
0
0 07
0
0
0
0
0
1
0
0
0
0
0
0
0
0
A
0
0
0.5
1.3
0
0
0
0
0
2
0.4
0
0
0
0
-1
0
0
0.9
0
0
0
0
1.6
00.
0
21

---

x
Z
Zz
Y!
x
73
Y?
X
Z
7
Y!
V
x2+
Zi+
Zi+
Y?
X
Z
Y
Figure 9; The rehearsal graph for synthetic data.
and the lagged influenced relations are recorded in:
0
0
0
0
0
0
0.66
0.6
0
0
0
0
0.6
0
0
0
0
0
0
B
000
0
0
0.7
0
0
0
0
0.2
0
0
0
0
0
0
0.3
0
0
0
0
0
0
0
0
0
0
0
4
0000
0
0
0
6
0
0
0
0
Last, the noise mean Ele,] é 0, with Cov[e,] e E
0
0
6
0
0
0
0
00
0
0
0
0
3
1.6
0
0
0
1.6
6
0
0
0
0
0
0
4
0
0
0
12
This say, in the demonstrated experimental results in the main paper, if obeys Gaussian distribution, then e, ~O, E); while if e, obeys Laplace distribution, then the scale parameter b for each dimension can be computed by 212 a2, and the covariance of mutually influenced variables can additionally computed.
Fig. 11 reports additional results on the synthetic dataset, including the variance reduction rate w.r.t. the time window length (left, in Thm. 3.5), the execution time w.r.t. T' (middle), and the excess risk ق| - 1/1/2 w.r.. the number of observational samples N (right, as in Thm. 3.6), which are similar those the Bermuda dataset in Sec. 5. Note that the execution time of both methods (O(z))
[|z]) for the GMuR method and O(tt, to) /z)]2]// [v|[v]) for the FarMuR method) significantly reduced compared to the previous methods (O ((T + 1)( for Du al. [7] with the iteration times of the ulterlized optimization algorithm, and & O ((T + V))). Vp E Z4, for Qin et al. [6]). Meanwhile, as discussed in Qin et al. [6], the performance of QW723 [6] is unstable on this dataset because outcome variable is not singleton, leading to an irregular execution time curve because its time complexity w.r.t. other factors, such VV. could be varying.
22

---

window length
length (L)
Executing time per decision (8)
Executing time per decision
Greedy Far-sighted
Greedy Far-sighted
1332
9.00
00000
1.004
1.002
0.900
Figure 10: Results on the Bermuda dataset (left) and the synthetic dataset (right), illustrating the average executing time w.r.t. T for our FarMuR method and GMuR method.
Meanwhile, Fig. 10 reports additional results on the Bermuda dataset (left) and the synthetic dataset (right), illustrating the average executing time w.r.t. L for our FarMuR method and the GMuR method. This results (combining with AUF probability in Tab. 1) illustrate trade-off between execution time and AUF performance for the GMuR and FarMuR approaches L increases.
The variables in the Bermuda dataset are illustrated Fig. 12, including:
Light (X+);: Light levels at the bottom;
Temp (X²): Temperature the bottom;
Sal (X Sea surface salinity;
DIC (Z!): Dissolved inorganic carbon of seawater;
TA (Z²): Total alkalinity of seawater;
PA (Z²): Saturation with respect to aragonite in seawater;
Nut (Z*): PC1 of NH4, NiO₂ + NiO3, SiO4;
Chla (2*): Chlorophyll-a at sea surface;
pHsw (Z"): pH of seawater;
CO2 (Z"): Pco. of seawater;
NEC (Y')): Net ecosystem calcification.
The parameters associated with instantaneous influence relations ((A)) and the noise covariance matrix (Cov[s,]) are estimated by fitting least-squares linear models to the real-world data [41, 43].
length
length (L)
samples
[Cov] the target
Booty All
joout Mec-all of Gready Methed
Greedy
G20 Pe 2 With
Greedy
Koviy JAre. of For Sighted Method
Tana grded
Far-Sigheed Method
Figure 11: Results on the synthetic dataset, showing the variance reduction w.r.t. the time horizon T. average executing time w.r.t. T, and gap | 2, 2|2 w.r.t the number of observational samples N.
23

---

Z
x;
Z
zi
V
x²
23
Z
Y!
x²
Z
Z
Z
X
7
74+
V
x
Z
2
Y
x
Zi
Z
Z
Figure 12: The rehearsal graph for Bermuda data.
while the parameters associated with the lagged influence relations (B) are manually determined:
0.6
0
0
0
0.6
0
0
0
0
0.6
0
0
0
0
0
0
0.1
0.23
0
0
0
0
0.25
0
0
0
0
B
0
0
0
0
0
0
0
0
0.1
0
0
0
0
00000
000
0
0
0
0000
00
0
-1.1
0.6
24

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope as illustrated in Sec. 3.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The paper discusses the limitations of the work in Sec. A.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
25

---

The paper provides the full set of assumptions in Ass. 3.3, and complete proof for each theoretical result as illustrated in Appx. C.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The code and data for the main experimental results are provided in the supplemental material. Experimental details are introduced in Sec. 5 and Appx. D.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
26

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
The code and data for the main experimental results are provided in supplementalmateriall The dataset used in the experiments is also accessible from the link the reference.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (kttps://nips.c cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
gwwdee/eueeeg Codeeubmiiss onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The code and data for the main experimental results provided in the supplemental material. Experimental details are introduced in Sec. and Appx. D.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Ye]]
Justification: The error bars suitably and correctly defined or other appropriate informa- tion about the statistical significance of the experiments as illustrated in figures and tables in Sec. 5 and Appx. D.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
27

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
The paper provides sufficient information on the computer resources (type of compute workers, time of execution) needed to reproduce the experiments as illustrated in Appx. D.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https:/ //te:///neeupp/ee wttppp....pppueee cc/public/Etht htw.ol/ite/hhiicccciieeeeeeeee
Answer: [Yes]
Justification: The research conducted in the paper conforms with the NeurIPS Code of Ethics in every respect.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper has discussed in Sec. 1 that our proposed algorithm could be used to make decisions in certain cases, which may potentially lead to some positive societal
impacts.
28

---

Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considcrations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning conrectly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper does not release any data models that have high risk for misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual- use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The datasct used in the experiment is properly cited including a URL in reference. Meanwhile, the original papers of models used in the competitive experiment properly cited.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
29

---

The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thwoee com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The code and model introduced in the paper are well documented, and the documentation is provided alongside the assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/mmodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor researe h with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
30

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification:
The paper does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https: / //nnruurpp.... wwc/ooneeeencee/2000000 for what should or should not be described.
31


Reviews:
[{"review_id": "TobuJLa25G", "reviewer": "Reviewer_JXfA", "summary": "The paper proposes to solve the sequential AUF problem of linear systems. The algorithm is to extend the current rehearsal algorithm to the sequential setting and is reduced to a QP formulation to as the core optimization problem. The work proves that it has relative tight risk bound and the variance is also well bounded by the time horizon. Empirical result show a proof-of concept numerical experiments on the algorithm on multiple linear systems.", "questions": "1. Can the formulation and equivalent results been extended to non-linear systems?\n2. Why the dynamics have the specific graph structure?\n3. What is the potential application of this problem?", "limitations": "None", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strenghts:\n1. the problem is an natural extension of previuos work and the formation reduction to QP is also reasonable.\n2. Though i cannot understand the proofs in detail, it looks correct to me intuitively.\n3. The experiments support the results on different linear systems, include the common assumptions on the noise distribution.\n\nWeekness:\n1. the work only focuses on the linear system, which limits its applicability. If this is not easy, empirical evidence on non-linear system should also be useful.\n2. the graph structure definition of the sequential problem is not straightforward, which potentially cannot cover all problem definitions.\n3. no practical application being mentioned.  it is unclear why this setting and problem is useful and worth research.", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for your detailed feedback and for recognizing the strengths of our work! We hope that our responses can adequately address your concerns.\n\n---\n\n**W1&Q1.** Extension to nonlinear systems.\n\n**A1.** Thanks for your insightful question. The core ideas of our proposed algorithm can be extended to general nonlinear settings. In principle, as long as the relationship between (i) the mean of the aggregated outcome $\\sum_i Y_i/T$ and (ii) the selected alteration sequence $\\\\{z_i\\\\}$ can be captured, *e.g.*, via a parameterized functional form $f$ such that $$\\sum_i Y_i/T = f(\\theta; \\\\{z_i\\\\}) + \\tilde{\\varepsilon},$$  for some parameter $\\theta$ and residual $\\tilde{\\varepsilon}$, then we can turn to optimize $$\\arg\\min_{\\\\{z_i\\\\}}\\\\|f(\\theta; \\\\{z_i\\\\}) - s_k\\\\|_2^2$$ at decision round $k$, where $s_k$ denotes the adjusted center of the desired region.\n\nMoreover, our main theoretical results can be adapted to such nonlinear settings. Taking the above example, if the surrogate loss $\\\\|f(\\theta; \\\\{z_i\\\\}) - s_k\\\\|_2^2$ is convex with respect to $\\\\{z_i\\\\}$, then Thm. 3.4 still applies. This is because the proof of Thm. 3.4 is based on residual analysis, with a key step showing that the aggregated error $\\tilde{\\varepsilon}$ remains within the log-concave distribution family (Prop. 3.2). Therefore, as long as the nonlinearity preserves this property, our main results remain valid.\n\nWe will incorporate this discussion in the revised version. Thanks!\n\n---\n\n**W2&Q2.** Concern about the graph structure.\n\n**A2.** Thanks for the thoughtful question. We are not entirely sure if we have fully understood your concern; if our response does not address your point accurately, we would be happy to further clarify during the upcoming author-reviewer discussion period.\n\nOur framework focuses on a specific class of decision problems in which structural relations among variables can be leveraged to improve decision-making, particularly when opportunities for interaction are limited. This line of work falls under the umbrella of rehearsal learning [1, 2, 3], with connections to causal bandits and causal RL [4, 5].\n\nFurthermore, even when structural information is not explicitly available in the raw data, it can often be inferred using well-established structure learning methods [6], including in sequential [7] or cyclic [8] settings. Our method can then be applied based on the learned structure.\n\nWe will incorporate this clarification into the revised paper. Thanks again!\n\n---\n\n**W3&Q3.** Potential applications.\n\n**A3.** Thanks for the question. We would like to address it in two parts:\n\n- First, our proposed decision-making method leverages structural information. Leveraging structural information to aid decision-making has broad applications in domains such as healthcare [9], finance [10], and environmental science [11]. For instance, Simpson’s paradox [12] illustrates that accurately evaluating the effect of a treatment $X$ on an outcome $Y$ requires accounting for confounding variables, highlighting the need for causal structure in certain decision-making scenario. Recent advances in causal bandits [4, 5] and rehearsal learning [2, 3] similarly emphasize the importance of structure-aware decision-making in complex environments.\n\n- Second, our work extends prior rehearsal learning frameworks by explicitly incorporating sequential information, which is crucial in many real-world scenarios. For example, a physician prescribing medication over time for a chronic disease, or a financial institution regularly updating trading strategies. In such cases, methods that do not guarantee single-step optimality may accumulate errors over time [2, 3]. In contrast, our method guarantees optimality at each step (Thm. 3.4) and controls variance across the full sequence (Thm. 3.5). Our experimental results further support the effectiveness of our method in these settings.\n\n---\n\nYour suggestions are constructive and important for further improving the paper. Thanks again!\n\n---\n\n**References:**\n\n[1] Rehearsal: Learning from prediction to decision. FCS 2022.\n\n[2] Rehearsal Learning for Avoiding Undesired Future. NeurIPS 2023.\n\n[3] Avoiding Undesired Future with Minimal Cost in Non-Stationary Environments. NeurIPS 2024.\n\n[4] Bandits with unobserved confounders: A causal approach. NIPS 2015.\n\n[5] Causal bandits: Learning good interventions via causal inference. NIPS 2016.\n\n[6] DAGs with NO TEARS: Continuous Optimization for Structure Learning. NIPS 2018.\n\n[7] DYNOTEARS: Structure learning from time-series data. AISTATS 2020.\n\n[8] Discovering Cyclic Causal Models by Independent Components Analysis. UAI 2008.\n\n[9] Causal inference and counterfactual prediction in machine learning for actionable healthcare. Nature Machine Intelligence 2020.\n\n[10] The causal impact of media in financial markets. Journal of Finance 2011.\n\n[11] Environmental controls on modern scleractinian coral and reef-scale calcification. Science Advances 2017.\n\n[12] Causation, Prediction, and Search. MIT Press 2000."}, {"review_id": "KdDgp0hLxp", "reviewer": "Reviewer_eWH7", "summary": "This paper introduces a new approach for \"Avoiding Undesired Future\" (AUF) problems in machine learning, extending existing methods to handle long-term decision-making scenarios. The authors propose a variance-reduced rehearsal learning framework that uses quadratic programming (QP) reformulation to transform intractable probabilistic optimization into a tractable one when making decisions to prevent predicted undesired outcomes over extended time horizons.", "questions": "1) How should practitioners handle model uncertainty and potential distribution shift over time?\n\n2) What monitoring and adaptation strategies are recommended when the system is deployed in dynamic environments?\n\n3) How can the approach be integrated with existing decision-making systems that may have additional constraints not captured in the current formulation?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n\n1) The paper provides rigorous theoretical guarantees including optimality of the QP reformulation under mild assumptions, variance reduction analysis, and O(1/√N) excess risk bounds\n\n2) The quadratic programming reformulation transforms an intractable probabilistic optimization into an efficiently solvable problem\n\n3) The far-sighted method achieves substantial performance gains (e.g., 95% vs 32% AUF probability on synthetic data)\n\nWeaknesses:\n\n1) The optimality guarantees require three specific assumptions (linear systems, unique targets, symmetric desired regions) that may not hold in many real-world scenarios\n\n2) Only two datasets (one synthetic, one environmental) are used for evaluation\n\n3) While theoretically efficient, the paper doesn't adequately address how the methods scale to high-dimensional problems or very long time horizons", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable feedback and appreciation of our work! We hope that our responses can address your concerns.\n\n---\n\n**W1.** Concerns about the assumptions.\n\n**A1.** Thanks for the insightful question. While our theoretical guarantees (Thm. 3.3) rely on certain assumptions, our method remains effective in practice even when these assumptions are partially violated, as evidenced by the new experimental results in **A2**. \n\nThe latter two assumptions, *i.e.*, *unique targets* and *symmetric desired regions*, are often satisfied in real-world applications, as they are typically determined by the decision-maker. As for the linear system assumption, we would like to clarify that we have already generalized the noise distribution beyond the Gaussian assumption used in prior works [1, 2], allowing for symmetric log-concave noise. \nMoreover, potential non-linearities could be addressed using standard techniques such as kernel methods [3, 4]. Due to space constraints, we kindly refer you to our response to Reviewer JXfA (**A1**) for a more detailed discussion of the linearity assumption.\n\nWe will incorporate this discussion into the revised paper. Thanks!\n\n---\n\n**W2.** Concerns about the experiments.\n\n**A2.** Thanks for your question. To address your concern, we conducted additional experiments on a new dataset to evaluate our method's performance when the assumptions are violated. \nThe table below shows the performance on this new dataset (where dimensions of $X$, $Z$, and $Y$ are 3, 2, and 2, respectively and the horizon is set to 9) when assumption (i) symmetric desired region and/or (ii) unique target assumptions are violated. To ensure a fair comparison, when violating assumption (i) (symmetric desired region), we used a desired region with a total area similar to that of the no violation setting.\n\n| Violation | QWZ23 [2] | MICNS [3] | Ours(Far-Sighted) |\n| - | - | - | - |\n|No|$0.342\\pm 0.017$ |$0.332\\pm 0.025$ |$0.765\\pm 0.128$ |\n| (i) |  $0.225\\pm 0.011$ | $0.250\\pm 0.037$ | $0.603\\pm 0.015$|\n| (ii) | $0.094\\pm 0.027$ | $0.052\\pm 0.020$ | $0.278\\pm 0.166$|\n| (i)&(ii) | $0.032\\pm 0.028$ | $0.062\\pm 0.028$ | $0.192\\pm 0.123$|\n\nAs the results show, while not explicitly discussed in previous work, violating these assumptions degrades performance of the rehearsal learning approaches, especially the unique target assumption. Although optimality is not guaranteed in these settings, our method still significantly outperforms the prior works.\n\nFinally, we would like to respectfully clarify that whenever Ass. 3.3 holds, our method is theoretically guaranteed to perform well, as established by Thms 3.4, 3.5, and 3.6.\n\n---\n\n**W3.** Concerns about the scalability.\n\n**A3.** Thanks for your thoughtful question. Indeed, in sequential decision-making problems, both dimensionality and the time horizon can be large. We would like to highlight that, as shown in Fig. 4(mid) of our paper, our method is significantly faster than existing rehearsal learning approaches [1, 2] for a long time horizon when the dimensionality is fixed. For further reducing the executing time in decision-making, we propose two feasible directions: (i) In high-dimensional settings, heuristic methods can be developed to select a key subset of variables $Z_s \\subset Z$ and approximately solve Eq. (5) over $Z_s$; (ii) For long horizons, the matrices ($M$, $N$, $H$ in Eq. (5)) that depend on the horizon length can be precomputed and cached offline to accelerate real-time inference.\n\nWe will incorporate this discussion into the revised paper. Thanks!\n\n---\n\n**Q1.** Potential distribution shift over time.\n\n**A4.** Thanks for your question. The rehearsal learning process involves two sequential steps:\n- **Step 1.** Estimate/Update the underlying system parameters $\\theta$ from historical observational data.\n- **Step 2.** Use $\\theta$ to construct system matrices (e.g., $A$, $B$ in Eq. (2)), and solve Eq. (5) for the decision alterations.\n\nThe model uncertainty and potential distribution shifts primarily affect the parameter estimation in Step 1. To handle model uncertainty (*e.g.*, a time-varying graph structure), one could segment the data based on different structures and learn a separate set of parameters for each. To address distribution shifts of the data, if the shift is gradual, some online learning methods can be employed to update the parameters [5]; and if the shift is abrupt, a restart mechanism can be implemented [6], *i.e.*, discard the old parameters upon detecting a major shift and re-estimate them from new samples.\n\n---\n\n**Q2.** How to deal with dynamic environments.\n\n**A5.** Thanks for your question. As noted in **A4**, our framework follows a two-step sequential process. The challenge of dynamic environments, where system parameters $\\theta$ change over time, primarily pertains to the parameter estimation in Step 1. This issue has been explored in prior work on rehearsal learning [2], and similar adaptive strategies could potentially be incorporated into our framework as well.\n\n---\n\n**Q3.** How to integrate inter additional constraints.\n\n**A6.** Thanks for the insightful suggestion. Prior works in rehearsal learning [1, 2] have considered integrating additional constraints (e.g., decision cost). If such constraints define a convex feasible set for the decision variables, they can be naturally incorporated into our QP reformulation (Eq. (5)) without compromising convexity, allowing for efficient optimization.\n\nWe will incorporate this discussion into the revised paper. Thanks!\n\n---\n\nWe also take this opportunity to sincerely thank you for the careful review. Your suggestions are very important for further improving the paper. Thanks again!\n\n---\n\n**References:**\n\n[1] Rehearsal Learning for Avoiding Undesired Future. NeurIPS 2023.\n\n[2] Avoiding Undesired Future with Minimal Cost in Non-Stationary Environments. NeurIPS 2024.\n\n[3] Kernel-based conditional independence test and application in causal discovery. UAI 2011.\n\n[4] Distinguishing Cause from Effect Using Observational Data: Methods and Benchmarks. JMLR 2016.\n\n[5] Adaptive online learning in dynamic environments. NIPS 2018.\n\n[6] Learning to optimize under non-stationarity. AISTATS 2019."}, {"review_id": "zHlb0exyPU", "reviewer": "Reviewer_2xvd", "summary": "The paper introduces the AUF (avoiding undesirable futures) problem for maximizing the probability of a sequence of future outcomes being in a certain set S. They work on a simple case of the problem - a linear markovian transition structure that is essentially a linear dynamical system. This allows them to formulate the problem as a QP, which they solve with both a greedy and a far-sighted method.", "questions": "1. Could you address the implicit questions within my discussion of the paper’s weaknesses?\n2. The noise in the trajectory is non-isotropic, so I find it a bit odd that the way to maximize the probability is to optimize an objective with L2 loss, instead of an L2 loss transformed by a matrix M.", "limitations": "The paper does not address the strength of the linearity assumption, and I think it also needs to spend some time verifying to what extent the assumption holds in its dataset (to what extent can the best fit linear transition model explain the observations in the dataset).", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The paper extends rehearsal learning from single-round AUF to multi-round, long-term settings, which better reflects many practical problems (e.g., portfolio management, sequential decision-making).\n2. It establishes optimality of the QP solution under reasonable (but strong) assumptions and derives variance reduction rates (Theorem 3.5) that are directly relevant for sequential AUF objectives.\n3. The quadratic programming reformulation provides a computationally efficient way to handle the long-term AUF problem, avoiding sampling-based or MILP approaches that scale poorly with the dimensionality of variables.\n4. The experiments, while limited, clearly show that the proposed far-sighted method achieves substantially higher AUF probabilities than both baselines and existing rehearsal learning methods across multiple datasets.\n\nWeaknesses:\n1. I think the linear assumption that crucially allows the paper to design a QP formulation of the AIUF problem is quite strong, and given the lack of experiments, it is unclear how well it holds up.\n2. The linear assumption also turns this problem into a linear dynamical system, unless I am mistaken. There is extensive literature in control theory, to my understanding, which addresses the problem of controlling a linear dynamical system to let it end up in a desirable set. I am not sure if the exact problem in this work is addressed in control theory literature, but the work is worth diving into and reference.\n3. Many arguments about the ineffectiveness of RL are about its stated “inability to use existing knowledge about systems.” I think this is incorrect for model-based RL. In fact, an experimental comparison with model-based RL methods under the same linear model is missing. The argument for reward drift seems incorrect to me - you can just have the reward be the indicator function of the desired set S, making it a deterministic reward with no distribution shift. The actual value of the reward now just depends on the observed Y_t.", "was_revised": true, "final_justification": "The empirical justification of their linearity assumption and the discussion of their relation to MPC have largely satisfied me, but I don’t think they have faithfully addressed their relation to causal RL, as I discuss in my response. So my score had gone from a 3 to a 4 but not to a 5.", "rebuttal": "Thanks for your detailed feedback and for recognizing the strengths of our work! We hope our responses below can address your concerns.\n\n---\n\n**W1&Limitation.** Concerns about the linear assumption.\n\n**A1.** Thanks for your constructive suggestion. First, we would like to clarify that our linear assumption generalizes the noise distribution to the log-concave family, which is a broader class than that considered in prior works [1, 2]. While this assumption is essential to enabling our QP reformulation, we also further discuss its generality and empirical validity as follows:\n\n- **Potential extensions to non-linear settings.**  \n  The core ideas behind our algorithm are not limited to linear models. Specifically, our decision-making objective only requires that the relationship between the aggregated outcome $\\sum_i Y_i/T$ and the sequence of alterations $\\\\{z_i\\\\}$ can be parameterized with additive aggregated noise. That is, if there exists a functional family $f$ such that $$\\sum_i Y_i/T = f(\\theta; \\\\{z_i\\\\}) + \\tilde{\\varepsilon},$$ for some parameters $\\theta$ and aggregated noise $\\tilde{\\varepsilon}$, then the optimization at each decision round $k$ can proceed by minimizing the surrogate loss $$\\\\|f(\\theta; \\\\{z_i\\\\}) - s_k\\\\|_2^2,$$ where $s_k$ denotes the adjusted center of the desired target region. This observation suggests that our formulation is not inherently tied to linear dynamics and may naturally extend to some general non-linear settings.\n\n- **Empirical validation of the linearity assumption.**  \n  To assess the adequacy of the linear assumption, we fitted the model using least squares and evaluated its fit using standard metrics: (i) per-coordinate $R^2$ scores, (ii) normalized mean squared error (NMSE), and (iii) normalized mean absolute error (NMAE).\n\n  The linear model achieves average $R^2$ scores of `0.869/0.686`, NMSE of `0.212/0.401`, and NMAE of `0.305/0.468` on the synthetic / real-world (Bermuda) datasets, respectively. For other datasets, this validation step can also be performed to verify the suitability of the linear assumption.\n\nTaken together, these points support both the current validity and future extensibility of our formulation. We thank the reviewer again for raising this important issue.\n\n---\n\n**W2.** Linear dynamical system and control theory.\n\n**A2.** Thanks for your insightful question. Our approach differs from classical control methods, such as online linear quadratic control (LQR) [3, 4], in several key ways:\n\n- **Different Motivations.** Traditional LQR assumes a linear dynamical system of the form $x_{t+1} = Ax_t+Bu_t+w_t$ and aims to control the state trajectory $\\{x_t\\}$ over time. In contrast, our goal is to maximize the probability that the **aggregated outcome** $\\sum_iY_i/T$ lies within a desired region, without being concerned about the specific trajectory of $X$, $Y$, or $Z$ at individual time points.\n\n- **Distinct Formulations.** Our system cannot be reduced to the standard LQR formulation, as the alteration can also affect variables *within* the same time point. Moreover, our problem is fundamentally defined by Eq. (3), and Eq. (5) is a heuristic reformulation under certain assumptions. Even so, the resulting QP in Eq. (5) differs from that in LQR: while online LQR decomposes into per-step QPs (w.r.t. control variables like $\\sum_t(x_t^\\top Qx_t+u_t^\\top Ru_t)$) due to additive structure, our objective involves aggregation of altered variables $z_t$ through matrix $F$, making the problem cannot be decomposed thus more challenging.\n\n- **Connection to MPC.** While our method shares a high-level similarity with Model Predictive Control (MPC) [5], in that we optimize a future sequence of actions but only execute the first, our predictive model is different from classic ones. We rely on structured dependencies (see **A3**) that go beyond the scope of standard MPC formulations.\n\nWe will incorporate this discussion into the revised version. Thanks again!\n\n---\n\n**W3.** Comparison with RL methods.\n\n**A3.** Thanks for your thoughtful comment. It appears there may be a misunderstanding of our problem setting, and we appreciate the opportunity to clarify the differences between our structural approach and conventional RL, including model-based RL.\n\nYour question highlights a key advantage of the rehearsal learning framework. Simply formulating a reward function with an indicator $\\mathbb{I}(Y \\in S)$ is insufficient for addressing decision problems with structural dependencies (like causal relations), especially those involving confounding. Even when the indicator function is deterministic, it is crucial to distinguish between *samples collected from different data distributions* (observation or interaction/intervention), as this distinction fundamentally impacts the decision-making process.\n\n- **Observation vs. Intervention.**  \n  Our setting relies solely on *observational data* to make immediate decisions, where structural (e.g., causal) relations exist. In contrast, standard RL methods assume *interventional data* from agent-environment interactions. Treating observational samples $\\langle x, z, \\mathbb{I}(Y \\in S) \\rangle$ as if they were standard RL tuples $\\langle s, a, r \\rangle$ implicitly assumes that the observational and interventional distributions are identical, an invalid assumption in the presence of confounding.\n\n    *E.g.*, suppose the structure includes: $X\\rightarrow Y$, $X\\rightarrow Z_a\\rightarrow Y$, $Z_a^\\prime\\rightarrow Z_u$ and $Z_a\\leftarrow Z_u\\rightarrow Y$, where $X$ can be viewed as the state in RL, $Z=(Z_a,Z_a^\\prime)$ are two actionable variables, and $Z_u$ is an observed but unactionable confounder. Observational data might show a strong correlation between $Z = z$ and $Y\\in S$ given $X=x$, leading an RL agent to learn $z$ as the optimal action incorrectly. In contrast, our method can identify the optimal action $z^\\star\\ne z$ without requiring costly exploration by leveraging structural information and applying *backdoor criteria* [10].\n\n- **Structural vs. Statistical Knowledge.**  \n  While model-based RL can incorporate prior knowledge, it typically captures **statistical transition dynamics** (e.g., $p(s_{t+1} \\mid s_t, a_t)$) rather than **fine-grained structural dependencies**. Our method leverages the latter, which is essential to distinguish causation/influence relations from purely correlation.\n\n- **New Experiments.** To empirically illustrate this distinction, we compared our method with both model-free (DDPG [6], SAC [7]) and model-based (MBPO [8]) RL agents. Each was allowed 100 environment interactions, the same sample budget available to our method from observational data.\n\n  |Method|Ours|[6]|[7]|[8]|\n  |-|-|-|-|-|\n  |AUF Probability|$0.827\\pm 0.044$|$0.207\\pm 0.055$|$0.182\\pm 0.039$|$0.228\\pm 0.027$|\n\nAll experiments were run on the same dataset, with the same region $S$ and horizon 1, across 100 rounds and 10 random seeds. RL methods underperform due to their inability to exploit *structural information*, resulting in slow convergence. Notably, SAC can eventually reach comparable performance, but only after ~6,000 interactions, underscoring the sample inefficiency of RL in our context.\n\nIn summary, our work is motivated by the need to incorporate structural knowledge, which is essential for making decisions in the presence of confounding. We hope this clarifies the distinction and illustrates why our approach is better suited to this class of problems than standard RL pipelines.\n\n---\n\n**Q2.** Non-isotropic noise.\n\n**A4.** Thank you for this insightful question. Your observation is very sharp and touches upon a key aspect of our theoretical contribution.\n\nIndeed, when the target variable $Y$ is multi-dimensional, the noise in the aggregated outcome $\\sum_i Y_i/T$ is possible to be non-isotropic. However, this does not affect our decision-making strategy. Let us revisit the core objective of our AUF problem: given an observation $X=x$, we aim to choose an alteration sequence $\\\\{z_i\\\\}$ to maximize the probability of $\\sum_i Y_i/T \\in S$. In this context, our Prop. 3.1 guarantees that the entire stochasticity of the aggregated outcome is captured by the aggregated zero-mean noise term, $F\\tilde{e}$, which is **unaffected** by the choice of the choosed action variables $\\\\{z_i\\\\}$.\n\nConsequently, our optimization in Eq.(5) is performed over the sequence of action variables $\\\\{z_i\\\\}$. The choice of norm for this optimization does not alter the stochastic term $F\\tilde{e}$ that governs the randomness of the outcome $\\sum_i Y_i/T$. The reason our method successfully maximizes the probability of the outcome falling into the desired region $S$ is guaranteed by established results in probability theory, involving key lemmas such as Anderson's Theorem [9]. Should you be interested, a detailed proof is provided in Appx. D.3.\n\n---\n\nWe hope our detailed responses have addressed your concerns. If any points remain unclear, we would be happy to elaborate further. In light of these clarifications, we would be grateful if you would reconsider your evaluation. Thank you again for your thoughtful feedback!\n\n---\n\n**Reference**\n\n[1] Rehearsal Learning for Avoiding Undesired Future. NeurIPS 2023.\n\n[2] Avoiding Undesired Future with Minimal Cost in Non-Stationary Environments. NeurIPS 2024.\n\n[3] Online Linear Quadratic Control. ICML 2018.\n\n[4] Naive Exploration is Optimal for Online LQR. ICML 2020.\n\n[5] Model predictive control: Theory and practice—A survey. Automatica 1989.\n\n[6] Continuous control with deep reinforcement learning. ICLR 2016.\n\n[7] Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. ICML 2018.\n\n[8] When to Trust Your Model: Model-Based Policy Optimization. NeurIPS 2019.\n\n[9] The integral of a symmetric unimodal function over a symmetric convex set and some probability inequalities. American Mathematical Society 1955.\n\n[10] Causality: Models, Reasoning and Inference."}, {"review_id": "54a9fFBiXO", "reviewer": "Reviewer_qTb1", "summary": "This paper introduces a novel approach to the Avoiding Undesired Future (AUF) problem by extending the rehearsal learning framework to long-term decision-making scenarios. It reformulates the intractable probabilistic AUF objective, maximizing the likelihood that the average of future outcomes falls within a desired region, into a tractable quadratic programming (QP) problem under mild assumptions such as linearity and log-concave noise. The authors propose two algorithms: a greedy method that optimizes each decision step independently, and a far-sighted method that accounts for future consequences to achieve better variance reduction and overall performance. Theoretical guarantees are provided for the optimality of the QP solution, variance reduction, and robustness when using estimated parameters. Empirical evaluations on synthetic and real-world datasets confirm the proposed methods significantly outperform existing approaches in both accuracy and efficiency.", "questions": "1. What is the practical computational limit (in terms of time horizon $𝑇$, dimensionality $|Z|$, and data size) for applying the far-sighted method in real-time or near real-time applications?\n2. Can the method accommodate settings where the decision horizon $T$ is not known in advance or varies over time?\n3. The paper proposes algorithms that compute optimal alterations in closed form (via QP), but it remains unclear how interpretable these solutions are. Given the practical relevance (e.g., in policy or finance), can the authors discuss whether the alteration vectors can be explained to domain experts, or whether some sparsity constraints or diagnostics could be incorporated?\n4. While the paper benchmarks against previous rehearsal-based methods (QWZ23, MICNS), the experimental comparison omits broader classes of structured decision-making or robust control algorithms. Can the authors clarify why alternatives were not considered? Could a simplified comparison or qualitative discussion be added to contextualize where rehearsal-based approaches stand in the broader decision-making landscape?", "limitations": "The authors have adequately addressed the limitations of their work in the Appendix B (Discussion) section. They explicitly acknowledge that their proposed QP reformulation may be sub-optimal under certain conditions. This reflects a clear effort to communicate the boundaries within which their methods can be expected to perform reliably.\n\nOne suggestion for improvement: It would be helpful to explicitly discuss potential negative consequences of automation in sensitive domains (e.g., economic systems, healthcare, or policy decision-making) where biased or poorly specified “undesired regions” (AUF targets) might lead to unintended harms if not designed responsibly.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths: (1) The paper is well-grounded in theory. The authors provide a principled extension of the AUF framework to long-term settings, supported by a rigorous QP reformulation. (2) The paper is generally well-organized and clearly written. Key concepts are introduced with sufficient motivation, and the technical derivations are presented with clarity. (3) The long-term AUF setting is practically important and aligns with real-world needs, where decisions often have delayed and cumulative consequences.\n\nWeaknesses: (1) While the QP reformulation is elegant, its optimality depends on a set of assumptions (e.g., linearity, unique targets, symmetric desired region). These are practical in many settings, but the paper could benefit from more detailed discussion and empirical evaluation of how violations of these assumptions affect performance. While a counterexample is provided, its role is limited to theoretical discussion. (2) The far-sighted method achieves superior performance but with increased computational cost. While this is acknowledged, a deeper analysis of scalability with respect to horizon length, dimensionality, and system size would improve practical understanding. (3) The comparison excludes reinforcement learning methods based on an assumption of interaction constraints. While justified, the paper would be stronger with at least some controlled comparisons or sensitivity analyses to confirm this assumption empirically, especially since many AUF problems in practice involve some feedback or batch interaction.", "was_revised": true, "final_justification": "I have reviewed their responses to other reviewers regarding non-linear settings, which have addressed some of the concerns I initially raised. Based on these clarifications and the improvements made, I am increasing my score to 4.", "rebuttal": "We sincerely thank you for your detailed feedback and for recognizing the strengths of our work, including its strong theoretical grounding, clear presentation, and the practical importance of the long-term AUF setting. We hope our responses below fully address your concerns.\n\n---\n\n**W1.** Empirical evaluation of the assumptions.\n\n**A1.** Thanks for this constructive suggestion. To empirically assess the impact of violating our assumptions, we conducted new experiments where the (i) *symmetric desired region* and/or (ii) *unique target* assumptions are violated. The table below presents results on a synthetic dataset (with dimensions of $X$, $Z$, and $Y$ set to 3, 2, and 2, respectively, and a decision horizon of 9). To ensure a fair comparison, when violating assumption (i), we chose a desired region with a total area similar to that in the no-violation setting.\n\n| Violation | QWZ23 [1] | MICNS [2] | Ours (Far-Sighted) |\n| - | - | - | - |\n| No | $0.342 \\pm 0.017$ | $0.332 \\pm 0.025$ | $0.765 \\pm 0.128$ |\n| (i) | $0.225 \\pm 0.011$ | $0.250 \\pm 0.037$ | $0.603 \\pm 0.015$ |\n| (ii) | $0.094 \\pm 0.027$ | $0.052 \\pm 0.020$ | $0.278 \\pm 0.166$ |\n| (i)&(ii) | $0.032 \\pm 0.028$ | $0.062 \\pm 0.028$ | $0.192 \\pm 0.123$ |\n\nThese results show that although performance degrades when assumptions are violated, our method remains substantially more robust than existing approaches [1, 2], particularly when the *unique target* assumption fails. This offers strong empirical support for the practical utility of our approach, even beyond the scope of our theoretical guarantees.\n\n---\n\n**W2&Q1.** A deeper analysis of scalability.\n\n**A2.** Thanks for raising this important point. We agree that understanding the computational scalability of our method is crucial, especially for large dimensions and long horizons. As shown in Fig. 4 (middle) of our paper, our approach is already significantly faster than prior rehearsal-based methods [1, 2] for long horizons with fixed dimensionality.\n\nTo further address scalability, we propose two practical strategies:\n(i) In high-dimensional settings, heuristic methods can be developed to select a key subset of variables $Z_s \\subset Z$ and approximately solve Eq. (5) over $Z_s$;\n(ii) For long horizons, the matrices ($M$, $N$, $H$ in Eq. (5)) that depend on the horizon length can be precomputed and cached offline to accelerate real-time inference.\n\nWe will incorporate this discussion into the revised version. Thank you!\n\n---\n\n**W3&Q4.** Empirical comparison with other methods.\n\n**A3.** Thanks for this insightful question. We agree that positioning our method within the broader decision-making landscape is important. While we primarily compare with rehearsal-based methods due to (i) the specific problem setting (detailed in Appx. A), and (ii) prior empirical evidence showing that alternative methods (e.g., RL) achieve limited effectiveness in this setting [1, 2], we conducted additional experiments in response to your suggestion.\n\nWe include comparisons with representative RL methods (DDPG [3], SAC [4]) and a recent causal bandit algorithm [5]. For fairness, we allowed the RL agents to interact with the environment for 100 episodes—the same sample budget available to our method from observational data.\n\n| Ours | [3] | [4] | [5] |\n| - | - | - | - |\n| $0.827 \\pm 0.044$ | $0.207 \\pm 0.055$ | $0.182 \\pm 0.039$ | $0.166 \\pm 0.022$ |\n\nAll experiments were conducted on the same dataset, with the same desired region $S$ and a fixed horizon 1, over 100 rounds and 10 random seeds. The table reports the AUF probability mean and variance. Our method clearly outperforms these baselines, as the rehearsal-based approach is more effective at leveraging fine-grained structural dependencies among variables and observed context $X$.\nWe also observed that SAC can eventually reach comparable performance after ~6,000 training steps. However, such extensive interaction is often not feasible in our problem setting.\n\n---\n\n**Q2.** Applicability to varying decision horizons.\n\n**A4.** Thanks for the thoughtful question.\n\nWe would like to respectfully clarify that the decision horizon $T$ is a user-specified parameter (thus should not be unknown). For instance, if a trader aims to maximize profit over the next 10 months, they would simply set $T=10$.\n\nFor a *varying* decision horizon setting, our main theoretical result (Thm. 3.4) can be straightforwardly applied to such scenarios. Thm. 3.4 provides a strong guarantee: for any finite future horizon $T$ and any decision step $t$, the solution to Eq.(5) is optimal in maximizing the AUF probability of the aggregated target $\\sum_i Y_i/T$, assuming Ass. 3.3 holds. If decision-makers choose to specify different horizons $T$ at different steps $t$, our method remains applicable and optimal in this context. One only needs to recompute the corresponding matrices ($M$, $N$, $H$) in Eq.(5) for the specified $T$.\n\n---\n\n**Q3.** Interpretability and sparsity constraints.\n\n**A5.** Thank you for highlighting this important issue. When the horizon is 1, the solution is easy to interpret: it directly maximizes the probability that the immediate outcome $Y$ lies within the desired region $S$. For longer horizons, interpretation becomes more subtle: the optimal alteration at each step maximizes the probability that the *average outcome* across the horizon falls in $S$. \nUsing a Texas Hold'em analogy: if the goal is to win the current hand, the strategy should reflect immediate hand strength. But if the goal is to maximize winnings over 100 hands, one may initially build a specific table image (e.g., tight or loose) that improves long-term return, even if it sacrifices short-term gains.\n\nRegarding sparsity, since Eq. (5) is a convex QP, we can include sparsity-inducing regularizers (e.g., L1 penalty, as in LASSO) without losing convexity. This helps generate more interpretable alteration vectors for domain experts.\n\nWe will add this discussion to the revised version. Thanks!\n\n---\n\n**Other questions.** Suggestions and formatting.\n\n**A6.** Thanks for your detail and helpful suggestions. We will include a discussion of potential negative consequences in the Appendix and Checklist of the revised version.\n\nRegarding reference formatting: we would like to respectfully clarify that our use of `\\citet` and `\\citep` is intentional, following standard citation practices to distinguish between in-text (e.g., Author [X]) and parenthetical ([X]) citations. Meanwhile, we have used the `\\bibliographystyle{unsrtnat}` style consistently throughout the paper. Thanks!\n\n---\n\nWe hope that our responses and additional experiments address your concerns. We would greatly appreciate your reconsideration of the paper in light of these clarifications. Thank you again for your valuable and constructive feedback.\n\n---\n\n**Reference**\n\n[1] Rehearsal Learning for Avoiding Undesired Future. NeurIPS 2023.\n\n[2] Avoiding Undesired Future with Minimal Cost in Non-Stationary Environments. NeurIPS 2024.\n\n[3] Continuous control with deep reinforcement learning. ICLR, 2016.\n\n[4] Soft actorcritic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. ICML, 2018.\n\n[5] Additive Causal Bandits with Unknown Graph. ICML, 2023."}]

Author rebuttal:

Meta-review: This work introduces a new approach for the “Avoiding Undesired Future” problems in linear systems. The authors propose a variance reduced rehearsal learning framework based on quadratic programming that can be used to make tractable the optimization problem at the heart of AUF. The paper goes on to prove a tight risk bound and empirical results show that their methods work well via numerical experiments in linear systems. The reviewers agreed this work meets the requirements for publication at Neurips.
Decision: Accept (poster)