Enabling Optimal Decisions in Rehearsal Learning under CARE Condition
Authors: Wen-Bo Du
Hao-Yi Lei
Lue Tao
Tian-Zuo Wang
Zhi-Hua Zhou
Venue: ICML 2025 poster (2025)
Keywords: decision-making
probabilistic optimization
structural model
Primary area: probabilistic_methods->graphical_models
TL;DR: We present a novel approach to identify optimal decision actions in rehearsal learning.

Abstract: In the field of machine learning (ML), an essential type of decision-related problem is known as AUF (Avoiding Undesired Future): if an ML model predicts an undesired outcome, how can decisions be made to prevent it? Recently, a novel framework called *rehearsal learning* has been proposed to address the AUF problem. Despite its utility in modeling uncertainty for decision-making, it remains unclear *under what conditions* and *how* optimal actions that maximize the *AUF probability* can be identified. In this paper, we propose *CARE* (CAnonical REctangle), a condition under which the maximum AUF probability can be achieved. Under the CARE condition, we present a projection-Newton algorithm to select actions and prove that the algorithm achieves superlinear convergence to the optimal one. Besides, we provide a generalization method for adopting the algorithm to AUF scenarios beyond the CARE condition. Finally, we demonstrate that a closed-form solution exists when the outcome is a singleton variable, substantially reducing the time complexity of decision-making. Experiments validate the effectiveness and efficiency of our method.

Enabling Optimal Decisions in Rehearsal Learning under CARE Condition
Wen-Bo Du 12 Yi Lei
Lue Tao 12 Tian-Zuo Wang" 12 Zhi-Hua Zhou 12
Abstract
Correlation
Prediction
In the field of machine learning (ML), an essential type of decision-related problem is known AUF (Avoiding Undesired Future): if an ML model pre- dicts undesired outcome, how can decisions be made to prevent it? Recently, a novel frame- work called rehearsal learning has been proposed to address the AUF problem. Despite its utility in modeling uncertainty for Dacissionmmkiige it remains unclear under what conditions and how optimal actions that maximize the AUF probabil- ity can be identified. In this paper, we propose CARE (CAnonical REctangle), condition un- der which the maximum AUF probability can be achieved. Under the CARE condition, we present a ioojection-Newtane algorithm to select actions and prove that the algorithm achieves superlinear convergence to the optimal one. Besides, we pro- vide a generalization method for adopting the al- gorithm to AUF scenarios beyond the CARE con- dition. Finally, we demonstrate that a closed-form solution exists when the outcome is a singleton variable, substantially reducing the time complex- ity of decision making.. Experiments validate the effectiveness and efficiency of our method.
Influence
Decision
Actionable
Causation
Discovery
Figure I. Relations between correlation, influence, and causation (reproduced from Zhou (2022)).
model that takes real-time environmental data (denoted as X) as input and predicts the risk of package loss (denoted as Y). For particular delivery, if the prediction indicates high risk of loss, the system will consider altering values of some intermediate control variables (denoted as Z), such as altitude, speed, or route, to mitigate the risk. Since ex- perimenting with real deliveries to explore better actions is costly and potentially results in lost packages, different alterations Z must be carefully considered. Let S denote the desired region of Y where the risk of package loss is sufficiently low; the AUF problem can be interpreted as find- ing the optimal alteration z that maximizes the probability of Y E S, which is called the AUF probability (IP AUP).
Generally, decision-makenn lack access to an explicit for- mula for IP PUU w.s. alterations, which complicates the se- lection of effective alterations. To model IP PAF,, need to understand the relations among variables (X, Z, Y While correlation is often sufficient for prediction in ML models, effective decision-making demands a more nuanced under- standing of variable relations. In certain decision decision-making contexts, causal relations are helpful to model these rela- tions (Bareinboim et al., 2015; Lattimore et al., 2016). In practice, identifying causal relations is essentially challeng- ing, as it typically relies on untestable assumptions (Spirtes et al., 2000; Schölkopf et al., 2021). However, com- plete causal understanding is not a prerequisite for effective decision-making, as evidenced by human decision-make makerss who could successfully make decisions without such com- prehensive knowledge (Zhou, 2022). Furthermore, even when causal factors are identified, they may not be action- able for decision decision-making purposes. Additionally, variables that influence outcomes are not necessarily causal factors. Recognizing that correlation is insufficient and causation is somewhat luxurious for decision maaiiin.mmaiing... Zhou (2023) propose the influence relation as an intermediate concept between correlation and causation, as depicted in Fig. 1.
1. Introduction
Machine Learning (ML) models have achieved great success in various real-world prediction tasks (LeCun et al., 2015). Instead of solely focusing on the prediction, Zhou (2022) emphasizes another important issue, i.e., if the prediction of ML model undesired, how to find effective actions to prevent it from happening. This problem is known the AUF (avoiding undesired future) problem (Zhou, 2022). Consider an autonomous drone delivery system scenario as an example. Imagine that the system has trained an MI.
'National Key Laboratory for Novel Software Technology, Nanjing University, China "School of Artificial Intelligence, Nan- jing University, China. Correspondence to: thi-Hua/ Zhou <zhouzh lamda.jj..cdu.nn>.
Proceedings of the 4nnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 20255 by the author(s).

---

Enabling Optimal Decisions in Rehcarsal Learning under CARE Condition
that maximizes PAUF. For cases where the CARE condition does not hold, we introduce an inner embedding technique to construct concave lower bound of log I' PAUP, which sat- isfies the CARE condition and can be used as surrogate for optimization. Last but not least, we demonstrate that the CARE condition always holds when the desired AUF region is an interval and constructively derive a closed-form solution for the optimal z when |Y| = significantly re- ducing the time complexity, illustrated in Tab. 1. Finally, experimental results validate the effectiveness and efficiency of our approach on both synthetic and real-world datasets.
Table 1. Time-complexity for decision-making, Y =1
ALG. QIN AI.,, 2023
DU AI.,, 2024
OURS
TIME ( yk E
(p*)
(p)
Building on this, Qin et al. (2023) develop the rehearsal learning framework to learn influence relations and provide a corresponding decision- decision-makinn algorithm based on the relations, and Du et al. (2024) consider the decision cost and enhance the computational efficiency. While the learned relations aid in selecting effective alterations, directly op- timizing FAUF remains challenging due to the lack of an analyzable functional form in most scenarios. Consequently, existing rehearsal learning methods typically identify plau- sible alterations by ensuring P AUP exceeds a predetermined threshold r, formulated as PAUP 2 T.
Our contributions can be summarized as follows:
1. We propose the CARE condition for AUF problem, under which the logged AUF probability can be proven to be a concave function w.r.f. the selected alteration x².
2. We present a projection-Newtor algorithm to select the optimal alteration, and we prove that the algorithm can achieve a superlinear convergence rate.
The existing rehearsal learning approaches suffer from two critical shortcomings. First, the threshold-based strategy for maximizing the AUF probability is unsuitable. Let C, represent the constraint I" AUF 2 and PA denote the maxi- mum achievable AUF probability through alterations. When T > PA the feasible domain of C, becomes empty, causing existing rehearsal learning approaches fail to find feasible alterations. Conversely, when 7 < PA, solutions satisfy- ing C, may be suboptimal, as they are not guaranteed to maximize the AUF probability. Second, the computational burden of constructing the feasible region satisfying C, is prohibitive, especially for high-dimensional problems. Specifically, when constructing the feasible domain of C,, the time complexity of existing sampling-basee based approach cannot be bounded by any polynomial function of p. and the method in Du et al. (2024) exhibits cubic complexity (p")) (where p represents the dimension of V) in cases where Y = 1, as illustrated in Tab. 1.
3. We provide an inner embedding methodology to handle the AUF problem not satisfying the CARE condition. Experiments validate the effectiveness of our methods. 4. We prove the existence of a closed-form solution when Y 1, in which case the time complexity can be significantly reduced for addressing the AUF problem.
2. Preliminaries and problem setup
A probabilistic graphical model called structural rehearsal model (SRM) is proposed by Qin et al. (2023) to charac- terize influence relations among variables in the AUF prob- lem. Influence relations can capture mutual dependencies between variables while accommodating dynamic environ- ments, making it more suitable for greision-making The SRM comprises a set of rehearsal graphs and associated parameters {(G,, 0.) Focusing the optimal action in specific decision rounds, we consider G, - G in this work, which can be straightforwardly generalized to multi-round scenarios. Detailed definition of SRM is listed in Appx. A.1.
To address the aforementioned shortcomings, we propose a novel approach to directly maximize the AUF probabil- ity F PAUF. Specificallyy, we establish the CARE (CAnonical REctangle) condition for the AUF problem, a condition that is practical in real-world tasks. Under the CARE condition, we show that PAU can be explicitly formulated as func- tion of the selected alteration z given observed variables x and influence relations parameterized by 0, which clim- inates the need to construct the feasible domain satisfying C, and facilitates direct optimization. Although this func- tion is not inherently concave, we prove that applying log transformation ensures concavity, thereby guaranteeing the existence of global maximum of IP PAUF While the trans- formed function remains complex, both the gradient vector and Hessian matrix are always computable. Leveraging these properties, we develop a projection- NNwtton method to select decision alterations for the AUF problem, which is proven to achieve superlinear convergence to the optimal z
The rehearsal graph G models the qualitative influence re- lations among variables, which is denoted by G = (V, Specifically, V represents the variable set of the AUF prob- lem and E represents the edges characterizing influence relations among variables. There are two types of edges in G, a directional edge X + Y means that X influences Y. and a bi-directional edge X ++ Y means that X and Y are mutually influenced. For example, sunlight unilaterally influences the plant growth, whereas rainfall and river flow are mutually influenced, as changes in either one affect the other. Besides, the corresponding structural equations of variable Vys can be parameterized by {33, o3) }=1=1 C 0:
[V]
Vy== (PA + +
(1)
2

---

Enabling Optimal Decisions in Rehcarsal Learning under CARE Condition
where V E V denotes the j-th vertex of the graph G, PAG & {ul u - V in G} represents parents of Vj, and the additive noise E follows (000 2 In this work, we focus on basic yet essential class of the AUF problem, characterized by linear structural equations fjs in Eq. (I), Le., V = 3 PA + Sj, where G, E RRPAğIIX consistent with prior studies (Qin et al., 2023; Du et al., 2024).
CARE condition, and prove that PAUF under this condition can be explicitly formulated and exhibits convexity w.z. the decision alterations after a negative log transformation.
3.1. CARE condition for AUF problem
In this subsection, we introduce the CAnonical REctangle (CARE) condition for the desired region S. The motivation of the condition stems from the observation that the func- tional form of AUF probability P. (Y E S x, 0, Rh(25)) depends on the shape of the desired region S. We start by defining the convex polytope in Def. 3. a superclass of the canonical rectangle in the CARE condition.
Besides, the ppcision-making process centers on identifying appropriate alterations, which are decision actions specified by human decision-makers as sets of vertex-value pairs. For instance, as shown in Fig. 6 (provided in Appx. A.1), & + 121 } in Fig. 6(b) demonstrates such an alteration. Meanwhile, the execution of an alteration is formalized through a rehearsal operation, denoted as Rt(()) which modifies the original graph structure as shown in Fig. 6(b)- Fig. 6(d). The rehearsal operation removes all incoming influence links that point into any vertices contained in &. and fixes the values in & to their associated vertices; while this operation maintains the influence relations among all other vertices in the resulting graph R
Definition 3.1 (Convex polytope). The region P is a con- vex polytope in R iff it could be written as P (M, d) & { E Rd Ms I d] with M E Raxd and E RTx1
In general, a convex polytope expresses a convex region that is formed by the intersection of multiple half-spaces divided by hyperplanes. In this sense, the empty set is contained in the definition and the convex polytope could be an open region. Based on this, we define the canonical rectangle in Def. 3.2, which is subclass of convex polytope.
For the AUF problem, the goal is to find an optimal alter- ation that can maximize the AUF probability after perform- ing it. As the generation of variables is parameterized by 0 in an SRM, and the observation of x happens before making decisions, the AUF probability is conditional on 0, x. Also, in practice, the alteration range is limited for the variables z. Thus, the AUF problem can be formulated as the following constrained optimization problem:
Definition 3.2 (Canonical rectangle). Let (M,d) de- note convex polytope in Rd G > 0 denote positive definite matrix in Rdxd P (M, d) is called to be canoni- cal reciangle w.r.f. matrix G iff the following holds:
P (M, d) expresses non-empy closed region; 2. M E Rd2dxd with full rank, d E 124x1, and permutation matrix Pa E 2d2x2d and diagonal ma- trix A E Rddx2d such that M = AP. (I, (I) Q, where G = =QTDQ is the eigen-decomposition of G.
PP xx,
arg max
z t. fie Z S 2iight
(2)
Remark 3.3. Note that if convex polytope P (M, d) is
The constraint in Eq. (2) uses a compact element-wise nota- tion, where z S zf S Light means that each element of z must satisfy its corresponding lower and upper bounds specified in 2 lett and 2f respectively.
canonical w.E.t. G E Rddxd then the polytope P (M, d) = {s E Rd. : Ms < d} expresses a (hyper-)rectangle region in Rd. We provide two geometric intuitions of the canonical rectangle. First, edges of the rectangle are parallel to axes of the coordinate system induced by Q, as shown in Fig. 2(a). Second, surfaces of the rectangle are parallel axial planes of the ellipse s Gs = 1, as shown in Fig. 2(b). Without loss of generality, for G = DQ. we assign P. = and A diag [ 4, D +) (thus M (I, -1) D QQ). since d can fit M by permutation and rescaling.
3. Our approach
In this section, we propose our approach to identify deci- sion actions that maximize PAUF. The primary challenge lies in the lack of an analyzable functional form and the potential irregularity of PP AuF, making direct optimization intractable. To address this, we propose a special type of desired region S, under which we prove that IP, |UU admits an analyzable functional form and exhibits convexity after applying negative log transformation, enabling efficient decision optimization. For AUF cases with irregular S, we present a generalization method to optimize an upper bound of the negative log AUF probability as a surrogate, thereby extending our approach to broader application scenarios.
Conrditate syown rilire by An eomncle centrics rechogle wrt.
Hspordiises 6,5
(a) axes induced by Q1
(b) ellipse s G2s 1
In what follows, we establish the key condition, i.e., the
Figure 2. Geometric intuitions of the canonical rectangle
3

---

Enabling Optimal Decisions in Rehcarsal Learning under CARE Condition
Recall from Eq. (2) that the goal is to find alterations maxi- mizing the AUF probability. Since optimizing a probabilis- tic target is generally intractable, we seek conditions under which the AUF probability becomes analyzable. Building on the canonical rectangle, we propose the CARE condition in Def. 3.5 to achieve this goal. Before that, we present Lemma 3.4, which links the target variables to the obser- vation x, the alteration zf, and parameters O.
3.2. Projection-Newton for addressing the AUF problem
In this subsection, we present a Neeeettion-Newtnn method to solve the AUF problem with theoretical guarantees. The proof of the theoretical result is provided in Appx. C.
As illustrated in Cor. 3.7, the AUF problem can be trans- formed into a convex optimization under the CARE con- dition. To solve the AUF problem, there remain two chal- lenges: (a) although Thm. 3.6 can express the AUF proba- bility by known elements, the functional expression of the associated Gaussian CDF (()) remains complex; and (b) AUF scenarios exist beyond the CARE condition, in which case the transformation in Eq. (3) does not hold anymore. In what follows, we propose an algorithm to accurately solve the transformed problem in Eq. (3) and provide a general- ization method of our algorithm for AUF scenarios that do not satisfy the CARE condition in the following subsection.
Lemma 3.4 (Qin et al., 2023). Given observed x and pa- rameter set ( {(,))s, E)) of the SRM, it holds that Y = Ax +Ce, where A, B, C are constant matrices of appropriate shapes based on 0, z are intermediate vari- ~ N (0, Σ). Definition 3.5 (CARE condition). The AUF problem satis- fies the CARE (CAnonical REctangle) condition iff the as- sociated desired region S is canonical rectangle S(M, d) W.E. matrix CΣC with C, Σ defined in Lemma 3.4.
ables with alteration E. and e = (e1,--.,6y
The formula of the gradient g and the Hessian H of the opti- mization target f(25) = Ei=1 - log { {2) is presented by Eq. (10) in Appx. C, which are associated with Gaussian CDF ((-) and Gaussian PDF (t-). Although the funetional expression of (+)) is too complex to analyze, its exact value at any specific point can be accurately approximated. Hence, gradient and Hessian H are al always computable. Mean- while, because Eq. (3) is a convex optimization, the Newton method (Boyd & Vandenherghe, 2004) could be considered to address the optimization. However, classical Newton (or Quasi-Newtor methods cannot be directly used, because they are designed for unconstrained optimization (Shanno, 1970), which is not the case of Eq. (3).
This condition is prevalent in real-world scenarios, par- ticularly when the dimensions of Y are mutually inde- pendent. Considering two-dimensional random variable Y = [(Y,, 12) as an example, if Y1 TL Y2 after alteration and the desired region S = {a= S Y1 I bj, 42 S Y2 S b2). then the AUF problem satisfies the CARE condition. Under the CARE condition, the AUF probability can be formulated as an analyzable function, and its negative log form exhibits convexity, as shown in the following Thm. 3.6.
Theorem 3.6. Under CARE condition that S & S(M, d), let PA denote AUF probability P(Y E S x,x, Rh(zE)), kj denote the j-th row of matrix MB, and by denote the j-th component of vector MAx- - d (A,B B defined in Lemma 3.4). Then the following asserts hold:
Due to this, we adapt the projection Newton method (Bert- sekas, 1982) to solve the optimization problem defined in Eq. (3). As outlined in Alg. 1, the iteration begins with the initial point z In each iteration t, we compute the gradient g(t). Hessian ((e), step size ((()). and construct D()) as a partial diagonal approximation of H After performing a Newton-like step, the point is projected into the feasible domain [Ziet, Zright]. The step size a(t) is de- termined using an Anmijo-like rule (Boyd & Vandenberghe, 2004). Detailed definitions of Diag DiagAppront (prox(()) Armijo(-), and Project [Ziett (.) are provided in Appx. C. For learning the structural model (0) from observational data, we employ the least square estimation (LSE) for parameters By, and estimating Σ based on residuals. Note that the time complexity of the decision-making step, specifically lines 5 to 11, can be bounded by 13 I.) (L is iteration rounds), as the dominant time-consuming operation in each iteration is computing the inverse of the matrix H(t) E RI x|=*|
1. Let| Y d, the AUF probability can be expressed as:
PA II 2 (- - bb.) 1 (KT + bl+d)) i=1
2. For any bounded alteration 2 PA > 0 and the func- tion (z5) = -lo xP)) is convex W.E.t z².
(4)-
Thm. 3.6 provides the functional form of the AUF proba- bility under the CARE condition defined in Def. 3.5. As consequence, Cor. 3.7 reveals that the AUF problem can be transformed into convex optimization.
Corollary 3.7. Let d denote Y). Under the CARE condi- tion, the AUF problem defined in Eq. (2) can be equivalently transformed into a convex optimization as follows:
arg min x
llgg (2E))
(3)
As a second-order optimization method, the presented pro- jection Newton method in Alg. 1 enjoys superlinear con- vergence speed by the following Thm. 3.8, similar to the classical Newton method (Boyd & Vandenberghe, 2004).
S. t.
Zieft S z < Zright:
where A1(25) = 1 (-kTzE - b)) - 1 (k + bbtd).
with kj and by defined as in Thm. 3.6.
4

---

Enabling Optimal Decisions in Rehcarsal Learning under CARE Condition
Algorithm 1 Projection Newton for selecting optimal action
Org desined reg on froer CART smbeddiing of
Drijin destee riginn 8 mer GARE enteiring of
Input: observed x; parameter 0; S(M, d) under CARE
1: Compute matrices A, B, C by {3,)) M 0
2: ((z²) - log {
> (z)) in Cor. 3.7
3: Initialize a(0) E Zleft, zrightl
(a) for non-convex S (red)
(b) for circular S (red)
4: for t = 0 to L 1 do
Figure 3. Examples of inner CARE embedding for different re- gions S with the same covariance matrix EC
5: Compute ((t)
dy l(z5)
a =i
6: Compute (()) Jrl ((z))
o²
2 (t)
7:
Computation formula in Appx. C
Definition 3.9 (Inner CARE embedding). Let S denote a non-empty region in Rd, and let Ior (M, d) denote a convex polytope as defined in Def. 3.1. Then Ier (M, d) is called the inner CARE embedding of S iff the following holds:
8:
(e))= = DiagAppx(HH, (1) (()) = Armijo (D())
9:
E *(+1)= = Project (2(0) a( Dn (())()))))))))
(Rints Zright!
(1) Ier (M, d) is a canonical rectangle w.E. CΣCT E Rdxd, where C and Σ are defined in Lemma 3.4; and
10:
if [0(e+1) - 2/00l < d then
11:
Set 2 (L2) 25 ((t1) and break
(2) Ier is the lagest subset of S satisfying (1) above.
12:
end if
13:
end for
With Def. 3.9, we always define region Ier (M, d) that satisfies the CARE condition for any non-empty desired region S. As illustrated in Fig. 3, we shade the inner CARE embedding in dark red for some irregular desired regions S, including a non-convex polytope as in Fig. 3(a) and a circular region as Fig. 3()). As the inner CARE embed- ding Ter always satisfies the CARE condition, leveraging the transformation in Cor. 3.7 could be considered, and the following Prop. 3.10 guarantees its feasibility.
Output: the optimal decision alteration (
Theorem 3.8. Let FA denote the maximal AUF probability P(Y E x, R)(23)) that can be achieved by alteration ae E [Mott, Zright], and let 12 k denote the sequence outputted by Alg. 1. The probability sequence (TP(Y E S 0,x, Rh, converges to PA with a superlinear convergence rate (at least quadratic).
Proposition 3.10. Let S denote a non-empty region in R and let Ter (M, d) denote the inner CARE embedding of S. Then, always holds that E(z5) I E(zz), where the loss function b(z5) = -log (IP(Y E E 1 RH(z())) and ((z²)= = =- (PIYY
As guaranteed in Thm. 3.8, the optimal alteration that max- imizes PA PAUF can be efficiently selected by Alg. 1. Note that PAUF in Thm. 3.8 conditioned on Ô rather than 0, this is because true parameters of the SRM are practically not available. We can only estimate the parameters from data, and then try to select the optimal decision alteration based on rather than true 0. Hence, the guarantee in Thm. 3.8 designed for Alg. 1 is conditioned on 0 as well, and if the ê used in Alg. can be changed as true 0, P PAU in Thm. 3.8 also turns to condition on O.
Ier 0, Rh(25)))
Prop. 3.10 guarantees that for any irregular desired region S in the AUF problem, the inner CARE embedding method provides a strict upper bound {(25) on the original loss function E(25). This upper bound serves as an effective sur- rogate optimization target, analogous to classical empirical risk minimization (ERM) (Mohri et al., 2018). Since inner CARE embedding always produces region satisfying the CARE condition and enabling the convex transformation
3.3. AUF scenarios beyond the CARE condition
Another crucial concern is that AUF scenarios exist beyond the CARE condition, where the transformation in Cor. 3.7 no longer holds. Analyzing the AUF probability in such cases is challenging due to irregular desired regions. To address this, we propose inner CARE embedding, which identifies an inner canonical rectangle Zer for the desired re- gion S, such that P(Y E I 0 x, Rh(2)) forms a strict lower bound for P(Y E S θ, R)) This ensures the transformation in Cor. 3.7 holds for the lower bound, allowing us to optimize it as a surrogate using Alg. 1. The formal definition of inner CARE embedding in Def. 3.9.
in Cor. 3.7, we can transform the intractable problem of optimizing PAUF without an analyzable formula into the more manageable task of finding the inner CARE embed- ding for S. It is worth noting that while the inner CARE embedding requires the largest subset of S satisfying the CARE condition, Prop. 3. 10 remains valid for any CARE- compliant subset of S. Finally, given that desired regions can vary substantially across applications, different inner CARE embedding methods need to be specifically designed for each distinct region type. Below, we provide a method for finding the inner CARE embedding for a eircular region.
5

---

Enabling Optimal Decisions in Rehcarsal Learning under CARE Condition
Proposition 3.11. Let S = (x Rd ||x note circular region on TR., and let C, Σ denote problem- related matrices as defined in Lemma 3.4. Then, the inner CARE embedding of S can be expressed by Ier (M, d) with:
1² de-
Algorithm 2 Closed-form solution for cases where YY|
Input: observed x; parameter θ; P,S(M,d) .) under CARE
I: Compute matrices A, B, C by {3,} [ j=1 E E
2: K = MB E R22 b = MAx - d E IR2x1
M M- = (I. AQ d= (L.( PAAL-r/VV+Ma
3: kJ +- the 2nd row vector of matrix K
(4)
4: b- (b) - b2)
> by is the i-th component of b 6: function CLOSED-SOLUTIONN b, Zlott Zright)
5: Select V
where Q AQ is the eigen-deccoposiiion of matrix CEC!,
DOOSEDD-OLUUIOO((K 6, l, r)
I E Th. Radxdi is the identity matrix and 1 E Rdx1 is the vector with all components equals 1.
7:
m = kJ kT((T(k 2 0) + 1(k < 0)
8:
M = k" .(I(k < 0) + [[kk 2 o))
9:
i S m then
From Prop. 3.11 presented above, for a circular desired re- gion S, we can directly apply Eq. (4) obtain its inner CARE embedding Ier (M, d). Although the original loss function, ie.,, f(zz) is generally not analyzable in this situation, Prop. 3.10 guar- antees that we can optimize its analyzable upper bound,
10:
z' = L[kk 2 0) o + 1(k < 0r
11:
else if 2 M then
12:
z I[k < 0) o 1 I(k > 0r or
log (P(YY E S 0,x, R)(2))))
13:
else x' +- 0, and do
14:
for j 0 to |z" do
15:
if ky = 0 then
i... - log (P(Y E Ier 0, x, Rh((2)) as surrogate by using the convex transformation defined in Cor. 3.7.
16:
continue
17:
else if b/k, E (l,,ry then
18:
2j = b/k, and break
19:
else
4. A closed-form solution for Y =1
20:
= T(b/k) < <bi0,, - T(b/k/ 2 ryyrr b b 2j
21:
In this section, we propose a closed-form solution for the formulated AUF problem in cases where the dimension of the concerned output equals (Y) 1) and the desired region of Y is an interval on IR. The proposed closed-form solution can greatly reduce the time complexity for selecting optimal alterations that maximize the AUF probability.
22:
end if
23:
end for
24: end if
25:
return z
Output: the optimal decision alteration z
Considering situations where the output dimension is sin- gleton, i.e., Y = 1, it can be verified that any interval region& S IR satisfies the CARE condition. In this case, recall from Thm. 3.6 that the AUF probability can always be expressed as PAUF = +b ( (-k] Rf - b1)
Intuitively, directly analyzing the stationary points of f(z²) can reduce execution time. As shown in Thm. 3.6, E(z5) is convex w.r.l. 25, so any stationary point zt of ((25), i.e. 2 where 8z. ((zz) ===z² 0, achieves the global minimum. In cases where Y 1, this idea is feasible, as Alg. 2 constructively provides zero point zt of Eq. (5).
+ (k]22 + b2), where k.T is the i-th row of matrix MB e |2 and 6, is
the i-th component of vector MAx- d E 22x1, Here, M,
A, B. d, and x are defined in Lemma 3.4 and Thm. 3.6. Fur- thermore, according to the discussion in Remark 3.3, we can assume without loss of generality that k,- = ) lead- ing to PAUF- / (k]2$ bi) + (k]2² + b2). Remark that b,, by always holds because (--) is a monotone in- creasing function and PAUF > 0 as guaranteed in Thm. 3.6.
As shown in Alg. 2, the function CLOSED-SOLUTION primarily utilizes if-else statements and iterates over the vector z". Therefore, this function can be represented in a closed form as complex equation, which depends on k, b, I, and using the indicator function I(()) and the summation operator Σ Specifically, Alg. 2 focuses on finding the root for Eq. (5), which suffices to be an optimal alteration that achieves the maximal AUF probability. Recognizing that k2 does not necessarily equal O, the point z is a root of Eq. (5) if and only if o (k]z b1) @ (kjz + b2). Given the even nature of (c-), which is monotonic on (-oc, O), and the fact that -b, > by previously discussed, it follows that kjxx x bi = -k bz, which simplifies to k! bu-b Alg. 2 is designed to solve this equation under the constraint z E [oth Zrigtt. where M and m represent the maximum and minimum values achievable by kj z under this constraint. If there exists a E [Zeett right]] such that
Using Alg. 1 to select the optimal alteration is also available in this case, which has a ([25| L) time complexity as discussed in Sec. 3.2. To decrease the executing time by leveraging the information of Y = 1, directly analyzing the gradient is worth considering. Specifically, the gradi- ent of f(zf) = log 4 (k]2² b1) & (kI25 + b2) }
in Eq. (3) can be derived as follows:
0 (15))
ò (kTzz b2) @ (k zt - b1)
Kg-
(5)
Oz
+ (k]zz b1) - $b (k,x zt - bz)
6

---

Enabling Optimal Decisions in Rehcarsal Learning under CARE Condition
k/z²= b this root is selected as shown in lines 14 to 20; otherwise, the boundary value is chosen described in
6. Related work
We discuss two types of works related to our study, including reinforcement learning (RL) and causality:
lines 10 to 13. Further discussions are provided in Appx. B, along with the proof for the following Thm. 4.1.
Reinforcement Learning (RL). RL methods have demon- strated remarkable success across various decision-making domains (Sutton & Barto, 2018). In general, classical RI. approaches (Lillicrap et al., 2016; Haarnoja et al., 2018) typically require extensive cost-free interactions, which are unavailable in AUF setting. While numbers of offline RI (Li et al., 2023; Qiao & Wang, 2023) and hybrid offline-online RL (Song et al., 2022; Pong et al., 2022) methods have been developed to mitigate interaction requirements, they remain ill-suited for AUF scenarios (which demands immediate de- cisions without interactions) and necessitate substantial of- fline datasets. Furthermore, when using MDP to model AUF, the reward function may differ substantially between offline and online data, as the distribution of Y given observed z (P((Y) 2)) can fundamentally differ from that under al- terations on (IP(Y RR(())), resulting in mismatched reward functions I(Y E S z) versus I(Y E S RR(()). In contrast, rehearsal learning leverages structural relations among variables, enabling it to recommend immediate deci- sions based solely on observational data without requiring interactions. Note that for addressing AUF, finding influ- ence relations is a key step, while RL could be helpful tool for finding the relations with proper adaptations.
Theorem 4.1. When Y-= 1, let PA denote the maximal AUF probability P(Y E [y,, vr] 0,, Rh(zF) that can be achieved by alteration z E [Zeent Zright], and let z² denote the alteration outputted by Alg. 2. Then the probability P(Y E [y0. yr] 0, x, Rh(z")) equals X-
As guaranteed in Thm. 4, the optimal alteration that maxi- mizes PAUF can be effectively determined by Alg. 2. The issue of conditioning on θ rather than true 0 follows the same discussion as Thm. 3.8. is worth noting that the time complexity of the decision-making step in this case, i.e., the time complexity of the function in Alg. 2, is bounded by ((zE)) I). because all operations can be reduced to iterating over elements of vectors with length |zz], including k. I, r, and zz. Compared with Alg. I, which has time complexity of O(22| L), the execution time for selecting the optimal alteration in Alg. 2 is greatly accelerated by leveraging the information that Y = 1. Last but not least, we would like to emphasize that Y = 1 is a crucial case, as most of the experiments in existing rehearsal learning research are es- tablished on this scenario (Qin et al., 2023; Du et al., 2024). Therefore, developing theoretically efficient approach for this case is essential for practical applications.
Causality. Extensive research has explored structural mod- els for decision decision-making, primarily based on structural causal models (Spirtes et al., 2000). This work encompasses two notable categories: (i) methods for identifying causal struc- tures or effects (Kocaoglu et al., 2017: Wang et al., 2020; 2023b; Qin et al., 2021; Wang et al., 2023a; Zhang et al., 2023; Qin al., 2024), which focus primarily on causal dis- covery and estimation; and (ii) approaches for determining optimal intervention points using causal bandits or causal RL methods (Lattimore et al., 2016; Sen et al., 2017; Lee & Bareinboim, 2018; Zhang & Bareinboim, 2019; Lu et al., 2021; Park et al., 2025), which incorporate additional util- ity considerations. These methods generally rely on causal modeling, which may be luxurious or restrictive in some real-world decision decision-making scenarios (Zhou, 2022). The strong assumptions required for causal identification often limit the feasibility of finding viable alterations. Conversely, correlation-baser approaches, which underpin most ML models, are generally not sufficient for decision-making Based on this insight, rehearsal is proposed (Zhou, 2022). which captures influence relations for addressing the AUF problem (Zhou, 2023). Qin et al. (2023) further propose the SRM and rehearsal-leariing framework, which can adapt to dynamically evolving decision systems. Note that this framework can recommend effective alterations across both time-varying systems and scenarios with mutual influences.
5. Discussion
Our approach aims to select optimal alteration d that maxi- mizes the AUF probability IP FAUF- Under the CARE condi- tion, it precisely determines the optimal alteration. The con- dition offers practical advantages, as oocision-maker can strategically design the desired region S to satisfy the CARE condition after estimating system parameters. For scenar- ios beyond CARE, our approach constructs inner CARE embeddings. While this construction remains challenging for arbitrary regions, we demonstrate its feasibility for spe- cific geometries such as circular regions in Prop. 3.11. For other region types, established techniques like maximum rectangle algorithms for polygons (Choi et al., 2021) pro- vide powerful tools, and future advances in these methods will further expand our approach's 's applicability.
Meanwhile, while our theoretical analysis is established with the Gaussian noise assumption that is widely adopted across various domains (Shumway et al., 2000; Cohen et al., 2018), our methodology remains effective in general cases through appropriate Gaussian approximations. For instance, Laplace's 's approximation (Bishop & Nasrabadi, 2006) in Bayesian statistics demonstrates how unimodal distributions can be effectively approximated by fitting a Gaussian distri- bution centered at their mode.
7

---

Enabling Optimal Decisions in Rehcarsal Learning under CARE Condition
7. Experiments
Table 2. Results of synthetie data (mean + std).
We evaluate our proposed approach on two datasets includ- ing synthetic dataset and a real-world dataset. We com- pare our method with two baseline methods and the existing rehearsal learning methods, including QWZ23 (Qin et al., 2023) and AUF-MICNS (Du et al., 2024). Besides, although we have discussed in Sec. 6 that RL methods are not well- suited for the AUF problem due to limited or even nonex- istent interactions with the decision environment, we still implement some classic RI. algorithms for comparison to mitigate potential concerns. The compared RL algorithms includes DDPG (Lillicrap al., 2016), PPO (Schulman et al., 2017), and SAC (Haarnoja et al., 2018). First, we briefly introduce the datasets, with details listed in Appx. D.
AUF PROB.(S)
100 100-RDs Succ. FREQ.
AvG. TIME (MS)
No ACTION
2.52 = 9.19
1.61 = 1.29
RANDOM
0.97 = 7.01
1.60 =: 1.52
DDPG
1.44 9.53
1.57 + 1.16
6.35 + 0.28
SAC
0,98 = 8.07
1.55 4 1.16
10.85
0.22
PPO
1.42 = 8.02
1.65 = 1.13
11.50 1.84
QWZ23
94.22
94.06
69.3
MICNS
96.96
96.99 8.44
4.02 1.28
OURS
99.01
98.93 1.26
5.86
Table Results of Bermuda data (mean + std).
METHOD
AUF PROB.(%)
(00-RDs Succ. FREQ.
AVG. TIME (MS)
No ACTION
8.77
3.36
8.94 1: 2.76
RANDOM
15.47
20.9
14.93 = 3.41
DDPG
14,46
20.70 5.52
15.17
Synthetic Data (Manage). The Manage dataset is designed to simulate a market management scenario where a market manager must make decisions to promote total profit (TPF) and customer numbers (NCT). Key variables affecting TPF and NCT include the cost of raw materials (C), product price (P), and competitor's s price (E), among others. We assume that P and C are the actionable variables in this context. There are mutually influenced relationships among these variables: setting P low leads to a competitive drop in E, and vice versa. The dimensions of X, Z, and Y are 2, 4, and 2, respectively. The desired region S for Y is a cireular region, and over 95% of value Y fails fall within 8 in natural conditions as shown in Fig. 5(a).
SAC
14,05 31.1
18.21 3,94
12.48
0.66
PPO
15.91 #3.8
17.15 3.71
13.31 2.59
QWZ23
73.47 7.11
72.62 = 4.21
567.7
129
MICNS
76.19 + 20.8
76.69 14.3
23.61
+ 13.2
OURS
82.76 =: 1.44
83.26
3.68
0.91 1 0.07
The comparison results are summarized in Tab. and Tab. 3, where the number of observational samples is set to 100. For immediate decisions, RL methods rely on default policies, resulting in performance similar random actions. With limited interactions, RL methods can gather information to update their policies but still fail to produce actions compa- rable to rehearsal methods due to limited interactions. In contrast, rehearsal methods can generate effective actions by exploiting structural relations among environment vari- ables. Our approach consistently outperforms others across all datasets under the AUF probability measure, which is the core concern of AUF problem. In terms of execution time, our approach has the same complexity (O(22)) ()) as AUF-MICNS when YY| > 1, but achieves efficient decision- making with a reduced complexity of O(zz) for Y = 1, leveraging the closed-form solution provided in Alg. 2. Fi- nally, the hyperparameter T for previous rehearsal-learinng methods is selected as the value that achieves the highest average AUF probability among various candidates.
Real-world Data (Bermuda). The Bermuda dataset, which records environmental variables in the Bermuda area, is described in ecology research (Courtney et al., 2017), with available generation order of variables (Andersson Bates, 2018). The dimensions of X, Z. and Y are 3, 7, and and the desired region S for Y is S (NEC E [0.5,2] 2|}. In natural conditions, over 90% of value Y fails to fall within S, as shown in Fig. 5(a). The parameters of structural equa- tions are derived from fitting linear models on normalized data (Qin et al., 2023). is assumed that 5 variables in Z are actionable (Aglietti et al., 2020), and feasible alteration values are set to [-,, 1] for each of them.
Training rounds
Mcanwhile, we repeat the experiment under 100 random seeds for each dataset, incluing 3 measures as follows:
00
AUF Probability
1. AUF Prob. given x. The probability that the selected action successfully make Y S (conditioned
2.6
PALF of SAC
x and 0). This is an immediate decision, and the probability is approximated by Monte Carlo method.
PauF of DDPG
0.4
2. 100-Rds AUF Freq. Success counts of making Y E S in a 100-round experiment (with different observations xs). This a sequential decision-makinn which the RL methods could update the policy.
02
scenario, in
2002
4000
6006
6000
3. Avg. Time. Average executing time for making one decision (the time only for the decision decision-making step).
Figure 4. RL convergence curves (DDPG and SAC) on Bermuda data with adequate number of interactions.
8

---

Enabling Optimal Decisions in Rehcarsal Learning under CARE Condition
Y Natural TPF Value
(Ys) TPF Value after Altering
Number observational samples
NET Value after Altering
Estimation Eroor LE 60
IT, (Yallvatura NC Value
AUF Probabilitt
QW223
Erorrjj 9]
destted region
Original dedred region
Samplis from natural distribution
Samples dat aktring
0.4
Natural Distributionn
Distr bution atter witering
Setmmttoon Error [E 81)
Desired interval
Desired
Empincal PDF for
AUF Prodabilltt
of rouch
OWZZ3
Emplnical
AUF-MICNS
Eree Fi Fjo-0]
Y Nature
Y Value after Altering
355
105
155
205
255
(a) S and natural Y distribution
(b) S and altered Y distribution
(c) PAUF,
B 0 and sample numbers
Figure 5. Results for the synthetic dataset (row 1) and Bermuda dataset (row 2), respectively. For each dataset, the desired region shaded in column (a) identical to that column (b). The bands column (c) represent the standard deviations.
Fig. 4 illustrates the convergence curves of DDPG and SAC on Bermuda data, demonstrating that classical RL methods perform well with sufficient interactions, despite their poor performance in the AUF setting where interactions are lim- ited (Tab. 3). Fig. presents: (a) the original distribution of outcome Y; (b) the distribution of Y after implementing al- terations; and (c) the relationship between AUF probability PAUF, estimation error E| 0|1, and the number of ob- servational samples. As evident in Fig. 5(b), our proposed approach successfully shifts the distribution Y toward the desired region through selected alterations. Furthermore, Fig. 5(c) shows that our method converges to the optimal solution as the number of observational samples increases, without requiring any interactions with the environment.
Acknowledgements
This research was supported by Jiangsu Science Foundation Leading-ede Technology Program (BK20232003), NSFC (62406137). Tian-Zuo Wang was supported by National Postdoctoral Program for Innovative Talent and Xiaomi Foundation. The authors thank Tian Qin discussions.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences our work, none which we feel must be specifically highlighted here.
References
8. Conclusion
Aglietti, V., Lu, X., Paleyes, A., and González, J. Causal Bayesian optimization. In Proceedings of the 23rd Inter- national Conference on Artificial Intelligence and Statis- tics, pp. 3155-3166 2020.
Rehearsal learning provides a promising framework for ad- dressing the AUF problem in practical applications. How- ever, existing rehearsal methods cannot identify the optimal decision alterations that maximize the AUF probability. To overcome this limitation, we proposed the CARE condi- tion, which enables the transformation of the AUF problem into a convex optimization. For solving this transformed problem, we developed the projection-Newe Newton algorithm that guarantees optimal decisions with superlinear conver- gence. Recognizing that some AUF scenarios fall outside introduced inner CARE embed-
Andersson, A. and Bates, N. In situ measurements used for coral and reef-scale calcification structural equation modeling including environmental and chemical measure- ments, and coral calcification rates in bermuda from 2010 to 2012 (BEACON project). http: http://ood //w////d/////////////////ooo looddbboo bco dmo.
org / id/ dat /tlisset/72070 720 788 2018
the CARE condition, ding to generalize our approach. Additionally, we proved that when Y = 1, AUF problem admits closed- fosed-form solution, enabling decision-making with time complexity (l2)), significatlly improving efficiency over prior meth- ods. Experiments validate the effectiveness of our approach.
Bareinboim, E., Forney, A., and Pearl, J. Bandits with unobserved confounders: A causal approach. Advances Neural Information Processing Systems 28, pp. 1342- 1350, 2015.
Bertsekas, D. On the goldstein-levitin-poole gradient

---

Enabling Optimal Decisions in Rehcarsal Learning under CARE Condition
projection method. JEEE Transactions on automatic con- trol, 21(2):174-184, 1976.
J., Zhang, E., Yin, M., Bai, Q. Wang, Y.-.,, and Wang, W. Y. Offline reinforcement learning with closed-form policy improvement operators. In Proceedings of the 40th International Conference on Machine Learning, PP. 20485-20528, 2023.
Bertsekas, D. Projected newton methods for optimiza- tion problems with simple constraints. SIAM Journal on control and Optimization, 20(2):2112446, 1982.
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. Continuous control with deep reinforcement learning. In Proceedings of the 4th International Conference on Learning Representa- tions, 2016
Bishop, C. M. and Nasrabadi, N. M. Pattern recognition and machine learning. Springer, 2006.
Boyd, S. and Vandenberghe, L. Conver optimization. Cam- bridge university press, 2004.
Lu, Y., Meisami, A., and Tewari, A. Causal bandits with unknown graph structure. In Advances in Neural Infor- mation Processing Systems 34, pp. 24817-24828, 2021.
Choi, Y., Lee, S., and Ahn, H.-K. Maximum-area and maximum- perimeter rectangles in polygons. Computa- tional Geometry, pp. 101710, 2021.
maximum-perimeter
Mohri, M., Rostamizadch, A., and Talwalkar, A. Founda- tions of machine learning. MIT press, 2018.
Cohen, A., Hasidim, A., Koren, T., Lazic, N., Mansour, Y., and Talwar, K. Online linear quadratic control. In Proceedings of the 35th International Conference on Ma- chine Learning, pp. 1029-00220000 1038, 2018.
Park, M. W., Arditi, A., Bareinboim, E., and Lee, S. Struc- tural causal bandits under markov equivalence. 2025.
Courtney, T. A., Lebrato, M. Bates, N. R., Collins, A., De Putron, S. J., Garley, R., Johnson, R., Molinero, J.-C., Noyes, J., Sabine, C. L.., et al. Environmental controls on modern scleractinian coral and reef-scale calcification. Science Advances, 3(1 1 1: 701356, 2017.
Pong, V. H., Nair, A. V., Smith, L. M., Huang, C., and Levine, S. Offline meta-reinforcemmnt learning with on- line self-supervision. In Proceedings of the 39th Inter- national Conference on Machine Learning, pp. 17811 17829, 2022.
Du, W.-B., Qin, T., Wang, T.-Z., and Zhou, Z.-H. Avoiding undesired future with minimal cost in non-stationary envi- ronments, In Advances in Neural Information Processing 2024.
Qiao, and Wang. Y.-X. Offline reinforcement learning with differential privacy. In Advances in Neural Informa- tion Processing Systems 36, pp. 61395-61436 2023.
Systems 37, Pp. 135741-135777
Qin, T., Wang, T.-Z., and Zhou, Z.-H. Budgeted heteroge- neous treatment effect estimation. In Proceedings of the 38th International Conference Machine Learning, pp. 8693-8702, 2021.
Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. Soft actor- critic: Off Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In Proceedings of the 35th International Conference on Machine Learning, pp. 1861-1870, 2018
Qin, T., Wang, T.-Z., and Zhou, Z.-H. Rehearsal learning for avoiding undesired future. In Advances in Neural Information Processing Systems 36, pp. 80517-80542, 2023.
Kocaoglu, M., Dimakis, A., and Vishwanath, S. Cost- optimal learning of causal graphs. In Proceedings of the 34th International Conference on Machine Learning, pp. 1875-1888 2017.
Qin, T., Li. L.-F., Wang, T.-Z., and Zhou, Z.-H. Tracking treaiment effect heterogeneity in evolving environments. Machine Learning, 11366):36333373, 2024.
Langley, P. Crafting papers on machine learning. In Proceed- ings of the 17th International Conference on Machine Learning, pp. 1207-1216, 2000.
Raffin, A., Hill, A., Gleave, A., Kanervisto, A., Ernestus, M., and Dormann, N. Stable-baselines3: Reliable rein- forcement learning implementations. Journal of Machine Learning Research, 22(268):1 2021.
Lattimore, F., Lattimore, T., and Reid, M. D. Causal bandits: Learning good interventions via causal inference. In Advances in Neural Information Processing Systems 29, pp. 1181-1189, 2016.
Schölkopf, B., Locatello, F., Bauer, S., Ke, N. R., Kalch- brenner, N., Goyal, A., and Bengio, Y. Toward causal representation learning. Proceedings of the IEEE, 109(5): 612-634, 2021.
LeCun, Y., Bengio, Y., and Hinton, G. Deep learning. Na- ture, 521(7553):436 444, 2015.
Lee, S. and Bareinboim, E. Structural causal bandits: Where to intervene? In Advances in Neural Information Pro- cessing Systems 31, pp. 2573 2583, 2018
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algorithms. CoRR, abs/1707.06133 2017.
10

---

Enabling Optimal Decisions in Rehcarsal Learning under CARE Condition
Sen, R., Shanmugam, K., Dimakis, A. G., and Shakkottai, S. Identifying best interventions through online impor- tance sampling. In Proceedings of the 34th International Conference on Machine Learning, pp. 3057-3066, 2017.
Shanno, D. F. Conditioning of quasi-newton methods for function minimization. Mathematics of computation, 24 (111):647-666, 1970.
Shumway, R. H., Stoffer, D. S., and Stoffer, D. Time series analysis and its applications. Springer, 2000.
Song. Y., Zhou, Y., Sekhari, A., Bagnell, J. A., Krishna- murthy, A., and Sun, W. Hybrid rl: Using both offline and online data can make RL efficient. In Proceedings of the 11th International Conference on Learning Represen- tations, 2022.
Spirtes, P., Glymour, C. N., and Scheines, R. Causation, Prediction, and Search. MIT press, 2000.
Sutton, R. S. and Barto, A. G. Reinforcement learning: An introduction. MIT press, 2018.
Wang, T.Z.,, Wu, X.-Z., Huang, S.-J, and Zhou, Z.-H. Cost- effectively identifying causal effects when only response variable is observable. In Proceedings of the 37th Inter- national Conference on Machine Learning, pp. 10060- 10069, 2020.
Wang, T.-Z., Qin, T., and Zhou, Z.-H. Estimating possi- ble causal effects with latent variables via adjustment. In Proceedings of the 40th International Conference on Machine Learning, pp. 36308-36335, 2023a.
Wang, T.-Z., Qin, T., and Zhou, Z.-H. Sound and complete causal identification with latent variables given local back- ground knowledge. Artificial Intelligence, 322: 103964, 2023b.
Zhang, J. and Bareinboim, E. Near-optimal reinforcement learning in dynamic treatment regimes. In Advances in Neural Information Processing Systems 32, pp. 13401- 13411,200 2019.
Zhang, J., Cammarata, I.,, Squires, C., Sapsis, T. P., and
Uhler, C. Active learning for optimal intervention design in causal models. Nature Machine Intelligence, pp. 1-10, 2023.
Zhou, Z. Rehearsal: Learning from prediction to deci- sion. Frontiers of Computer Science, 16(4):1-3, 2022.
Zhou, Z.-H. Rehearsal: Learning from prediction to deci- sion. Keynote at the CCF Conference on AI, Urumqi, China, 2023.

---

Enabling Optimal Decisions in Rehcarsal Learning under CARE Condition
A. Definitions
A.1. Details about Structural Rehearsal Models
In this section, we provide comprehensive definitions and discussions of the Structural Rehearsal Model (SRM), a proba- bilistic graphical model introduced by (Qin et al., 2023) to characterize influence relations among variables in AUF problem. The original definitions and discussions of the SRM can be found in Qin et al. (2023), and in this paper, we do not consider the time since we focus on the immediate decision making.
Definition of the rehearsal graph
Definition A.1 (Mixed graph, (Qin et al., 2023)). Let G (V, E) be a graph, where V denotes the vertices and E the edges. G is mixed graph if for any distinct vertices u, U E V, there is at most one edge connecting them, and the edge is either directional u + or < bi-directional (u ++ v).
Definition A.2 (Bi-directional clique, (Qin et al., 2023)). A bi-directional clique C C= (Vo; E²) of mixed graph G (V, E) is complete subgraph induced by V C V such that any edge E E is bi-directional. C is maximal if adding any other vertex does not induce bi-directional clique.
Definition A.3 (Rehearsal graph. (Qin et al., 2023)). G = (V, E) be mixed graph. Let (C.) denote all maximal bi-directional cliques of G, where C, (Vi EG). G a rehearsal graph if and only if:
1. V n Vj = 0 for any i * j.
2. Vi E [4]. if there is any edge pointing from some u E V\V; to some v e Vi, then Ve E Vi, u + v.
3. There exists a topological ordering for (C+)' following the directions of directional edges between Cjs.
Note that the topological ordering for bi-directional cliques (c:)/=1 in G reflects the generation order of involved variables.
The associated structural equations
Associated with the graphical representation, the struc- tural equations are defined over the bi-directional cliques
X1
Z
Y
Yi
(C.) Specifically, these equations are parameterized by 0, which comprises the set of parameter matrices {G.) and the covariance matrix Σ; for each clique Cj in the rehearsal graph G. Notably, the quantitative influence of a directed edge A + B, where A E Ca and B E C.. is captured in the parameter matrix 3r. This is due to the topological ordering of the bi-directional cliques reflecting the temporal sequence of the generation process, In contrast, the influence of a bi- directional edge D1 + D2 for D1, D2 E Cd is encoded in the covariance matrix Ed, as mutual influence between variables ceases once the system reaches equilibrium.
X2
Iz
<
Zg
Y2
X2
Zu
Zg
Y2
X
Z
Y
X
Z
Y
(a) original rehearsal graph G
(b) G
(2
2
Yi
Y
X2
Y2
X2
X
Z
Y
X
Z
Y
For clarity in the main paper, we simplify the definition of dynamic structural equations in Eq. (1) to operate at the variable level rather than the clique level. For each variable
(c) RRhlZ
(d) G 2z) (a1.a))
Figure 6. Fig. 6(a) a rehearsal graph, Fig. 6(b)~ Fig. 6(d) the corresponding alteration graphs with alterations. When an alteration occurs to certain variables, all incoming arrows to those variables are removed while other graph structures preserved.
V, E C, the associated parameter vector can be extracted directly from the corresponding parameter matrices {.G.). Similarly, the variance of for each variable V, E C, can be obtained from the respective covariance matrix Σ,
Lastly, while the dynamic structural equations in Eq. (1) assume Gaussian noise for all V, E V, it is worth emphasizing that for variables in cliques without parents, their noise terms can follow arbitrary distributions. This flexibility does not affect our proposed approach, as the columns of the matrix C corresponding to such variables are zero (as per Lemma 3.4). Consequently, both the theoretical results and the proposed algorithms remain unaffected.
12

---

Enabling Optimal Decisions in Rehcarsal Learning under CARE Condition
B. Proofs
In this section, we prove the asserts appeared the paper.
B.1. Proof of Theorem 3.6
Lemma B.1. Let h (x, A) = - log ( (c + A) - +b (cTx)) where X > O and c is a non-zero constant vector. Then (x, A) convex w.E.t x, since the Hessian matrix V (x, A) = 0.
Proof. First, the gradient and Hcssian w.r.t x can be derived as:
o (c'xx x + A) 6 (c ( + A) (ccxx'
Vxh
(6)
V2h = ( (c(xx+M) 2) + (cTx)) ( [(cx A) (c x + x (c_x)) + (0 (x(c (cTx))
cc (7)
( (c x + X) 1 (cTx))
Let f(x,1) A) = () (
xx + - Notice that in Eq. (7),
(o (c'x 2
0 and cc > always hold. Hence, it suffices to prove that V 2 0 if prove that ((z,A) > 0 holds for vz E R when > 0. Notice that:
o(x f(x A) +(2 (2, A), where
f1 A) = - A) (x
-(xx) (z)) +(x -)(())
(z + (x
Since Φ(x) > 0 and 6(x + A) > 0 always holds, thus when > 0, if we can prove that f1 A), f2 A) > 0 holds for Va E IR, it suffices to prove that f(x, A) > holds for Vxr RR Notice that when > can be derived that:
Vafi = -x [ +)
+ (z A)o(= = A0(x + 00
VAfe 1 (x+
(z) + (z () (x A)o(z A) (a (a+
12 (x) O.
Hence, for Vo E RR, (x f1 0) = 0 and f2 A) > f2(2, = 0, which suffices to prove that V 2 O.
Theorem 3.6. We call the desired region S(M, d) satisfies the CARE condition isf S(M, d) is canonical w.r.l. CΣCT. Let
PA denote P(Y E S 0, x, Rh(zE)) )), k denote the j-th row of MB and by denote the j-th component of MAx - d, the following asserts hold under the CARE condition:
1. Let Y d, the AUF probability can be expressed
FA = II (-K bi) (k]+d2 + bxta)};
i=1
2. For any bounded alteration zf, PA > 0 and the function (z+) = log (PA)) convex W.E.f. z
Proof. Since S(M, d) canonical wet. CΣCT as discussed in Remark 3.3, we define without loss of generality that:
M M [ -rJ' D'-D-ig Q E 2pxp,
where p= |Y, CΣCT = DD is the eigen decomposition of matrix CΣCT, Meanwhile, from Lemma 3.4, we know that ~ (Ax Bz, CECT). Hence, it can be derived that:
MY < + + MY M (Ax + Bz d M M(Ax (Ax BzA)
# M ( - Ax Bz < d M (Ax + Bz
* (1, (1.-1)' 1 Q Ax
Bz < - M (Ax + B
+ (I.-I) d M
(A BB where v ~ NN00
13

---

Enabling Optimal Decisions in Rehcarsal Learning under CARE Condition
That is say, it holds that:
^ (K S
P(Y E 0, Rh(z))
Sus -k,2 bi}
P (k]+p2* 26 + bi+p SM -K - bi)
② (-k 25 - bi) + (k 20 bitp))
where kJ j-th row MB, and by j-th component of MAx d. Besides, we prove the convexity of e (zt) as follows. First, it can be derived that:
8 (zf) = log P(Y E 0,x, Rh(zf))
Σ log (4 (-k ze b.) - 2 (k + bb40))
Let & (zf) denote - log (4) (-k, - b:) 4 (k, 2 + b(tp)). If we could prove that & (zt)s are all convex w.r.t. zf, it suffices to prove e (zf) is convex W.r.t. z because e (z5) Em=1 (z5).
It can be observed that k; = kitp always holds due to the definition of M. Meanwhile, it has to hold that ditp bb because (MM, d) is canonical rectangle as defined in Def. 3.2. Without loss of generality, assume the j-th component of k; is non-zero, then define the affine transformation:
a4 (z5) = z + bitp e,,
ej, where e, is zero vector except j-th component set to 1.
kij
Notice that by definition, can be derived that:
&s (as (z))
log ( (-k. ai (z5) - b,) - 1 (-k a; (z5) bi+p))
log ( (-kT TTx ff bitn bi) (-k]22) 2))
Because -bitp - b, > as explained above, & (a: (z+) is convex w.r.t. by Lemma B. Since a;(-) is an affine transformation, 4, (z5) is also convex w.r.t. z (for Vi), which suffices to prove that (z5) is convex w.r. z
B.2. Proof of Theorem 3.8
The proof of Thm. 3.8 relies on the detailed definition of the gradient, the Hessain matrix, the SlagApprox operator and the Armijo-like rule. Hence, we provide the proof in Appx. C after the details of Alg.
B.3. Proof of Proposition 3.10
Proposition 3.10. Let S denote non-empty region in Rd,, and let Ior (M,d) denote the inner CARE embedding of
S. Then, it always holds that ((z)) < ε(z²), where the loss function E(z²) = - llog (P(Y E S | Rh(zz)) and
((zz) log (P(Y E Ior | x, Rh(2))))
Proof. By definition of the inner CARE embedding as in Def. 3.9, it always holds that Ior (M, d) S. Hence, it suffices
prove that for any given Ú, x, and 2f, must hold that IP(Y E Iar Ó, x, Rh(z5)) P(Y S 0 Rh(zE)) by the definition of probability theory. Consequently, we always have {(z5) S E(z6)
14

---

Enabling Optimal Decisions in Rehcarsal Learning under CARE Condition
B.4. Proof of Proposition 3.11
Proposition 3.11. Let {x Rd || =²} denote a circular region on R and let C, E denote problem-related
matrices defined in Lemma 3.4. Then, the inner CARE embedding of S can be expressed by Ler (M, d) with:
M = (I, AQ
== AL./V
where QT is the eigen-decompositton of matrix CEC², I Rdxd is the identity matrix and 1 Rd is the vector with all components equals I.
Proof. The inner CARE rectangle of circle must be (hyper-)square, Let denote the edge length of the square, then by pythagorean theorem, it can be derived that:
2r
d. d= =
Vd
Hence, the inner CARE square w.r.t. the original axes coordinate system should be:
{x (1, (1. s (1.1)
Vdl
we rotate the coordinate system using the positive definite matrix Q, the inner CARE square w.st Q can be expressed as:
(I, (I,-I) - o) (1,1) Vd
As introduced Sec. 3.1, we add the shrinkage matrix A, and can derive that:
(I. (I.-I)AA(x o) o (I.1) A1 x
vd
from which we can assert that M = (I, -I,-I)"AQ and d (I,I) A1 r/Vd+ + Mo for region (x Mx < d].
B.5. Proof of Theorem 4.1
Theorem 4.1. When Y 1, let TPA denote the maximal AUF probability P(Y [1,r] 0,x, Rh(25)) that can be
achieved by alteration z E [Ziett Zright]. and let z denote the alteration outputted by Alg. 2. Then the probability P(Y [[r]] 0, x. Rh(z*)) equals the maximal one P
Proof. Recall from Sec. 4 that when the dimension of Y 1, we can know that:
((zb) = - -log (kJ - b1) + (k + b2))
In this case, to find of arg min 2 E arigh,
((25), we first define function f(x) as follws:
f(x) = log (T (x b1) (a b2))
Let m = kJ (I(k, > 0) Zich + I(k2 < 0) Zright) and M = kJ (I(k2 < 0) Ziett + I(k2 0) Zright). it can be derived that kj zf E [m, .M] and k z0 can continuously choose the value in this interval. Hence, we turn to use a two stage method to find 2f: first, we find v. arg min fx): and second, we find appropriate zf such that kJ I+.
xc M
-( which can be analyzed that f'(x) 0 (-o0, b رجا and f'(x) > 0 on b b2 +00). Thus, it can be derived that bb-ba < m, then v. m; and bi-ba 2 M, then I. M. For these two cases, we can directly set
can be derived that ('(x) a(- -
bi bb- too). Hence, f(x) is monotone decreasing on (-o,, b b1-b2] and is monotone increasing
15

---

Enabling Optimal Decisions in Rehcarsal Learning under CARE Condition
2. = = > 0) Kjett (1k 0) Zright for x. m and set z 11(k2 (0) ziett I(k 0) Zright for a. M such that kg = I. can always hold.
For the remaining case that < bi-b < M. it holds that æ, - In this case, to find an appropriate z, such 2
that kzzz. b we can start from z and perform an iteration that for each dimension j of z check if there exist 2j E [eletes Fright.5] such that substitute the j-th dimension of 2, by 2j can lead to kīz,, 11 b1 The checking process listed in line to line 14 in Alg. (banke m, 77
we have discussed how to determine z, for all the possible cases b1= .M and b= Z M), has been proven that Alg. 2 always outputs the alteration that can achieve min E((5). i..., the one that can achieve the maximum AUF probability.
ELEcan Brigha)
C. Details of projection Newton method
We aim to optimize the following objective function:
(z
log { (-k] - bi) $ (-KT + bb+d)]
We can compute both the gradient and the second-order Hessian matrix of the objective function as follows:
k. zf b.) 0 (-kk z + bitd)
8 (2 2 (Z (z5)
k;
$
k2 of
b.) (-k, 25 bs+d)
1=1
(-kk 26 b:) - 2 (-kk 25 + bitd) )²
(2 2 (zf)
Σ kg k;
(10)
(Tb (-k, z b.) + (-k, 2F ((sdd)
i=1
{( ((-k, kj 인 bi) Φ (-k z - bi) (-k, z + a)) Φ (-k. z + bb+d))
+b [-k,] z b.) - 1 k, z + bi+d)
Before proceeding, we make a few simple observations regarding g(z²) and HzZ2). Both of these quantities admit closed- form expressions, enabling us to leverage second-orde Hessian information to facilitate the optimization of ((z)), and we employ a modified projection Newton method. The method can be expressed as follows:
Z (ax,)),,
where the subscript & + and k are identical to (t (L++) 1) and (t) defined in the main body of the paper, and
(a) = Proj [z - aD V (zx)]-
For all IR", we define the projection operator Proj[z] with coordinates given by:
Light S z², if 2 z¹ Lright
right
Projíz]'=
z'
Zieft
if S z
To clarify the definitions of Dy and ag in the above equation, first introduce the subset of indices Proo
IProl = =(51 zi.. =
OE (zx).
0 zight Ex z Light and Ol (zz)
2 S 2 EK and
Oz < 0
Oz
and the matrix D& is defined as:
16

---

Enabling Optimal Decisions in Rehcarsal Learning under CARE Condition
D2 =
H H
where H2 is the matrix with elements Hy given by
0
if + j, and either i E IProj or j E I Is
H
((x,)) Oz'
otherwise.
The algorithm utilizes scalar e > 0 (typically small), fixed diagonal positive definite matrix M (such the identity matrix), and two parameters 3 (0, 1) and a E (0, 2). which are used in conjunction with Armijo-like stepsize rule (first introduced in (Bertsekas, 1976)). The scalar EK is defined as
6 min fe, 1z
[2x-
(VE
The matrix D3 is positive definite, and M is fixed diagonal positive definite matrix. The stepsize oy is given by:
a3 5m
where
Ph: D2 DV
and my is the first nonnegative integer mn such that:
Ol (Z+)
Ol (2x))
E (zx) - E( (8 2
gm
Σ Proj
Pk+t +
Pk
[z
2 (Bm))
(11)
Oz
Oz
Proposition C.I. We have the following observations:
The gradient Ve is Lipschitz continuous on each bounded set of RK;;; i.e., given any bounded set C R there exists scalar L (depending on S such that
V((z,) V((zz)] < L|z1 Zzl
Vz1, Z2 E S.
(12)
There exist positive scalars A1, X2 and nonnegative integers Q1 92 such that
Au 1 12/2
z D'D
A2 |z|2 Vz E R", ==
(13)
where
MVI (z:) Pros
wk
Zi
[2k
3 The local minimum z" of problem is such that for some 5 > O. E is twice continuously differentiable in the open sphere {z iz z²1 < 8}, and there exist positive scalars m1, m2 such that
my mz|z/ 2//2 n 2 m2 m2 Vz such that 1z z" 8 and z + 0 such that z' 0, ((z)).
Furthermore,
88 (z")
> 0
Vi E (z").
(14)
Oz'
17

---

Enabling Optimal Decisions in Rehcarsal Learning under CARE Condition
Proof. Consider bounded set 2 IR: for z5, Because k,, by and bi+d are fixed, the quantities k, 2f b and -k! 25 + betd vary over bounded interval of ranges over the bounded set Z.
Since ( is continuous, &b (-k, z - b.) and + (-k]* 2f + bi+a) are also continuous functions of 25, and hence uniformly continuous on the compact set Z.
We write mnin = minlggid 1bi+d - bil, and let
Di (z6) = 1 (-kk + bi-a) (-k 2f -bi).
(15)
Without loss of generality, suppose bitd > by. Then for all 2 E Z:
KT < k. 2f + bi+d.
Since (-)) is strictly increasing, follows that D; (25) > 0. Because 25 restricted to bounded set Z, the values -k 25 lie within some finite interval. Hence, (-k, zf b,) and -k, 2f + bi+d) also lie in some bounded intervals. Since +(.) is continuous and strictly increasing, the difference & (-k, z bi+a) @ (-k, z - b.) attains strictly positive minimum on the compact set 2. Alternatively, one can use Lipschitz continuity argument: ((.) Lipschitz continuous with some constant L S V2x"
so
Dj (x5) = (v)) (xx) 2 L/y x/p
where x = -k," z b, and y = -k 2 + bitds and L is determined by S. Thus,
ly x = =lbsta bil 2 Umin-
Therefore,
Dj (z)) 2 Lbmin
Vz E Z
For the numerator:
N₂ (zt) o (-kk -b.) ① (-K betd).
Since (6-) the standard normal PDF, it is bounded above by 1 Thus:
V2*
2
[N, (25) <
v 2
Combining these results:
N: (zs)
C1
<
22
Dj (25)
L.buin
bmin
where C₁ 2 is a constant independent of 25, Since k; are fixed vectors, let V2nl.
C2 = max |kill
isidd
then each term the sum has uniform bound:
18

---

Enabling Optimal Decisions in Rehcarsal Learning under CARE Condition
N (zf)
C1 C2. bmin
k. Di (z5)
$
Summing over i = I,...,d d:
d
N₂ (2S) Di (25)
N₂
C1 Arin
C1C2 bmin
lg (xf)l
k;
k;
C2
Dj (z5)
This constants do not depend on zf.
For the second one, From this definition, H2 is essentially diagonal except possibly for som modifications dictated by the sets Phu and I Crucially, if an index i falls into 2. indicates that z is close to the boundary and that the sign of 0f(x)) such that pushing beyond the boundary would reduce, hence the zeroing out of certain off-diagonal terms ensures 0z*
stable modification.
Because f(() is twice continuously differentiable, the second derivatives Bg are continuous and thus bounded within any compact region. This ensures that each nonzero diagonal element of H₃ stays within certain positive bounds, except potentially when i E Prou where the proximity to the boundary and the definition of H₃. allows the diagonal entries to shrink.
8C
Indices in the Proi correspond coordinates near the boundary where the step is adjusted by the projection. The deviation Wy controls how significant this projection adjustment is. As 2₁ approaches the boundary, the difference between Z₁ and [ MVI (zz)) Proj reflects how the Hessian's 's diagonal elements may shrink or grow.
By suitable scaling arguments and using continuity of the second derivatives, we can show that there exist nonnegative integers G1, (2 and positive constants λ1 X2 such that every nonzero diagonal element of Hg (and thus its inverse D₁ ) can be sandwiched as follows:
A S H S A2
Since D₁- Hy² these inequalities invert to give:
A/ W
Taking all coordinates z into account and using the fact that D₁ is diagonal (due the structure imposed by zeroing certain offdiagonal terms), for all IR,, we obtain:
A s z'D < Az
For the third one, E(-) is twice continuously differentiable at z" By definition, this means that there exists a >> such that is C3-smooth in the open ball
{z 1 1- <
This ensures that both the gradient V((z) and the Hessian (²((z) are continuous functions in that neighborhood. Since x is a local minimizer of E(.), the second-order sufficient condition for strict local minimum states that the Hessian at x", iee. 2 (z"), ), is positive definite.
Because (z((z") is a symmetric positive definite matrix, it has a full set of positive eigenvalues. Let Amin (z") and Amax (2*) denote the smallest and largest eigenvalues of 2 (z*), respectively. We then have:
19

---

Enabling Optimal Decisions in Rehcarsal Learning under CARE Condition
0 Amin (z*) h ('V"(2z" Amax (z*)
1hl
And the Hessian (2)(z) is continuous in z. Therefore, there exists a > 0 such that for all z with ||z == < 6, the Hessian ((z( remains close to V2 (z*) in operator norm. This implies that the eigenvalues of (2((z) are also close to those of 2 (z*).
Since 2 (z*) is positive definite, we find constants mi, m2 such that for all with ||z z=| < <
m
(())) m
where I is the identity matrix and x denotes the Loewner (matrix) order. Equivalently:
m1 12 2 < z V (()
/2|2 t/2 + O with ||z 2 6.
This shows both the boundedness from above and below of the quadratic form defined by the Hessian in a neighhorhood of the minimizer. 2t(x')
Vi E B (2*) is obviously since =" is local minimum and 06(x")
>
0
Oz is continous.
Oz
Notice that in:
(6 k. of - 61) 0 (-kk 2 + b(+d)
H (2) = Σ k; k;
( (-k] 2f 6.) 4 (-k, 2 + b
i=1
(-k k z - -bi) Φ (-k 2 - b)) - (-k] 2$ + bitd 2 k 2 + beta)]
(-k] zf bx) (-k] 2 + bitd)
since k,,, has only rank 1, the upper and lower bounds of the eigenvalues of (z5) can be only depended on term:
(o b.) 0 (-k, z + bi+d))² ( k. TT b.) + (-kT 28 + by+d)) )
(-k,Tzf 10--bb) bb)) (-k, z bi) - (-k, z + bute) of d(-k) 2 + bita) ① (-k, z - b.) +b (-k] z + bitd)
Next, we will prove that every limit point of a sequence { generated by iteration is a critical point with respect to Eq. (3). First, we need the following lemma, which was first proven in (Bertsekas, 1982).
Lemma C.2. (Bertsekas, 1982).
Let z 2 0 and D be positive definite symmetric matrix that is diagonal with respect to (P)) Define
z(a)= Projíz aDV(())
Ya 2 O.
The vector is critical point with respect problem (1) and only if
z = z(a) Va 2 0.
If is not critical point with respect to problem (1), there exists sealar ā > 0 such that
{[z(a)) ((z))
Ya E (0,
20

---

Enabling Optimal Decisions in Rehcarsal Learning under CARE Condition
Note that for all we have
_ 5 IProj (zx),
so the matrix D is diagonal with respect to /Proj (z+). can be shown that for all m 2 0, the right-hand side of Eq. (11) nonnegative and positive if and only if žy is not critical point. Indeed, since D₃ is positive definite and diagonal with Proj we have
respect kk
(zk)
Σ
Oz Pk
Ak 0,1, 0.
(16)
ir
while for all E 1 Proj given that at(x")
O, we have pk and thus
8x
x z (a)
Va 0, E k
Proj
k= 0,1,""
de (Zx)
(17)
[z] (a))
Ya 2 0,
iE I
Proj
k 0,1,
Oz
This shows that the right side of Eq. (11) is nonnegative. If 2g is not a critical point, then it is clear that one of the inequalities Eq. (16)s strict for a > O, making the right side of Eq. (11) positive for all m 2 0. A slight modification of the proof of Proposition ((b) also shows that 2₁ is not a critical point, then Eq. (11) will satisfied for all sufficiently large m, ensuring that the stepsize ag is well-defined and can be determined via finite number of arithmetic operations. If z₁ is critical point, then, by Lemma C.2, have 2₃ 2ạ(a)) for all 0. Furthermore, the argument given in the proof of Proposition ((a) shows that:
OE (ax) Oz pk = 0,
so both terms on the right side of Eq. (1I) are zero. Since also Z (a) for all a 2 0, it follows that Eq. (1I) is satisfied for m = 0. thereby implying that:
2g+1 = 2x(1) = 2 if 2 critical.
Now assume that there exists a subsequence { converging vector z which is not critical. Since {e (Zx)) is decreasing and f is continuous, it follows that {e (zx)} converges fzz) and therefore:
[e (zx) (z++1)] + 0.
Since each the sums on the right-hand hand side of Eq. (11) is nonnegative (cf. Eq. (16), Eq. (17)), we must have
al (Z+)
axR
Oz Pk - 0;
(18)
I Pro)
and
dR (2x)
Σ REFR
[z
2/ (ax))
-
Oz
Also, since ž is not critical and M is diagonal, we have Iž= - iz MV((z))Froi * 0, so Eq. (13) implies that the eigenvalues of (D*) k are uniformly bounded above and away from zero. view of the fact that D is diagonal w.t. I Prooj it follows that there exist positive scalars λ1, X2 such that for all l E K that sufficiently large, it holds that:
de (2x)
Ol (Zx)
X1
S
ph
<
A2
Vi E I Proi
0z'
Oz'
de (zz)
dl (zx)
all (zz)
<
Pk
< Ao
Oz'
Oz'
Oz'
NE
TeT
VEI
21

---

Enabling Optimal Decisions in Rehcarsal Learning under CARE Condition
We will show that hypotheses so far lead to the conclusion that
lim inf og = O.
(19)
R
kER
Indeed, since z is not a critical point, there must exist an index such that either
0f(z)
z'
0
and
Oz + 0,
(20)
or
(f Oz'
z' 0 and
0.
(21)
i & I Pro) for an infinite number of indices K, then Eq. (19) follows from Eq. (18).Eq. (20) and Eq. (21). If I Pro] for an infinite number of indices k K, then for all those indices must have 0((z)) Therefore, from Eq. (20),
0, so Eq. (21) cannot hold.
8x
(l Oz
> 0 and
>
(22)
Since we have [ef. Eg (17) for all & E K for which
E I
Proj
it holds that:
dl (nx)
al (Zx)
2 (ax)
2
[z] 2 (ax)) 2 O,
Oz'
Oz
JETx
which follows from Eq. (18) and Eq. (22) that
1 KEK
lim
z (ax)
= O.
Using the above relation, we obtain Eq. (19).
We will complete the proof by showing that {aw}x is bounded away from zero, thereby the subsequences {Zo}Kr K- {pR}x. and (a)) K. [0, 1]. are uniformly bounded. Hence, by 12, there exists scalar L >> 0 such that for all t [0.
[0, 1]. and & E K, we have
Vl(z)) - Vl [2x - -( zx(a)]] S tL 1zz - zz(a))
For all k K and [0, 1], we have
e [(xx(a)]= = E (2x) + V (2N)' (zn(a) - 2x] + S/' tve (zx) (V((z))
Ve [ex ( [2x zu (a))]] di [zx zx(a)].
Hence,
8 (zx) - - [2xx(a]] = (zx)) [zz 2x(a)] + [" (VE [z - t [2 - (x)]] Ve (zx)) dt [zx 2x(a)
/-' viee [2t -t/z [x(l)]] - Vl (2x)) dt (z 2a)l
Ve 12t (a))
L
2 ((())) [Zx - 2* -
2N 2z(a))2
2
For i IProj have 2/ (a)
[2 app] Proi > 2 -op and p 0. 500 2 - -z((a) oph. It follows.
Ol (2x)
Σ
1 - 2 2 < Eph - (a)] S aa² Σ
Oz'
LE14
1E
ver
22

---

Enabling Optimal Decisions in Rehcarsal Learning under CARE Condition
Consider the sets
I1,6 {i {il
de (zx) > 0, $ Proj j.
I2 {al
de (zx) Oz
I 0, i & x
Proj
Oz
For all i E 2i we must have zi. > cu, otherwise we would have i Ix Proj Since |z - [z - -V * 0, we must have limg-+0, inf EK > 0 and E > 0 for all k. Let E > 0 be such that E I E for all E K, and let denote constant such that [p S for all and E K. Then for all a E [0, BJ. we have z (ax) = z - opk for all E Li so follows that:
Σ i
de (Zk)
12 - - a Σ
0l (l()),
Va E 0, c ] Pi
On'
Oz
vEli.
Also, for all 2 O, we have 2 z (a) S app. and since
0e(s,) oz
< for all E 12,4, we obtain
de (zk)
de (Zx)
[zt
2h(aa]
2 a Σ
Oz Pk.
Oz'
1.
iEl2,
Combining these two equations above, we obtain
Ol (Zx)
Ol (Zx)
Σ
[2]s - 21(00]]
2 a
Oz' Ph
va E
0,
Oz
B
ZE
<EIx
For all a 2 0, we also have
2// - za(aa S a |p
=i= 1, , n.
Furthermore, it is easily seen using Eq. (13) that there exists
> such that
de (2x)
(pi)
Oz ph Vk E K.
TT
TE
Combining the last two relations, we obtain for all a 2 0. holds that:
de (zz).
Σ 1 (a)a) a222 iEZ
Oz ph Wk E K.
TEZ
We now obtain for all E (0, B)] and ls E K, it holds that:
a²\L
de (zx)
Oz Pl. + 11 adpL
2L (Zx)
é (zx) - e [Zx(aa)] 2 a
Σ
[z]
21(a)]
2
2
Oz
T
1ET
Suppose is chosen such that:
adL
at²L
0 3 a
20,
1 1
2 a as1;
B'
2
2
or equivalently,
E 2(1 - 0) 2(1 - 0)
0 S < min
-1
(23)
B
AL
XzL
23

---

Enabling Optimal Decisions in Rehcarsal Learning under CARE Condition
Then for all k E K, have:
dE (Zx)
dl (Zx)
(2x) (((z((aa 20
a
Pkc
+
[2)
Oz
Oz'
TE
T.p
This means that if Eq. (23) is satisfied with nThh = a, then the inequality Eq. (11) of the Armio-like rule will be satisfied. follows from the way the stepsize reduced that og. satisfies:
4k: K. E 2(1 o)) 2(1 -1-m)111
11
ag B min {
B' AL
A2L 112L
This contradicts Eq. (19). From Eq. (14), since f is twice differentiable on {z 1z -="l < d}, it follows that there exist scalars L > O and 61 E (0, 6] such that for all y, with z 201 S and ly V S L - y/. 1/
z² S 51, we have
[e(())
Also, for 2ą sufficiently close to z", the scalar wi
'2k MVL )
is arbitrarily close Thus have:
de (Zx)
Proj
= (2
Oz
where ye is the i-th diagonal element of M. It follows that for sufficiently close to 2", we have
/ S wk = Vi E while z > EN vi 4 B (z").
This implies that there exists 82 E (0, 51] such that
B (z")) Proj
Vk such that Ize. z31 S 82.
Also, there exist scalars > and 83 E (0, 82] such that
z E Vi & (z") and such that |zg z²| S 83.
By essentially repeating the argument in the proof of above that, we find that there exists a scalar ā >> 0 such that
2 vk such that |ze 2²1 S 83.
follows that
dl (2c))
0 <<> A1
S ph
Vi e (z²) and such that |2, - 2²1 S 6z.
Oz
While there exists a scalar À > such that
0e (zx) 2
Σ
Pk
Σ
vk such that Izz zz S 8z.
Oz'
i ((x))
ie (x))
Since 0((x") Ezze
for Vi B (z²) and 0f(2') Oz'
0 for Vi & B (zz),, it follows that there exists a scalar 84 E (0, 5x] such that
B (z")) B (2)+1) and |z241 - z=1 S 83; Vk such that [2s z²1 S de.
Thus have
B (zz)) B (Z++1) =
Proj k+1
vk such that [z z²| S 81.
This to say, when Izz =2 < 6,, we have Zb+1 2 < 83. B (z²) = B 11),),, and the (k + 1)-st iteration of the algorithm reduces to an iteration of an unconstrained minimization algorithm on the subspace binding constraints at z". Given the well- quadratic convergence rate the unconstrained Newton method, we thus obtain a guarantce on the convergence rate.
24

---

Enabling Optimal Decisions in Rehcarsal Learning under CARE Condition
D. Experiments
The experiments are run on a Nvidia Tesla A100 GPU and two Intel Xeon Platinum 8358 CPUs. Besides, the RL results (Fig. are obtained by using the stable-baseling library (Raffin et al., 2021).
D.I. Synthetic Data
In this section, we provide details about the Market-Manage data. The variables included in the generation process are:
Fe
The feature used to predict the raw cost of our market;
Featurecpt: The feature used to predict the cost of the competitor market;
Cour: The raw cost of our market;
Copt' The raw cost of the competitor market;
Pour: The product price of our market;
Pept: The product price of the competitor market;
NCT: Customer numbers of market;
TPF: Total profit of our market.
The rehearsal graph for the variables is illustrated in Fig. 7. The presumed actionable variables that can be altered by the manager are Cour and Pam Paur-
Cour
feature,
NCT
P Pour
Popt
TPF
feature
Cep
X,
Z:
Y,
Figure 7. The rehearsal graph for market manage data.
25

---

Enabling Optimal Decisions in Rehcarsal Learning under CARE Condition
D.2. Bermuda Data
In this section, we provide details about the Bermuda data. The Bermuda data an environment dataset that involves some environmental variables in Bermuda (Courtney et al., 2017). The variables included in the generation process are:
Light: Light levels at the bottom;
Temp: Temperature at the bottom;
Sal: Sea surface salinity:
DIC: Dissolved inorganic carbon of seawater;
TA: Total alkalinity of seawater;
2r Saturation with respect to aragonite in seawater;
Chla: Chlorophyll-a sea surface;
Nut: PCI of NH,, NiO₂ + NiOg, SiO,;
pHsw: pH of seawater;
CO2: Pco, of seawater;
NEC: Net ecosystem calcification.
The rehearsal graph for the variables is illustrated in Fig. 8. The presumed actionable variables that can be altered by the decision-maker are DIC, TA, Sa Chla, and Nut according to Aglietti et al. (2020); Qin et al. (2023); Du et al. (2024).
Light
DIC
Chla
Nut
Temp
NEC
pHsw
Sal
CO2
X
Y
Figure 8. The rehearsal graph for Bermuda data.
26


Reviews:
[{"review_id": "p37zV9Uiux", "reviewer": "Reviewer_YMZh", "summary": "The paper proposes an algorithm for decision making that helps avoid undesirable future (AUF), i.e., increasing the AUF probability. The new algorithm is shown to reduce time complexity compared to prior work and has showed performance improvement compared to a few baselines.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your detailed feedback! We hope our responses address your concerns.\n\n**Q1.** Relevance to RL research.\n\n**A1.** Thanks for your insightful question. Below, we clarify the connection between RL and AUF problem and explain distinctions.\n\n- **Connection between RL and AUF.** When interactions are available, AUF can be formulated as an MDP (or reduced to a Bandit if no state transitions) and solved using RL methods. Specifically, at round $t$, state $x_t$ is observed, then an action is taken by altering $z_t$, and the environment provides a reward: $r=1$ if $y_t\\in S$ else $r=0$. This aligns with our RL baselines (Tab. 2&3) and additional experiments in **A2**.\n\n- **Distinctions between RL and rehearsal learning.** Rehearsal learning focuses on a specialized decision-making setting where interactions are limited or even unavailable. In this case, variables **X,Z,Y** follow a structured generative process parameterized by $\\theta$ (in MDP formulation, transition dynamics and reward function also depend on $\\theta$). Leveraging this fine-grained structure, reliable decisions can be made using only a small set of observational samples (for estimating $\\hat\\theta$) without necessarily requiring interactions with the environment. In contrast, online RL methods rely on extensive interactions for effective policy learning. While offline/hybrid online-offline RL might seem applicable, direct applications would also be unsuitable, as actions significantly shift the distribution of **Y** as discussed in Sec. 2, rendering the reward functions different between offline and online data.\n\nWhile RL/MDP provides a general framework for decision-making, real-world interactions can be costly or even unethical. E.g., in healthcare, doctors cannot freely experiment with untested treatments. In such cases, leveraging structural knowledge enables effective policy without interaction-based exploration while also enhancing interpretability.\n\nFinally, integrating a rehearsal-learning policy as an initialization for online RL could serve as a potential bridge between our work and the broader RL community, offering a promising yet challenging direction to improve RL’s sample efficiency in certain cases.\n\nWe will incorporate this discussion into the revised paper. Thanks!\n\n---\n\n**Q2.** RL baseline&Discussion on $\\hat{\\theta}$.\n\n**A2.** Thanks for your question. We conduct additional experiments in Fig. 3 of [Anon. Link](https://default-anno-bucket.s3.us-west-1.amazonaws.com/rebuttal.pdf) to show that policy gradient  RL methods can be effective for AUF when sufficient interactions are available. Note that Fig. 4c in paper illustrates the performance of rehearsal methods w.r.t. the number of ***offline observational samples***, making it unsuitable for evaluating RL methods (including offline/hybrid online-offline RL as discussed in **A1**).\n\nFurthermore, we address implications of using an $\\epsilon$-approximate model $\\hat{\\theta}$, i.e., $||\\hat{\\theta}-\\theta||\\leq \\epsilon$:\n\n- **Empirical validation.** As shown in [1], $||\\hat{\\theta}-\\theta||$ decreases as the number of observational samples increases. Hence, Fig. 4c shows that our approach's performance improves as $\\epsilon$ decreases in practice.\n\n- **Theoretical analysis.** We would like to clarify that our theoretical guarantees are established on the AUF probability conditioned on $\\hat\\theta$ rather than on the true parameter $\\theta$, as discussed below Thm. 3.8. Moreover, deriving a regret bound similar to those in RL literature is generally challenging because the rehearsal-learning policy is obtained by solving a probabilistic optimization (Eq. 2). In this case, although the output action $z$ depends on $\\theta$, properties such as L-Lipschitz continuity are extremely difficult to establish, making regret analysis nontrivial. However, if the function $\\ell \\circ z(\\cdot)$ can be assumed to be L-Lipschitz continuous, then the following bound can be derived:\n$$\\ell(z(\\hat{\\theta}))-\\ell(z^*)\\leq L||\\hat{\\theta}-\\theta||_2\\lesssim\\mathcal{O}(\\frac{1}{\\sqrt{n}}),$$\nwhere $n$ is the number of observational samples. The last inequality follows from [1]. Finally, online updating of $\\hat{\\theta}$ can also be incorporated in our approach via an offline parameter-update step after each decision round.\n\nWe will refine this discussion in the revised version. Thanks!\n\n---\n\n**Q3.** The meaning of $p$.\n\n**A3.** Thanks for your feedback. In Tab. 1,  $p$ represents the dimensionality of actionable variables $z^\\xi$.\n\nRehearsal learning has two-stages: (i) estimate $\\theta$ from historical samples; (ii) make decisions after observing $x$. Only stage (ii) requires immediate actions, and Tab. 1 reports its time complexity. The sensitivity of theoretical results is discussed in **A2**.\n\nWe will incorporate this clarification in the revised version. Thanks again!\n\n---\n\n**References:**\n\n[1] Avoiding Undesired Future with Minimal Cost in Non-Stationary Environments, NeurIPS 2024.", "claims_and_evidence": "The theoretical claims in the paper seem to be supported by proper arguments. There are also limited empirical evidence supporting the proposed method.", "methods_and_evaluation_criteria": "The main empirical evaluation criterion is the probability of AUF, as measured by the empirical results.", "theoretical_claims": "I did not check the correctness of the proof.", "experimental_designs_or_analyses": "The experimental design seems reasonable though I think it might be good to add a few more baseline methods.", "relation_to_broader_scientific_literature": "I think the work is related to reinforcement learning and especially its deployment in real applications.", "essential_references_not_discussed": "NA", "other_strengths_and_weaknesses": "The paper proposes a more efficient algorithm compared to prior work, which is very desirable and this has been established theoretically. In practice there are also empirical improvements compared to prior work. I think the main weakness is how the methodology here relates more broadly to the RL community, both in terms of the methodology relevance and application relevance.", "other_comments_or_suggestions": "NA", "questions_for_authors": "=== *relevance to RL literature* ====\n\nFrom the looks of the definition of AUF, it seems that we can think of it as a RL problem where success gets $r=1$ and failure gets $r=0$, by maximizing the average reward we essentially maximize AUF. I wonder if framing the problem this way can help bridge the work's relevance to more general RL community. Do the authors agree with this characterization?\n\n=== *RL algorithm baseline* ===\n\nIf the above characterization is proper, then I wonder whether there can be more RL related baselines to compare against in Fig 4 (c), such as reinforce policy gradient. \n\nI also find that the methodology proposed in this work has a rather strong \"model based|\" flavor, in that in order to obtain the solution we need to already know the graphical model behind the transition dynamics. This access to the ground truth model provides an advantage to the proposed method and I wonder what happens if the assumed model deviates from the real world domain, can we characterize the performance regret in that case? and what if we get to update the model based on real life applications? these ablations or discussions or bring the proposed method closer to the problems that RL community is well-versed in.\n\n=== *time complexity* ===\n\nAs a minor point, I wonder what $p$ stands for in the time complexity comparison in Fig 1. I also assume that this complexity is obtained under assuming access to the ground truth graphical model of the problem, and I wonder what is the sensitivity of the theoretical result to the correctness of the graphical model (ie what happens if the model is $\\epsilon$ away from the ground truth model. I think characterizing those will be more meaningful for practitioners.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "NA"}}, {"review_id": "mVQdbeTUiO", "reviewer": "Reviewer_XTv5", "summary": "This paper addresses the AUF (Avoiding Undesired Future) problem in machine learning decision-making, where the goal is to identify actions that prevent undesirable outcomes predicted by ML models.  It introduces the CARE condition (CAnonical REctangle), a novel assumption under which the AUF probability—i.e., the probability that a post-decision outcome falls within a desired region—can be explicitly expressed and transformed (via a negative log operation) into a convex function. This convexity enables efficient optimization. They present a projection-Newton algorithm that achieves superlinear convergence to the optimal decision alteration and an inner embedding technique for cases where the CARE condition does not hold. Additionally, the paper provides a closed-form solution when the outcome is a singleton, significantly reducing time complexity. The experimental results on both synthetic and real-world datasets demonstrate that the proposed approach not only improves the AUF probability but also enhances computational efficiency compared to existing rehearsal learning and reinforcement learning methods.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable feedback and appreciation of our work! We hope that our responses can address your concerns.\n\n**W1.** Further Discussion on the CARE Condition.\n\n**A1.** Thanks for your insightful question. In practice, the dimensions of $\\mathbf{Y}$ are often dependent, meaning that the common region defined as {$a_i \\leq Y_i \\leq b_i$}, $i=1,\\dots,|\\mathbf{Y}|$ does not necessarily form a canonical rectangle w.r.t. the covariance matrix of $\\mathbf{Y}$. However, the CARE condition remains useful because the desired region $\\mathcal{S}$ can be manually specified by the decision-maker, as decisions are made after obtaining some system information. The overall decision-making process in rehearsal learning proceeds as follows:\n\n- **Step 1.** Initially, historical observational data samples are available, enabling estimation of the underlying system parameters $\\theta$.  \n- **Step 2.** Consequently, matrices such as $C$ and $\\Sigma$ (for Def. 3.5 of the CARE condition) are available before decision-making. Since the desired properties of the target $\\mathbf{Y}$ are typically determined by the decision-maker, the target region can be manually adjusted based on the estimated $C$ and $\\Sigma$ to ensure compliance with the CARE condition.\n- **Example.** In portfolio optimization, defining a desired region such as $\\{a_1 \\leq \\text{returns} \\leq b_1\\}$ and $\\{a_2 \\leq \\text{risk} \\leq b_2\\}$ ensures a balanced strategy with both favorable returns and controlled risk. Alternatively, using desired region such as $\\{a \\leq \\alpha\\cdot\\text{returns}+\\beta\\cdot\\text{risk}\\leq b\\}$ and $\\{c \\leq \\beta\\cdot\\text{returns}-\\alpha\\cdot\\text{risk} \\leq d\\}$ (where $\\alpha, \\beta\\neq 0$ and depend on $C$, $\\Sigma$) can also achieve a balanced strategy while satisfying the CARE condition, allowing for a theoretically optimal solution.  \n\nHence, the CARE condition can be satisfied by appropriately adjusting the desired region if such adjustments are feasible. Additionally, the inner CARE embedding of {$a_i \\leq Y_i \\leq b_i$} can be computed using techniques for identifying axis-parallel rectangles within polygons [1]. By defining a basis aligned with the specific matrix space, the CARE embedding of the polygon {$a_i \\leq Y_i \\leq b_i$} can be derived. Additional experimental results provide support of this argument as detailed in **A2**.\n\nWe will incorporate this discussion in the revised version. Thanks!  \n\n---\n\n**W2.** Additional case studies of inner CARE embedding.\n\n**A2.** Thanks for your thoughtful question. We conducted additional experiments for cases where the dimensions of $\\mathbf{Y}$ remain **dependent** even after alterations, demonstrating that our proposed inner CARE embedding method is effective and flexible. Specifically, the experiments include two types of desired regions:  (i) Circular desired region, with the inner CARE embedding computed using Eq. (4); (ii) Axis-aligned rectangular region {$a_i \\leq Y_i \\leq b_i$}, with the inner CARE embedding computed using [1]. The visualizations are presented in Fig. 2 of [Anon. Link](https://default-anno-bucket.s3.us-west-1.amazonaws.com/rebuttal.pdf), and the detailed performance, measured by AUF probability, is listed below:\n\n| Region types|      No action      | QWZ23 [2]        | MICNS [3]        | Ours             |\n| --------------------------- | :--------------: | ---------------- | ---------------- | ---------------- |\n| Circular region| $0.010\\pm 0.036$ | $0.961\\pm 0.013$ | $0.957\\pm 0.024$ | $0.983\\pm 0.018$ |\n| {$a_i \\leq Y_i \\leq b_i$} | $0.004\\pm 0.021$ | $0.885\\pm 0.028$ | $0.893\\pm 0.044$ | $0.922\\pm 0.083$ |\n\nThese results show that our proposed generalization method, i.e., inner CARE embedding, is effective in several irregular or non-canonical scenarios.  \n\nAdditionally, we would like to emphasize that finding a maximal axis-parallel inner rectangle for an irregular region is a challenging geometric problem [1], especially in high-dimensional spaces. Hence in practice, one can instead identify an inner canonical rectangle (not necessarily the maximal one) and optimize using this inner rectangle as a surrogate, which still enables a deterministic transformation in Cor. 3.7. This approach remains valid, as any inner region of the original retains a probability mass less than or equal to that of the original region, ensuring consistency with Prop. 3.10 due to the non-negativity of the probability density function.\n\nWe will incorporate these results and discussions into the revised paper. Thanks!\n\n---\n\n**References:**\n\n[1] Finding the largest area axis-parallel rectangle in a polygon, Computational Geometry 1997.\n\n[2] Rehearsal Learning for Avoiding Undesired Future, NeurIPS 2023.\n\n[3] Avoiding Undesired Future with Minimal Cost in Non-Stationary Environments, NeurIPS 2024.\n\n---\n\nWe also take this opportunity to sincerely thank you for the careful review. Your suggestions are important for further improving the paper. Thanks again!", "claims_and_evidence": "Most of the claims are well supported by both theoretical derivations and empirical results. However, the assertion that the CARE condition is prevalent in real-world scenarios deserves additional discussion. The authors claim that this condition particularly holds when the dimensions of Y, such as Y₁ and Y₂, are mutually independent. In many practical situations, however, these objectives tend to be dependent—improvements in one may lead to declines in another. For example, in an autonomous drone delivery system, reducing the risk of package loss may require higher altitudes or longer routes, which in turn increase delivery times; similarly, in healthcare treatment planning, enhancing treatment efficacy often comes with more severe side effects. In portfolio optimization, striving to maximize returns usually entails accepting higher risk, illustrating that objectives are frequently interdependent rather than mutually independent.  \n\nThis dependency calls for further clarification or evidence to convincingly support the claim regarding the prevalence of the CARE condition, which underpins the paper. \n\n**Generalization Beyond CARE via Inner CARE Embedding:**  \nWhile the paper introduces an inner CARE embedding technique to handle scenarios where the CARE condition does not naturally hold, the evidence here is somewhat less extensive. Although Propositions 3.10 and 3.11 provide a theoretical basis and a demonstration for circular regions, the empirical evaluation of this generalization is more limited. In practical settings where the desired region S is irregular or non-canonical, additional experiments or case studies might be necessary to fully convince readers of its broad applicability.", "methods_and_evaluation_criteria": "The proposed methods are well-aligned with the AUF problem, featuring a projection-Newton algorithm, an inner CARE embedding for irregular cases, and a closed-form solution for unidimensional outcomes. The evaluation criteria—including AUF probability, success frequency over multiple rounds, and decision-making time—directly reflect the challenges of immediate, interaction-free decisions in rehearsal learning. Additionally, the use of both synthetic and real-world datasets, along with comparisons to state-of-the-art methods, provides a comprehensive and practical benchmark for the proposed approach.", "theoretical_claims": "The theoretical proofs looks sound to me.", "experimental_designs_or_analyses": "The experimental design and analyses looks good to me.", "relation_to_broader_scientific_literature": "Regarding the rehearsal learning and AUF problem characterized by linear structural equations scientific literature, this paper introduces the CARE condition which imposes the concave structure on the logged AUF probability, making this problem more tractable and paving the way to structured algorithmic solution like the projection-Newton method proposed herein.", "essential_references_not_discussed": "Not that I know of.", "other_strengths_and_weaknesses": "The paper is solid, and clear, with good theoretical and experimental results. However, the authors should backup the CARE condition with more real-world examples.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "22JbPRY0Aa", "reviewer": "Reviewer_UMM3", "summary": "The paper introduces a CAnonical REctangle (CARE) condition for the Avoiding Undesired Future (AUF) problem. Under this CARE condition, along with additional assumptions on the problem structure and the noise term, the AUF problem can be reformulated as a convex optimization problem. The authors propose a projection-Newton-based algorithm with a provable sublinear convergence rate and extend the approach to cases where the CARE condition does not hold. Furthermore, for the special case where the target variable $Y$ is single-dimensional, the paper derives a closed-form solution that significantly reduces time complexity. Numerical experiments on both synthetic and real-world datasets are provided to demonstrate the effectiveness and efficiency of the proposed algorithms.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the valuable feedback! We hope our responses can address your concerns.\n\n**Q1.** Extension for non-Gaussian noise.\n\n**A1.** Thanks for your insightful question. We would like to clarify that the Gaussian noise assumption is primarily used to establish theoretical guarantees. For cases where this assumption does not hold, AUF can also be addressed effectively by:\n\n- **Gaussian approximation.** It is common to approximate irregular distributions using a Gaussian distribution. E.g., Laplace's approximation [1] can approximate unimodal distributions by fitting a Gaussian distribution centered at the mode. To validate the empirical effectiveness of our method under non-Gaussian noise, we provide additional experiments as shown in Fig. 1 of [Anon. Link](https://default-anno-bucket.s3.us-west-1.amazonaws.com/rebuttal.pdf).\n\n- **A numerical solution.** An alternative empirical approach can be used to solve AUF in Eq. (2) by training a sampler (e.g., normalizing flow) for the potentially non-Gaussian noise using residuals. This method involves sampling $n$ noise realizations from the trained sampler and selecting $z^\\xi$ to maximize the number of instances where $Y\\in S$. This can be formulated as the following mixed-integer linear program:\n$$\\begin{aligned}\\max_{e_i\\in\\{0,1\\}, z^\\xi}\\ &\\sum_{i=1}^n e_i\\\\\\\\s.t.\\ &M(Ax+Bz^\\xi+C\\epsilon_i)-d\\leq(1-e_i)\\alpha,\\ i\\in[n]\\end{aligned}$$\nwhere $e_i$ indicates whether $i$-th sample is successful, $\\alpha$ is a sufficiently large constant vector to tolerate failed samples, $M,d,A,B,C$ are defined in Thm. 3.6. This approach can be empirically applied with non-Gaussian noise.\n\nWe will incorporate this discussion into the revised paper. Thanks!\n\n---\n\n**Q2.** Potential distribution family making Thm. 3.6 valid.\n\n**A2.** Thanks for your question. The validity of Thm. 3.6 stems from the fact that Gaussian belongs to both elliptical and exponential families. Hence, the decomposition (Eq. 8) and the log-concavity (lines 611–628) can be established.\n\nThe main challenge in extending these results to general distributions, such as the exponential family, is that simultaneously satisfying both the decomposition and log-concavity is difficult. Additionally, the PDF of a general distribution can be complex, limiting the applicability of existing analytical techniques. Recognizing the importance of extending theoretical results to non-Gaussian cases, we are currently investigating a similar theoretical foundation for the log-concave family (including Gaussian, uniform, and Laplace distributions, etc.) as part of our future work. In this case, we find that techniques used in this paper are insufficient, and the Prékopa–Leindler theorem [2] may provide a useful tool for proving log-concavity of generalized distribution families. However, explicit characterization for these distributions remains a significant challenge.\n\nFinally, we would like to emphasize that proving theoretical optimality for probabilistic optimization is challenging, even under Gaussian noise. Establishing theoretical guarantees under Gaussian noise provides insights that extend beyond Gaussian scenarios and is a common practice in studies on previous rehearsal learning [3, 4], time series analysis [5], and control problems [6], among others.\n\nWe will incorporate this discussion in the revised paper. Thanks!\n\n---\n\n**Q3.** Detailed data information.\n\n**A3.** Thanks for your question.\n- **For synthetic data**. Let $V=[feature_1,feature_2,C_{our},C_{cpt},P_{our},P_{cpt},NCT,TPF]$ with variables defined in Fig. 5. The data generation follows $V=PV+\\epsilon,\\epsilon\\sim\\mathcal{N}(0,\\Sigma)$:\n$$P=\\begin{pmatrix}0&0&0&0&0&0&0&0\\\\\\\\0&0&0&0&0&0&0&0\\\\\\\\10&0&0&0&0&0&0&0\\\\\\\\0&10&0&0&0&0&0&0\\\\\\\\0&0&2.0&0.4&0&0&0&0\\\\\\\\0&0&0.5&1.3&0&0&0&0\\\\\\\\0&0&1.6&0&-0.9&0&0&0\\\\\\\\0&0&-1&0&0.9&0&0&0\\end{pmatrix},\\Sigma=10^{-2}\\begin{pmatrix}4&0&0&0&0&0&0&0\\\\\\\\0&4&0&0&0&0&0&0\\\\\\\\0&0&6&0&0&0&0&0\\\\\\\\0&0&0&3&16&0&0&0\\\\\\\\0&0&0&16&6&0&0&0\\\\\\\\0&0&0&0&0&6&0&0\\\\\\\\0&0&0&0&0&0&4&0\\\\\\\\0&0&0&0&0&0&0&12\\end{pmatrix}.$$\nAs $feature_{1,2}$ have no parents, their covariance can be marginalized thus omitted. Parameters are identical to those used in the code provided in Supp. Material and experimental results can be reproduced by running the code.\n\n- **For real-world data**. The dataset records values of environment variables in Bermuda, and the decision target is to maintain a high NEC (net coral ecosystem calcification). Due to space limits, we welcome further questions on the dataset and will provide additional details in Appx. E.2 of the revised paper.\n\nThanks again!\n\n---\n\n**References:**\n\n[1] Pattern Recognition and Machine Learning, 2006\n\n[2] On logarithmic concave measures and functions, 1973\n\n[3] Avoiding Undesired Future with Minimal Cost in Non-Stationary Environments, NeurIPS 2024\n\n[4] Rehearsal Learning for Avoiding Undesired Future, NeurIPS 2023\n\n[5] Time series analysis and its applications, 2000\n\n[6] Online Linear Quadratic Control, ICML 2018", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "I reviewed the proofs of Theorem 3.6 and Theorem 3.8, and they appear to be correct to the best of my knowledge.", "experimental_designs_or_analyses": "While the overall experimental design seems sound, the paper does not provide sufficient details on the experimental setup in its appendix. For example, the procedure for generating the synthetic data is not clearly explained.", "relation_to_broader_scientific_literature": "The results presented can be applied to general AUF problems under specific conditions.", "essential_references_not_discussed": "Not sure.", "other_strengths_and_weaknesses": "Strengths: The paper is well written and provides clear remarks and visualizations that effectively illustrate the theoretical results.\n\nWeaknesses: The discussion regarding the additive noise is limited; it appears that the results may only hold under Gaussian noise assumptions. Additionally, the experimental section would be clearer with more detailed descriptions of the experimental setup.", "other_comments_or_suggestions": "None.", "questions_for_authors": "a) If the additive noise in equation (1) does not follow a Gaussian distribution, do the proposed methods remain (numerically) effective? A discussion on potential performance variations or issues under non-Gaussian noise, or a potential extension method which can be applied in non-Gaussian noises would help clarify the broader applicability of the results.\n\nb) Related to the previous question, if Theorem 3.6 is valid only under the assumption of Gaussian noise, are there other families of distributions (e.g., those within the exponential family) for which the results could still hold? The limitation of Gaussian noises is my major concern.\n\nc) Could you provide more detailed information on how the synthetic data was generated, as well as a summary of the characteristics of the real-world dataset used in your experiments?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper received mildly positive reviews. There are strengths and weaknesses:

**Strengths**

- Novelty: Introduces the CARE condition to make the AUF problem convex and tractable.

- Clarity: Well-written with clear visualizations.

- Practical Performance: Outperforms baselines on both synthetic and real datasets.

**Weaknesses**

- CARE Assumption: Realism of the CARE condition is questionable.

- Noise Assumptions: Results rely on Gaussian noise; unclear how well it generalizes for what concerns theory.

Overall, I think the strengths outweigh the weaknesses, and thus I am inclined toward acceptance.
Decision: Accept (poster)