SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Thm. 16 shows that a rehearsal graph has several Bayesian network counterparts that have the same skeleton and encode the same conditional independence relations. So by learning a Bayesian network or a Markov equivalence class of Bayesian networks from data, we can determine the skeleton of the rehearsal graph. Then we can get a rehearsal graph or a set of rehearsal graphs that cannot be distinguished from observational data by adding bi-directional edges to the edges whose directions are not determined. Assuming that the observational distribution factorizes according to a rehearsal graph and that all conditional independence relations of the distribution are encoded by the rehearsal graph, Algorithm 2 lists all valid rehearsal graphs. The first three steps of Algorithm 2 are simply the IC algorithm for Bayesian network learning [12]. Then we list all Bayesian networks and map them back to rehearsal graphs by replacing directional edges with bi-directional ones.
Algorithm 2 Learning rehearsal graphs from observational data
Input: Observational distribution  $P(\mathbf{V})$ .
1: For each pair of variables  $a$  and  $b$  in  $\mathbf{V}$ , search for a set  $S_{ab}$  such that  $a \perp b \mid S_{ab}$  holds. Construct an undirected graph  $G$  such that vertices  $a$  and  $b$  are connected with an edge if and only if no set  $S_{ab}$  can be found.
2: For each pair of nonadjacent variables  $a$  and  $b$  with a common neighbor  $c$ , check if  $c \in S_{ab}$ . If it is, then continue. If it is not, then add arrowheads pointing at  $c$ , i.e.,  $a \rightarrow c \leftarrow b$ .
3: In the partially directed graph that results, orient as many of the undirected edges as possible subject to two conditions:
(i) Any alternative orientation would yield a new  $v$ -structure;
(ii) Any alternative orientation would yield a directed cycle.
4: List all Markov equivalent directed graphs from the partially directed graph. Denote the set of Markov equivalent graphs with  $\mathcal{G}_1$ .
5: For each listed graph  $g$  in  $\mathcal{G}_1$ , build a set of mixed graphs  $\mathcal{G}_2^g$  by enumerating all possible combinations of cliques and replace the directional edges therein with bi-directional edges.
6: Check if the graphs in each  $\mathcal{G}_2^g$  are valid rehearsal graphs. If they are not, remove them from corresponding  $\mathcal{G}_2^g$ .
Output: Markov equivalent rehearsal graphs  $\mathcal{G}_1 \cup (\cup_{g \in \mathcal{G}_1} \mathcal{G}_2^g)$ .
B Candidate Alteration Selection
B.1 Size-1 Alteration
Algorithm 1 in the main text shows the main steps of finding candidate alterations, where details on finding alteration values that satisfy the performance requirement are omitted. Here we give more details in Algorithm 3. The main idea is to sort the found intervals in increasing order, then keep those that appear more than  $n \cdot \tau$  times. Sorting the array in Line 17 on average costs  $O(n \log n)$  time if using quicksort. Other operations cost  $O(n)$  time. Thus, the overall average running time of Algorithm 3 is  $O(|\mathbf{Z}_t| \cdot n \log n)$ .
B.2 Size- $k$  Alteration
We propose to build a finite candidate set by solving a mixed-integer linear programming problem:
$$
\max  _ {\boldsymbol {e} \in \{0, 1 \} ^ {n ^ {\prime}}, \boldsymbol {z} ^ {\xi}} \sum_ {i} \boldsymbol {e} _ {i}
$$
$$
\text {s . t .} \mathbf {M} \left(\mathbf {A} _ {i} \mathbf {x} + \mathbf {B} _ {i} \mathbf {z} ^ {\xi} + \mathbf {C} _ {i} \varepsilon_ {i}\right) - \mathbf {d} \leq \left(1 - e _ {i}\right) \boldsymbol {\alpha}, i \in [ n ^ {\prime} ],
$$
where  $e_i$  is a binary decision variable that equals 1 if the  $i$ -th inequality is satisfied and 0 otherwise, and  $\alpha$  is a constant vector that upper bounds the left-hand side. If the found solution satisfies  $\sum_{i} e_i \geq n \cdot \tau$ , corresponding  $\mathbf{z}^\xi$  will be joined in the candidate set. The above programming is run for every combination of at most  $k$  alteration targets multiple times to obtain multiple alterations. The hard constraints satisfied by previous programs for a common  $\mathbf{Z}^\xi$  are randomly dropped in following iterations to encourage diverse solutions, thus the  $n' \leq n$  constraints instead of  $n$ .
C Mutual Information Maximization
Here we detail the estimation of the mutual information objective in Eq. (5). We have
$$
I \left(\mathbf {G}, \boldsymbol {\Theta}; \mathbf {Z} _ {t}, \mathbf {Y} _ {t} \mid D _ {t}, \mathbf {x} _ {t}, \xi_ {t}\right) = H \left(\mathbf {Z} _ {t}, \mathbf {Y} _ {t} \mid D _ {t}, \mathbf {x} _ {t}, \xi_ {t}\right) - H \left(\mathbf {Z} _ {t}, \mathbf {Y} _ {t} \mid \mathbf {G}, \boldsymbol {\Theta}, D _ {t}, \mathbf {x} _ {t}, \xi_ {t}\right).
$$
In order to estimate the mutual information, we only need to estimate  $\mathrm{H}(\mathbf{Z}_t,\mathbf{Y}_t\mid D_t,\mathbf{x}_t,\xi_t)$  and  $\mathrm{H}(\mathbf{Z}_t,\mathbf{Y}_t\mid \mathbf{G},\boldsymbol {\Theta},D_t,\mathbf{x}_t,\xi_t)$ . We give a detailed estimation procedure in Algorithm 4. The entropy estimation method ENTROPYESTIMATE that estimates information entropy with samples is used as a black-box method since various methods are available, e.g., Sricharan et al. [59].
Let  $g \triangleq \operatorname{I}(\mathbf{G}, \Theta; \mathbf{Z}_t, \mathbf{Y}_t \mid D_t, \mathbf{x}_t, \xi_t)$  denote the target optimization function to be used in Bayesian optimization [22]. A Gaussian process is used to attain the posterior of  $g$ . Denote the posterior
Algorithm 3 Find candidate single alterations
Input:  $S_{tr} = \{\langle G_i,\theta_i,\varepsilon_i\rangle \}_{i = 1}^n$ $S_{val} = \{\langle G_i,\theta_i,\varepsilon_j\rangle \}_{i = n + 1}^{2n}$    
1:  $\mathcal{C}\gets \emptyset$  ▷Store candidate alterations   
2: for  $Z_{i}\in \mathbf{Z}$  do   
3: int1  $\leftarrow$  FIND  $(S_{tr},Z_{i},\tau)$  ▷Training   
4: int2  $\leftarrow$  FIND(Sval,Zi,τ)   
5:  $\mathcal{C}\gets \mathcal{C}\cup \{\langle Z_i,\mathrm{int}1\cap \mathrm{int}2\rangle \}$  ▷Validation   
6: function FIND(S,Z,τ) ▷Find alteration values   
7: int  $\leftarrow \emptyset$  ▷Store valid intervals   
8: for  $\langle G,\pmb {\theta},\pmb {\varepsilon}\rangle \in S$  do   
9:  $[u,v]\leftarrow$  solution of  $\mathbf{M}(\mathbf{Ax} + \mathbf{Bz}^{\xi} + \mathbf{C}\varepsilon)\leq \mathbf{d}$  ▷A,B,C are determined by  $G,\theta ,Z$    
10: int  $\leftarrow$  int U  $\{[u,v]\cap \Delta (Z)\}$    
11: return FILTERINTERVALS(int，[nτ])   
12: function FILTERINTERVALS(，k) ▷Find elements that appear more than  $k$  times in I   
13: arr  $\leftarrow \emptyset$    
14: for [u,v]  $\in \mathcal{I}$  do   
15: arr  $\leftarrow$  arr U  $\{(u,0)\}$    
16: arr  $\leftarrow$  arr U  $\{(v,1)\}$  ▷0 or 1 indicates the value is taken from the left or the right   
endpoints of an interval   
17: arr  $\leftarrow$  sorted(arr) ▷Sorted in increasing order with the first element as the key   
18: cnt  $\leftarrow 0$    
19: for  $0\leq i <   |\mathrm{arr}|$  do   
20: if arr[i][1] = 0 then ▷If the value is a left endpoint   
21: cnt  $\leftarrow$  cnt + 1   
22: else   
23: cnt  $\leftarrow$  cnt - 1   
24: arr[i][1]  $\leftarrow$  cnt ▷Record the number of intervals that intersect at arr[i][0]   
25: rtn  $\leftarrow \emptyset$    
26: left  $\leftarrow$  NULL,right  $\leftarrow$  NULL   
27: for  $0\leq i <   |\mathrm{arr}|$  do ▷Scan the array and record valid intervals   
28: right  $\leftarrow$  arr[i][0]   
29: if arr[i][1] ≥ k then   
30: if left is NULL then   
31: left  $\leftarrow$  arr[i][0]   
32: else   
33: if left is not NULL then   
34: rtn  $\leftarrow$  rtn U  $\{[\mathrm{left},\mathrm{right}]\}$    
35: left  $\leftarrow$  NULL   
36: return rtn   
Output: The found alterations C
mean and variance at an unseen  $\mathbf{z}_{l + 1}$  by  $\mu_g(\mathbf{z}_{l + 1})$  and  $\sigma_g^2 (\mathbf{z}_{l + 1})$  respectively. Given queried points  $\{\mathbf{z}^1,\dots ,\mathbf{z}^l\}$ , we use upper confidence bound [60] as a querying criterion to select the next point:
$$
\mathbf{z}^{l + 1} = \operatorname *{arg  max}_{\mathbf{z}^{l + 1}\in \mathcal{C}(\mathbf{Z}^{\xi})}\mu_{g}(\mathbf{z}^{l + 1}) + \gamma \cdot \sigma_{g}(\mathbf{z}^{l + 1}),
$$
where  $\gamma$  is a constant and  $\mathcal{C}(\mathbf{Z}^{\xi})$  is the feasible region of  $\mathbf{Z}^{\xi}$  defined by  $\mathcal{C}$ . We run this optimization process for every alteration target in  $\mathcal{C}$  to find the corresponding best alteration values. Finally, the alteration, including both target variables and values, that has maximum mutual information is selected as the recommended output.
Algorithm 4 Mutual information estimation
Input: The posterior estimate  $p(G,\theta \mid D_t)$ , the candidate graph set  $\mathcal{G}$ , the observed  $\mathbf{x}$ , the alteration  $\xi$ , the number of samples for estimating an entropy term  $n$ , the number of samples for structural parameters  $n'$ .  
1: srms  $\leftarrow \emptyset$   
2: for  $G \in \mathcal{G}$  do  
3: for  $i \in [[n \cdot p(G \mid D_t)]]$  do  
4: Sample  $\theta$  from  $p(\theta \mid G, D_t)$   
5: srms  $\leftarrow$  srms  $\cup \{\langle G, \theta_i \rangle\}$   
6: samples  $\leftarrow \emptyset$   
7: for  $\langle G, \theta \rangle \in$  srms do  
8: Sample  $\mathbf{z}$  and  $\mathbf{y}$  from  $p(\mathbf{z}, \mathbf{y} \mid G, \theta, \mathbf{x}, \xi)$   
9: samples  $\leftarrow$  samples  $\cup \{(\mathbf{z}, \mathbf{y})\}$   
10:  $\hat{H}_1 \gets$  ENTROPYESTIMATE(samples)  
11:  $\hat{H}_2 \gets 0$   
12: for  $G \in \mathcal{G}$  do  
13: entropies  $\leftarrow \emptyset$   
14: for  $i \in [n']$  do  
15: Sample  $\theta$  from  $p(\theta \mid G, D_t)$   
16: samples  $\leftarrow \emptyset$   
17: for  $i \in [n]$  do  
18: Sample  $\mathbf{z}$  and  $\mathbf{y}$  from  $p(\mathbf{z}, \mathbf{y} \mid G, \theta, \mathbf{x}, \xi)$   
19: samples  $\leftarrow$  samples  $\cup \{(\mathbf{z}, \mathbf{y})\}$   
20: entropies  $\leftarrow$  entropies  $\cup$  {ENTROPYESTIMATE(samples)}  
21: cond_entropy  $\leftarrow$  average of entropies  
22:  $\hat{H}_2 \gets \hat{H}_2 +$  cond_entropy  $\cdot p(G \mid D_t)$   
23:  $\hat{I} \gets \hat{H}_1 - \hat{H}_2$   
Output: The mutual information estimate  $\hat{I}$
In this section, we provide proof for claims in the main text.
$$
\begin{array}{l} \mathbb {P} (\mathbf {y} \mid \mathbf {x}, R h (\xi); G _ {i}, \boldsymbol {\theta} _ {i}) \\ = \sum_ {\mathbf {z}} \mathbb {P} (\mathbf {z} \mid \mathbf {x}, R h (\boldsymbol {\xi}); G _ {i}, \pmb {\theta} _ {i}) \mathbb {P} (\mathbf {y} \mid \mathbf {z}, \mathbf {x}, R h (\boldsymbol {\xi}); G _ {i}, \pmb {\theta} _ {i}), \quad i = 1, 2. \\ \end{array}
$$
Since  $(G_{1},\theta_{1})$  and  $(G_{2},\theta_{2})$  only differ in parameters controlling  $\mathbb{P}(\mathbf{x})$ , we have
$$
\mathbb {P} (\mathbf {z} \mid \mathbf {x}, R h (\xi); G _ {1}, \boldsymbol {\theta} _ {1}) = \mathbb {P} (\mathbf {z} \mid \mathbf {x}, R h (\xi); G _ {2}, \boldsymbol {\theta} _ {2}),
$$
$$
\mathbb {P} (\mathbf {y} \mid \mathbf {z}, \mathbf {x}, R h (\xi); G _ {1}, \boldsymbol {\theta} _ {1}) = \mathbb {P} (\mathbf {y} \mid \mathbf {z}, \mathbf {x}, R h (\xi); G _ {2}, \boldsymbol {\theta} _ {2}).
$$
Combining the above equations gives
$$
\mathbb {P} \left(\mathbf {y} \mid \mathbf {x}, R h (\xi); G _ {1}, \boldsymbol {\theta} _ {1}\right) = \mathbb {P} \left(\mathbf {y} \mid \mathbf {x}, R h (\xi); G _ {2}, \boldsymbol {\theta} _ {2}\right).
$$
Proposition 5. Let  $\xi = \{(Z_{a_i},z_{a_i})\}_{i = 1}^k$  be a valid alteration where  $Z_{a_i}\in \mathbf{Z}_t$  and  $z_{a_i}\in \Delta (Z_{a_i})$ . Given  $\langle G,\theta ,\varepsilon \rangle$  and  $\mathbf{x}_t$ , the outcome variables are linear in the alteration values. We have
Proof. The proposition is immediate by noting that if the parent of a set of variables can be described by
$$
\mathbf {P} \mathbf {A} = \mathbf {A} ^ {\prime} \mathbf {x} + \mathbf {B} ^ {\prime} \mathbf {z} ^ {\xi} + \mathbf {C} ^ {\prime} \varepsilon ,
$$
$$
\mathbf {V} = \boldsymbol {\beta} ^ {T} \mathbf {P} \mathbf {A} + \varepsilon_ {\mathbf {V}} = \boldsymbol {\beta} ^ {T} \mathbf {A} ^ {\prime} \mathbf {x} + \boldsymbol {\beta} ^ {T} \mathbf {B} ^ {\prime} \mathbf {z} ^ {\xi} + \boldsymbol {\beta} ^ {T} \mathbf {C} ^ {\prime} \varepsilon + \varepsilon_ {\mathbf {V}}
$$
can be described in a similar linear form.
Proof. We prove this by contradiction. Assume that the problem is solvable with a subroutine  $f$  that runs in polynomial time. Then, given any infeasible linear system  $\Sigma : \{\mathbf{A}\mathbf{x} \leq \mathbf{b}\}$ , we can find a feasible subsystem containing as many inequalities as possible by finding a maximized  $\tau$  that gives a valid solution for the problem of finding a  $\tau$  that satisfies at least  $n \cdot \tau$  inequalities. And the process of finding a maximized  $\tau$  can be finished by running a binary search on  $\tau$  and calling  $f$ , which will be called at most  $O(\log n)$  times. Thus the overall running time will still be polynomial. However, finding a maximum feasible subsystem is known to be NP-hard [21], which cannot be solved within polynomial time if  $\mathrm{P} \neq \mathrm{NP}$ . Therefore we have a contradiction and conclude that the problem does not admit a polynomial time algorithm.
$$
\max  \left\{p, \hat {p} - \sqrt {\ln (2 / \delta) / 2 n} \right\} \leq \mathbb {P} \left(\mathbf {Y} _ {t} \in \mathcal {S} | D _ {t}, \mathbf {x} _ {t}, R h (\xi)\right) \leq \min  \left\{\bar {p}, \hat {p} + \sqrt {\ln (2 / \delta) / 2 n} \right\}.
$$
Proof. The proof is mainly based on the scenario approach [23] and adapted from Badings et al. [24]. The desired region for  $\mathbf{Y}$  is given by  $\mathcal{S} = \{\mathbf{y} \in \mathbb{R}^{|\mathbf{Y}|} \mid \mathbf{M}\mathbf{y} \leq \mathbf{d}\}$ . We define a scaled version of  $\mathcal{S}$  by
$$
R (\lambda) = \left\{\mathbf {y} \in \mathbb {R} ^ {| \mathbf {Y} |} \mid \mathbf {M} \mathbf {y} \leq \lambda \mathbf {d} + (1 - \lambda) \mathbf {M} \mathbf {h} \right\}, \tag {19}
$$
where  $h \in \mathbb{R}$  is a Chebyshev center of  $\mathcal{S}$  [61]. We have  $R(1) = \mathcal{S}$  and  $R(\lambda_1) \subset R(\lambda_2)$  for any  $0 \leq \lambda_1 < \lambda_2$ . Using Prop. 5, we express each  $\mathbf{y}^i$  with
$$
\mathbf {y} ^ {i} = \mathbf {A} _ {i} \mathbf {x} + \mathbf {B} _ {i} \mathbf {z} ^ {\xi} + \mathbf {C} _ {i} \varepsilon_ {i}. \tag {20}
$$
We use the scenario approach to estimate the probability of the happening of undesired  $\mathbf{Y}$  with the following convex scenario optimization problem with discarded samples:
$$
\min  _ {\lambda \geq 0} \lambda \tag {21}
$$
$$
\begin{array}{l} \text {s . t .} \quad \mathbf {y} ^ {i} \in R (\lambda) \forall i \in \{1, \dots , n \} \backslash Q, \end{array}
$$
which has a scalar decision variable  $\lambda$ . We denote the optimal solution of the above optimization program with  $\lambda_{|Q|}^{*}$ .  $Q$  is a subset of samples whose constraints have been discarded according to the following rule [24, Lem. 1]: The sample removal set  $Q \subseteq \{1, \dots, n\}$  is obtained by iteratively removing the active constraints from (21). Thus, given  $N$  samples and any two removal sets with cardinalities  $|Q_1| < |Q_2|$ , it holds that  $Q_1 \subset Q_2$ . Moreover, any discarded sample  $i \in Q$  violates the solution  $\lambda_Q^*$  to Eq. (21), i.e.  $\mathbf{y}^i \notin R_j(\lambda_{|Q|}^*)$ , with probability one. When discarding  $|Q| = n_o$  samples, it holds that  $R(\lambda_{n_o}^*) \subseteq S$ . When discarding  $|Q| = n_o - 1$  samples ( $n_o > 0$ ),  $R(\lambda_{n_o - 1}^*) \supset S$ .
We further assume that given a sample  $\langle G_i, \pmb{\theta}_i, \pmb{\varepsilon}_i \rangle$  from  $\mathbb{P}(G, \pmb{\theta}, \pmb{\varepsilon} \mid D_t)$ , the probability that the generated  $\mathbf{y}^i$  is on the boundary of any polytope  $R(\lambda)$  for any  $\lambda \geq 0$  is zero.
Based on the results of Romao et al. [62, Thm. 5] and Badings et al. [24, Thm. 1], we have
$$
\mathbb {P} \left\{P (\mathbf {y} \notin R \left(\lambda_ {| Q |} ^ {*}\right)) \leq \epsilon \right\} = F (\epsilon ; | Q | + 1, n - | Q |), \tag {22}
$$
where  $F(\cdot ;\alpha ,\beta)$  is the cumulative distribution function of the beta distribution with parameters  $\alpha$  and  $\beta$ ,  $|Q| < n$ . Equivalently, we have
$$
\mathbb {P} \left\{P (\mathbf {y} \notin R \left(\lambda_ {| Q |} ^ {*}\right)) > \epsilon \right\} = 1 - F (\epsilon ; | Q | + 1, n - | Q |). \tag {23}
$$
Solving  $1 - F(\epsilon; |Q| + 1, n - |Q|) = \frac{\delta}{2n}$ , we have
$$
\epsilon = F ^ {- 1} \left(1 - \frac {\delta}{2 n}; | Q | + 1, n - | Q |\right). \tag {24}
$$
For notational convenience, let  $F_{i}(\cdot) = F(\cdot ;i + 1,n - i)$ . We have
$$
\mathbb {P} \left\{P (\mathbf {y} \notin R \left(\lambda_ {| Q |} ^ {*}\right)) > F _ {| Q |} ^ {- 1} \left(1 - \frac {\delta}{2 n}\right) \right\} = \frac {\delta}{2 n}. \tag {25}
$$
Using Boole's inequality, we know that
$$
\mathbb {P} \left\{\bigcup_ {i = 0} ^ {n - 1} P \left(\mathbf {y} \notin R \left(\lambda_ {i} ^ {*}\right)\right) > F _ {i} ^ {- 1} \left(1 - \frac {\delta}{2 n}\right) \right\} \leq \sum_ {i = 0} ^ {n - 1} \mathbb {P} \left\{P \left(\mathbf {y} \notin R \left(\lambda_ {i} ^ {*}\right)\right) > F _ {i} ^ {- 1} \left(1 - \frac {\delta}{2 n}\right) \right\} = \frac {\delta}{2}. \tag {26}
$$
$$
\mathbb {P} \left\{\bigcap_ {i = 0} ^ {n - 1} P \left(\mathbf {y} \notin R \left(\lambda_ {i} ^ {*}\right)\right) \leq F _ {i} ^ {- 1} \left(1 - \frac {\delta}{2 n}\right) \right\} \geq 1 - \frac {\delta}{2}. \tag {27}
$$
After observing  $\mathbf{y}^i$  s at hand, we replace  $|Q|$  by  $n_o$ . Since  $R(\lambda_{n_o}^*) \subset S$  and  $\bigcap_{i=0}^{n-1} P(\mathbf{y} \notin R(\lambda_i^*)) \leq F_i^{-1}(1 - \frac{\delta}{2n})$  implies  $P(\mathbf{y} \notin R(\lambda_{n_o}^*)) \leq F_{n_o}^{-1}(1 - \frac{\delta}{2n})$ , we have
$$
\mathbb {P} \left\{P (\mathbf {y} \notin \mathcal {S}) \leq F _ {n _ {o}} ^ {- 1} \left(1 - \frac {\delta}{2 n}\right) \right\} \geq \mathbb {P} \left\{P \left(\mathbf {y} \notin R \left(\lambda_ {n _ {o}} ^ {*}\right)\right) \leq F _ {n _ {o}} ^ {- 1} \left(1 - \frac {\delta}{2 n}\right) \right\} \geq 1 - \frac {\delta}{2}, \tag {28}
$$
$$
\mathbb {P} \left\{P (\mathbf {y} \in \mathcal {S}) \geq 1 - F _ {n _ {o}} ^ {- 1} \left(1 - \frac {\delta}{2 n}\right) \right\} \geq 1 - \frac {\delta}{2}. \tag {29}
$$
When  $n_o = n$ ,  $P(\mathbf{y} \in \mathcal{S}) \geq 0$  trivially holds. Thus,  $\mathfrak{p} \leq \mathbb{P}\left(\mathbf{Y}_t \in \mathcal{S} \mid D_t, \mathbf{x}_t, Rh(\xi)\right)$  holds with probability at least  $1 - \frac{\delta}{2}$ . For the other half of the lower bound, we apply Hoeffding's inequality on binary variables  $D_i \triangleq \mathbb{I}(\mathbf{y}^i \in \mathcal{S})$ . Note that  $\sum_{i=1}^{n} D_i = n_o$  and  $\mathbb{E}(D_i) = \mathbb{P}\left(\mathbf{Y}_t \in \mathcal{S} \mid D_t, \mathbf{x}_t, Rh(\xi)\right)$ , we have
$$
\mathbb {P} \left\{n _ {o} - n \cdot \mathbb {P} \left(\mathbf {Y} _ {t} \in \mathcal {S} \mid D _ {t}, \mathbf {x} _ {t}, R h (\xi)\right) \geq \sqrt {\frac {n}{2} \ln \frac {2}{\delta}} \right\} \leq \frac {\delta}{2}, \tag {30}
$$
$$
\mathbb {P} \left\{\mathbb {P} \left(\mathbf {Y} _ {t} \in \mathcal {S} \mid D _ {t}, \mathbf {x} _ {t}, R h (\xi)\right) \geq \hat {p} - \sqrt {\frac {\ln (2 / \delta)}{2 n}} \right\} \geq 1 - \frac {\delta}{2}. \tag {31}
$$
Combining (29) and (31), we have with probability at least  $1 - \frac{\delta}{2}$ ,
$$
\max  \left\{\mathbf {p}, \hat {p} - \sqrt {\frac {\ln (2 / \delta)}{2 n}} \right\} \leq \mathbb {P} \left(\mathbf {Y} _ {t} \in \mathcal {S} \mid D _ {t}, \mathbf {x} _ {t}, R h (\xi)\right). \tag {32}
$$
For the upper bound, we adopt a similar approach. From Eq. (22), by noting that  $P\left(\mathbf{y} \notin R(\lambda_{|Q|}^{*})\right) + P\left(\mathbf{y} \in R(\lambda_{|Q|}^{*})\right) = 1$ , we get
$$
\mathbb {P} \left\{P \left(\mathbf {y} \in R \left(\lambda_ {| Q |} ^ {*}\right)\right) \geq 1 - \epsilon \right\} = F _ {| Q |} (\epsilon). \tag {33}
$$
Substituting  $F_{|Q|}(\epsilon)$  with  $\frac{\delta}{2n}$ , we have
$$
\mathbb {P} \left\{P \left(\mathbf {y} \in R \left(\lambda_ {| Q |} ^ {*}\right)\right) \geq 1 - F _ {| Q |} ^ {- 1} \left(\frac {\delta}{2 n}\right) \right\} = \frac {\delta}{2 n}. \tag {34}
$$
Again via Boole's inequality,
$$
\mathbb {P} \left\{\bigcap_ {i = 0} ^ {n - 1} P \left(\mathbf {y} \in R \left(\lambda_ {i} ^ {*}\right)\right) \leq 1 - F _ {i} ^ {- 1} \left(\frac {\delta}{2 n}\right) \right\} \geq 1 - \frac {\delta}{2}. \tag {35}
$$
When  $n_o > 0$ , since  $R(\lambda_{n_o - 1}^*) \supset S$  and  $\bigcap_{i=0}^{n-1} P(\mathbf{y} \in R(\lambda_i^*)) \leq 1 - F_i^{-1}\left(\frac{\delta}{2n}\right)$  implies  $P(\mathbf{y} \in R(\lambda_{n_o - 1}^*)) \leq 1 - F_{n_o - 1}^{-1}\left(\frac{\delta}{2n}\right)$ , we have
$$
\mathbb {P} \left\{P (\mathbf {y} \in \mathcal {S}) \leq 1 - F _ {n _ {o} - 1} ^ {- 1} \left(\frac {\delta}{2 n}\right) \right\} \geq \mathbb {P} \left\{P \left(\mathbf {y} \in R \left(\lambda_ {n _ {o} - 1} ^ {*}\right)\right) \leq 1 - F _ {n _ {o} - 1} ^ {- 1} \left(\frac {\delta}{2 n}\right) \right\} \geq 1 - \frac {\delta}{2}. \tag {36}
$$
When  $n_o = 0$ ,  $P(\mathbf{y} \in S) \leq 1$  trivially holds. Thus,
$$
\mathbb {P} \left(\mathbf {Y} _ {t} \in \mathcal {S} \mid D _ {t}, \mathbf {x} _ {t}, R h (\xi)\right) \leq \bar {p} \tag {37}
$$
holds with probability at least  $1 - \frac{\delta}{2}$ . Applying Hoeffding's inequality from the other direction of Eq. (30), we get
$$
\mathbb {P} \left\{\mathbb {P} \left(\mathbf {Y} _ {t} \in \mathcal {S} \mid D _ {t}, \mathbf {x} _ {t}, R h (\xi)\right) \leq \hat {p} + \sqrt {\frac {\ln (2 / \delta)}{2 n}} \right\} \geq 1 - \frac {\delta}{2}. \tag {38}
$$
Combining the above results gives the upper bound that holds with probability at least  $1 - \frac{\delta}{2}$ :
$$
\mathbb {P} \left(\mathbf {Y} _ {t} \in \mathcal {S} \mid D _ {t}, \mathbf {x} _ {t}, R h (\xi)\right) \leq \min  \left\{\bar {p}, \hat {p} + \sqrt {\frac {\ln (2 / \delta)}{2 n}} \right\}. \tag {39}
$$
Combining the lower bound (32) and the upper bound (39) gives the overall bound.
E Data Details
E.1 Ride-Hailing Data
We give details about the Ride-Hailing data in this section. The variables included in the generation process are
- weather: weather condition;
- #user: number of users in the neighborhood;
- recommend: recommendation level of the ride-hailing app for a specific route;
- congestion: traffic congestion on the route;
time: time spent for the ride;
- discount: discount provided by the app;
- rating: user rating for the ride.
The rehearsal graph for the variables is given in Fig. 6. The presumed actionable variables that can be altered by the app are 'recommend' and 'discount'.
Figure 6: The rehearsal graph for ride-hailing data.
E.2 Bermuda Data
We give details about the Bermuda data in this section. The Bermuda data involves a set of environmental variables [57, 58, 37]. The variables included in the generation process are
- Chla: sea surface chlorophyll a;
- Sal: sea surface salinity;
- TA: seawater total alkalinity;
DIC: seawater dissolved inorganic carbon;
$CO_2$  : seawater  $\mathrm{P_{CO_2}}$
- Temp: bottom temperature;
- NEC: net ecosystem calcification;
- Light: bottom light levels;
- Nut: PC1 of  $NH_4$ ,  $NiO_2 + NiO_3$ ,  $SiO_4$ ;
$\cdot \mathrm{pHsw}$  : seawater pH
-  $\Omega_{A}$ : seawater saturation with respect to aragonite.
The rehearsal graph for the variables is given in Fig. 7. The presumed actionable variables that can be altered are: DIC, TA,  $\Omega_A$ , Chla, and Nut.
Figure 7: The rehearsal graph for Bermuda data.