SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\min _ {\mathbf {s} \in \mathcal {I}} N _ {t _ {i}} (\mathbf {s}) \geq \frac {i - j}{| \mathcal {I} |} \geq \frac {\epsilon i}{2},
$$
which creates a contradiction.
B.2 Supporting Lemmas (Concentration Inequalities)
The following concentration inequalities are instrumental for regret analysis. For bandit with feedback drawn from arbitrary discrete distributions, a concentration bound on the empirical distribution is presented in [35, Lemma 6]. In the structured causal bandit problem with unobserved confounders, the actions space is  $\mathcal{I}$ , and the discrete support of feedback is  $\Omega(\mathbf{V})$ . At each time  $t$ , for each intervention  $\mathbf{s} \in \mathcal{I}$ , recall  $\bar{P}_{\mathbf{s},t}$  is the empirical interventional distribution of  $\mathbf{V}$  and  $N_{t}(\mathbf{s})$  is the number of times the intervention  $do(\mathbf{S} = \mathbf{s})$  is taken till  $t$ . For each intervention  $\mathbf{s} \in \mathcal{I}$ , the true interventional distribution is  $P_{\mathbf{s}}$ .
Lemma 7 (Concentration Inequality for Information Distance [35]). Let  $\delta \geq |\mathcal{I}|(|\Omega(V)| - 1)$ . Then for any  $t > 0$
$$
\mathbb {P} \left[ \sum_ {\mathbf {s} \in \mathcal {I}} N _ {t} (\mathbf {s}) D (\bar {P} _ {t, \mathbf {s}} \| P _ {\mathbf {s}}) \geq \delta \right] \leq \left(\frac {\delta \lceil \delta \ln t + 1 \rceil 2 e}{| \mathcal {I} | (| \Omega (V) | - 1)}\right) ^ {| \mathcal {I} | (| \Omega (V) | - 1)} \exp (1 - \delta).
$$
llowing Corollary 8 is a result of Lemma 7.
Corollary 8. For any  $\gamma >0$ , there exists suitably large universal constant  $c$  such that
$$
\sum_ {t = 1} ^ {\infty} \mathbb {P} \left(\sum_ {\mathbf {s} \in \mathcal {I}} N _ {t} (\mathbf {s}) D \left(\bar {P} _ {\mathbf {s}, t} \| P _ {\mathbf {s}}\right) \geq (1 + \gamma) \ln t\right) \leq c.
$$
Proof. We select  $t'$  such that  $(1 + \gamma)\ln t' \geq |\mathcal{I}|(|\Omega(V)| - 1)$ . We apply Lemma 7 to get
$$
\begin{array}{l} \sum_ {t = 1} ^ {\infty} \mathbb {P} \left(\sum_ {\mathbf {s} \in \mathcal {I}} N _ {t} (\mathbf {s}) D \left(\bar {P} _ {\mathbf {s}, t} \| P _ {\mathbf {s}}\right) \geq (1 + \gamma) \ln t\right) \\ \leq t ^ {\prime} + \sum_ {t \geq t ^ {\prime}} \mathbb {P} \left[ \sum_ {\mathbf {s} \in \mathcal {I}} N _ {t} (\mathbf {s}) D \left(\bar {P} _ {t, \mathbf {s}} \| P _ {\mathbf {s}}\right) \geq (1 + \gamma) \ln t \right] \\ \leq t ^ {\prime} + \sum_ {t \geq t ^ {\prime}} \frac {e}{t ^ {1 + \gamma}} \left(\frac {(1 + \gamma) \ln t \lceil (1 + \gamma) (\ln t) ^ {2} + 1 \rceil 2 e}{| \mathcal {I} | (| \Omega (V) | - 1)}\right) ^ {| \mathcal {I} | (| \Omega (V) | - 1)}, \\ \end{array}
$$
which remains finite. We conclude our proof.
Lemma 9 proposed in [9, Lemma 4.3] extends Hoeffding's inequality to provide an upper bound on the deviation of the empirical mean sampled at a stopping time. The time SCM-AAM enters the exploration phase is a stopping time, and the time each intervention is selected is also a stopping time. It will be used in multiple circumstances in regret analysis.
Lemma 9 (Extension of Hoeffding's Inequality [9]). Let  $\{Z_t\}_{t \in \mathbb{N}_{>0}}$  be a sequence of independent random variables with values in  $[0,1]$ . Let  $\mathcal{F}_t$  be the  $\sigma$ -algebra such that  $\sigma(Z_1, \ldots, Z_t) \subset \mathcal{F}_t$  and the filtration  $\mathcal{F} = \{\mathcal{F}_t\}_{t \in \mathbb{N}_{>0}}$ . Consider  $s \in \mathbb{N}$ , and  $T \in \mathbb{N}_{>0}$ . We define  $S_t = \sum_{j=1}^{t} \epsilon_j (Z_j - \mathbb{E}[Z_j])$ , where  $\epsilon_j \in \{0,1\}$  is a  $\mathcal{F}_{j-1}$ -measurable random variable. Further define  $N_t = \sum_{j=1}^{t} \epsilon_j$ . Define  $\phi \in \{1, \ldots, T+1\}$  a  $\mathcal{F}$ -stopping time such that either  $N_\phi \geq s$  or  $\phi = T+1$ . Then we have that
$$
P \left[ S _ {\phi} \geq N _ {\phi} \delta \right] \leq \exp \left(- 2 s \delta^ {2}\right).
$$
As a consequence,
$$
P [ | S _ {\phi} | \geq N _ {\phi} \delta ] \leq 2 \exp (- 2 s \delta^ {2}).
$$
In Corollary 10, we extend Lemma 9 to bound the  $L_{1}$  deviation of the empirical distribution.
Corollary 10 ( $L_{1}$  deviation of the empirical distribution). Let  $\mathcal{A}$  denote finite set  $\{1, \ldots, a\}$ . For two probability distribution  $Q$  and  $Q'$  on  $\mathcal{A}$ , let  $\|Q' - Q\|_{1} = \sum_{k=1}^{a} |Q'(k) - Q(k)|$ . Let  $X_{t} \in \mathcal{A}$  be a sequence of independent random variables with common distribution  $Q$ . Let  $\mathcal{F}_{t}$  be the  $\sigma$ -algebra such that  $\sigma(X_{1}, \ldots, X_{t}) \subset \mathcal{F}_{t}$  and the filtration  $\mathcal{F} = \{\mathcal{F}_{t}\}_{t \in \mathbb{N}_{>0}}$ . Let  $\epsilon_{t} \in \{0, 1\}$  is a  $\mathcal{F}_{t-1}$ -measurable random variable. We define
$$
N _ {t} = \sum_ {j = 1} ^ {t} \epsilon_ {j}, S _ {t} (i) = \sum_ {j = 1} ^ {t} \epsilon_ {j} \mathbb {1} \{X _ {j} = i \}, a n d \bar {Q} _ {t} (i) = \frac {S _ {t} (i)}{N _ {t}}, \forall i \in \mathcal {A}.
$$
For  $s \in \mathbb{N}$ , and  $T \in \mathbb{N}_{>0}$ , let  $\phi \in \{1, \dots, T + 1\}$  be a  $\mathcal{F}$ -stopping time such that either  $N_{\phi} \geq s$  or  $\phi = T + 1$ . Then we have
$$
P \left(\left\| \bar {Q} _ {\phi} - Q \right\| _ {1} \geq \delta\right) \leq (2 ^ {a} - 2) \exp \left(\frac {- s \delta^ {2}}{2}\right).
$$
Proof. It is known that for any distribution  $Q^{\prime}$  on  $\mathcal{A}$ ,
$$
\| Q ^ {\prime} - Q \| _ {1} = 2 \max  _ {A \subseteq A} (Q ^ {\prime} (A) - Q (A)).
$$
Then we apply a union bound to get
$$
\begin{array}{l} P \left(\left\| \bar {Q} _ {\phi} - Q \right\| _ {1} \geq \delta\right) \leq \sum_ {A \subseteq A} P \left(\bar {Q} _ {\phi} (A) - Q (A) \geq \frac {\delta}{2}\right) \\ \leq \sum_ {A \subseteq A: A \neq A \text {o r} \emptyset} P \left(\bar {Q} _ {\phi} (A) - Q (A) \geq \frac {\delta}{2}\right) \\ \leq \left(2 ^ {a} - 2\right) \exp \left(\frac {- s \delta^ {2}}{2}\right), \\ \end{array}
$$
B.3 Supporting Lemmas (Continuity of Upper Confidence Bound)
For a constrained optimization problem, Berge's maximum theorem imposes additional restrictions on the objective function and constraint set to guarantee that the problem's solution varies smoothly with parameters. Specifically, it studies the following optimization problem
$$
V (\theta) = \max  _ {x \in G (\theta)} F (x, \theta),
$$
where  $F: X \times \Theta \to \mathbb{R}$  the objective function, and  $G(\theta)$  is the set of feasible values for  $X$  given  $\theta$  defined by the multi-valued correspondence  $G: \Theta \Rightarrow X$ . We assume that  $G(\theta) \subseteq X$  is compact for all  $\theta \in \Theta$ .
Definition 4. The compact valued correspondence  $G$  is continuous at  $\theta \in \Omega(\theta)$  if it is both upper and lower hemicontinuous at  $\theta$ . The upper and lower hemicontinuous can be verified as below.
- Upper hemicontinuous: the correspondence  $G$  is upper hemicontinuous at  $\theta \in \Theta$  if  $G(\theta)$  is nonempty and if, for every sequence  $\{\theta_j\}$  with  $\theta_j \to \theta$  and every sequence  $\{x_j\}$  with  $x_j \in G(\theta_j)$  for all  $j$ , there exists a convergent subsequence  $\{x_{j_k}\}$  such that  $x_{j_k} \to x \in G(\theta)$ .
- Lower hemicontinuous: the correspondence  $G$  is lower hemicontinuous at  $\theta \in \Theta$  if  $G(\theta)$  is nonempty and if, for every  $x \in G(\theta)$  and every sequence  $\{\theta_j\}$  such that  $\theta_j \to \theta$ , there is a  $J \geq 1$  and a sequence  $\{x_j\}$  such that  $x_j \in G(\theta_j)$  for all  $j \geq J$  and  $x_j \to x$ .
The correspondence  $G$  is continuous if it is continuous at every  $\theta \in \Omega (\theta)$
Lemma 11 (Berge's Maximum Theorem [16, Ch. 17.31]). Let  $X \subseteq \mathbb{R}^n$  and  $\Theta \subseteq \mathbb{R}^m$ . Let  $F: X \times \Theta \to \mathbb{R}$  be a continuous function, and let  $G: \Theta \to X$  be a compact valued and continuous correspondence. Then the maximum value function
$$
V (\theta) = \max  _ {x \in G (\theta)} F (x, \theta)
$$
is well-defined and continuous, and the optimal correspondence
$$
x ^ {*} (\theta) = \{x \in G (\theta) \mid F (x, \theta) \}
$$
is nonempty, compact valued, and upper hemicontinuous.
We want to leverage Berge's maximum theorem to prove the UCB index  $\tilde{\mu}_{\mathbf{s}}(\mathcal{C}_{\mathbf{s}}(\eta_{\mathbf{s}},\mathcal{P})) = \sum_{\mathbf{v}\in \Omega (\mathbf{V})}v_n\max_{\mathcal{P}'\in \mathcal{C}_{\mathbf{s}}(\eta_{\mathbf{s}},\mathcal{P})}P_{\mathbf{s}}'(v)$  is continuous with respect to  $\eta_{\mathbf{s}}$  and  $\mathcal{P}$ . Recall in Theorem 2, for given interventional distribution tuple  $\mathcal{P}\in \mathcal{M}_{\mathcal{G}}$  and  $\eta_{\mathrm{s}} > 0$ ,  $\mathcal{C}_{\mathbf{s}}(\eta_{\mathbf{s}},\mathcal{P})$  is the set of feasible interventional distribution tuples  $\mathcal{P}'$ , and it is defined as
$$
\mathcal {C} _ {\mathbf {s}} (\eta_ {\mathbf {s}}, \mathcal {P}) = \left\{\mathcal {P} ^ {\prime} \in \mathcal {M} _ {\mathcal {G}} \Big | \eta_ {\mathbf {s}} D (P _ {\mathbf {s}} \| P _ {\mathbf {s}} ^ {\prime}) + \epsilon \eta_ {\mathbf {s}} / 2 \sum_ {\mathbf {x} \in \mathcal {I} _ {- \mathbf {s}}} D (P _ {\mathbf {x}} \| P _ {\mathbf {x}} ^ {\prime}) \leq 1 \right\}.
$$
Lemma 12. For any  $\mathbf{s} \in \mathcal{I}$ , the correspondence  $\mathcal{C}_{\mathbf{s}}: \mathbb{R}_{\geq 0} \times \mathcal{M}_{\mathcal{G}} \Rightarrow \mathcal{M}_{\mathcal{G}}$  is a compacted valued continuous correspondence.
Proof. The space of interventional distribution tuples  $\mathcal{M}_{\mathcal{G}}$  is compact as has been explained below Definition 2. Besides, for any  $\mathcal{P} \in \mathcal{M}_{\mathcal{G}}$  and  $\eta_{\mathbf{s}} \geq 0$ , the constrain  $\eta_{\mathbf{s}}D(P_{\mathbf{s}} \| P_{\mathbf{s}}^{\prime}) + \epsilon \eta_{\mathbf{s}} / 2 \sum_{\mathbf{x} \in \mathcal{I}_{-s}} D(P_{\mathbf{x}} \| P_{\mathbf{x}}^{\prime}) \leq 1$  defines a closed compact set for  $\mathcal{P}^{\prime} = \langle P_{\mathbf{s}}^{\prime} \rangle_{\mathbf{s} \in \mathcal{I}}$ . Since the intersection of two compact sets is compact if one of them is also closed,  $\mathcal{C}_{\mathbf{s}}(\eta_{\mathbf{s}}, \mathcal{P})$  is compact. It remains to show  $\mathcal{C}_{\mathbf{s}}$  is both upper hemicontinuous and lower hemicontinuous.
Upper hemicontinuous: Fix arbitrary  $\eta_{\mathbf{s}} \geq 0$  and  $\mathcal{P} \in \mathcal{M}_{\mathcal{G}}$ , and let  $\{(\eta_{\mathbf{s},i}, \mathcal{P}_i)\}_{i \in \mathbb{N}_{>0}}$  be a sequence such that  $\eta_{\mathbf{s},i} \geq 0$ ,  $\mathcal{P}_i \in \mathcal{M}_{\mathcal{G}}$ , and  $(\eta_{\mathbf{s},i}, \mathcal{P}_i) \to (\eta_{\mathbf{s}}, \mathcal{P})$ . Notice that each  $\mathcal{C}_{\mathbf{s}}(\eta_{\mathbf{s},i}, \mathcal{P}_i)$  is non-empty since  $\mathcal{P}_i \in \mathcal{C}_{\mathbf{s}}(\eta_{\mathbf{s},i}, \mathcal{P}_i)$ . Therefore, there is a sequence  $\{\mathcal{P}_i'\}_{i \in \mathbb{N}_{>0}}$  such that  $\mathcal{P}_i' \in \mathcal{C}_{\mathbf{s}}(\eta_{\mathbf{s},i}, \mathcal{P}_i)$ . Since  $\eta_{\mathbf{s},i} \to \eta_{\mathbf{s}}$ , there exists a closed and bounded set  $\Theta \subset \mathbb{R}_{\geq 0}$  such that  $\eta_{\mathbf{s}} \in \Theta$ , and for some  $N \geq 1$ ,  $\eta_{\mathbf{s},i} \in \Theta$  for all  $i \geq N$ . Moreover,  $\mathcal{P}_i$  and  $\mathcal{P}_i'$  are also bounded since every interventional probability tuple in  $\mathcal{M}_{\mathcal{G}}$  is bounded. Since each  $(\eta_{\mathbf{s},i}, \mathcal{P}_i, \mathcal{P}_i')$  lies in a closed and bounded subset for  $i \geq N$  by Bolzano-Weierstrass theorem, the sequence  $\{(\eta_{\mathbf{s},i}, \mathcal{P}_i, \mathcal{P}_i'),\}$  is in  $\mathbb{N}_{>0}$  has a convergent subsequence  $\{(\eta_{\mathbf{s},i_k}, \mathcal{P}_{i_k}, \mathcal{P}_{i_k}')\}_{k \in \mathbb{N}_{>0}}$  with limit point  $(\eta_{\mathbf{s}}, \mathcal{P}, \mathcal{P}')$ . Since each element of this sequence satisfies
$$
h (\eta_ {\mathbf {s}, i _ {k}}, \mathcal {P} _ {i _ {k}}, \mathcal {P} _ {i _ {k}} ^ {\prime}) := \eta_ {\mathbf {s}, i _ {k}} D (P _ {\mathbf {s}, i _ {k}} \parallel P _ {\mathbf {s}, i _ {k}} ^ {\prime}) + \epsilon \eta_ {\mathbf {s}, i _ {k}} / 2 \sum_ {\mathbf {x} \in \mathcal {I} _ {- \mathbf {s}}} D (P _ {\mathbf {x}, i _ {k}} \parallel P _ {\mathbf {x}, i _ {k}} ^ {\prime}) \leq 1,
$$
the inequality holds at the limit point  $(\eta_{\mathbf{s}},\mathcal{P}^{\prime})$  , i.e.,
$$
h (\eta_ {\mathbf {s}}, \mathcal {P}, \mathcal {P} ^ {\prime}) = \eta_ {\mathbf {s}} D (P _ {\mathbf {s}} \| P _ {\mathbf {s}} ^ {\prime}) + \epsilon \eta_ {\mathbf {s}} / 2 \sum_ {\mathbf {x} \in \mathcal {I} _ {- \mathbf {s}}} D (P _ {\mathbf {x}} \| P _ {\mathbf {x}} ^ {\prime}) \leq 1,
$$
due to that  $h$  is a continuous function. We get  $\mathcal{C}_{\mathbf{s}}$  is upper hemicontinuous.
Lower hemicontinuous: For arbitrary  $\eta_{\mathbf{s}} \geq 0$  and  $\mathcal{P} \in \mathcal{M}_{\mathcal{G}}$ , we fix  $\mathcal{P}' \in \mathcal{C}_{\mathbf{s}}(\eta_{\mathbf{s}}, \mathcal{P})$ . Let  $\{(\eta_{\mathbf{s},i}, \mathcal{P}_i)\}_{i \in \mathbb{N}_{>0}}$  be a sequence such that  $\eta_{\mathbf{s},i} \geq 0$ ,  $\mathcal{P}_i = \langle P_{\mathbf{s},i} \rangle_{\mathbf{s} \in \mathcal{I}} \in \mathcal{M}_{\mathcal{G}}$ , and  $(\eta_{\mathbf{s},i}, \mathcal{P}_i) \to (\eta_{\mathbf{s}}, \mathcal{P})$ . We show there exists a sequence  $\{\mathcal{P}_i'\}_{i \in \mathbb{N}_{>0}}$  such that  $\mathcal{P}_i' \in \mathcal{C}_{\mathbf{s}}(\eta_{\mathbf{s},i}, \mathcal{P}_i)$  for all  $i$ , and  $\mathcal{P}_i' \to \mathcal{P}'$ .
Toward this end, for each  $(\eta_{\mathbf{s},i},\mathcal{P}_i)$ , if  $h(\eta_{\mathbf{s},i},\mathcal{P}_i,\mathcal{P}') \leq 1$ , we set  $\mathcal{P}_i' = \mathcal{P}'$  so that  $\mathcal{P}_i' \in \mathcal{C}_{\mathbf{s}}(\eta_{\mathbf{s},i},\mathcal{P}_i)$ . If  $c_i := h(\eta_{\mathbf{s},i},\mathcal{P}_i,\mathcal{P}') > 1$ , we construct  $\mathcal{P}_i' = \langle P_{\mathbf{s},i}'\rangle_{\mathbf{s} \in \mathcal{I}}$  as the following. With Definition 2, there exist  $\{\mathbf{p}_1^i,\dots,\mathbf{p}_{n_c(\mathcal{G})}^i\}$  and  $\{\mathbf{p}_1',\dots,\mathbf{p}_{n_c(\mathcal{G})}'\}$  associated with the interventional distribution tuple  $\mathcal{P}_i$  and  $\mathcal{P}'$  respectively. Let  $\mathcal{P}_i'$  be associated with
$$
\left\{\frac {c _ {i} - 1}{c _ {i}} \mathbf {p} _ {1} ^ {i} + \frac {1}{c _ {i}} \mathbf {p} _ {1} ^ {\prime}, \dots , \frac {c _ {i} - 1}{c _ {i}} \mathbf {p} _ {n _ {\mathrm {c}} (\mathcal {G})} ^ {i} + \frac {1}{c _ {i}} \mathbf {p} _ {n _ {\mathrm {c}} (\mathcal {G})} ^ {\prime} \right\}, \tag {16}
$$
so that  $\mathcal{P}_i^\prime \in \mathcal{M}_{\mathcal{G}}$ . For each  $\mathbf{s}\in \mathcal{I}$  and  $\mathbf{v}\in \Omega (\mathbf{V})$ , we have
$$
\begin{array}{l} P _ {\mathbf {s}, i} (\mathbf {v}) \ln \frac {P _ {\mathbf {s} , i} (\mathbf {v})}{P _ {\mathbf {s} , i} ^ {\prime} (\mathbf {v})} \\ = P _ {\mathbf {s}, i} (\mathbf {v}) \ln \frac {\prod_ {j = 1} ^ {n _ {\mathrm {c}} (\mathcal {G})} b _ {\mathcal {G} , \mathbf {s} [ \mathbf {V} ^ {j} ]} ^ {\top} (\mathbf {v} ^ {j} , \mathsf {p a} _ {\mathbf {V} ^ {j}}) \mathbf {p} _ {j} ^ {i}}{\prod_ {j = 1} ^ {n _ {\mathrm {c}} (\mathcal {G})} b _ {\mathcal {G} , \mathbf {s} [ \mathbf {V} ^ {j} ]} ^ {\top} (\mathbf {v} ^ {j} , \mathsf {p a} _ {\mathbf {V} ^ {j}}) \left[ \frac {c _ {i} - 1}{c _ {i}} \mathbf {p} _ {j} ^ {i} + \frac {1}{c _ {i}} \mathbf {p} _ {j} ^ {\prime} \right]} \\ = \prod_ {j = 1} ^ {n _ {\mathrm {c}} (\mathcal {G})} b _ {\mathcal {G}, \mathbf {s} [ \mathbf {V} ^ {j} ]} ^ {\top} (\mathbf {v} ^ {j}, \mathsf {p a} _ {\mathbf {V} ^ {j}}) \mathbf {p} _ {j} ^ {i} \sum_ {k = 1} ^ {n _ {\mathrm {c}} (\mathcal {G})} \ln \frac {b _ {\mathcal {G} , \mathbf {s} [ \mathbf {V} ^ {k} ]} ^ {\top} (\mathbf {v} ^ {k} , \mathsf {p a} _ {\mathbf {V} ^ {k}}) \mathbf {p} _ {k} ^ {i}}{b _ {\mathcal {G} , \mathbf {s} [ \mathbf {V} ^ {k} ]} ^ {\top} (\mathbf {v} ^ {k} , \mathsf {p a} _ {\mathbf {V} ^ {k}}) \left[ \frac {c _ {1} - 1}{c _ {i}} \mathbf {p} _ {k} ^ {i} + \frac {1}{c _ {i}} \mathbf {p} _ {k} ^ {\prime} \right]} \\ = \sum_ {k = 1} ^ {n _ {\mathrm {c}} (\mathcal {G})} \prod_ {j \neq k} b _ {\mathcal {G}, \mathbf {s} [ \mathbf {V} ^ {j} ]} ^ {\top} (\mathbf {v} ^ {j}, \mathsf {p a} _ {\mathbf {V} ^ {j}}) \mathbf {p} _ {j} ^ {i} \underbrace {b _ {\mathcal {G} , \mathbf {s} [ \mathbf {V} ^ {k} ]} ^ {\top} (\mathbf {v} ^ {k} , \mathsf {p a} _ {\mathbf {V} ^ {k}}) \mathbf {p} _ {k} ^ {i} \ln \frac {b _ {\mathcal {G} , \mathbf {s} [ \mathbf {V} ^ {k} ]} ^ {\top} (\mathbf {v} ^ {k} , \mathsf {p a} _ {\mathbf {V} ^ {k}}) \mathbf {p} _ {k} ^ {i}}{b _ {\mathcal {G} , \mathbf {s} [ \mathbf {V} ^ {k} ]} ^ {\top} (\mathbf {v} ^ {k} , \mathsf {p a} _ {\mathbf {V} ^ {k}}) \left[ \frac {c _ {i} - 1}{c _ {i}} \mathbf {p} _ {k} ^ {i} + \frac {1}{c _ {i}} \mathbf {p} _ {k} ^ {\prime} \right]}} _ {(a)} \\ \leq \frac {1}{c _ {i}} \sum_ {k = 1} ^ {n _ {\mathrm {c}} (\mathcal {G})} \prod_ {j = 1} ^ {n _ {\mathrm {c}} (\mathcal {G})} b _ {\mathcal {G}, \mathbf {s} [ \mathbf {V} ^ {j} ]} ^ {\top} (\mathbf {v} ^ {j}, \mathsf {p a} _ {\mathbf {V} ^ {j}}) \mathbf {p} _ {j} ^ {i} \ln \frac {b _ {\mathcal {G} , \mathbf {s} [ \mathbf {V} ^ {k} ]} ^ {\top} (\mathbf {v} ^ {k} , \mathsf {p a} _ {\mathbf {V} ^ {k}}) \mathbf {p} _ {k} ^ {i}}{b _ {\mathcal {G} , \mathbf {s} [ \mathbf {V} ^ {k} ]} ^ {\top} (\mathbf {v} ^ {k} , \mathsf {p a} _ {\mathbf {V} ^ {k}}) \mathbf {p} _ {k} ^ {\prime}} = \frac {1}{c _ {i}} P _ {\mathbf {s}} (\mathbf {v}) \ln \frac {P _ {\mathbf {s}} (\mathbf {v})}{P _ {\mathbf {s}} ^ {\prime} (\mathbf {v})}, \\ \end{array}
$$
where we apply the log sum inequality to  $(a)$ . Thus, for any  $\mathbf{s} \in \mathcal{I}$ , we have
$$
D (P _ {\mathbf {s}, i} \parallel P _ {\mathbf {s}, i} ^ {\prime}) = \sum_ {\mathbf {v} \in \Omega (\mathbf {V})} P _ {\mathbf {s}} (\mathbf {v}) \ln \frac {P _ {\mathbf {s}} (\mathbf {v})}{P _ {\mathbf {s} , i} ^ {\prime} (\mathbf {v})} \leq \frac {1}{c _ {i}} D (P _ {\mathbf {s}} \parallel P _ {\mathbf {s}} ^ {\prime}).
$$
As a result,  $h(\eta_{\mathbf{s},i},\mathcal{P}_i,\mathcal{P}_i')\leq \frac{1}{c_f} h(\eta_{\mathbf{s},i},\mathcal{P}_i,\mathcal{P}') = 1$  , which means  $\mathcal{P}_i^\prime \in \mathcal{C}_{\mathbf{s}}(\eta_{\mathbf{s},i},\mathcal{P}_i)$
It remains to show  $\mathcal{P}_i' \to \mathcal{P}'$  by construction. If  $h(\eta_{\mathbf{s}}, \mathcal{P}, \mathcal{P}') < 1$ , in the sequence  $\{(\eta_{\mathbf{s},i}, \mathcal{P}_i)\}_{i \in \mathbb{N}_{>0}}$ , there exist finite instances such that  $h(\eta_{\mathbf{s},i}, \mathcal{P}_i, \mathcal{P}') > 1$ . Otherwise,  $h(\eta_{\mathbf{s}}, \mathcal{P}, \mathcal{P}') = 1$  indicates  $c_i \to 1$ , resulting in (16)  $\rightarrow \{\mathbf{p}_1', \dots, \mathbf{p}_{n_c}^{\prime}(\mathcal{G})\}$ . Therefore,  $\mathcal{P}_i' \to \mathcal{P}'$  and  $\mathcal{C}_{\mathbf{s}}$  is lower hemicontinuous.
With correspondence  $\mathcal{C}_{\mathbf{s}}$  being continuous, Lemma 13 gives a continuous property of  $\tilde{\mu}_{\mathbf{s}}(\mathcal{C}_{\mathbf{s}}(\eta_{\mathbf{s}},\mathcal{P}))$
Lemma 13. For any  $\mathcal{P} \in \mathcal{M}_{\mathcal{G}}$  and for any any  $\mathbf{s} \in \mathcal{I}$ , the UCB index  $\tilde{\mu}_{\mathbf{s}}(\mathcal{C}_{\mathbf{s}}(\eta_{\mathbf{s}}, \mathcal{P}))$  is a continuous monotonically nonincreasing function of  $\eta_{\mathbf{s}} \geq 0$ .
Proof. Since  $\mathcal{C}_{\mathbf{s}}$  is a compacted valued continuous correspondence, we apply Berge's maximum theorem in Lemma 11 to get that  $\max_{\mathcal{P}^{\prime}\in \mathcal{C}_{\mathbf{s}}(\eta_{\mathbf{s}},\mathcal{P})}P_{\mathbf{s}}^{\prime}(\mathbf{v})$  is continuous with  $\eta_{\mathbf{s}}$ . So  $\tilde{\mu}_{\mathbf{s}}(\mathcal{C}_{\mathbf{s}}(\eta_{\mathbf{s}},\mathcal{P})) = \sum_{\mathbf{v}\in \Omega (\mathbf{V})}v_n\max_{\mathcal{P}^{\prime}\in \mathcal{C}_{\mathbf{s}}(\eta_{\mathbf{s}},\mathcal{P})}P_{\mathbf{s}}^{\prime}(\mathbf{v})$  is also continuous with  $\eta_{\mathbf{s}}$ . For any  $a > b > 0$ , since  $\mathcal{C}_{\mathbf{s}}(a,\mathcal{P})\subseteq \mathcal{C}_{\mathbf{s}}(b,\mathcal{P}), \tilde{\mu}_{\mathbf{s}}(\mathcal{C}_{\mathbf{s}}(a,\mathcal{P}))\leq \tilde{\mu}_{\mathbf{s}}(\mathcal{C}_{\mathbf{s}}(b,\mathcal{P}))$ . Therefore,  $\tilde{\mu}_{\mathbf{s}}(\mathcal{C}_{\mathbf{s}}(\eta_{\mathbf{s}},\mathcal{P}))$  monotonically nonincreasing with respect to  $\eta_{\mathbf{s}}$ .
The SCM-AAM utilizes the empirical interventional distributions to compute UCB and mean reward in exploration steps. Lemma 14 indicates that only a small ratio of exploration  $\kappa$  is wasted if the gap between empirical distribution and the ground truth is as small as  $\delta(\kappa)$ . This fact is the cornerstone of regret analysis since the ground truth is unavailable.
Lemma 14. Given a causal graph  $\mathcal{G} = \langle \mathbf{V},\mathcal{E},\mathcal{B}\rangle$ , for any interventional distribution tuple  $\mathcal{P}' = \langle P_{\mathbf{s}}' \rangle_{\mathbf{s} \in \mathcal{I}} \in \mathcal{M}_{\mathcal{G}}$ , let  $\mu_{\mathbf{s}}(\mathcal{P}')$  be the mean reward of each intervention  $\mathbf{s} \in \mathcal{I}$  according to  $\mathcal{P}'$ , and define  $\mu_{*}(\mathcal{P}') = \max_{\mathbf{s} \in \mathcal{I}} \mu_{\mathbf{s}}(\mathcal{P}')$ . Let  $\mathcal{P} = \langle P_{\mathbf{s}} \rangle_{\mathbf{s} \in \mathcal{I}} \in \mathcal{M}_{\mathcal{G}}$  be the true interventional distribution tuple with a unique optimal intervention  $\mathbf{s}_{*} \in \mathcal{I}$ . For each  $\mathbf{s} \in \mathcal{I}_{-\mathbf{s}_{*}}$ , let
$$
\zeta_ {\mathbf {s}} (\mathcal {P}) = \max  _ {\eta_ {\mathbf {s}} \geq 0} \eta_ {\mathbf {s}} \text {s . t .} \tilde {\mu} _ {\mathbf {s}} \left(\mathcal {C} _ {\mathbf {s}} \left(\eta_ {\mathbf {s}}, \mathcal {P}\right)\right) \geq \mu_ {*} (\mathcal {P}), \tag {17}
$$
in which  $\mathcal{C}_{\mathbf{s}}(\eta_{\mathbf{s}},\mathcal{P})\coloneqq \left\{\mathcal{P}^{\prime}\in \mathcal{M}_{\mathcal{G}}\mid \eta_{\mathbf{s}}D(P_{\mathbf{s}}\parallel P_{\mathbf{s}}^{\prime}) + \epsilon \eta_{\mathbf{s}} / 2\sum_{\mathbf{x}\in \mathcal{I}_{-s}}D(P_{\mathbf{x}}\parallel P_{\mathbf{x}}^{\prime})\leq 1\right\}$ . For any  $\kappa >0$  there exists  $\delta (\kappa)$  such that for any  $\bar{\mathcal{P}} = \langle \bar{P}_{\mathbf{s}}\rangle_{\mathbf{s}\in \mathcal{I}}\in \mathcal{M}_{\mathcal{G}}$  if  $\forall :\mathbf{s}\in \mathcal{I}:\left\| \bar{P}_{\mathbf{s}} - P_{\mathbf{s}}\right\| _1\leq \delta (\kappa)$ ,
- Intervention  $\mathbf{s}_{*}$  is also optimal according to  $\bar{\mathcal{P}}$ , i.e.,  $\mu_{\mathbf{s}_{*}}(\bar{\mathcal{P}}) \geq \mu_{\mathbf{s}}(\bar{\mathcal{P}})$ ,  $\forall \mathbf{s} \in \mathcal{I}_{-\mathbf{s}_{*}}$ .
- For any  $\mathbf{s} \in \mathcal{I}_{-\mathbf{s}_*}$ ,  $\zeta_{\mathbf{s}}(\bar{\mathcal{P}}) \leq (1 + \kappa)\zeta_{\mathbf{s}}(\mathcal{P})$ .
Proof. For each  $\mathbf{s} \in \mathcal{I}$ , it can be seen that  $\lim_{\eta_{\mathbf{s}} \to \infty} \tilde{\mu}_{\mathbf{s}}(\mathcal{C}_{\mathbf{s}}(\eta_{\mathbf{s}}, \mathcal{P})) = \mu_{\mathbf{s}}(\mathcal{P})$  since  $\mathcal{P}'$  in  $\mathcal{C}_{\mathbf{s}}(\eta_{\mathbf{s}}, \mathcal{P})$  is required to be identical to  $\mathcal{P}$  as  $\eta_{\mathbf{s} \to \infty}$ . For each  $\mathbf{s} \in \mathcal{I}_{-\mathbf{s}_*}$ , if  $\tilde{\mu}_{\mathbf{s}}(\mathcal{C}_{\mathbf{s}}(0, \mathcal{P})) > \mu_*(\mathcal{P})$ , since  $\tilde{\mu}_{\mathbf{s}}(\mathcal{C}_{\mathbf{s}}(\eta_{\mathbf{s}}, \mathcal{P}))$  is a continuous monotonically nonincreasing function of  $\eta_{\mathbf{s}}$  according to Lemma 13, there exists  $\eta_{\mathbf{s}} > 0$  such that  $\tilde{\mu}_{\mathbf{s}}(\mathcal{C}_{\mathbf{s}}(\eta_{\mathbf{s}}, \mathcal{P})) = \mu_*(\mathcal{P})$ . Setting  $\eta_{\mathbf{s}}^* = \max \{\eta_{\mathbf{s}} \geq 0 \mid \tilde{\mu}_{\mathbf{s}}(\mathcal{C}_{\mathbf{s}}(\eta_{\mathbf{s}}, \mathcal{P})) = \mu_*(\mathcal{P})\}$ , we have  $\eta_{\mathbf{s}}^* > 0$  and  $\zeta_{\mathbf{s}}(\mathcal{P}) = \eta_{\mathbf{s}}^*$ . For any  $\kappa > 0$ ,  $\tilde{\mu}_{\mathbf{s}}(\mathcal{C}_{\mathbf{s}}(\sqrt{1 + \kappa} \eta_{\mathbf{s}}^*, \mathcal{P})) < \mu_*(\mathcal{P})$  due to the definition of  $\eta_{\mathbf{s}}^*$ . We define