SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\frac {\mathbb {E} \left[ Y _ {\tilde {X}} - Y _ {X ^ {(t)}} \mid \bar {\boldsymbol {V}} _ {t} , \bar {\boldsymbol {v}} \right]}{\sqrt {I _ {P (\cdot | \bar {\boldsymbol {V}} _ {t} , \bar {\boldsymbol {v}})} (\tilde {X} ; \boldsymbol {V} _ {X ^ {(t)}})}} \leqslant \Gamma_ {t}.
$$
By writing  $\Gamma \geqslant \Gamma_t, \forall t$ , applying this bound yields
$$
\begin{array}{l} \mathbb {E} \left[ R _ {T} \mid \bar {\boldsymbol {v}} \right] = \mathbb {E} \sum_ {t = 1} ^ {T} \sqrt {I _ {P (\cdot | \bar {\boldsymbol {V}} _ {t} , \bar {\boldsymbol {v}})} (\tilde {X} ; \boldsymbol {V} _ {X ^ {(t)}})} \frac {\mathbb {E} \left[ Y _ {\tilde {X}} - Y _ {X ^ {(t)}} \mid \bar {\boldsymbol {V}} _ {t} , \bar {\boldsymbol {v}} \right]}{\sqrt {I _ {P (\cdot | \bar {\boldsymbol {V}} _ {t} , \bar {\boldsymbol {v}})} (\tilde {X} ; \boldsymbol {V} _ {X ^ {(t)}})}} \\ \leqslant \mathbb {E} \sum_ {t = 1} ^ {T} \Gamma_ {t} \sqrt {I _ {P (\cdot | \bar {\boldsymbol {V}} _ {t} , \bar {\boldsymbol {v}})} (\tilde {X} ; \boldsymbol {V} _ {X ^ {(t)}})} \\ \leqslant \Gamma \mathbb {E} \sum_ {t = 1} ^ {T} \sqrt {I _ {P (\cdot | \bar {\boldsymbol {V}} _ {t} , \bar {\boldsymbol {v}})} (\tilde {X} ; \boldsymbol {V} _ {X ^ {(t)}})} \\ \leqslant \Gamma \sqrt {T \mathbb {E} \sum_ {t = 1} ^ {T} I _ {P (\cdot | \tilde {\boldsymbol {V}} _ {t} , \tilde {\boldsymbol {v}})} (\tilde {\boldsymbol {X}} ; \boldsymbol {V} _ {X ^ {(t)}})} \\ \leqslant \Gamma \sqrt {T \mathcal {H} (\tilde {X} \mid \bar {\boldsymbol {v}})}, \\ \end{array}
$$
where the second to last inequality follows from the Cauchy-Schwartz inequality and the last inequality follows from the fact that,
$$
\begin{array}{l} \mathbb {E} \sum_ {t = 1} ^ {T} I _ {P (\cdot | \bar {\boldsymbol {V}} _ {t}, \bar {\boldsymbol {v}})} (\tilde {X}; \boldsymbol {V} _ {X ^ {(t)}}) = \sum_ {t = 1} ^ {T} I _ {P (\cdot | \bar {\boldsymbol {v}})} (\tilde {X}; \boldsymbol {V} _ {X ^ {(t)}} \mid \bar {\boldsymbol {V}} _ {t}) (18) \\ = I _ {P (\cdot | \vec {\mathbf {v}})} \left(\tilde {X}; \left(\boldsymbol {V} _ {X ^ {(T)}}, \dots , \boldsymbol {V} _ {X ^ {(1)}}\right)\right) (19) \\ = \mathcal {H} (\tilde {X} \mid \bar {\boldsymbol {v}}) - \mathcal {H} (\tilde {X} \mid \boldsymbol {V} _ {X ^ {(T)}}, \dots , \boldsymbol {V} _ {X ^ {(1)}}, \bar {\boldsymbol {v}}) (20) \\ \leqslant \mathcal {H} (\tilde {X} \mid \bar {\boldsymbol {v}}), (21) \\ \end{array}
$$
by the chain rule for the mutual information and given the non-negativity of the entropy.
Proposition 2 (restated). Let  $R_T$  denote the regret incurred following the policy defined by Alg. 1. For any  $T \in \mathbb{N}$ , if Eq. (7) holds with  $\Gamma \geqslant \Gamma_t$  for all  $t$ ,
$$
\mathbb {E} \left[ R _ {T} \mid \bar {\boldsymbol {v}} \right] \leqslant \Gamma \sqrt {T I _ {P (\cdot | \bar {\boldsymbol {v}})} \left(\boldsymbol {\theta} , \boldsymbol {\xi} ; \boldsymbol {V} _ {X ^ {(1)}}, \dots , \boldsymbol {V} _ {X ^ {(T)}}\right)} + \sum_ {t = 1} ^ {T} \mathbb {E} \left[ \epsilon_ {t} \right].
$$
Proof. This proof uses a different characterization of the per round regret to explicitly consider the influence of model parameters  $(\theta, \xi)$  on information gain. Assume instead that there exists  $\Gamma_t$  such that
$$
\mathbb {E} \left[ Y _ {\bar {X}} - Y _ {X ^ {(t)}} \mid \bar {V} _ {t}, \bar {v} \right] \leqslant \Gamma_ {t} \sqrt {I _ {P (\cdot | \bar {V} _ {t} , \bar {v})} \left(\boldsymbol {\theta} , \boldsymbol {\xi} ; \boldsymbol {V} _ {X ^ {(t)}}\right)} + \epsilon_ {t}, \tag {22}
$$
where  $\epsilon_t > 0$  is an additional slack term. The proof strategy follows that of [25, Prop. 2] where unconditional regret bounds were shown. Given Eq. (22) the following derivation holds,
$= \Gamma \sqrt{TI_{P(\cdot|\vec{\pmb{v}})}(\pmb{\theta},\pmb{\xi};\pmb{V}_{X^{(1)}},\dots,\pmb{V}_{X^{(T)}})} + \mathbb{E}\left[\sum_t\epsilon_t\right]$  by the chain rule for mutual information.
$$
\begin{array}{l} \mathbb {E} \left[ R _ {T} \mid \bar {\boldsymbol {v}} \right] = \mathbb {E} \left[ \sum_ {t = 1} ^ {T} Y _ {\bar {X}} - Y _ {X ^ {(t)}} \mid \bar {\boldsymbol {v}} \right] \\ = \mathbb {E} \left[ \sum_ {t = 1} ^ {T} \mathbb {E} \left[ Y _ {\bar {X}} - Y _ {X ^ {(t)}} \mid \bar {\boldsymbol {v}}, \bar {V} _ {t} \right] \right] \\ \leqslant \mathbb {E} \left[ \sum_ {t = 1} ^ {T} \Gamma_ {t} \sqrt {I _ {P (\cdot | \bar {\boldsymbol {V}} _ {t} , \bar {\boldsymbol {v}})} \left(\boldsymbol {\theta} , \boldsymbol {\xi} ; \boldsymbol {V} _ {X ^ {(t)}}\right)} + \epsilon_ {t} \right] \\ \leqslant \Gamma \sqrt {T \sum_ {t = 1} ^ {T} \mathbb {E} \left[ I _ {P (\cdot | \bar {V} _ {t} , \bar {v})} \left(\boldsymbol {\theta} , \boldsymbol {\xi}; \boldsymbol {V} _ {X ^ {(t)}}\right) \right]} + \mathbb {E} \left[ \sum_ {t} \epsilon_ {t} \right] \\ = \Gamma \sqrt {T \sum_ {t = 1} ^ {T} I _ {P (\cdot | \bar {\boldsymbol {v}})} \left(\boldsymbol {\theta} , \boldsymbol {\xi} ; \mathbf {V} _ {X ^ {(t)}} \mid \bar {\mathbf {V}} _ {t}\right)} + \mathbb {E} \left[ \sum_ {t} \epsilon_ {t} \right] \\ \end{array}
$$
by definition
by the law of iterated expectations
by definition of  $\Gamma_t$  and  $\epsilon_t$
by Cauchy-Schwarz's inequality
by definition of the expectation of  $I_P$
Proposition 3 (restated). Fix  $\delta > 0$  and choose  $\Gamma_t$  such that  $|Y_x - \mathbb{E}[Y_x|\bar{V}_t,\bar{v} ]|\leqslant \frac{\Gamma_t}{2}\sqrt{I_{P(\cdot|\bar{V}_t,\bar{v})}(\pmb{\theta},\pmb{\xi};Y_x)}$  for all  $x\in \Omega_X$  simultaneously with probability greater than  $1 - \delta$ . Then Alg. 1 chooses actions  $X^{(t)}$  that satisfy
$$
\mathbb {E} \left[ Y _ {\tilde {X}} - Y _ {X ^ {(t)}} \mid \bar {\boldsymbol {V}} _ {t}, \bar {\boldsymbol {v}} \right] \leqslant \Gamma_ {t} \sqrt {I _ {P (\cdot | \bar {V} _ {t} , \bar {\boldsymbol {v}})} \left(\boldsymbol {\theta} , \boldsymbol {\xi} ; \boldsymbol {V} _ {X ^ {(t)}}\right)} + \delta B,
$$
Proof. This proposition extends [25, Lem. 3] to account for the case in which prior data is conditioned upon. Thompson sampling, by definition, samples an action according to its probability of being optimal with the current distribution of parameter values,
$$
\mathbb {E} \left[ Y _ {\tilde {X}} - Y _ {X ^ {(t)}} \mid \bar {\boldsymbol {V}} _ {t}, \bar {\boldsymbol {v}} \right] = \mathbb {E} \left[ \mathbb {E} _ {P _ {\hat {M}}} \left[ Y _ {X ^ {(t)}} \mid \bar {\boldsymbol {V}} _ {t}, \bar {\boldsymbol {v}} \right] - \mathbb {E} _ {P _ {\pi^ {*}}} \left[ Y _ {X ^ {(t)}} \mid \bar {\boldsymbol {V}} _ {t}, \bar {\boldsymbol {v}} \right] \right],
$$
where  $\hat{M} := M(\hat{\pmb{\xi}},\hat{\pmb{\theta}})$  defines the SCM that is obtained with the current sample  $\hat{\pmb{\xi}},\hat{\pmb{\theta}}\sim P(\pmb {\xi},\pmb {\theta}\mid \bar{\pmb{v}}_t,\bar{\pmb{v}})$  from the posterior distribution at round  $t$ , while  $M_{\pi^*}$  refers to the true underlying SCM of the environment. Expectations with respect to the distributions  $P_{\hat{M}}$  and  $P_{\pi^*}$  are conditioned on a specific model of the environment. Define,
$$
\mathcal {E} := \left\{(\pmb {\xi}, \pmb {\theta}) \in \Omega_ {\pmb {\xi}} \times \Omega_ {\pmb {\theta}}: | Y _ {x, M (\pmb {\xi}, \pmb {\theta})} - \mathbb {E} _ {M \sim P _ {\mathcal {M} (\mathcal {G} _ {\pi^ {*}})}} [ Y _ {x, M} ] | \leqslant \Gamma_ {t} \sqrt {I _ {P (\cdot | \bar {\mathbf {V}} _ {t} , \bar {\mathbf {v}})} (\pmb {\theta} , \pmb {\xi} ; Y _ {x})}, \forall x \in \Omega_ {X} \right\},
$$
where  $Y_{x,M(\pmb {\xi},\pmb {\theta})}$  refers to the random variable  $Y$  in  $\operatorname {SCM}M(\pmb {\xi},\pmb {\theta})$  in which we intervene to set  $X$  to  $x$ , and  $Y_{x,M}$  refers to the random variable  $Y$  in  $\operatorname {SCM}M$  in which we set  $X$  to  $x$ .
The probability matching property of Thompson sampling implies that  $P(\hat{\xi}, \hat{\theta} \in \mathcal{E} \mid \bar{v}_t, \bar{v}) \geqslant 1 - \delta / 2$ . It follows then, using the same proof strategy as [25, Lemma 3], that
$$
\begin{array}{l} \mathbb {E} \left[ Y _ {\tilde {X}} - Y _ {X ^ {(t)}} \mid \bar {\boldsymbol {V}} _ {t}, \bar {\boldsymbol {v}} \right] \leqslant \mathbb {E} _ {P (\cdot | \bar {\boldsymbol {V}} _ {t}, \bar {\boldsymbol {v}})} \left[ \mathbb {1} \left\{\left(\hat {\boldsymbol {\xi}}, \hat {\boldsymbol {\theta}}\right), (\boldsymbol {\xi}, \boldsymbol {\theta}) \in \mathcal {E} \right\} \left(Y _ {X ^ {(t)}, \hat {M}} - Y _ {X ^ {(t)}, M}\right) \right] + \delta B \\ \leqslant \mathbb {E} _ {P (\cdot | \bar {\boldsymbol {V}} _ {t}, \bar {\boldsymbol {v}})} \left[ \Gamma_ {t} \sum_ {x \in \Omega_ {X}} \mathbb {1} \{X ^ {(t)} = x \} \sqrt {I _ {P (\cdot | \bar {\boldsymbol {V}} _ {t} , \bar {\boldsymbol {v}})} (\boldsymbol {\theta} , \boldsymbol {\xi} ; Y _ {x})} \right] + \delta B \\ \leqslant \Gamma_ {t} \sum_ {x \in \Omega_ {X}} P (X ^ {(t)} = x \mid \bar {V} _ {t}, \bar {\boldsymbol {v}})) \sqrt {I _ {P (\cdot | \bar {V} _ {t} , \bar {\boldsymbol {v}})} (\boldsymbol {\theta} , \boldsymbol {\xi} ; Y _ {x})} + \delta B \\ \leqslant \Gamma_ {t} \sqrt {\sum_ {x \in \Omega_ {X}} P (X ^ {(t)} = x \mid \bar {V} _ {t} , \bar {v})) I _ {P (\cdot | \bar {V} _ {t} , \bar {v})} (\boldsymbol {\theta} , \boldsymbol {\xi} ; Y _ {x})} + \delta B \\ = \Gamma_ {t} \sqrt {\sum_ {x \in \Omega_ {X}} P (X ^ {(t)} = x \mid \bar {\boldsymbol {V}} _ {t} , \bar {\boldsymbol {v}})) I _ {P (\cdot | \bar {\boldsymbol {V}} _ {t} , \bar {\boldsymbol {v}})} (\boldsymbol {\theta} , \boldsymbol {\xi} ; Y _ {X ^ {(t)}} \mid X ^ {(t)} = x)} + \delta B \\ = \Gamma_ {t} \sqrt {I _ {P (\cdot | \bar {\boldsymbol {V}} _ {t} , \bar {\boldsymbol {v}})} (\boldsymbol {\theta} , \boldsymbol {\xi} ; Y _ {X ^ {(t)}} \mid X ^ {(t)})} + \delta B \\ = \Gamma_ {t} \sqrt {I _ {P (\cdot | \bar {\boldsymbol {V}} _ {t} , \bar {\boldsymbol {v}})} (\boldsymbol {\theta} , \boldsymbol {\xi} ; Y _ {X ^ {(t)}} , X ^ {(t)})} + \delta B, \\ \end{array}
$$
where the last equalities follow from the conditional independence between  $X^{(t)}$  and  $(\pmb {\theta},\pmb {\xi})$  conditioned on  $\bar{\pmb{V}}_t,\bar{\pmb{v}}$
This proposition might guide us to obtain a per-period regret bound that in particular determines a choice for the noise term  $\epsilon_{t}$ , which results in an additive term inProp. 2. A common choice for  $\epsilon_{t}$  is  $1 / T$  where  $T$  is the number of experimentation rounds and therefore the additive term in the regret is simply a constant. This is also the choice made by [25]. In this way, the regret bound is tuned with knowledge of the horizon, which is common in online learning.
C.3 Information ratio computation for specific example
The theoretical statements presented in the main body of this work only implicitly relate bounds on regret with the assumed environment discrepancies assumed in selection diagrams. It is challenging in general to provide more explicit connections. Although similarity in structure between domains is explicitly encoded in selection diagrams, the association between discrepancies and posterior distributions on parameters or reward probabilities learned from prior data for an arbitrary input selection diagram are complex. For instance, it doesn't necessarily hold that few discrepancies lead to narrow reward probability posterior distributions, and conversely it doesn't necessarily hold that large discrepancies, e.g. many selection nodes, lead to wide reward probability posterior distributions.
In particular, consider an extreme example in which source and target domains are equal and described by  $\mathcal{G} := \{X \to Y, X \leftarrow \dots \rightarrow Y\}$ . Even with infinite data and no discrepancy, it can be shown that posterior reward probabilities  $P^{*}(y \mid do(x))$  are bounded in the interval  $[P(x,y), P(x,y) + 1 - P(x)]$  which might be arbitrarily wide depending on the underlying SCM. In contrast, in other examples with certain graphs in which source and target domains differ on an arbitrarily large set of causal mechanisms (that are, however, irrelevant for the computation of posterior reward probabilities) posterior reward probabilities could be made arbitrarily narrow with sufficient data.
The information ratio  $\Gamma$  can highlight the trade-off between regret and graph structure. For specific graphs, a bound on the information ratio  $\Gamma$  can be analytically computed and used to convey some of the performance gains to be expected, for intuition. Recall that the information ratio is defined as a scalar  $\Gamma_{t}$  such that,
$$
\mathbb {E} \left[ Y _ {\tilde {X}} - Y _ {X ^ {(t)}} \mid \bar {\boldsymbol {V}} _ {t}, \bar {\boldsymbol {v}} \right] = \sqrt {\Gamma_ {t} I _ {P (\cdot | \bar {\boldsymbol {V}} _ {t} , \bar {\boldsymbol {v}})} \left(\tilde {X} ; \boldsymbol {V} _ {X ^ {(t)}}\right)}, \tag {23}
$$
The information ratio can be bounded:  $1/2 \leqslant \Gamma_t \leqslant |\Omega_X|/2$ , and describes how much information is revealed about reward distributions when an arm is pulled: on one end  $\Gamma_t = 1/2$  in problems with full information arise when the outcome  $Y_x$  is perfectly revealed by observing  $Y_{x'}$ , and on the other end  $\Gamma_t = |\Omega_X|/2$  in problems where random variables  $Y_x$  and  $Y_{x'}$  are independent for any
$x \neq x'$ . Bounds on the information ratio have also been shown for linear bandits [35]. With our parameterization of  $P_{\pi^*}(y_x)$  the probability of exogenous variables encoded in  $\theta$  is shared while  $\xi$  is not, thus some information about reward distributions is expected to be gained on all arms in each MAB round, and  $1/2 < \Gamma_t < |\Omega_X|/2$ .
In the following we will quantify the amount of sharing between reward distributions and compute a corresponding smaller upper bound for the information ratio for a specific graph in which the computation is tractable following the proof strategy of [35, Proposition 5]. This illustrates more precisely the regret gains that can be expected from having a shared parameterization for reward distributions.
Figure 10:  $\mathcal{G}^{a,*}$
We consider the selection diagram with domain-specific confounding given in Fig. 10 and the computation of  $P^{*}(y \mid do(x)) = \sum_{z \in \Omega_Z} P^a(y \mid x, z) P^*(z)$  given knowledge of  $P^a(x, y, z)$ . This graph is interesting because we may directly parameterize the conditional distributions involved in  $P^{*}(y \mid do(x))$  instead of considering its underlying SCM. Let  $Y$  be binary for simplicity (all arguments hold also more generally). For the purposes of this discussion, we will assume that  $P^a(y \mid x, z)$  can be approximated arbitrarily well from sufficient prior data. It holds that
$$
\begin{array}{l} P ^ {*} (y \mid d o (x)) = \sum_ {z \in \Omega_ {Z}} P ^ {a} (y \mid x, z) P ^ {*} (z) (24) \\ = \phi (x) ^ {T} \boldsymbol {p}, (25) \\ \end{array}
$$
where  $\phi(x) \in \mathbb{R}^{|\Omega_z|}$  is a column vector with  $i$ -th entry  $P_{\pi^a}(Y = 1 \mid x, z = i)$  and  $\pmb{p} \in \mathbb{R}^{|\Omega_z|}$  is a column vector with  $i$ -th entry  $P(Z = i)$ . The latter has some distribution in the interval  $[0, 1]$  and in particular we denote its mean by  $\mu := \mathbb{E}[p]$  and its mean conditioned on the optimal action being  $x_j$  by  $\mu_j := \mathbb{E}[p \mid \tilde{X} = x_j]$ . Recall that  $\tilde{X}$  denotes the optimal action.
Let  $\alpha_{i} := P(\tilde{X} = x_{i})$  and define  $M \in \mathbb{R}^{|\Omega_{X}| \times |\Omega_{X}|}$  by its  $(i,j)$ -th entry,
$$
\begin{array}{l} M _ {i, j} := \sqrt {\alpha_ {i} \alpha_ {j}} \left(\mathbb {E} \left[ Y _ {x _ {i}} \mid \tilde {X} = x _ {j} \right] - \mathbb {E} \left[ Y _ {x _ {i}} \right]\right) (26) \\ = \sqrt {\alpha_ {i} \alpha_ {j}} \left(\phi \left(x _ {i}\right) ^ {T} \mathbb {E} [ \boldsymbol {p} \mid \tilde {X} = x _ {j} ] - \phi \left(x _ {i}\right) ^ {T} \mathbb {E} [ \boldsymbol {p} ]\right) (27) \\ = \sqrt {\alpha_ {i} \alpha_ {j}} \left(\mathbb {E} [ \boldsymbol {p} \mid \tilde {X} = x _ {j} ] - \mathbb {E} [ \boldsymbol {p} ]\right) ^ {T} \phi (x _ {i}). (28) \\ \end{array}
$$
All expectations are taken with respect to  $P(\cdot \mid \bar{V}_t, \bar{v})$ .  $M$  can therefore be written as a product of two matrices of rank  $|\Omega_X|$ . With this definition, it was shown by [35, Proposition 5] that,
$$
\mathbb {E} \left[ Y _ {\tilde {X}} - Y _ {X ^ {(t)}} \mid \bar {V} _ {t}, \bar {v} \right] ^ {2} = \operatorname {T r a c e} (M), \tag {29}
$$
and that,
$$
I _ {P \left(\cdot | \bar {\mathbf {V}} _ {t}, \bar {\mathbf {v}}\right)} (\tilde {X}; \mathbf {V} _ {X ^ {(t)}}) \geqslant 2 \| M \| _ {F} ^ {2}, \tag {30}
$$
leading to the fact that the information ratio,
$$
\Gamma_ {t} := \frac {\mathbb {E} \left[ Y _ {\tilde {X}} - Y _ {X ^ {(t)}} \mid \bar {V} _ {t} , \bar {v} \right] ^ {2}}{I _ {P (\cdot | \bar {V} _ {t} , \bar {v})} (\tilde {X} ; (X , Y _ {X}))} \leqslant \frac {\operatorname {R a n k} (M)}{2} = \frac {| \Omega_ {Z} |}{2}. \tag {31}
$$
Now,  $|\Omega_Z| < \langle \Omega_X|$  in certain applications which shows that the information ratio may be smaller than the worst-case value of  $|\Omega_X| / 2$  due to sharing of  $P^{*}(z)$  across different actions  $x$ .
D Details on the data generating mechanisms
Bow graph with domain-specific confounding. To generate source data we choose an SCM  $M^a$  compatible with the graph specified as follows:  $P(u_Z), P(u_X, P(u_Y, P(u_{XY}))$  are given by independent Gaussian distributions with mean 0 and variance 1, and each observation  $(z, x, y)$  is generated from  $(u_Z, u_X, u_Y, u_{XY})$  using the structural assignments:  $z \gets \mathbb{1}\{u_Z < 1\}$ ,  $x \gets \mathbb{1}\{z + u_{XY} - u_X > 0\}$ ,  $y \gets \mathbb{1}\{x - 0.5z + 2u_{XY} > 0\}$ .
The deployment domain  $\pi^{*}$  is given by a different SCM  $M^{*}$  that varies in the causal mechanism relating to  $Z$ . It is given by the following SCM:  $P(u_{Z}), P(u_{X}, P(u_{Y}, P(u_{XY})$  are given by independent Gaussian distributions with mean 0 and variance 1, and each observation  $(z,x,y)$  is generated from  $(u_{Z}), u_{X}, u_{Y}, u_{XY})$  using the structural assignments:  $z \gets \mathbb{1}\{u_{Z} < 0\}, x \gets \mathbb{1}\{z + u_{XY} - u_{X} > 0\}, y \gets \mathbb{1}\{x - 0.5z + 2u_{XY} > 0\}$ .
Hypertension example. To generate data from the patient population  $\pi^a$  we choose an SCM  $M^a$  compatible with the graph specified as follows:  $P(u_Z), P(u_X, P(u_W, P(u_{XY}))$  are given by independent Gaussian distributions with mean 0 and variance 1, and each observation  $(z, x, w, y)$  is generated from  $(u_Z), u_X, u_Y, u_{XY})$  using the structural assignments:  $z \gets \mathbb{1}\{u_Z > 0\}$ ,  $x \gets \mathrm{int}\{0.5z + \mathbb{1}\{u_{XY} > 0\} + 2\mathbb{1}\{u_{XY} > 0\} - \mathbb{1}\{u_X > 0\} > 0\} + 2\mathbb{1}\{u_X > 1\} + 1\}$ ,  $w \gets \mathbb{1}\{0.3x - u_W - 0.8 > 0\}$ ,  $y \gets \mathbb{1}\{w - 0.5z + u_{XY} > 0\}$ .  $\mathrm{int}\{\cdot\}$  stands for the integer part of the content of the brackets.
The domain  $\pi^*$  in which the MAB is deployed is given by a different SCM  $M^*$  that varies in the causal mechanism relating to  $Z$  and  $W$  with respect to  $M^b$ , and varies in  $W$  with respect to  $M^a$ . It is defined by:  $P(u_Z), P(u_X, P(u_W, P(u_{XY})$  are given by independent Gaussian distributions with mean 0 and variance 1, and each observation  $(z,x,w,y)$  is generated from  $(u_Z), u_X, u_Y, u_{XY}$  using the structural assignments:  $z \gets \mathbb{1}\{u_Z > 0\}$ ,  $x \gets \mathrm{int}\{0.5z + \mathbb{1}\{u_{XY} > 0\} + 2\mathbb{1}\{u_{XY} > 0\} - \mathbb{1}\{u_X > 0\} + 2\mathbb{1}\{u_X > 1\} + 1\}$ ,  $w \gets \mathbb{1}\{0.2x - u_W - 1.8 > 0\}$ ,  $y \gets \mathbb{1}\{w - 0.5z + u_{XY} > 0\}$ .
Digital advertising example. This example considers data from two domains. Data from the source domain  $\pi^a$  is given by a SCM  $M^a$  compatible with the graph that is specified as follows:  $P(u_Z), P(u_X), P(u_W), P(u_{XW}), P(u_{WY}), P(u_A)$  are given by independent Gaussian distributions with mean 0 and variance 1, and each observation  $(z,x,w,a,y)$  is generated from a sample  $u_Z, u_X, u_W, u_{XW}, u_{WY}, u_A$  using the structural assignments:  $w \gets \mathbb{1}\{u_{XW} + u_{WY} > 0\}, z \gets \mathbb{1}\{u_Z + w > 0\}, a \gets \mathbb{1}\{u_A > 0\}, x \gets \mathrm{int}\{0.5z - 0.5a + \mathbb{1}\{u_{XW} > 0\} + 2\mathbb{1}\{u_{XW} > 0.5\} - \mathbb{1}\{u_X > 0\} + 1\}, y \gets \mathbb{1}\{0.2x - 0.5a + u_{WY} - 1 > 0\}$ .
The domain  $\pi^{*}$  in which the MAB is deployed is given by a different SCM  $M^{*}$  that varies in the causal mechanism relating to  $A$  with respect to  $M^a$ . It is defined by:  $P(u_{Z}), P(u_{X}, P(u_{W}, P(u_{XW}, P(u_{WY}, P(u_{A})$  are given by independent Gaussian distributions with mean 0 and variance 1, and each observation  $(z,x,w,a,y)$  is generated from a sample  $u_{Z}, u_{X}, u_{W}, u_{XW}, u_{WY}, u_{A}$  using the structural assignments:  $w \gets \mathbb{1}\{u_{XW} + u_{WY} > 0\}$ ,  $z \gets \mathbb{1}\{u_{Z} + w > 0\}$ ,  $a \gets \mathbb{1}\{u_{A} > 0.5\}$ ,  $x \gets \mathrm{int}\{0.5z - 0.5a + \mathbb{1}\{u_{XW} > 0\} + 2\mathbb{1}\{u_{XW} > 0.5\} - \mathbb{1}\{u_{X} > 0\} + 1\}$ ,  $y \gets \mathbb{1}\{0.2x - 0.5a + u_{WY} - 1 > 0\}$ .
E Gibbs sampling
This section gives the derivation of all conditionals using our parameterization of causal effects for the bow graph with confounding example.
The query of interest is given by  $P^{*}(y_{x} = 1)$  which is first approximated using source data  $\bar{v}$  and then updated using interventional data  $\pmb{v}_{x^{(1)}},\pmb{v}_{x^{(2)}},\dots$  collected by the MAB in the deployment environment  $\pi^{*}$ .
Its parameterization, following Corol. 1, is given by
$$
P ^ {*} \left(y _ {x} = 1\right) = \sum_ {u _ {x y}, u _ {z}} \mathbb {1} \left\{\xi_ {Y} ^ {(x, z, u _ {x y})} = y \right\} \mathbb {1} \left\{\xi_ {Z} ^ {\left(u _ {z}\right)} = z \right\} \theta_ {u _ {x y}} \theta_ {u _ {z}}, \tag {32}
$$
where  $\xi_{Y}^{(x,z,u_{xy})}$  and  $\theta_{u_{xy}}$  parameters are shared across source and deployment environments, while  $\mathbb{1}\{\xi_Z^{(u_z)} = z\}$  and  $\theta_{u_z}$  is specific to the deployment environment. We start by approximating the posterior of all relevant parameters, that is  $\xi_Y^{(x,z,u_{xy})}$  and  $\theta_{u_{xy}}$ , given  $\bar{v}$ , before interacting with the deployment environment  $\pi^*$ . Note that prior data  $\bar{v}$  is not relevant for estimating  $\mathbb{1}\{\xi_Z^{(u_z)} = z\}$  and  $\theta_{u_z}$  and thus we maintain uniform prior distributions over these parameters at this stage. In the following, we give the derivation of the complete conditionals over  $\xi_Y^{(x,z,u_{xy})}$  and  $\theta_{u_{xy}}$ .
1. Sampling from  $P(\bar{u}_{xy}, \bar{u}_z \mid \bar{v}, \xi, \theta)$ . Let  $\bar{u}_{xy} = \{u_{xy}^{(n)}, n = 1, \dots, N\}$  and  $\bar{u}_z = \{u_z^{(n)}, n = 1, \dots, N\}$  denote  $N$  independent samples of  $U_{xy}$  and  $U_z$  respectively, one corresponding to each observation  $v^{(n)} = (x^{(n)}, y^{(n)}, z^{(n)})$ , where  $n$  is the number of prior data samples. The complete conditional can be derived following the functional dependencies in the underlying SCM given by the causal graph,
$$
\begin{array}{l} P (u _ {x y} ^ {(n)}, u _ {z} ^ {(n)} \mid \bar {\boldsymbol {v}}, \boldsymbol {\xi}, \boldsymbol {\theta}) = P (u _ {x y} ^ {(n)}, u _ {z} ^ {(n)} \mid \boldsymbol {v} ^ {(n)}, \boldsymbol {\xi}, \boldsymbol {\theta}) \propto P (u _ {x y} ^ {(n)}, u _ {z} ^ {(n)}, \boldsymbol {v} ^ {(n)} \mid \boldsymbol {\xi}, \boldsymbol {\theta}) \\ = P (y ^ {(n)} \mid x ^ {(n)}, z ^ {(n)}, u _ {x y} ^ {(n)}) P (x ^ {(n)} \mid z ^ {(n)}, u _ {x y} ^ {(n)}) P ((z ^ {(n)} \mid u _ {z} ^ {(n)}) P (u _ {z} ^ {(n)}) P (u _ {x y} ^ {(n)}) \\ = \mathbb {1} \{\xi_ {Y} ^ {(x ^ {(n)}, z ^ {(n)}, u _ {x y})} = y ^ {(n)} \} \mathbb {1} \{\xi_ {X} ^ {(z ^ {(n)}, u _ {x y} ^ {(n)})} = x ^ {(n)} \} \mathbb {1} \{\xi_ {Z} ^ {(u _ {z} ^ {(n)})} = (z ^ {(n)} \} \theta_ {u _ {x y} ^ {(n)}} \theta_ {u _ {z} ^ {(n)}}, \\ \end{array}
$$
where we have replaced the probabilities with the corresponding parameters that are used to define them.