SlowGuess/NeurIPS_2023_OCR huggingface dataset:
2. Sampling from  $P(\xi_Y^{(x,z,u_{xy})} \mid \bar{v}, \bar{u}, \theta)$ . Similarly, for fixed  $x, z, u_{xy}$ , parameter  $\xi_Y^{(x,z,u_{xy})}$  is mutually independent of any other parameter in  $\xi$  given  $\bar{v}, \bar{u}, \theta$  and can be sampled separately. Recall that by definition of the underlying SCM  $\xi_Y^{(x,z,u_{xy})}$  represent a deterministic mapping between inputs  $x, z, u_{xy}$  and output  $y \in \Omega_Y$ . The value  $\xi_Y^{(x,z,u_{xy})} \in \Omega_Y$  is therefore implicitly determined by the current values  $\bar{v}, \bar{u}$ : if there exists a tuple  $(x^{(n)} = x, z^{(n)} = z, u_{xy}^{(n)} = u_{xy}, y^{(n)} = y)$  for some  $n = 1, \ldots, N$ , then by definition  $\xi_Y^{(x,z,u_{xy})} := y$  with probability 1. If no such tuple exist, then the distribution of  $\xi_Y^{(x,z,u_{xy})}$  remains uniform over its domain  $\Omega_Y$  as none of the data points carries information as to the mapping  $(x, z, u_{xy}) \mapsto y$ .
3. Sampling from  $P(\theta_{u_{xy}} \mid \bar{v}, \bar{u}, \theta)$ . The conditional distribution over  $\theta_{u_{xy}}$  given  $\bar{v}, \bar{u}$  is given by a Dirichlet distribution following the conjugacy of it with regard to categorical distributions of  $U_{xy}$  and can be updated by adding the counts of each outcome  $\bar{u}_{xy}$  of  $U_{xy}$  in the current sample to the corresponding prior concentration parameters of the Dirichlet prior distribution.
This process eventually forms a chain of samples from the correct posterior distribution of each parameter. At this stage, the MAB is deployed in  $\pi^{*}$  and all parameters may be updated both with prior data  $\bar{v}$  as well as with additional samples  $v_{x}$  collected online in every round of experimentation. Offline and online data points are different in kind, and contribute to updating parameters differently. For shared parameters, both types of data may be used while for parameters specific to  $\pi^{*}$  only newly collected interventional data samples will be relevant in posterior computations.
$\xi_{Y}^{(x,z,u_{xy})}$  is updated using the posterior  $P(\xi_{Y}^{(x,z,u_{xy})} \mid \bar{\pmb{v}}, \pmb{v}_{x^{(1)}}, \dots, \pmb{v}_{x^{(t-1)}}, \bar{\pmb{u}}, \pmb{u}_{x^{(1)}}, \dots, \pmb{u}_{x^{(t-1)}})$ , while  $\xi_{Z}^{(u_{z})}$  is updated using the posterior  $P(\xi_{Z}^{(u_{z})} \mid \bar{\pmb{v}}, \pmb{v}_{x^{(1)}}, \dots, \pmb{v}_{x^{(t-1)}}, \bar{\pmb{u}}, \pmb{u}_{x^{(1)}}, \dots, \pmb{u}_{x^{(t-1)}}) =$
$P(\xi_{Z}^{(u_{z})}\mid \pmb{v}_{x^{(1)}},\dots ,\pmb{v}_{x^{(t - 1)}},\pmb{u}_{x^{(1)}},\dots ,\pmb{u}_{x^{(t - 1)}})$  using the same intuition as described above. Similarly,  $\theta_{u_{xy}}$  is updated using the posterior  $P(\theta_{u_{xy}}\mid \bar{\pmb{u}},\pmb{u}_{x^{(1)}},\dots ,\pmb{u}_{x^{(t - 1)}})$  while  $\theta_{u_z}$  is updated using the posterior  $P(\theta_{u_z}\mid \bar{\pmb{u}},\pmb{u}_{x^{(1)}},\dots ,\pmb{u}_{x^{(t - 1)})} = P(\theta_{u_z}\mid \pmb{u}_{x^{(1)}},\dots ,\pmb{u}_{x^{(t - 1)}})$ .