SlowGuess/NeurIPS_2023_OCR huggingface dataset:
from certain MCMC algorithms. The specific details of the MCMC algorithm will depend on the Bayesian model used, but generating posterior-predictive samples while imputing unobserved data, as required in (20), is a common objective of such algorithms (see e.g. [15, Chapter 18]). Let us denote the  $K$  samples as
$$
\left(Y _ {i, r} ^ {\{k \}} (0)\right) _ {i \in \mathcal {T} _ {r}, r \in [ t - 1 ]}, \left(Y _ {i, t} ^ {\{k \}} (0)\right), Y _ {i = 1, t} ^ {\{k \}} (1), Y _ {i = 2, t} ^ {\{k \}} (0), Y _ {i = 2, t} ^ {\{k \}} (1), \quad k = 1, \dots , K \tag {21}
$$
These will give us  $K$  samples from  $[h_{i = 1,t},h_{i = 2,t},R_{t - 1}\mid \mathcal{F}_{t - 1}]$  as follows:
$$
\begin{array}{l} \left(\hat {h} _ {i = 1, t} ^ {\{k \}}, \hat {h} _ {i = 2, t} ^ {\{k \}}, \hat {R} _ {t - 1} ^ {\{k \}}\right) = \left(h _ {t} \left(Y _ {i = 1, t} ^ {\{k \}} (1) - Y _ {i = 1, t} ^ {\{k \}} (0)\right), h _ {t} \left(Y _ {i = 2, t} ^ {\{k \}} (1) - Y _ {i = 2, t} ^ {\{k \}} (0)\right), \right. \\ \left. \sum_ {r = 1} ^ {t - 1} \sum_ {i \in \mathcal {T} _ {r}} h _ {r} \left(Y _ {i, r} ^ {\{k \}} (1) - Y _ {i, r} ^ {\{k \}} (0)\right)\right), \quad k = 1, \dots , K \\ \end{array}
$$
Then we can estimate  $\mathbb{P}\left(R_{t - 1} \geq B \mid \mathcal{F}_{t - 1}\right)$  by
$$
\mathbb {P} \left(R _ {t - 1} \geq B \mid \mathcal {F} _ {t - 1}\right) \gets \hat {\varphi} _ {t} (0) = \frac {1}{K} \sum_ {k = 1} ^ {K} \mathbb {I} \left(\hat {R} _ {t - 1} ^ {\{k \}} \geq B\right)
$$
$$
\mathcal {L} _ {t} := \left\{k \in [ K ]: \widehat {R} _ {t - 1} ^ {\{k \}} \geq B \right\} \subset [ K ]
$$
which denotes the subset of the  $K$  Monte-Calor samples for which the budgets are not depleted.
If  $\hat{\varphi}_t(0) = 0 \iff \mathcal{L}_t = \emptyset$ , we can simply out  $m_t = 0$  since this corresponds to the case that the condition (i) does not hold, i.e.  $\mathbb{P}(R_t \leq b_t \mid R_{t-1} > B, \mathcal{F}_{t-1}) \approx 0$ . Otherwise, we continue to construct  $\hat{\varphi}_t(1), \ldots, \hat{\varphi}_t^{(6)}$  as follows:
$$
\mathbb {E} \left(h _ {i = 1, t} \mid R _ {t - 1} \geq B, \mathcal {F} _ {t - 1}\right) \gets \hat {\varphi} _ {t} (1) = \frac {1}{| \mathcal {L} _ {t} |} \sum_ {k \in \mathcal {L} _ {t}} \hat {h} _ {i = 1, t} ^ {\{k \}}
$$
$$
\mathbb {E} \left(R _ {t - 1} \mid R _ {t - 1} \geq B, \mathcal {F} _ {t - 1}\right) \gets \hat {\varphi} _ {t} ^ {(2)} = \frac {1}{| \mathcal {L} _ {t} |} \sum_ {k \in \mathcal {L} _ {t}} \hat {R} _ {t - 1} ^ {\{k \}}
$$
$$
\mathbb {V} \left(h _ {i = 1, t} \mid R _ {t - 1} \geq B, \mathcal {F} _ {t - 1}\right) \leftarrow \hat {\varphi} _ {t} ^ {(3)} = \frac {1}{| \mathcal {L} _ {t} |} \sum_ {k \in \mathcal {L} _ {t}} \left(\hat {h} _ {i = 1, t} ^ {k k \}}\right) ^ {2} - (\hat {\varphi} _ {t} (1)) ^ {2}
$$
$$
\operatorname {C o v} \left(h _ {i = 1, t}, h _ {i = 2, t} \mid R _ {t - 1} \geq B, \mathcal {F} _ {t - 1}\right) \tag {22}
$$
$$
\leftarrow \hat {\varphi} _ {t} ^ {(4)} = \frac {1}{| \mathcal {L} _ {t} |} \sum_ {k \in \mathcal {L} _ {t}} \hat {h} _ {i = 1, t} ^ {\{k \}} \hat {h} _ {i = 2, t} ^ {\{k \}} - \hat {\varphi} _ {t} (1) \left(\frac {1}{| \mathcal {L} _ {t} |} \sum_ {k \in \mathcal {L} _ {t}} \hat {h} _ {i = 2, t} ^ {\{k \}}\right)
$$
$$
\mathbb {V} \left[ R _ {t - 1} \mid R _ {t - 1} \geq B, \mathcal {F} _ {t - 1} \right] \gets \hat {\varphi} _ {t} ^ {(5)} = \frac {1}{| \mathcal {L} _ {t} |} \sum_ {k \in \mathcal {L} _ {t}} \left(\hat {R} _ {t - 1} ^ {\{k \}}\right) ^ {2} - \left(\hat {\varphi} _ {t} ^ {(2)}\right) ^ {2}
$$
$$
\operatorname {C o v} \left(h _ {i = 1, t}, R _ {t - 1} \mid R _ {t - 1} \geq B, \mathcal {F} _ {t - 1}\right) \leftarrow \hat {\varphi} _ {t} ^ {(6)} = \frac {1}{| \mathcal {L} _ {t} |} \sum_ {k \in \mathcal {L} _ {t}} \hat {h} _ {i = 1, t} ^ {\{k \}} \hat {h} _ {i = 2, t} ^ {\{k \}} - \hat {\varphi} _ {t} (1) \hat {\varphi} _ {t} ^ {(2)}
$$
From (18), (19) and the Monte-Carlo estimates above, we then have estimators for  $\mathbb{E}\left[R_t\mid R_{t - 1}\geq B,\mathcal{F}_{t - 1}\right],\mathbb{V}\left[R_t\mid R_{t - 1}\geq B,\mathcal{F}_{t - 1}\right]$  in terms of  $\hat{\varphi}_t(1),\ldots ,\hat{\varphi}_t^{(6)}$  as follows
$$
\mathbb {E} \left[ R _ {t} \mid R _ {t - 1} \geq B, \mathcal {F} _ {t - 1} \right] \leftarrow m _ {t} \cdot \hat {\varphi} _ {t} (1) + \hat {\varphi} _ {t} ^ {(2)}
$$
$$
\mathbb {V} \left[ R _ {t} \mid R _ {t - 1} \geq B, \mathcal {F} _ {t - 1} \right] \gets \left(m _ {t} \cdot \hat {\varphi} _ {t} ^ {(3)} + m _ {t} (m _ {t} - 1) \cdot \hat {\varphi} _ {t} ^ {(4)}\right) + \hat {\varphi} _ {t} ^ {(5)} + m _ {t} \cdot \hat {\varphi} _ {t} ^ {(6)}
$$
The two inequalities in (17) then become
$$
m _ {t} \cdot \hat {\varphi} _ {t} (1) + \hat {\varphi} _ {t} ^ {(2)} \geq b _ {t} \tag {23a}
$$
$$
\frac {1}{1 + \frac {\left(m _ {t} \cdot \hat {\varphi} _ {t} (1) + \hat {\varphi} _ {t} ^ {(2)} - b _ {t}\right) ^ {2}}{\left(m _ {t} \cdot \hat {\varphi} _ {t} ^ {(3)} + m _ {t} (m _ {t} - 1) \cdot \hat {\varphi} _ {t} ^ {(4)}\right) + \hat {\varphi} _ {t} ^ {(5)} + m _ {t} \cdot \hat {\varphi} _ {t} ^ {(6)}}} \leq \Delta_ {t} \tag {23b}
$$
respectively. Assume that  $\Delta_t > 0$  or else set  $m_t = 0$  directly. Observe that (23b) can be written as, with  $q_t \coloneqq \Delta_t^{-1} - 1$ ,
$$
A _ {t} m _ {t} ^ {2} + B _ {t} m _ {t} + C _ {t} \geq 0
$$
$$
A _ {t} := \left(\hat {\varphi} _ {t} (1)\right) ^ {2} - q _ {t} \hat {\varphi} _ {t} ^ {(4)}
$$
$$
B _ {t} := 2 \widehat {\varphi} _ {t} (1) \left(\widehat {\varphi} _ {t} ^ {(2)} - b _ {t}\right) - q _ {t} \widehat {\varphi} _ {t} ^ {(3)} + q _ {t} \widehat {\varphi} _ {t} ^ {(4)} - q _ {t} \widehat {\varphi} _ {t} ^ {(6)} \tag {24}
$$
$$
C _ {t} := \left(\hat {\varphi} _ {t} ^ {(2)} - b _ {t}\right) ^ {2} - q _ {t} \hat {\varphi} _ {t} ^ {(5)}
$$
Then one can choose  $m_t$  to be the largest, positive integer in the range defined by
$$
m _ {t} \cdot \hat {\varphi} _ {t} (1) + \hat {\varphi} _ {t} ^ {(2)} \geq b _ {t}, A _ {t} m _ {t} ^ {2} + B _ {t} m _ {t} + C _ {t} \geq 0
$$
If the range does not contain any positive integer, we set  $m_t = 0$ . Note that the range can be easily identified after solving the quadratic equation  $A_t m_t^2 + B_t m_t + C_t = 0$ . Algorithm 2 gives the algorithm that outputs ramp sizes adaptively. Note that by construction, it gives a  $(\delta, B)$ -RRC experiments if the Monte-Carlo estimators are sufficiently accurate.
Algorithm 2 Output ramp size adaptively
Input:  $B < 0, \delta \in [0,1)$   
1: Initialize  $t \gets 1, \prod_{r=1}^{0}(1 - \Delta_r) \gets 1$   
2: while  $\prod_{r=1}^{t-1}(1 - \Delta_r) > 1 - \delta$  do  
3: choose  $\Delta_t \in \left[0, \frac{1 - \delta}{\prod_{r=1}^{t-1}(1 - \Delta_r)} - 1\right], b_t \geq B$   
4: run MCMC to obtain posterior samples in (21) and computes  $\hat{\varphi}_t(0)$   
5: if  $\hat{\varphi}_t(0) \gets 0$  then  $m_t \gets 0$   
6: else  
7: compute  $\hat{\varphi}_t(1), \ldots, \hat{\varphi}_t^{(6)}$  using (22) and then  $A_t, B_t, C_t$  by (24)  
8: find  $\mathcal{V}_t \gets \left\{m \in \mathbb{N}_+ \cap [0, N_t/2] : m \cdot \hat{\varphi}_t(1) + \hat{\varphi}_t^{(2)} \geq b_t, A_t m^2 + B_t m + C_t \geq 0\right\}$   
9: if  $\mathcal{V}_t \neq \emptyset$  then  
10:  $m_t \gets \max \mathcal{V}_t$   
11: else  
12:  $m_t \gets 0$   
13: end if  
14: end if  
15: Output  $m_t$  and then conduct stage  $t$ -experiment and observe the outcomes  
16: update  $t \gets t + 1$   
17: end while
We have conducted preliminary simulations of the proposed procedure for a multivariate Gaussian outcome model with Gaussian-inverse-Wishart prior, and observed satisfactory results. However, we defer presenting numerical results until future work when a more systematic investigation of Monte-Carlo based procedures can be conducted.
F LinkedIn experiment data
In Table 1 below,  $\mu_{\mathrm{true}}(w), \sigma(w)^2, w = 0, 1$  are sample statistics from the actual LinkedIn experiment.  $N_t$  are incoming population size reduced by  $10^4$  factor for tractability on a personal computer.
Stages t	1	2	3	4	5	6
μtrue(0)	0.3648	0.3780	0.3752	0.2317	0.4009	0.3930
μtrue(1)	0.3659	0.3788	0.3754	0.2317	0.4010	0.3941
σ(0)2	2.0993	2.2769	2.0909	1.1165	2.2705	2.3982
σ(1)2	2.0923	2.2248	2.0135	1.0526	2.2476	2.4430
Nt	10,756	10,460	10,598	7,580	10,550	10,688
Table 1: LinkedIn experiment data
G Thompson-sampling based Bayesian bandit
This algorithm is developed in [27, Section 4] for clinical trials. The algorithm assigns a user  $i$  at stage  $t \geq 1$  to treatment with probability
$$
\mathbb {P} \left(i \in \mathcal {T} _ {t}\right) = \frac {\mathbb {P} \left(\mu_ {\text {t r u e}} (1) > \mu_ {\text {t r u e}} (0) \mid \mathcal {F} _ {t - 1}\right) ^ {c}}{\mathbb {P} \left(\mu_ {\text {t r u e}} (1) > \mu_ {\text {t r u e}} (0) \mid \mathcal {F} _ {t - 1}\right) ^ {c} + \mathbb {P} \left(\mu_ {\text {t r u e}} (1) \leq \mu_ {\text {t r u e}} (0) \mid \mathcal {F} _ {t - 1}\right) ^ {c}}
$$
for tuning parameter  $c > 0$ . Under Definition 3.1, by (15d), we have that
$$
\mathbb {P} \left(\mu_ {\mathrm {t r u e}} (1) > \mu_ {\mathrm {t r u e}} (0) \mid \mathcal {F} _ {t - 1}\right) = \Phi \left(\frac {\mu_ {p , t} (1) - \mu_ {p , t} (0)}{\sqrt {\sigma_ {p , t} (0) ^ {2} + \sigma_ {p , t} (1) ^ {2}}}\right).
$$