SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\limsup_{t\to \infty}\frac{1}{t}\sum_{i = 1}^{t}\mathbb{1}\left\{W_{i}\cdot \operatorname {sign}\left[ g _{i}(Z_{i})\right] <   0\right\} \stackrel {\text{a.s.}}{<  } \frac{1}{2}.
$$
Since  $\mathbb{1}\left\{x < 0\right\} = (1 - \mathrm{sign}[x]) / 2$  we get that:
$$
\lim _ {t \to \infty} \sup _ {t \to \infty} \frac {1}{t} \sum_ {i = 1} ^ {t} \left(\frac {1}{2} - \frac {W _ {i} \cdot \mathrm {s i g n} [ g _ {i} (Z _ {i}) ]}{2}\right) ^ {\mathrm {a . s .}} <   \frac {1}{2},
$$
which, after rearranging and multiplying by two, implies that:
$$
\liminf_{t\to \infty}\frac{1}{t}\sum_{i = 1}^{t}W_{i}\cdot \operatorname {sign}\left[ g_{i}(Z_{i})\right] > 0.
$$
Hence, a sufficient condition for consistency (59) holds, and we conclude that the result is true.
(b) Recall that
$$
f _ {i} ^ {\mathrm {s}} \left(Z _ {i}, W _ {i}\right) = W _ {i} \cdot g _ {i} \left(Z _ {i}\right),
$$
and Assumption 2 states that:
$$
\limsup _ {t \rightarrow \infty} \frac {1}{t} \sum_ {i = 1} ^ {t} \left(g _ {i} (Z _ {i}) - W _ {i}\right) ^ {2} \stackrel {\mathrm {a . s}} {<  } 1,
$$
$$
\operatorname * {l i m s u p} _ {t \rightarrow \infty} \frac {1}{t} \sum_ {i = 1} ^ {t} \left(g _ {i} ^ {2} (Z _ {i}) - 2 W _ {i} \cdot g _ {i} (Z _ {i})\right) ^ {\mathrm {a . s}} <   0.
$$
It is easy to see that the above, in turn, implies that:
$$
\liminf_{t\to \infty}\frac{1}{t}\sum_{i = 1}^{t}W_{i}\cdot g_{i}(Z_{i})\overset {\mathrm{a.s}}{>0}.
$$
D.4 Proofs for Appendix A
Proof. 1. We trivially have that the payoff function (27) is bounded:  $\forall (x,y,w)\in \mathcal{X}\times \mathcal{Y}\times \{-1,1\}$ , it holds that  $f_{\star}^{\mathrm{r}}(x,y,w)\in [-1,1]$ . Further, under the null  $H_0$  in (26a), it trivially holds that  $\mathbb{E}_{H_0}[f_\star^{\mathrm{r}}(X_t,Y_t,W_t)\mid \mathcal{F}_{t - 1}] = 0$ , where  $\mathcal{F}_{t - 1} = \sigma (\{(X_i,Y_i,W_i)\}_{i\leq t - 1})$ . Since ONS betting fractions  $\left(\lambda_t^{\mathrm{ONS}}\right)_{t\geq 1}$  are predictable, we conclude that the resulting wealth process is a nonnegative martingale. The assertion of the Theorem then follows directly from Ville's inequality (Proposition 2) when  $a = 1 / \alpha$ .
$$
\lim  _ {t \rightarrow \infty} \inf  _ {i = 1} ^ {t} \frac {1}{t} \sum_ {i = 1} ^ {t} f _ {\star} ^ {\mathrm {r}} \left(X _ {i}, Y _ {i}, W _ {i}\right) ^ {\text {a . s .}} > 0. \tag {60}
$$
$$
\frac {1}{t} \sum_ {i = 1} ^ {t} f _ {\star} ^ {\mathrm {r}} (X _ {i}, Y _ {i}, W _ {i}) = \frac {1}{t} \sum_ {i = 1} ^ {t} \tanh \left(s _ {\star} \cdot W _ {i} \ell (g _ {\star} (X _ {i}), Y _ {i})\right) \stackrel {\mathrm {a . s .}} {\rightarrow} \mathbb {E} \left[ \tanh \left(s _ {\star} \cdot W \ell (g _ {\star} (X), Y)\right)\right].
$$
Note that for any  $x \in \mathbb{R} : \tanh(x) \geq x - \frac{1}{3} \cdot \max\{x^3, 0\}$ . Hence, for any  $s > 0$ , it holds that:
$$
\begin{array}{l} \mathbb {E} \left[ \tanh  \left(s \cdot W \ell (g _ {\star} (X), Y)\right) \right] \geq s \mathbb {E} \left[ W \ell (g _ {\star} (X), Y) \right] - \frac {1}{3} \mathbb {E} \left[ \max  \left\{s ^ {3} \cdot W (\ell (g _ {\star} (X), Y)) ^ {3}, 0 \right\} \right] \\ = s \mathbb {E} [ W \ell (g _ {\star} (X), Y) ] - \frac {s ^ {3}}{3} \mathbb {E} [ (\ell (g _ {\star} (X), Y)) ^ {3} \cdot \max  \{W, 0 \} ] \\ = s \mathbb {E} [ W \ell (g _ {\star} (X), Y) ] - \frac {s ^ {3}}{6} \mathbb {E} [ (1 + W) \cdot (\ell (g _ {\star} (X), Y)) ^ {3} ], \tag {61} \\ \end{array}
$$
where we used that  $\max \{W,0\} = (W + 1) / 2$  since  $W\in \{-1,1\}$ . Maximizing the RHS of (61) over  $s > 0$  yields  $s_\star$  defined in (28a). Hence,
$$
\begin{array}{l} \mathbb {E} \left[ \tanh  \left(s _ {\star} \cdot W \ell \left(g _ {\star} (X), Y\right)\right) \right] \geq s _ {\star} \mathbb {E} \left[ W \ell \left(g _ {\star} (X), Y\right) \right] - \frac {s _ {\star} ^ {3}}{6} \mathbb {E} \left[ (1 + W) \cdot \left(\ell \left(g _ {\star} (X), Y\right)\right) ^ {3} \right] \\ = s _ {\star} \left(\mathbb {E} \left[ W \ell \left(g _ {\star} (X), Y\right) \right] - \frac {s _ {\star} ^ {2}}{6} \mathbb {E} \left[ (1 + W) \cdot \left(\ell \left(g _ {\star} (X), Y\right)\right) ^ {3} \right]\right) \\ = s _ {\star} \left(\mathbb {E} \left[ W \ell \left(g _ {\star} (X), Y\right) \right] - \frac {1}{3} \mathbb {E} \left[ W \ell \left(g _ {\star} (X), Y\right) \right]\right) \\ = \frac {2 s _ {\star}}{3} \mathbb {E} [ W \ell (g _ {\star} (X), Y) ] > 0. \\ \end{array}
$$
Hence, we conclude that the oracle regression-based IT is consistent since the sufficient condition (62) holds.
Proof. 1. We trivially have that the payoff function (29) is bounded:  $\forall (x,y,w)\in \mathcal{X}\times \mathcal{Y}\times \{-1,1\}$ , it holds that  $f_{t}^{\mathrm{r}}(x,y,w)\in [-1,1]$ . Further, under the null  $H_0$  in (26a), it trivially holds that  $\mathbb{E}_{H_0}[f_t^{\mathrm{r}}(X_t,Y_t,W_t)\mid \mathcal{F}_{t - 1}] = 0$ , where  $\mathcal{F}_{t - 1} = \sigma (\{(X_i,Y_i,W_i)\}_{i\leq t - 1})$ . Since ONS betting fractions  $(\lambda_{t}^{\mathrm{ONS}})_{t\geq 1}$  are predictable, we conclude that the resulting wealth process is a nonnegative martingale. The assertion of the Theorem then follows directly from Ville's inequality (Proposition 2) with  $a = 1 / \alpha$ .
$$
\lim  _ {t \rightarrow \infty} \inf  _ {t} \frac {1}{t} \sum_ {i = 1} ^ {t} f _ {t} ^ {\mathrm {r}} \left(X _ {i}, Y _ {i}, W _ {i}\right) ^ {\text {a . s .}} > 0. \tag {62}
$$
(a) Step 1. Consider a predictable sequence of scaling factors  $(s_t)_{t\geq 1}$ , defined in (30a), and the corresponding sequences  $(\mu_t)_{t\geq 1}$  and  $(\nu_{t})_{t\geq 1}$ , defined in (30b) and (30c) respectively. For  $t\geq 1$ , let  $\mathcal{F}_t\coloneqq \sigma (\{(X_i,Y_i,W_i)\}_{i\leq t})$ . Since the losses are bounded, we have that:
$$
\left. \left(W _ {i} \cdot \ell \left(g \left(X _ {i}; \theta_ {i}\right), Y _ {i}\right) - \mathbb {E} \left[ W _ {i} \cdot \ell \left(g \left(X _ {i}; \theta_ {i}\right), Y _ {i}\right) \mid \mathcal {F} _ {i - 1}\right]\right) _ {i \geq 1}, \right.
$$
is a bounded martingale difference sequence (BMDS). By the Strong Law of Large Numbers for BMDS, it follows that:
$$
\frac {1}{t} \sum_ {i = 1} ^ {t} \left(W _ {i} \cdot \ell \left(g \left(X _ {i}; \theta_ {i}\right), Y _ {i}\right) - \mathbb {E} \left[ W _ {i} \cdot \ell \left(g \left(X _ {i}; \theta_ {i}\right), Y _ {i}\right) \mid \mathcal {F} _ {i - 1} \right]\right) \stackrel {\mathrm {a . s .}} {\rightarrow} 0.
$$
Since  $((X_{t},Y_{t},W_{t}))_{t\geq 1}$  is a sequence of i.i.d. observations, we can write
$$
\frac {1}{t} \sum_ {i = 1} ^ {t} \mathbb {E} \left[ W _ {i} \cdot \ell \left(g \left(X _ {i}; \theta_ {i}\right), Y _ {i}\right) \mid \mathcal {F} _ {i - 1} \right] = \frac {1}{t} \sum_ {i = 1} ^ {t} \mathbb {E} \left[ W \cdot \ell \left(g (X; \theta_ {i}), Y\right) \mid \theta_ {i} \right],
$$
where  $(X,Y,W)\perp (\theta_t)_{t\geq 1},\theta_\star$  . Using Assumption 3, we get that:
$$
\begin{array}{l} \left| \frac {1}{t} \sum_ {i = 1} ^ {t} \mathbb {E} \left[ W \cdot \ell (g (X; \theta_ {i}), Y) \mid \theta_ {i} \right] - \mathbb {E} \left[ W \cdot \ell (g (X; \theta_ {\star}), Y) \mid \theta_ {\star} \right] \right| \\ \leq \frac {1}{t} \sum_ {i = 1} ^ {t} \sup  _ {\substack {x \in \mathcal {X} \\ y \in \mathcal {Y}}} | \ell (g (x; \theta_ {i}), y) - \ell (g (x; \theta_ {\star}), y) | \tag{63} \\ \leq \frac {1}{t} \sum_ {i = 1} ^ {t} L _ {2} \sup  _ {x \in \mathcal {X}} | g (x; \theta_ {i}) - g (x; \theta_ {\star}) | \\ \leq \frac {1}{t} \sum_ {i = 1} ^ {t} L _ {2} \cdot L _ {1} \cdot \| \theta_ {i} - \theta_ {\star} \| \xrightarrow {\mathrm {a . s .}} 0, \\ \end{array}
$$
since  $\| \theta_{i} - \theta_{\star}\| \stackrel {\mathrm{a.s.}}{\longrightarrow}0$  by Assumption 4. In particular, this implies that  $\mu_t \stackrel {\mathrm{a.s.}}{\longrightarrow} \mathbb{E}\left[W\ell (g(X;\theta_{\star}),Y)\mid \theta_{\star}\right]$ . Similar argument can be used to show that  $\nu_{t} \stackrel {\mathrm{a.s.}}{\longrightarrow} \mathbb{E}\left[(1 + W)\cdot (\ell (g(X;\theta_{\star}),Y))^{3}\mid \theta_{\star}\right]$ , and hence,
$$
s _ {t} \stackrel {\text {a . s .}} {\rightarrow} \sqrt {\frac {2 \mathbb {E} [ W \ell (g (X ; \theta_ {\star}) , Y) \mid \theta_ {\star} ]}{\mathbb {E} [ (1 + W) \cdot (\ell (g (X ; \theta_ {\star}), Y)) ^ {3} \mid \theta_ {\star} ]}} =: s _ {\star}. \tag {64}
$$
Note that  $s_{\star}$  is a random variable which is positive (almost surely) by Assumption 4.
(b) Step 2. Recall that for any  $x \in \mathbb{R} : \tanh(x) \geq x - \frac{1}{3} \cdot \max\{x^3, 0\}$  and that  $\max\{W, 0\} = (W + 1)/2$  since  $W \in \{-1, 1\}$ . We have:
$$
\begin{array}{l} \frac {1}{t} \sum_ {i = 1} ^ {t} f _ {i} ^ {\mathrm {r}} \left(X _ {i}, Y _ {i}, W _ {i}\right) \\ = \frac {1}{t} \sum_ {i = 1} ^ {t} \tanh  \left(s _ {i} \cdot W _ {i} \ell \left(g \left(X _ {i}; \theta_ {i}\right), Y _ {i}\right)\right) \\ \geq \frac {1}{t} \sum_ {i = 1} ^ {t} \left(s _ {i} \cdot W _ {i} \cdot \ell \left(g \left(X _ {i}; \theta_ {i}\right), Y _ {i}\right) - \frac {s _ {i} ^ {3}}{6} \cdot \left(1 + W _ {i}\right) \cdot \left(\ell \left(g \left(X _ {i}; \theta_ {i}\right), Y _ {i}\right)\right) ^ {3}\right). \\ \end{array}
$$
Note that  $\theta_{i}$  and  $s_i$  are  $\mathcal{F}_{i-1}$ -measurable (see Step 1 for the definition of  $\mathcal{F}_{i-1}$ ). Under a minor technical assumption that  $(s_t)_{t \geq 1}$  is a sequence of bounded scaling factors (the lower bound is trivially zero and the upper bound also holds if  $\nu_t$  are bounded away from zero almost surely which is reasonable given the definition of  $\nu_t$ ), we can use analogous arguments regarding a BMDS in Step 1 to deduce that:
$$
\begin{array}{l} \liminf_{t\to \infty}\frac{1}{t}\sum_{i = 1}^{t}f_{i}^{\mathrm{r}}\left(X_{i},Y_{i},W_{i}\right) \\ \geq \lim  _ {t \rightarrow \infty} \frac {1}{t} \sum_ {i = 1} ^ {t} \left(s _ {i} \cdot \mathbb {E} [ W \cdot \ell (g (X; \theta_ {i}), Y) \mid \theta_ {i} ] - \frac {s _ {i} ^ {3}}{6} \mathbb {E} [ (1 + W) \cdot (\ell (g (X; \theta_ {i}), Y)) ^ {3} \mid \theta_ {i} ]\right). \tag {65} \\ \end{array}
$$
Using argument analogous to (63), we can show that:
$$
\frac {1}{t} \sum_ {i = 1} ^ {t} \mathbb {E} \left[ (1 + W) \cdot \left(\ell \left(g \left(X; \theta_ {i}\right), Y\right)\right) ^ {3} \mid \theta_ {i} \right] \xrightarrow {\text {a . s .}} \mathbb {E} \left[ (1 + W) \cdot \left(\ell \left(g \left(X; \theta_ {\star}\right), Y\right)\right) ^ {3} \mid \theta_ {\star} \right]. \tag {66}
$$
Combining (63), (64) and (66), we deduce that
$$
\frac {1}{t} \sum_ {i = 1} ^ {t} \left(s _ {i} \cdot \mathbb {E} [ W \cdot \ell (g (X; \theta_ {i}), Y) \mid \theta_ {i} ] - \frac {s _ {i} ^ {3}}{6} \mathbb {E} [ (1 + W) \cdot (\ell (g (X; \theta_ {i}), Y)) ^ {3} \mid \theta_ {i} ]\right)
$$
$$
\stackrel {\mathrm {a . s .}} {\rightarrow} \quad s _ {\star} \cdot \mathbb {E} [ W \cdot \ell (g (X; \theta_ {\star}), Y) | \theta_ {\star} ] - \frac {s _ {\star} ^ {3}}{6} \cdot \mathbb {E} [ (1 + W) \cdot (\ell (g (X; \theta_ {\star}), Y)) ^ {3} | \theta_ {\star} ]
$$
$$
= \frac {2 s _ {\star}}{3} \cdot \mathbb {E} [ W \cdot \ell (g (X; \theta_ {\star}), Y) \mid \theta_ {\star} ].
$$
Hence, from (65) it follows that:
$$
\liminf_{t\to \infty}\frac{1}{t}\sum_{i = 1}^{t}f_{i}^{\mathrm{r}}(X_{i},Y_{i},W_{i})\geq \frac{2s_{\star}}{3}\cdot \mathbb{E}\left[ W\cdot \ell (g(X;\theta_{\star}),Y)\mid \theta_{\star}\right],
$$
where the RHS is a random variable which is positive almost surely. Hence, a sufficient condition for consistency (62) holds which concludes the proof.
D.5 Proofs for Appendix B
Two-Sample Testing with Unbalanced Classes. Note that  $(g(z) = 2\eta (z) - 1)$
$$
\begin{array}{l} (1 - \lambda_ {t}) \cdot 1 + \lambda_ {t} \cdot \frac {(\eta (Z _ {t})) ^ {\mathbb {1} \left\{W _ {t} = 1 \right\}} (1 - \eta (Z _ {t})) ^ {1 - \mathbb {1} \left\{W _ {t} = 1 \right\}}}{(\pi) ^ {\mathbb {1} \left\{W _ {t} = 1 \right\}} (1 - \pi) ^ {1 - \mathbb {1} \left\{W _ {t} = 1 \right\}}} \\ = (1 - \lambda_ {t}) \cdot 1 + \lambda_ {t} \cdot \frac {\left(\frac {1 + g (Z _ {t})}{2}\right) ^ {\mathbb {1} \{W _ {t} = 1 \}} \left(\frac {1 - g (Z _ {t})}{2}\right) ^ {1 - \mathbb {1} \{W _ {t} = 1 \}}}{\left(\pi\right) ^ {\mathbb {1} \{W _ {t} = 1 \}} \left(1 - \pi\right) ^ {1 - \mathbb {1} \{W _ {t} = 1 \}}} \\ = (1 - \lambda_ {t}) \cdot 1 + \frac {\lambda_ {t}}{2} \cdot \frac {(1 + g (Z _ {t})) ^ {\mathbb {1} \left\{W _ {t} = 1 \right\}} (1 - g (Z _ {t})) ^ {1 - \mathbb {1} \left\{W _ {t} = 1 \right\}}}{(\pi) ^ {\mathbb {1} \left\{W _ {t} = 1 \right\}} (1 - \pi) ^ {1 - \mathbb {1} \left\{W _ {t} = 1 \right\}}} \\ = \left(1 - \lambda_ {t}\right) \cdot 1 + \frac {\lambda_ {t}}{2} \cdot \frac {1 + W _ {t} g (Z _ {t})}{\left(\pi\right) ^ {\mathbb {1} \left\{W _ {t} = 1 \right\}} \left(1 - \pi\right) ^ {1 - \mathbb {1} \left\{W _ {t} = 1 \right\}}} \\ = (1 - \lambda_ {t}) \cdot 1 + \frac {\lambda_ {t}}{2} \cdot \frac {2}{1 + W _ {t} (2 \pi - 1)} \cdot (1 + W _ {t} g (Z _ {t})) \\ = \left(1 - \lambda_ {t}\right) \cdot 1 + \frac {\lambda_ {t}}{1 + W _ {t} (2 \pi - 1)} \cdot \left(1 + W _ {t} g \left(Z _ {t}\right)\right) \\ = 1 + \lambda_ {t} \cdot \frac {W _ {t} (g (Z _ {t}) - (2 \pi - 1))}{1 + W _ {t} (2 \pi - 1)}. \\ \end{array}
$$
Payoff for the Case of Unbalanced Classes (known  $\pi$ ). To see that the payoff function (37) is lower bounded by negative one, note that:
$$
f _ {t} ^ {\mathrm {u}} (z, 1) = \frac {g _ {t} (z) - (2 \pi - 1)}{2 \pi} \geq \frac {- 1 - (2 \pi - 1)}{2 \pi} = - 1,
$$
$$
f _ {t} ^ {\mathrm {u}} (z, - 1) = \frac {- g _ {t} (z) + (2 \pi - 1)}{2 (1 - \pi)} \geq \frac {- 1 + (2 \pi - 1)}{2 (1 - \pi)} = - 1.
$$
To see that such payoff is fair, note that:
$$
\mathbb {E} _ {H _ {0}} \left[ f _ {t} ^ {\mathrm {u}} (Z _ {t}, W _ {t}) \mid \mathcal {F} _ {t - 1} \right] = \mathbb {E} _ {P} \left[ \pi \cdot \frac {g _ {t} (Z _ {t}) - (2 \pi - 1)}{2 \pi} \right] - \mathbb {E} _ {Q} \left[ (1 - \pi) \cdot \frac {g _ {t} (Z _ {t}) - (2 \pi - 1)}{2 (1 - \pi)} \mid \mathcal {F} _ {t - 1} \right] = 0,
$$
where  $\mathcal{F}_{t - 1} = \sigma \left(\{(Z_i,W_i)\}_{i\leq t - 1}\right)$
Proof. First, we show that  $(\mathcal{K}_t^{\mathrm{u}})_{t\geq 0}$  is a nonnegative supermartingale. For any  $t\geq 1$ , the wealth  $\mathcal{K}_{t - 1}$  is multiplied at round  $t$  by
$$
1 + \lambda_ {t} f _ {t} ^ {\mathrm {u}} \left(\left\{(Z _ {b (t - 1) + i}, W _ {b (t - 1) + i}) \right\} _ {i \in \{1, \ldots , b \}}\right) = (1 - \lambda_ {t}) \cdot 1 + \lambda_ {t} \cdot \frac {\prod_ {i = b (t - 1) + 1} ^ {b t} \left(1 + W _ {i} g _ {t} (Z _ {i})\right)}{\prod_ {i = 1} ^ {b} \left(1 + W _ {i} (2 \hat {\pi} _ {t} - 1)\right)}.
$$
Since  $\lambda_t \in [0, 0.5]$ , we conclude that the process  $(\mathcal{K}_t^{\mathrm{u}})_{t \geq 0}$  is nonnegative. Next, note that since  $\hat{\pi}_t$  is the MLE of  $\pi$  computed from a  $t$ -th minibatch, it follows that:
$$
\begin{array}{l} 1 + \lambda_ {t} f _ {t} ^ {\mathrm {u}} \left(\left\{(Z _ {b (t - 1) + i}, W _ {b (t - 1) + i}) \right\} _ {i \in \{1, \dots , b \}}\right) \leq (1 - \lambda_ {t}) \cdot 1 + \lambda_ {t} \cdot \frac {\prod_ {i = b (t - 1) + 1} ^ {b t} \left(1 + W _ {i} g _ {t} (Z _ {i})\right)}{\prod_ {i = b (t - 1) + 1} ^ {b t} \left(1 + W _ {i} (2 \pi - 1)\right)} \\ = (1 - \lambda_ {t}) \cdot 1 + \lambda_ {t} \cdot \prod_ {i = b (t - 1) + 1} ^ {b t} \left(\frac {1 + W _ {i} g _ {t} (Z _ {i})}{1 + W _ {i} (2 \pi - 1)}\right). \\ \end{array}
$$
Recall that  $\mathcal{F}_{t - 1} = \sigma \left(\{Z_i,W_i\}_{i\leq b(t - 1)}\right)$ . It suffices to show that if  $H_{0}$  is true, then
$$
\mathbb {E} _ {H _ {0}} \left[ \prod_ {i = b (t - 1) + 1} ^ {b t} \left(\frac {1 + W _ {i} g _ {t} \left(Z _ {i}\right)}{1 + W _ {i} (2 \pi - 1)}\right) \mid \mathcal {F} _ {t - 1} \right] = 1.
$$
Note that the individual terms in the above product are independent conditional on  $\mathcal{F}_{t - 1}$ . Hence,
$$
\mathbb {E} _ {H _ {0}} \left[ \prod_ {i = b (t - 1) + 1} ^ {b t} \left(\frac {1 + W _ {i} g _ {t} (Z _ {i})}{1 + W _ {i} (2 \pi - 1)}\right) \mid \mathcal {F} _ {t - 1} \right] = \prod_ {i = b (t - 1) + 1} ^ {b t} \mathbb {E} _ {H _ {0}} \left[ \frac {1 + W _ {i} g _ {t} (Z _ {i})}{1 + W _ {i} (2 \pi - 1)} \mid \mathcal {F} _ {t - 1} \right].
$$
For any  $i\in \{b(t - 1) + 1,\ldots ,bt\}$ , it holds that:
$$
\begin{array}{l} \mathbb {E} _ {H _ {0}} \left[ \frac {1 + W _ {i} g _ {t} (Z _ {i})}{1 + W _ {i} (2 \pi - 1)} \mid \mathcal {F} _ {t - 1} \right] = \mathbb {E} _ {H _ {0}} \left[ \pi \cdot \frac {1 + g _ {t} (Z _ {i})}{1 + (2 \pi - 1)} + (1 - \pi) \cdot \frac {1 - g _ {t} (Z _ {i})}{1 - (2 \pi - 1)} \mid \mathcal {F} _ {t - 1} \right] \\ = \mathbb {E} _ {H _ {0}} \left[ \frac {1 + g _ {t} \left(Z _ {i}\right)}{2} + \frac {1 - g _ {t} \left(Z _ {i}\right)}{2} \mid \mathcal {F} _ {t - 1} \right] \\ = 1. \\ \end{array}
$$
Hence, we conclude that  $(\mathcal{K}_t^{\mathrm{u}})_{t\geq 0}$  is a nonnegative supermartingale adapted to  $(\mathcal{F}_t)_{t\geq 0}$ . The time-uniform type I error control of the resulting test then follows from Ville's inequality (Proposition 2).
E Additional Experiments and Details
E.1 Modeling Details
CNN Architecture and Training. We use CNN with 4 convolutional layers (kernel size is taken to be  $3 \times 3$ ) and 16, 32, 32, 64 filters respectively. Further, each convolutional layer is followed by max-pooling layer  $(2 \times 2)$ . After flattening, those layers are followed by 1 fully connected layer with 128 neurons. Dropout  $(p = 0.5)$  and early stopping (with patience equal to ten epochs and  $20\%$  of data used in the validation set) is used for regularization. ReLU activation functions are used in each layer. Adam optimizer is used for training the network. We start training after processing twenty observations, and update the model parameters after processing every next ten observations. Maximum number of epochs is set to 25 for each training iteration. The batch size is set to 32.
Single-stream Sequential Kernelized 2ST. The construction of this test is the extension of 2ST of Shekhar and Ramdas [2023] to the case when at each round an observation only from a single distribution  $(P$  or  $Q)$  is revealed. Let  $\mathcal{G}$  denote an RKHS with positive-definite kernel  $k$  and canonical feature map  $\varphi (\cdot)$  defined on  $\mathcal{Z}$ . Recall that instances from  $P$  as labeled as  $+1$  and instances from  $Q$  are labeled as  $-1$  (characterized by  $W$ ). The mean embeddings of  $P$  and  $Q$  are then defined as
$$
\begin{array}{l} \hat {\mu} _ {P} ^ {(t)} = \frac {1}{N _ {+} (t)} \sum_ {i = 1} ^ {t} \varphi \left(Z _ {i}\right) \cdot \mathbb {1} \left\{W _ {i} = + 1 \right\}, \\ \hat {\mu} _ {Q} ^ {(t)} = \frac {1}{N _ {-} (t)} \sum_ {i = 1} ^ {t} \varphi \left(Z _ {i}\right) \cdot \mathbb {1} \left\{W _ {i} = - 1 \right\}, \\ \end{array}
$$
where  $N_{+}(t) = |i \leq t : W_{i} = +1|$  and  $N_{-}(t) = |i \leq t : W_{i} = -1|$ . The corresponding payoff function is
$$
f _ {t} ^ {\mathrm {k}} \left(Z _ {t + 1}, W _ {t + 1}\right) = W _ {t + 1} \cdot \hat {g} _ {t} \left(Z _ {t + 1}\right),
$$
$$
\text {w h e r e} \quad \hat {g} _ {t} = \frac {\hat {\mu} _ {P} ^ {(t)} - \hat {\mu} _ {Q} ^ {(t)}}{\left\| \hat {\mu} _ {P} ^ {(t)} - \hat {\mu} _ {Q} ^ {(t)} \right\| _ {\mathcal {G}}}.
$$
To make the test computationally efficient, it is critical to update the normalization constant efficiently. Suppose that at round  $t + 1$ , an instance from  $P$  is observed. In this case,  $\hat{\mu}_{Q}^{(t + 1)} = \hat{\mu}_{Q}^{(t)}$ . Note that:
$$
\begin{array}{l} \hat {\mu} _ {P} ^ {(t + 1)} = \frac {1}{N _ {+} (t + 1)} \sum_ {i = 1} ^ {t + 1} \varphi (Z _ {i}) \cdot \mathbb {1} \left\{W _ {i} = + 1 \right\} \\ = \frac {1}{N _ {+} (t) + 1} \sum_ {i = 1} ^ {t + 1} \varphi \left(Z _ {i}\right) \cdot \mathbb {1} \left\{W _ {i} = + 1 \right\} \\ = \frac {1}{N _ {+} (t) + 1} \varphi \left(Z _ {t + 1}\right) + \frac {1}{N _ {+} (t) + 1} \sum_ {i = 1} ^ {t} \varphi \left(Z _ {i}\right) \cdot \mathbb {1} \left\{W _ {i} = + 1 \right\} \\ = \frac {1}{N _ {+} (t) + 1} \varphi (Z _ {t + 1}) + \frac {N _ {+} (t)}{N _ {+} (t) + 1} \hat {\mu} _ {P} ^ {(t)}. \\ \end{array}
$$
$$
\begin{array}{l} \left\| \hat {\mu} _ {P} ^ {(t + 1)} - \hat {\mu} _ {Q} ^ {(t + 1)} \right\| _ {\mathcal {G}} ^ {2} = \left\| \hat {\mu} _ {P} ^ {(t + 1)} - \hat {\mu} _ {Q} ^ {(t)} \right\| _ {\mathcal {G}} ^ {2} \\ = \left\| \hat {\mu} _ {P} ^ {(t + 1)} \right\| _ {\mathcal {G}} ^ {2} - 2 \left\langle \hat {\mu} _ {P} ^ {(t + 1)}, \hat {\mu} _ {Q} ^ {(t)} \right\rangle_ {\mathcal {G}} + \left\| \hat {\mu} _ {Q} ^ {(t)} \right\| _ {\mathcal {G}} ^ {2}. \\ \end{array}
$$
$$
\begin{array}{l} \left\langle \hat {\mu} _ {P} ^ {(t + 1)}, \hat {\mu} _ {Q} ^ {(t)} \right\rangle_ {\mathcal {G}} = \left\langle \frac {1}{N _ {+} (t) + 1} \varphi (Z _ {t + 1}) + \frac {N _ {+} (t)}{N _ {+} (t) + 1} \hat {\mu} _ {P} ^ {(t)}, \hat {\mu} _ {Q} ^ {(t)} \right\rangle_ {\mathcal {G}} \\ = \frac {1}{N _ {+} (t) + 1} \left\langle \varphi (Z _ {t + 1}), \hat {\mu} _ {Q} ^ {(t)} \right\rangle_ {\mathcal {G}} + \frac {N _ {+} (t)}{N _ {+} (t) + 1} \left\langle \hat {\mu} _ {P} ^ {(t)}, \hat {\mu} _ {Q} ^ {(t)} \right\rangle_ {\mathcal {G}}. \\ \end{array}
$$
$$
\left\langle \varphi (Z _ {t + 1}), \hat {\mu} _ {Q} ^ {(t)} \right\rangle_ {\mathcal {G}} = \frac {1}{N _ {-} (t)} \sum_ {i = 1} ^ {t} k (Z _ {t + 1}, Z _ {i}) \cdot \mathbb {1} \left\{W _ {i} = - 1 \right\}.
$$
Next, we assume for simplicity that  $k(x,x) = 1,\forall x$  which holds for RBF kernel. Observe that:
$$
\begin{array}{l} \left\| \hat {\mu} _ {P} ^ {(t + 1)} \right\| _ {\mathcal {G}} ^ {2} = \left\langle \hat {\mu} _ {P} ^ {(t + 1)}, \hat {\mu} _ {P} ^ {(t + 1)} \right\rangle_ {\mathcal {G}} \\ = \frac {1}{(N _ {+} (t) + 1) ^ {2}} + \frac {2 N _ {+} (t)}{(N _ {+} (t) + 1) ^ {2}} \left\langle \varphi (Z _ {t + 1}), \hat {\mu} _ {P} ^ {(t)} \right\rangle_ {\mathcal {G}} + \frac {(N _ {+} (t)) ^ {2}}{(N _ {+} (t) + 1) ^ {2}} \left\| \hat {\mu} _ {P} ^ {(t)} \right\| _ {\mathcal {G}} ^ {2}. \\ \end{array}
$$
By caching intermediate results, we can compute the normalization constant using linear in  $t$  number of kernel evaluations. We start betting once at least one instance is observed from both  $P$  and  $Q$ . For simulations, we use RBF kernel and the median heuristic with first 20 instances to compute the kernel hyperparameter.
MLP Training Scheme We begin training after processing twenty datapoints from  $P_{XY}$  which gives a training dataset with 40 datapoints (due to randomization). When updating a model, we use previous parameters as initialization. We use the following update scheme: we start after next  $n_0 = 10$  datapoints from  $P_{XY}$  are observed. Once  $n_0$  becomes less than  $1\%$  of the size of the existing training dataset, we increase it by ten, that is,  $n_t = n_{t-1} + 10$ . When we fit the model, we set the maximum number of epochs to be 25 and use early stopping with patience of 3 epochs.
Kernel Hyperparameters for Synthetic Experiments. For SKIT, we use RBF kernels:
$$
k (x, x ^ {\prime}) = \exp \left(- \lambda_ {X} \| x - x ^ {\prime} \| _ {2} ^ {2}\right), \quad l (y, y ^ {\prime}) = \exp \left(- \lambda_ {Y} \| y - y ^ {\prime} \| _ {2} ^ {2}\right).
$$
For simulations on synthetic data, we take kernel hyperparameters to be inversely proportional to the second moment of the underlying variables (the median heuristic yields similar results):
$$
\lambda_ {X} = \frac {1}{2 \mathbb {E} \left[ \| X - X ^ {\prime} \| _ {2} ^ {2} \right]}, \quad \lambda_ {Y} = \frac {1}{2 \mathbb {E} \left[ \| Y - Y ^ {\prime} \| _ {2} ^ {2} \right]}.
$$
1. Spherical model. By symmetry, we have:  $P_{X} = P_{Y}$ , and hence we take  $\lambda_{X} = \lambda_{Y}$ . We have
$$
\mathbb {E} \left[ (X - X ^ {\prime}) ^ {2} \right] = 2 \mathbb {E} \left[ X ^ {2} \right] = \frac {2}{d}.
$$
2. HTDD model. By symmetry, we have:  $P_{X} = P_{Y}$ , and hence we take  $\lambda_{X} = \lambda_{Y}$ . We have
$$
\mathbb {E} \left[ (X - X ^ {\prime}) ^ {2} \right] = 2 \mathbb {E} \left[ X ^ {2} \right] = \frac {2 \pi^ {2}}{3}.
$$
3. Sparse signal model. We have