SlowGuess/NeurIPS_2023_OCR huggingface dataset:
One can view this problem as sequential two-sample testing under distribution drift (due to incremental learning of  $(g_{t})_{t\geq 1}$ ). Hence, one may use either Seq-C-2ST from Section 2 or sequential kernelized 2ST of Shekhar and Ramdas [2023] on the resulting sequence of the losses on observations from  $P_{XY}$  and  $P_{X}\times P_{Y}$ . In what follows, we analyze a direct approach where testing is performed by comparing the losses on instances drawn from the two distributions. A critical difference with a construction of Seq-C-2ST is that to design a valid betting strategy one has to ensure that the payoff functions are lower bounded by negative one.
A.1 Proxy Regression-based Independence Test
To avoid cases when some expected values are not well-defined, we assume for simplicity that  $\mathcal{X}$  is a bounded subset of  $\mathbb{R}^d$  for som  $d\geq 1$ :  $\mathcal{X} = \left\{x\in \mathbb{R}^d:\| x\| _2\leq B_1\right\}$  for some  $B_{1} > 0$ . Similarly, we assume that  $\mathcal{V}$  is a bounded subset of  $\mathbb{R}$ :  $\mathcal{V} = \{y\in \mathbb{R}:|y|\leq B_2\}$  for some  $B_{2} > 0$ . We note that the construction of the regression-based IT will not require explicit knowledge of constants  $B_{1}$  and  $B_{2}$ . First, we consider a setting where an instance either from the joint distribution or an instance from the product of the marginal distributions is observed at each round.
Definition 3 (Proxy Setting). Suppose that we observe a stream of i.i.d. observations  $((X_{t},Y_{t},W_{t}))_{t\geq 1}$ , where  $W_{t}\sim$  Rademacher(1/2), the distribution of  $(X_{t},Y_{t})\mid W_{t} = +1$  is  $P_{X}\times P_{Y}$ , and that of  $(X_{t},Y_{t})\mid W_{t} = -1$  is  $P_{XY}$ . The goal is to design a test for the following pair of hypotheses:
$$
H _ {0}: P _ {X Y} = P _ {X} \times P _ {Y}, \tag {26a}
$$
$$
H _ {1}: P _ {X Y} \neq P _ {X} \times P _ {Y}. \tag {26b}
$$
Oracle Proxy Sequential Regression-based IT. To construct an oracle test, we assume having access to the oracle predictor  $g_{\star} : \mathcal{X} \to \mathcal{Y}$ , e.g., the minimizer of the squared risk is  $g_{\star}(x) = \mathbb{E}[Y \mid X = x]$ . Formalizing the above intuition, we use  $\mathbb{E}[W\ell(g_{\star}(X), Y)]$  as a natural way for measuring dependence between  $X$  and  $Y$ . To enforce boundedness of the payoff functions, we use ideas of the tests for symmetry from [Ramdas et al., 2020, Shekhar and Ramdas, 2023, Podkopaev et al., 2023, Shaer et al., 2023], namely we use a composition with an odd function:
$$
f _ {\star} ^ {r} \left(X _ {t}, Y _ {t}, W _ {t}\right) = \tanh  \left(s _ {\star} \cdot W _ {t} \cdot \ell \left(g _ {\star} \left(X _ {t}\right), Y _ {t}\right)\right) \in [ - 1, 1 ], \tag {27}
$$
where  $s_{\star} > 0$  is an appropriately selected scaling factor<sup>4</sup>. Since under  $H_0$  in (26a),  $s_{\star} \cdot W_t \cdot \ell(g_{\star}(X_t), Y_t)$  is a random variable that is symmetric around zero, it follows that  $\mathbb{E}[f_{\star}^{\mathrm{r}}(X_t, Y_t, W_t)] =$
4We note that rescaling is important for arguing about consistency and not the type I error control.
0, and, using the argument analogous to the proof of Theorem 1, we can easily deduce that a sequential IT based on  $f_{\star}^{\mathrm{r}}$  controls the type I error control. The scaling factor  $s_{\star}$  is selected in a way that guarantees that, if  $H_{1}$  in (26b) is true and if  $\mathbb{E}\left[W\ell(g_{\star}(X),Y)\right] > 0$ , then  $\mathbb{E}\left[f_{\star}^{\mathrm{r}}(X,Y,W)\right] > 0$ , which is a sufficient condition for consistency of the oracle test. In particular, we show that it suffices to consider:
$$
s _ {\star} = \sqrt {\frac {2 \mu_ {\star}}{\nu_ {\star}}}, \tag {28a}
$$
where  $\mu_{\star} = \mathbb{E}\left[W\ell (g_{\star}(X),Y)\right],$  (28b)
$$
\nu_ {\star} = \mathbb {E} \left[ (1 + W) (\ell \left(g _ {\star} (X), Y\right)) ^ {3} \right]. \tag {28c}
$$
Without loss of generality, we assume that  $\nu_{\star}$  is bounded away from zero (which is a very mild assumption since  $\nu_{\star}$  essentially corresponds to a cubic risk of  $g_{\star}$  on data drawn from the product of the marginal distributions  $P_{X} \times P_{Y}$ ). Let the oracle regression-based wealth process  $\left(\mathcal{K}_t^{\mathrm{r},\star}\right)_{t \geq 0}$  be defined by using the payoff function (27) with a scaling factor defined in (28a), along with a predictable sequence of betting fractions  $(\lambda_t)_{t \geq 1}$  selected via the ONS strategy (Algorithm 1). We have the following result about the oracle regression-based IT, whose proof is deferred to Appendix D.4.
Theorem 3. The following claims hold for the oracle sequential regression-based IT based on  $\left(\mathcal{K}_t^{\mathrm{r},\star}\right)_{t\geq 0}$ :
1. Suppose that  $H_0$  in (26a) is true. Then the test ever stops with probability at most  $\alpha$ :  $\mathbb{P}_{H_1}(\tau < \infty) \leq \alpha$ .
2. Suppose that  $H_{1}$  in (26b) is true. Further, suppose that:  $\mathbb{E}\left[W\ell(g_{\star}(X), Y)\right] > 0$ . Then the test is consistent:  $\mathbb{P}_{H_1}(\tau < \infty) = 1$ .
Practical Proxy Sequential Regression-based IT. To construct a practical test, we use a sequence of predictors  $(g_{t})_{t\geq 1}$  that are updated sequentially as more data are observed. We write  $\mathcal{A}_{\mathrm{r}}:\left(\cup_{t\geq 1}(\mathcal{X}\times \mathcal{Y})^{t}\right)\times \mathcal{G}\to \mathcal{G}$  to denote a chosen regressor learning algorithm which maps a training dataset of any size and previously used predictor, to an updated predictor. We start with  $\mathcal{D}_0 = \emptyset$  and some initial guess  $g_{1}\in \mathcal{G}$ . At round  $t$ , we use the payoff function:
$$
f _ {t} ^ {\mathrm {r}} \left(X _ {t}, Y _ {t}, W _ {t}\right) = \tanh  \left(s _ {t} \cdot W _ {t} \cdot \ell \left(g _ {t} \left(X _ {t}\right), Y _ {t}\right)\right). \tag {29}
$$
where a sequence of predictable scaling factors  $(s_t)_{t\geq 1}$  is defined as follows: we set  $s_0 = 0$  and define:
$$
s _ {t} = \sqrt {\frac {2 \mu_ {t}}{\nu_ {t}}}, \tag {30a}
$$
where  $\mu_t = \left(\frac{1}{t - 1}\sum_{i = 1}^{t - 1}W_i\cdot \ell (g_i(X_i),Y_i)\right)\lor 0,$  (30b)
$$
\nu_ {t} = \frac {1}{t - 1} \sum_ {i = 1} ^ {t - 1} \left(1 + W _ {i}\right) \cdot \left(\ell \left(g _ {i} \left(X _ {i}\right), Y _ {i}\right)\right) ^ {3}. \tag {30c}
$$
After  $(X_{t},Y_{t},W_{t})$  has been used for betting, we update a training dataset:  $\mathcal{D}_t = \mathcal{D}_{t - 1}\cup \{(X_t,Y_t,W_t)\}$ , and an existing predictor:  $g_{t + 1} = \mathcal{A}_{\mathrm{r}}(\mathcal{D}_t,g_t)$ . We summarize this practical sequential 2ST in Algorithm 3.
For simplicity, we consider a class of functions  $\mathcal{G} \coloneqq \{g_{\theta} : \mathcal{X} \to \mathcal{Y}, \theta \in \Theta\}$  for some parameter set  $\Theta$  which we assume to be a subset of a metric space. In this case, a sequence of predictors  $(g_t)_{t \geq 1}$  is associated with the corresponding sequence of parameters  $(\theta_t)_{t \geq 1}$ : for  $t \geq 1$ ,  $g_t(\cdot) = g(\cdot; \theta_t)$  for some  $\theta_t \in \Theta$ . To argue about the consistency of the resulting test, we make two assumptions.
Assumption 3 (Smoothness). We assume that:
- Predictors in  $\mathcal{G}$  are  $L_{1}$ -Lipschitz smooth:
$$
\sup  _ {x \in \mathcal {X}} | g (x; \theta) - g (x; \theta^ {\prime}) | \leq L _ {1} \| \theta - \theta^ {\prime} \|, \quad \forall \theta , \theta^ {\prime} \in \Theta . \tag {31}
$$
Algorithm 3 Proxy Sequential Regression-based IT
Input: significance level  $\alpha \in (0,1)$ , data stream  $((X_{t},Y_{t},W_{t}))_{t\geq 1}$ ,  $g_{1}(z) \equiv 0$ ,  $\mathcal{A}_{\mathrm{r}}$ ,  $\mathcal{D}_0 = \emptyset$ ,  $\lambda_1^{\mathrm{ONS}} = 0$ ,  $s_1 = 0$ .  
for  $t = 1,2,\ldots$  do  
    Evaluate the payoff  $f_t^{\mathrm{r}}(X_t,Y_t,W_t)$  as in (29);  
    Using  $\lambda_t^{\mathrm{ONS}}$ , update the wealth process  $\mathcal{K}_t^{\mathrm{r}}$  as in (5);  
    if  $\mathcal{K}_t^{\mathrm{r}} \geq 1/\alpha$  then  
        Reject  $H_0$  and stop;  
    else  
        Update the training dataset:  $\mathcal{D}_t := \mathcal{D}_{t-1} \cup \{(X_t,Y_t)\}$ ;  
        Update predictor:  $g_{t+1} = \mathcal{A}_{\mathrm{r}}(\mathcal{D}_t,g_t)$ ;  
        Compute  $s_{t+1}$  as in (30a);  
        Compute  $\lambda_{t+1}^{\mathrm{ONS}}$  (Algorithm 1) using  $f_t^{\mathrm{r}}(X_t,Y_t,W_t)$ ;
- The loss function  $\ell$  is  $L_{2}$ -Lipschitz smooth:
$$
\sup  _ {\substack {x \in \mathcal{X} \\ y \in \mathcal{Y}}} | \ell (g(x; \theta), y) - \ell (g(x; \theta^{\prime}), y) | \leq L_{2} \sup  _ {x \in \mathcal{X}} | g(x; \theta) - g(x; \theta^{\prime})|, \quad \forall \theta , \theta^{\prime} \in \Theta . \tag{32}
$$
In words, Assumption (31) states that the outputs of predictors, whose parameters are close, will also be close. Assumption (32) states that the losses of two predictors, whose outputs are close, will also be close. For example, if  $\mathcal{G}$  is a class of linear predictors:  $g_{\theta}(x) = \theta^{\top}x$ ,  $x \in \mathcal{X}$ , then Assumption 3 will be trivially satisfied for the squared and the absolute losses if  $\mathcal{X}$  and  $\mathcal{Y}$  are bounded. Note that we do not need an explicit knowledge of  $L_{1}$  or  $L_{2}$  for designing a test. Second, we make a learnability assumption about algorithm  $\mathcal{A}_{\mathrm{r}}$ .
Assumption 4 (Learnability). Suppose that  $H_{1}$  in (26b) is true. We assume that the regressor learning algorithm  $\mathcal{A}_{\mathrm{r}}$  is such that for the resulting sequence of parameters  $(\theta_t)_{t \geq 1}$ , it holds that  $\theta_t \stackrel{\mathrm{a.s.}}{\rightarrow} \theta_\star$ , where  $\theta_\star$  is a random variable taking values in  $\Theta$  and  $\mathbb{E}[W \ell(g(X; \theta_\star), Y) | \theta_\star] > 0$ , where  $(X, Y, W) \perp \theta_\star$ .
We conclude with the following result for the practical proxy sequential regression-based IT, whose proof is deferred to Appendix D.4.
Theorem 4. The following claims hold for the proxy sequential regression-based IT (Algorithm 3):
1. Suppose that  $H_0$  in (26a) is true. Then the test ever stops with probability at most  $\alpha$ :  $\mathbb{P}_{H_0}(\tau < \infty) \leq \alpha$ .
2. Suppose that  $H_{1}$  in (26b) is true. Further, suppose that Assumptions 3 and 4 are satisfied. Then the test is consistent:  $\mathbb{P}_{H_1}(\tau < \infty) = 1$ .
Sequential Regression-based Independence Test (Seq-R-IT). Next, we instantiate this test for the sequential independence testing setting (as per Definition 2) where we observe sequence  $((X_{t},Y_{t}))_{t\geq 1}$ , where  $(X_{t},Y_{t})\stackrel {\mathrm{iid}}{\sim}P_{XY}$ ,  $t\geq 1$ . Analogous to Section 2, we bet on the outcome of two observations drawn from the joint distribution  $P_{XY}$ . To proceed, we derandomize the payoff function (29) and consider
$$
\begin{array}{l} f _ {t} ^ {\mathrm {r}} \big ((X _ {2 t - 1}, Y _ {2 t - 1}), (X _ {2 t}, Y _ {2 t}) \big) = \frac {1}{4} \left(\tanh  \left(s _ {t} \cdot \ell \left(g _ {t} (X _ {2 t - 1}), Y _ {2 t}\right)\right) + \tanh  \left(s _ {t} \cdot \ell \left(g _ {t} (X _ {2 t}), Y _ {2 t - 1}\right)\right)\right) \\ - \frac {1}{4} \left(\tanh  \left(s _ {t} \cdot \ell \left(g _ {t} \left(X _ {2 t}\right), Y _ {2 t}\right)\right) - \tanh  \left(s _ {t} \cdot \ell \left(g _ {t} \left(X _ {2 t - 1}\right), Y _ {2 t - 1}\right)\right)\right). \tag {33} \\ \end{array}
$$
After betting on the outcome of the  $t$ -th pair of observations from  $P_{XY}$ , we update a training dataset:
$$
\mathcal {D} _ {t} = \mathcal {D} _ {t - 1} \cup \left\{\left(X _ {2 t - 1}, Y _ {2 t - 1}\right), \left(X _ {2 t}, Y _ {2 t}\right) \right\},
$$
and a predictive model:  $\hat{g}_{t + 1} = \mathcal{A}_{\mathrm{r}}(\mathcal{D}_t,\hat{g}_t)$
A.2 Synthetic Experiments
To evaluate the performance of Seq-R-IT, we consider the Gaussian linear model. Let  $(X_{t})_{t\geq 1}$  and  $(\varepsilon_{t})_{t\geq 1}$  denote two independent sequences of i.i.d. standard Gaussian random variables. For  $t\geq 1$  we take
$$
\left(X _ {t}, Y _ {t}\right) = \left(X _ {t}, X _ {t} \beta + \varepsilon_ {t}\right),
$$
where  $\beta \neq 0$  implies nonzero linear correlation (hence dependence). We consider 20 values of  $\beta$  equally spaced in  $[0,1/2]$ . For the comparison, we use:
1. Seq-R-IT with ridge regression. We use ridge regression as an underlying model:  $\hat{g}_t(x) = \beta_0^{(t)} + x\beta_1^{(t)}$ , where
$$
\left(\beta_ {0} ^ {(t)}, \beta_ {1} ^ {(t)}\right) = \underset {\beta_ {0}, \beta_ {1}} {\arg \min } \sum_ {i = 1} ^ {2 (t - 1)} \left(Y _ {i} - X _ {i} \beta_ {1} - \beta_ {0}\right) ^ {2} + \lambda \beta_ {1} ^ {2}.
$$
2. Seq-C-IT with QDA. Note that  $P_{XY} = \mathcal{N}(\mu, \Sigma^{+})$  and  $P_{X} \times P_{Y} = \mathcal{N}(\mu, \Sigma^{-})$ , where
$$
\mu = \left( \begin{array}{c} 0 \\ 0 \end{array} \right), \quad \Sigma^ {+} = \left( \begin{array}{c c} 1 & \beta \\ \beta & 1 + \beta^ {2} \end{array} \right), \quad \Sigma^ {-} = \left( \begin{array}{c c} 1 & 0 \\ 0 & 1 + \beta^ {2} \end{array} \right).
$$
For this problem, an oracle predictor which minimizes the misclassification risk is
$$
g ^ {\star} (x, y) = \frac {\varphi ((x , y) ; \mu^ {+} , \Sigma^ {+}) - \varphi ((x , y) ; \mu^ {-} , \Sigma^ {-})}{\varphi ((x , y) ; \mu^ {-} , \Sigma^ {-}) + \varphi ((x , y) ; \mu^ {+} , \Sigma^ {+})} \in [ - 1, 1 ], \tag {34}
$$
where  $\varphi((x, y); \mu, \Sigma)$  denotes the density of the Gaussian distribution  $\mathcal{N}(\mu, \Sigma)$  evaluated at  $(x, y)$ . Recall that  $\mathcal{D}_{t-1} = \{(Z_i, +1)\}_{i \leq 2(t-1)} \cup \{(Z_i', -1)\}_{i \leq 2(t-1)}$  denotes the training dataset that is available at round  $t$  for training a predictor  $\hat{g}_t: \mathcal{X} \times \mathcal{Y} \to [-1, 1]$ . We deploy Seq-C-IT with an estimator  $\hat{g}_t$  of (34), obtained by using plug-in estimates of  $\mu^+, \Sigma^+, \mu^-, \Sigma^-$ , computed from  $\mathcal{D}_{t-1}$ :
$$
\hat {\mu} _ {t} ^ {+} = \frac {1}{2 (t - 1)} \sum_ {Z \in \mathcal {D} _ {t - 1} ^ {+}} Z, \quad \hat {\Sigma} _ {t} ^ {+} = \left(\frac {1}{2 (t - 1)} \sum_ {Z \in \mathcal {D} _ {t - 1} ^ {+}} Z Z ^ {\top}\right) - (\hat {\mu} _ {t} ^ {+}) (\hat {\mu} _ {t} ^ {+}) ^ {\top},
$$
and  $\hat{\mu}_t^{-}$ ,  $\hat{\Sigma}_t^{-}$  are computed similarly from  $\mathcal{D}_t^{-}$ .
In addition, we also include HSIC-based SKIT to the comparison and defer the details regarding kernel hyperparameters to Appendix E.1. We set the monitoring horizon to  $T = 5000$  points from  $P_{XY}$  and aggregate the results over 200 sequences of observations for each value of  $\beta$ . We illustrate the result in Figure 5: while Seq-R-IT has high power for large values of  $\beta$ , we observe its inferior performance against Seq-C-IT (and SKIT) under the harder settings. Improving regression-based betting strategies, e.g., designing better scaling factors that still yield a provably consistent test, is an open question for future research.
B Two-sample Testing with Unbalanced Classes
In Section 2, we developed a sequential 2ST under the assumption at each round, an instance from either  $P$  or  $Q$  is revealed with equal probability. Such assumption was reasonable for designing Seq-C-IT, where external randomization produced two instances from  $P_{XY}$  and  $P_X \times P_Y$  at each round. Next, we generalize our sequential 2ST to a more general setting of unbalanced classes.
Definition 4 (Sequential two-sample testing with unbalanced classes). Let  $\pi \in (0,1)$ . Suppose that we observe a stream of i.i.d. observations  $((Z_{t},W_{t}))_{t\geq 1}$ , where  $W_{t}\sim \mathrm{Rademacher}(\pi)$ , the distribution of  $Z_{t}\mid W_{t} = +1$  is denoted  $P$ , and that of  $Z_{t}\mid \bar{W}_{t} = -1$  is denoted  $Q$ . We set the goal of designing a sequential test for the following pair of hypotheses:
$$
H _ {0}: P = Q, \tag {35a}
$$
$$
H _ {1}: P \neq Q. \tag {35b}
$$
(b)
Figure 5: Comparison between Seq-R-IT, Seq-C-IT and HSIC-based SKIT under the Gaussian linear model. Inspecting Figure 5a at  $\beta = 0$  confirms that all tests control the type I error. Non-surprisingly, kernel-based SKIT performs better than predictive tests under this model (no localized dependence). We also observe that Seq-C-IT performs better than Seq-R-IT.
For what follows, we will focus on the payoff based on the squared risk due to its relationship to the likelihood-ratio-based test (Remark 3). In particular, after correcting the likelihood under the null in (20) to account for a general positive class proportion  $\pi$ , we can deduce that (see Appendix D.5):
$$
\left(1 - \lambda_ {t}\right) \cdot 1 + \lambda_ {t} \cdot \frac {\left(\eta_ {t} \left(Z _ {t}\right)\right) ^ {\mathbb {1} \left\{W _ {t} = 1 \right\}} \left(1 - \eta_ {t} \left(Z _ {t}\right)\right) ^ {\mathbb {1} \left\{W _ {t} = 0 \right\}}}{\left(\pi\right) ^ {\mathbb {1} \left\{W _ {t} = 1 \right\}} \left(1 - \pi\right) ^ {\mathbb {1} \left\{W _ {t} = 0 \right\}}} = 1 + \lambda_ {t} \cdot \frac {W _ {t} \left(g _ {t} \left(Z _ {t}\right) - (2 \pi - 1)\right)}{1 + W _ {t} (2 \pi - 1)}, \tag {36}
$$
where  $\eta_t(z) = (g_t(z) + 1)/2$ , and hence, a natural payoff function for the case with unbalanced classes is
$$
f _ {t} ^ {\mathrm {u}} \left(Z _ {t}, W _ {t}\right) = \frac {W _ {t} \left(g _ {t} \left(Z _ {t}\right) - (2 \pi - 1)\right)}{1 + W _ {t} (2 \pi - 1)}. \tag {37}
$$
Note that the payoff for the balanced case (22b) is recovered by setting  $\pi = 1/2$ . It is easy to check that (see Appendix D.5): (a)  $f_{t}^{\mathrm{u}}(z,w)\geq -1$  for any  $(z,w)\in \mathcal{Z}\times \{-1,1\}$ , and (b) if  $H_0$  in (35a) is true, then  $\mathbb{E}_{H_0}[f_t^{\mathrm{u}}(Z_t,W_t)\mid \mathcal{F}_{t - 1}] = 0$ , where  $\mathcal{F}_{t - 1} = \sigma (\{(Z_i,W_i)\}_{i\leq t - 1})$ . This in turn implies that a wealth process that relies on the payoff function  $f_{t}^{\mathrm{u}}$  in (37) is a nonnegative martingale, and hence, the corresponding sequential 2ST is valid. However, the positive class proportion  $\pi$ , needed to use the payoff function (37), is generally unknown beforehand. First, let us consider the case when  $\lambda_{t} = 1,t\geq 1$ . In this case, the wealth of a gambler that uses the payoff function (37) after round  $t$  is
$$
\mathcal {K} _ {t} = \frac {\prod_ {i = 1} ^ {t} \left(\eta_ {i} \left(Z _ {i}\right)\right) ^ {\mathbb {1} \left\{W _ {i} = 1 \right\}} \left(1 - \eta_ {i} \left(Z _ {i}\right)\right) ^ {\mathbb {1} \left\{W _ {i} = 0 \right\}}}{\prod_ {i = 1} ^ {t} \pi^ {\mathbb {1} \left\{W _ {i} = 1 \right\}} \left(1 - \pi\right) ^ {\mathbb {1} \left\{W _ {i} = 0 \right\}}}. \tag {38}
$$
$$
\hat {\pi} _ {t} := \frac {1}{t} \sum_ {i = 1} ^ {t} \mathbb {1} \left\{W _ {t} = 1 \right\} = \underset {\pi \in [ 0, 1 ]} {\arg \max } \left(\prod_ {i = 1} ^ {t} \pi^ {\mathbb {1} \left\{W _ {i} = 1 \right\}} (1 - \pi) ^ {\mathbb {1} \left\{W _ {i} = 0 \right\}}\right),
$$
is the MLE for  $\pi$  computed from  $\{W_i\}_{i \leq t}$ . In particular, if we consider a process  $(\tilde{\mathcal{K}}_t)_{t \geq 0}$ , where
$$
\tilde {\mathcal {K}} _ {t} := \frac {\prod_ {i = 1} ^ {t} \left(\eta_ {i} (Z _ {i})\right) ^ {\mathbb {1} \{W _ {i} = 1 \}} \left(1 - \eta_ {i} (Z _ {i})\right) ^ {\mathbb {1} \{W _ {i} = 0 \}}}{\prod_ {i = 1} ^ {t} \left(\hat {\pi} _ {t}\right) ^ {\mathbb {1} \{W _ {i} = 1 \}} \left(1 - \hat {\pi} _ {t}\right) ^ {\mathbb {1} \{W _ {i} = 0 \}}}, \quad t \geq 1,
$$
it follows that  $\tilde{\mathcal{K}}_t\leq \mathcal{K}_t,\forall t\geq 1$  , meaning that  $(\tilde{\mathcal{K}}_t)_{t\geq 0}$  is a process that is upper bounded by a nonnegative martingale with initial value one. This in turn implies that a test based on  $(\tilde{\mathcal{K}}_t)_{t\geq 0}$  is a valid level-  $\alpha$  sequential 2ST for the case of unknown class proportions. This idea underlies the running MLE sequential likelihood ratio test of Wasserman et al. [2020] and has been recently considered in the context of two-sample testing by Pandeva et al. [2022]. In case of nontrivial betting fractions:  $(\lambda_t)_{t\geq 1}$  , representation of the wealth process (38) no longer holds, and to proceed, we modify the rules of the game and use minibatching. A bet is placed on every  $b$  (say, 5 or 10) observations, meaning
that for a given minibatch size  $b \geq 1$ , at round  $t$  we bet on  $\{(Z_{b(t-1)+i}, W_{b(t-1)+i})\}_{i \in \{1, \ldots, b\}}$ . The MLE of  $\pi$  computed from the  $t$ -th minibatch is
$$
\hat {\pi} _ {t} = \frac {1}{b} \sum_ {i = b (t - 1) + 1} ^ {b t} \mathbb {1} \left\{W _ {i} = + 1 \right\}.
$$
We consider a payoff function of the following form:
$$
\left. f _ {t} ^ {\mathrm {u}} \left(\left\{\left(Z _ {b (t - 1) + i}, W _ {b (t - 1) + i}\right) \right\} _ {i \in \{1, \dots , b \}}\right) = \prod_ {i = b (t - 1) + 1} ^ {b t} \left(\frac {1 + W _ {i} g _ {t} \left(Z _ {i}\right)}{1 + W _ {i} \left(2 \hat {\pi} _ {t} - 1\right)}\right) - 1. \right. \tag {39}
$$
In words, the above payoff essentially compares the performance of a predictor  $g_{t}$ , trained on  $\{(Z_{i},W_{i})\}_{i\leq b(t - 1)}$  and evaluated on the  $t$ -th minibatch, to that of a trivial baseline predictor to form a bet. In particular, setting  $b = 1$  yields a valid, yet a powerless test. Indeed, we have  $\hat{\pi}_t = \mathbb{1}\left\{W_t = 1\right\} = (W_t + 1) / 2$ . In this case, the payoff (39) reduces to
$$
\frac {W _ {t} \left(g _ {t} \left(Z _ {t}\right) - \left(2 \hat {\pi} _ {t} - 1\right)\right)}{1 + W _ {t} \left(2 \hat {\pi} _ {t} - 1\right)} = \frac {W _ {t} g _ {t} \left(Z _ {t}\right) - 1}{2} \enspace \enspace \enspace \enspace \enspace \enspace \enspace \enspace \enspace \enspace \enspace \enspace \enspace \enspace \enspace \enspace \enspace \enspace \enspace \enspace \enspace \enspace \enspace \enspace \enspace \enspace \enspace \enspace \enspace \enspace \enspace \enspace \enspace \enspace a. s. \enspace [ - 1, 0 ],
$$
implying that the wealth can not grow even if the null is false. Define a wealth processes  $(\mathcal{K}_t^{\mathrm{u}})_{t\geq 0}$  based on the payoff functions (39) along with a predictable sequence of betting fractions  $(\lambda_{t})_{t\geq 1}$  selected via ONS strategy (Algorithm 1). Let  $\mathcal{F}_t = \sigma (\{(Z_i,W_i)\}_{i\leq bt})$  for  $t\geq 1$ , with  $\mathcal{F}_0$  denoting a trivial sigma-algebra. We conclude with the following result, whose proof is deferred to Appendix D.5.
Theorem 5. Suppose that  $H_0$  in (35a) is true. Then  $(\mathcal{K}_t^{\mathrm{u}})_{t\geq 0}$  is a nonnegative supermartingale adapted to  $(\mathcal{F}_t)_{t\geq 0}$ . Hence, the sequential 2ST based on  $(\mathcal{K}_t^{\mathrm{u}})_{t\geq 0}$  satisfies:  $\mathbb{P}_{H_0}(\tau < \infty)\leq \alpha$ .
C Testing under Distribution Drift
First, we define the problem of two-sample testing when at each round instances from both distributions are observed.
Definition 5 (Sequential two-sample testing). Suppose that we observe that a stream of observations:  $((X_{t},Y_{t}))_{t\geq 1}$ , where  $(X_{t},Y_{t})\stackrel {\mathrm{iid}}{\sim}P_{X}\times P_{Y}$  for  $t\geq 1$ . The goal is to design a sequential test for
$$
H _ {0}: \left(X _ {t}, Y _ {t}\right) \stackrel {\text {i i d}} {\sim} P _ {X} \times P _ {Y} \text {a n d} P _ {X} = P _ {Y}, \tag {40a}
$$
$$
H _ {1}: \left(X _ {t}, Y _ {t}\right) \stackrel {\text {i i d}} {\sim} P _ {X} \times P _ {Y} \text {a n d} P _ {X} \neq P _ {Y}. \tag {40b}
$$
Under the two-sample testing setting (Definition 5), we label observations from  $P_{Y}$  as positive (+1) and observations from  $P_{X}$  as negative (-1). We write  $\mathcal{A}_{\mathrm{c}}^{2\mathrm{ST}}: (\cup_{t \geq 1} (\mathcal{X} \times \{-1, +1\})^t) \times \mathcal{G} \to \mathcal{G}$  to denote a chosen learning algorithm which maps a training dataset of any size and previously used predictor, to an updated predictor. We start with  $\mathcal{D}_0 = \emptyset$  and  $g_1: g_1(x) = 0, \forall x \in \mathcal{X}$ . At round  $t$ , we bet using derandomized versions of the payoffs (22), namely