SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Alessio Mazzetto
Eli Upfal
We develop and analyze a general technique for learning with an unknown distribution drift. Given a sequence of independent observations from the last  $T$  steps of a drifting distribution, our algorithm agnostically learns a family of functions with respect to the current distribution at time  $T$ . Unlike previous work, our technique does not require prior knowledge about the magnitude of the drift. Instead, the algorithm adapts to the sample data. Without explicitly estimating the drift, the algorithm learns a family of functions with almost the same error as a learning algorithm that knows the magnitude of the drift in advance. Furthermore, since our algorithm adapts to the data, it can guarantee a better learning error than an algorithm that relies on loose bounds on the drift. We demonstrate the application of our technique in two fundamental learning scenarios: binary classification and linear regression.
Standard statistical learning models (such as PAC learning) assume independent and identically distributed training set, and evaluate the performance of their algorithms with respect to the same distribution as the training set [Vapnik, 1998, van de Geer, 2000, Shalev-Shwartz and Ben-David, 2014, Wainwright, 2019]. However, in many practical applications, such as weather forecast, finance prediction or consumer preference analysis, the training data is drawn from a non-stationary distribution that drifts in time. In this work, we consider a more general setting where the samples are still independent, but their distribution can change over time. To obtain accurate results, the learning algorithm needs to adjust to the distribution drift occurring in the input.
This framework has been extensively studied in the literature [Helmbold and Long, 1991, Bartlett, 1992, Helmbold and Long, 1994, Barve and Long, 1996, 1997]. This line of research culminated in showing that as long as the total variation distance of two consecutive distributions is bounded by  $\Delta$ , there exists an algorithm that agnostically learns a family of binary classifiers with VC dimension  $\nu$  with expected error  $O((\nu \Delta)^{1/3})$  [Long, 1998], which can be shown to be tight. These results were generalized in the work of Mohri and Muñoz Medina [2012] to address any family of functions with bounded Rademacher complexity and to use a finer problem-dependent distance between distributions called discrepancy, originally introduced in the context of domain adaptation [Mansour et al., 2009].
The core idea of the aforementioned work is to learn by using a number of previous samples that minimizes the trade-off between the error due to the variance of the estimation (statistical error), and the error due to the drifting of the samples with respect to the current distribution (drift error). If the algorithm trains using only a few recent observations, the statistical error will be large. If the algorithm uses a larger training set, including not very recent observations, the drift error will be large. For example, if the algorithm uses the most recent  $r$  training point, the hypothesis class has VC-dimension  $\nu$ , and the distribution drift in each step is bounded by  $\Delta$ , then the statistical error is  $O(\sqrt{\nu / r})$  and the drift error is  $O(r\Delta)$ . The trade-off with respect to  $r$  is optimized for  $r = \Theta((\Delta^2 /\nu)^{-1 / 3})$  giving  $O((\nu \Delta)^{1 / 3})$  error, as mentioned before. This, as well as similar approaches
in the literature, requires an upper bound to the magnitude of the drift and resolves the trade-off between the statistical and drift errors based on this knowledge. As noted in previous work [Hanneke and Yang, 2019], it is an open problem to develop an algorithm that adapts to the training set and does not rely on prior knowledge about the drift, whose solution would lead to the practical applicability of those ideas.
Our work resolves this open problem. Our algorithm does not require any prior knowledge of the magnitude of the drift, and it adapts based on the input data. Without explicitly estimating the drift (which is often impossible), the algorithm agnostically learns a family of functions with the same error guarantee as an algorithm that knows the exact magnitude of the drift in advance. Our approach has two advantages: it eliminates the, often unrealistic, requirement of having a bound on the drift, and it gives better results when the drift bounds are not tight. We showcase our algorithm in two important learning settings: binary classification and linear regression.
Let  $(\mathcal{Z},\mathcal{A})$  be a measurable space. Let  $Z_{1},\ldots ,Z_{T}$  be a sequence of mutually independent random variables on  $\mathcal{Z}$  distributed according to  $P_{1},\ldots ,P_{T}$  respectively, i.e.  $Z_{t}\sim P_{t}$  for  $t\leq T$ . For  $r\leq T$  we denote by  $P_T^r$  the average distribution of the most recent  $r$  distributions  $P_{T - r + 1},\dots ,P_T$ :
$$
P _ {T} ^ {r} (A) \doteq \frac {1}{r} \sum_ {t = T - r + 1} ^ {T} P _ {t} (A) \quad \forall A \in \mathcal {A}.
$$
We set  $\mathbb{P}_T^r$  to be the corresponding empirical distribution over the random variables  $Z_{T - r + 1},\ldots ,Z_T$ :
$$
\mathbb {P} _ {T} ^ {r} (A) \doteq \frac {\left| \left\{Z _ {t} \in A : T - r + 1 \leq t \leq T \right\} \right|}{r} \quad \forall A \in \mathcal {A}.
$$
Let  $\mathcal{F}$  be a class of measurable functions from  $\mathcal{Z}$  to  $\mathbb{R}$ . For  $f\in \mathcal{F}$ , and any distributions  $P,Q$  on  $(\mathcal{Z},\mathcal{A})$ , we let
$$
P (f) \doteq \underset {Z \sim P} {\mathbb {E}} f (Z) = \int f d P, \quad \| P - Q \| _ {\mathcal {F}} \doteq \sup  _ {f \in \mathcal {F}} | P (f) - Q (f) |.
$$
The norm  $\| \cdot \|_{\mathcal{F}}$  is a notion of discrepancy introduced by the work of Mohri and Muñoz Medina [2012] to quantify the error due to the distribution shift with respect to a family of functions  $\mathcal{F}$ , and it is based on previous work on domain adaptation [Mansour et al., 2009].
The goal is to estimate  $P_T(f)$  for all  $f \in \mathcal{F}$  using the random variables  $Z_1, \ldots, Z_T$ . Let  $1 \leq r \leq T$ , and suppose that we do this estimate by considering the empirical values induced by the most recent  $r$  random variables. Then, by using triangle inequality, we have the following decomposition
$$
\mathbb {E} \| P _ {T} - \mathbb {P} _ {T} ^ {r} \| _ {\mathcal {F}} \leq \mathbb {E} \underbrace {\| P _ {T} ^ {r} - \mathbb {P} _ {T} ^ {r} \| _ {\mathcal {F}}} _ {\text {s t a t i s t i c a l e r r o r}} + \underbrace {\| P _ {T} - P _ {T} ^ {r} \| _ {\mathcal {F}}} _ {\text {d r i f t e r r o r}}. \tag {1}
$$
The first term of the upper bound is the expected statistical error of the estimation, and quantifies how accurately the empirical values  $\{\mathbb{P}_T^r (f):f\in \mathcal{F}\}$  approximate the expectation of each function  $f$  according to  $P_T^r$ . The second term  $\| P_T - P_T^r\|_{\mathcal{F}}$  of the upper bound represents the drift error. Intuitively, the statistical error decreases by considering more samples, whereas the drift error can potentially increase since we are considering distributions that are further away from our current distribution. We are looking for the value of  $r$  that balances this trade-off between statistical error and drift error.
2.1 Statistical Error
Since our results revolve around learning a class of functions  $\mathcal{F}$ , we first need to assume that  $\mathcal{F}$  is "learnable", i.e. the statistical error can be bounded as a function of  $r$ . For concreteness, we use the following standard assumption that the family of  $\mathcal{F}$  satisfies the standard machine learning uniform convergence requirement with rate  $O(1 / \sqrt{r})$ .
Assumption 1. (Uniform Convergence). There exists non-negative constants  $C_{\mathcal{F},1}$  and  $C_{\mathcal{F},2}$  such that for any fixed  $r \leq T$  and  $\delta \in (0,1)$ , it holds
$$
\mathbb {E} \| P _ {T} ^ {r} - \mathbb {P} _ {T} ^ {r} \| _ {\mathcal {F}} \leq \frac {C _ {\mathcal {F} , 1}}{\sqrt {r}} ,
$$
and with probability at least  $1 - \delta$ , we have
$$
\| P _ {T} ^ {r} - \mathbb {P} _ {T} ^ {r} \| _ {\mathcal {F}} \leq \frac {C _ {\mathcal {F} , 1}}{\sqrt {r}} + C _ {\mathcal {F}, 2} \sqrt {\frac {\ln (1 / \delta)}{r}}.
$$
The learnability of a family of functions  $\mathcal{F}$  is an extensively studied topic in the statistical learning literature (e.g., [Bousquet et al., 2003, Wainwright, 2019]). For a family of binary functions, the above assumption is equivalent to  $\mathcal{F}$  having a finite VC-dimension, in which case  $C_{\mathcal{F},1} = O(\nu)$  and  $C_{\mathcal{F},2} = O(1)$ . For a general family of functions  $\mathcal{F}$ , a sufficient requirement is that the Rademacher complexity of the first  $r$  samples is  $O(r^{-1/2})$  and the range of any function in  $\mathcal{F}$  is uniformly bounded. There is nothing special about the rate  $O(r^{-1/2})$ . It is possible to adapt our analysis to any rate  $O(r^{-\alpha})$  with  $\alpha \in (0,1)$  by modifying the constants of our algorithm.
2.2 Quantifying the Drift Error
While for many classes  $\mathcal{F}$ , it is possible to provide an upper bound to the statistical error by using standard statistical learning theory tools, the drift error is unknown and challenging to estimate. The literature used different approaches to quantify the drift error. By using triangle inequality, it is possible to show that for any  $r \leq T$ , we have the following upper bounds to the drift error:
$$
\| P _ {T} - P _ {T} ^ {r} \| _ {\mathcal {F}} \leq \frac {1}{r} \sum_ {t = 1} ^ {r} \| P _ {T} - P _ {T - t} \| _ {\mathcal {F}} \leq \max  _ {t <   r} \| P _ {T} - P _ {T - t} \| _ {\mathcal {F}} \leq \sum_ {t = 1} ^ {r - 1} \| P _ {T - t} - P _ {T - t + 1} \| _ {\mathcal {F}}. \tag {2}
$$
In a long line of research (e.g., [Bartlett, 1992, Long, 1998, Mohri and Muñoz Medina, 2012, Hanneke and Yang, 2019]), it is assumed that an upper bound to the drift error is known apriori. One of the most used assumption is that there exists a known upper bound  $\Delta$  to the discrepancy between any two consecutive distributions, in which case we can upper bound  $\sum_{t=1}^{r-1} \| P_{T-t} - P_{T-t+1} \|_{\mathcal{F}}$  with  $r\Delta$ . In this case, for a binary family  $\mathcal{F}$  with VC-dimension  $\nu$ , we obtain that:
$$
\mathbb {E} \| P _ {T} - P _ {T} ^ {r} \| _ {\mathcal {F}} \lesssim \sqrt {\frac {\nu}{r}} + r \Delta ,
$$
and we can choose the value of  $r$  that minimizes this upper bound. Since these algorithms rely on an unrealistic assumption that an upper bound to the drift is known a priori, they are not usable in practice. It is an open problem to provide a competitive algorithm that can choose  $r$  adaptively and it is oblivious to the magnitude of the drift.
Another sequence of work [Mohri and Muñoz Medina, 2012, Awasthi et al., 2023] relaxes the problem setting assuming that the algorithm can observe multiple samples from each distribution  $P_{1}, \ldots, P_{T}$ . In this case, they can provably estimate the discrepancies between different distributions, and compute a weighting of the samples that minimizes a trade-off between the statistical error and the estimated discrepancies. This strategy does not apply to our more general setting, as we can have access to at most one sample from each distribution.
Surprisingly, we show that we can adaptively choose the value of  $r$  to minimize the trade-off between statistical error and drift error without explicitly estimating the discrepancy. Noticeably, our method does not require any additional assumption on the drift. The only requirement for our algorithm is that we can compute the norm  $\| \cdot \|_{\mathcal{F}}$  from a set of samples. This is formalized as follows.
Assumption 2. (Computability). There exists a procedure that computes  $\| \mathbb{P}_T^r -\mathbb{P}_T^{2r}\|_{\mathcal{F}}$  for any  $r\leq T / 2$
In general, the hardness of computing the norm  $\| \cdot \|_{\mathcal{F}}$  depends on the family  $\mathcal{F}$ . This challenge is also common in previous work that uses this norm to quantify the distribution drift [Mansour et al., 2009, Awasthi et al., 2023]. In this paper, we provide two examples of important learning settings where this assumption is satisfied: binary classification with zero-one loss and linear regression with squared loss.
3 Main Result
Under those assumption, we prove the following theorem, which is our main result.
Theorem 1. Let  $\delta \in (0,1)$ . Let Assumptions 1 and 2 hold. Given  $Z_{1},\ldots,Z_{T}$ , Algorithm 1 outputs a value  $\hat{r} \leq T$  such that with probability at least  $1 - \delta$ , it holds that
$$
\| P _ {T} - \mathbb {P} _ {T} ^ {\hat {r}} \| _ {\mathcal {F}} = O \left(\min  _ {r \leq T} \left[ \frac {C _ {\mathcal {F} , 1}}{\sqrt {r}} + \max  _ {t <   r} \| P _ {T} - P _ {T - t} \| _ {\mathcal {F}} + C _ {\mathcal {F}, 2} \sqrt {\frac {\log (\log (r + 1) / \delta)}{r}} \right]\right)
$$
In order to appreciate this theorem, we can observe the following. If we learn using the most recent  $r$  samples, similarly to (1) we have the following error decomposition
$$
\| P _ {T} - \mathbb {P} _ {T} ^ {r} \| _ {\mathcal {F}} \leq \| P _ {T} ^ {r} - \mathbb {P} _ {T} ^ {r} \| _ {\mathcal {F}} + \| P _ {T} - P _ {T} ^ {r} \| _ {\mathcal {F}}
$$
By using Assumption 1 and (2), we have that with probability at least  $1 - \delta$  it holds that
$$
\left\| P _ {T} - \mathbb {P} _ {T} ^ {r} \right\| _ {\mathcal {F}} \leq \frac {C _ {\mathcal {F} , 1}}{\sqrt {r}} + C _ {\mathcal {F}, 2} \sqrt {\frac {\log (1 / \delta)}{r}} + \max  _ {t <   r} \left\| P _ {T} - P _ {T - t} \right\| _ {\mathcal {F}}. \tag {3}
$$
Theorem 1 guarantees a learning error that is essentially within a multiplicative constant factor as good as the upper bound obtained by selecting the optimal choice of  $r$  in (3). This result provides an affirmative answer to the open problem posed by Hanneke and Yang [2019], that asked if it was possible to adaptively choose the value of  $r$  that minimizes (3)  $^{1}$ . The upper bound of the theorem contains a negligible additional factor  $\log (r + 1)$  within the logarithm due to the union bound required to consider multiple candidate values of  $r$ . As further evidence of the efficiency of our algorithm, assume that there is no drift, and we are in the usual i.i.d. setting where  $P_{1} = \ldots = P_{T}$ . In this setting, the following corollary immediately follows from Theorem 1 by setting the drift error equal to 0.
Corollary 2. Consider the setting of Theorem 1, and also assume that  $P_{1} = \ldots = P_{T}$ . With probability at least  $1 - \delta$ , the window size  $\hat{r}$  computed by Algorithm 1 satisfies
$$
\| P _ {T} - \mathbb {P} _ {T} ^ {\hat {r}} \| _ {\mathcal {F}} = O \left(\frac {C _ {\mathcal {F} , 1}}{\sqrt {T}} + C _ {\mathcal {F}, 2} \sqrt {\frac {\log (\log (T + 1) / \delta)}{T}}\right)
$$
Observe that in the i.i.d. case, Assumption 1 implies that with probability at least  $1 - \delta$ , it holds  $\|P_T - \mathbb{P}_T^T\|_{\mathcal{F}} \leq C_{\mathcal{F},1} / \sqrt{T} + C_{\mathcal{F},2} \sqrt{\log(1 / \delta) / T}$ . Corollary 2 shows that with our algorithm we obtain a result that is competitive except for a negligible extra factor  $O(\log T)$  within the logarithm. We want to emphasize that our algorithm does not know in advance whether the data is i.i.d. or drifting, and this factor is the small cost of our algorithm to adapt between those two cases.
4 Algorithm
We describe an algorithm that attains the results of Theorem 1. Throughout the remaining of this section, we let Assumptions 1 hold. In particular, the algorithm has access to constants  $C_{\mathcal{F},1}$  and  $C_{\mathcal{F},2}$  that satisfy this assumption. The main challenge is the fact that the drift error  $\| P_T - \mathbb{P}_T^r\|_{\mathcal{F}}$  is unknown for any  $r > 1$ , and it is challenging to quantify since we have only a single sample  $Z_T \sim P_T$ .
We first provide an informal description of the algorithm. Our algorithm revolves around the following strategy. We do not try to estimate the drift. Instead, we try to assess whether increasing the sample size can yield a better upper bound on the error. Recall that given the most recent  $r$  samples, we have the following upper bound on the error,
$$
\mathbb {E} \| P _ {T} - \mathbb {P} _ {T} ^ {r} \| _ {\mathcal {F}} \leq \frac {C _ {\mathcal {F} , 1}}{\sqrt {r}} + \| P _ {T} - P _ {T} ^ {r} \| _ {\mathcal {F}}. \tag {4}
$$
The algorithm cannot evaluate (4) since the drift component,  $\| P_T - P_T^r\|_{\mathcal{F}}$ , is unknown. Our key idea is to compare the difference in the upper bound on the error when using the latest  $2r$  or  $r$  samples:
$$
\left(\frac {C _ {\mathcal {F} , 1}}{\sqrt {2 r}} - \frac {C _ {\mathcal {F} , 1}}{\sqrt {r}}\right) + \left(\| P _ {T} - P _ {T} ^ {2 r} \| _ {\mathcal {F}} - \| P _ {T} - P _ {T} ^ {r} \| _ {\mathcal {F}}\right) \leq \left(\frac {C _ {\mathcal {F} , 1}}{\sqrt {2 r}} - \frac {C _ {\mathcal {F} , 1}}{\sqrt {r}}\right) + \| P _ {T} ^ {r} - P _ {T} ^ {2 r} \| _ {\mathcal {F}} \tag {5}
$$
<sup>1</sup>We prove a stronger result. The original formulation of the question uses the looser upper bound  $\sum_{t=1}^{r-1} \| P_{T-t} - P_{T-t+1} \|_{\mathcal{F}}$  to the drift error in (3), which can be significantly worse as shown in an example of Section 5
The last step follows from the triangle inequality:  $\| P_T - P_T^{2r} \|_{\mathcal{F}} \leq \| P_T - P_T^r \|_{\mathcal{F}} + \| P_T^r - P_T^{2r} \|_{\mathcal{F}}$ . By considering the latest  $2r$  samples rather than  $r$  samples, we can see from (5) that the statistical error decreases, and the difference in drift error can be upper bounded by  $\| P_T^{2r} - P_T^r \|_{\mathcal{F}}$ . We can estimate  $\| P_T^r - P_T^{2r} \|_{\mathcal{F}}$  using  $\| \mathbb{P}_T^r - \mathbb{P}_T^{2r} \|_{\mathcal{F}}$  within expected error  $O(C_{\mathcal{F},1} / \sqrt{r})$  as it depends on  $\geq r$  samples. This suggests the following algorithm. Let  $r$  be the current sample size considered by the algorithm. The algorithm starts from  $r$  equal to 1, and doubles the sample size as long as  $\| \mathbb{P}_T^r - \mathbb{P}_T^{2r} \|_{\mathcal{F}}$  is small. If  $\| \mathbb{P}_T^r - \mathbb{P}_T^{2r} \|_{\mathcal{F}}$  is big enough, a substantial drift must have occurred in the distributions  $P_{T-2r+1}, \ldots, P_T$ , and we can show that this implies that  $\max_{t < 2r-1} \| P_T - P_{T-t} \|_{\mathcal{F}}$  is also large. When this happens, we can stop our algorithm and return the current sample size.
For a formal description of the algorithm, let  $\delta \in (0,1)$  denote the probability of failure of our algorithm, and let  $r_i = 2^i$ , for  $i \geq 0$ , be the size of the training set used by the algorithm at iteration  $i + 1$ . For ease of notation, we set
$$
C _ {\mathcal {F}, \delta} \doteq C _ {\mathcal {F}, 1} + C _ {\mathcal {F}, 2} \sqrt {2 \ln (\pi^ {2} / (6 \delta))},
$$
$$
S (r, \delta) \doteq \frac {C _ {\mathcal {F} , \delta}}{\sqrt {r}} + C _ {\mathcal {F}, 2} \sqrt {\frac {2 \ln (\log_ {2} (r) + 1 0)}{r}}. \tag {6}
$$
The proofs of the following propositions appear in the Appendix A.
Proposition 3. With probability at least  $1 - \delta$ , the following event holds:
$$
\| P _ {T} ^ {r _ {i}} - \mathbb {P} _ {T} ^ {r _ {i}} \| _ {\mathcal {F}} \leq S (r _ {i}, \delta) \quad \forall i \geq 0
$$
We assume that the event of Proposition 3 holds, otherwise our algorithm fails (with probability  $\leq \delta$ ). Our algorithm considers the following inflated upper bound  $U(r,\delta)$  to  $\| \mathbb{P}_T^r - P_T\|_{\mathcal{F}}$  defined as follows
$$
U (r, \delta) \doteq 2 1 \cdot S (r, \delta) + \| P _ {T} - P _ {T} ^ {r} \| _ {\mathcal {F}}. \tag {7}
$$
Proposition 3 implies that with probability at least  $1 - \delta$  for any  $i\geq 0$ , we have
$$
U (r _ {i}, \delta) = 2 1 \cdot S (r _ {i}, \delta) + \| P _ {T} - P _ {T} ^ {r _ {i}} \| _ {\mathcal {F}} \geq \| \mathbb {P} _ {T} ^ {r _ {i}} - P _ {T} ^ {r _ {i}} \| _ {\mathcal {F}} + \| P _ {T} - P _ {T} ^ {r _ {i}} \| _ {\mathcal {F}} \geq \| P _ {T} - \mathbb {P} _ {T} ^ {r _ {i}} \| _ {\mathcal {F}}
$$
We use this value as an upper bound that our algorithm guarantees if we choose a sample size  $r_i$ . We observe that with respect to (4), the upper bound of the algorithm also contains an additional term that is proportional to  $\sqrt{\log(1 / \delta)}$  for the high-probability guarantee, and a term proportional to  $\sqrt{\log\log r_i}$  that is necessary for the union bound across all possible window sizes  $r_i$  for  $i \geq 0$ . This union bound is required as we need to have a correct estimation for all possible sample sizes  $r_i$  in order to assure that the algorithm takes a correct decision at each step. The constant factor in front of the upper bound on the statistical error  $S(r_i, \delta)$  is a technical detail that allows taking into account the error of the estimation of the difference in drift error.
We want to compare the upper bound  $U(r_i, \delta)$  with the upper bound  $U(r_{i+1}, \delta)$  obtained by doubling the current sample size  $r_i$ . As we previously discussed, it is possible to show that if  $\| \mathbb{P}_T^{r_i} - \mathbb{P}_T^{r_{i+1}} \|_{\mathcal{F}}$  is sufficiently small, then  $U(r_{i+1}, \delta) \leq U(r_i, \delta)$ , and this intuition is formalized in the following proposition.
Proposition 4. Assume that the event of Proposition 3 holds and let  $i \geq 0$ .
$$
I f \left\| \mathbb {P} _ {T} ^ {r _ {i}} - \mathbb {P} _ {T} ^ {r _ {i + 1}} \right\| _ {\mathcal {F}} \leq 4 S (r _ {i}, \delta) t h e n U (r _ {i + 1}, \delta) \leq U (r _ {i}, \delta).
$$
The algorithm works as follows. Starting from  $i = 0$ , we iteratively increase  $i$  by one while  $\| \mathbb{P}_T^{r_i} - \mathbb{P}_T^{r_{i+1}} \|_{\mathcal{F}} \leq 4S(r_i, \delta)$ . There are two cases. In the first case, we reach  $r_i = \lfloor \log_2 T \rfloor$ , and this implies that  $r_{i+1} > T$ . In this case, the algorithm returns  $r_i$ , and Proposition 4 guarantees that the sample size returned by the algorithm is as good as any previously considered sample size. In the second case, we reach a value of  $i$  such that  $\| \mathbb{P}_T^{r_i} - \mathbb{P}_T^{r_{i+1}} \|_{\mathcal{F}} > 4S(r_i, \delta)$ . In this case, we return  $r_i$ , and we can still prove that this is a good choice. In fact, as shown in the next proposition, this terminating condition implies a lower bound  $\max_{t < r_{i+1}} \| P_T - P_{T-i} \|_{\mathcal{F}}$ , thus any estimation with a number of recent samples greater or equal to  $r_{i+1}$  could have a non-negligible drift error.
Algorithm 1: Adaptive Learning Algorithm under Drift
$i\gets 0$    
while  $r_i\leq T / 2$  do if  $\| \mathbb{P}_T^{r_i} - \mathbb{P}_T^{r_i + 1}\|_{\mathcal{F}}\leq 4S(r_i,\delta)$  then  $i\gets i + 1$  end else return  $r_i$  end   
end   
return  $r_i$
Proposition 5. Assume that the event of Proposition 3 holds, and assume that there exists  $i \geq 0$  such that  $\| \mathbb{P}_T^{r_i} - \mathbb{P}_T^{r_i + 1}\|_{\mathcal{F}} > 4S(r_i,\delta)$ , then  $\max_{t < r_{i + 1}} \| P_T - P_{T - t}\|_{\mathcal{F}} > S(r_i,\delta)$ .
The pseudo-code of the algorithm is reported in Algorithm 1. The algorithm receives in input  $\delta$ , the samples  $Z_{1},\ldots,Z_{T}$ , and returns an integer  $\hat{r} \in \{1,\dots,T\}$  that satisfies Theorem 1.
Proof of Theorem 1. We assume that the event of Proposition 3 holds. If it doesn't, we say that our algorithm fails, and this happens with probability  $\leq \delta$ . Let  $\hat{r} = r_{j} = 2^{j}$  for  $j \geq 0$  be the value returned by the algorithm when it terminates. We remind that our algorithm guarantees an upper bound  $U(r_{j}, \delta) \geq \|P_{T} - \mathbb{P}_{T}^{r_{j}}\|_{\mathcal{F}}$  to the learning error by using  $r_{j}$  samples. Let  $r^{*}$  be the value of  $r$  that minimizes this expression
$$
r ^ {*} = \operatorname * {a r g m i n} _ {r \leq T} \left(2 1 \cdot S (r, \delta) + \max  _ {t <   r} \| P _ {T} - P _ {T - t} \| _ {\mathcal {F}}\right),
$$
and let  $B^{*}$  be the minimum value of this expression, i.e.