Title: An Adaptive Algorithm for Learning with Unknown Distribution Drift
Venue: NeurIPS 2023
Accepted: yes
Keywords: statistical learning

learning theory

machine learning

supervised learning

non-stationary

transfer learning

distribution drift
Abstract: We develop and analyze a general technique for learning with an unknown distribution drift. Given a sequence of independent observations from the last $T$ steps of a drifting distribution, our algorithm agnostically learns a family of functions with respect to the current distribution at time $T$. Unlike previous work, our technique does not require prior knowledge about the magnitude of the drift. Instead, the algorithm adapts to the sample data. Without explicitly estimating the drift, the algorithm learns a family of functions with almost the same error as a learning algorithm that knows the magnitude of the drift in advance. Furthermore, since our algorithm adapts to the data, it can guarantee a better learning error than an algorithm that relies on loose bounds on the drift. We demonstrate the application of our technique in two fundamental learning scenarios: binary classification and linear regression.

An Adaptive Algorithm for Learning with Unknown Distribution Drift

Abstract

We develop and analyze a general technique for learning with an unknown distribution drift. Given a sequence of independent observations from the last T steps of a drifting distribution, our algorithm agnostically learns a family of functions with respect to the current distribution at time T . Unlike previous work, our technique does not require prior knowledge about the magnitude of the drift. Instead, the algorithm adapts to the sample data. Without explicitly estimating the drift, the algorithm learns a family of functions with almost the same error as a learning algorithm that knows the magnitude of the drift in advance. Furthermore, since our algorithm adapts to the data, it can guarantee a better learning error than an algorithm that relies on loose bounds on the drift. We demonstrate the application of our technique in two fundamental learning scenarios: binary classification and linear regression.

1 Introduction

Standard statistical learning models (such as PAC learning) assume independent and identically distributed training set, and evaluate the performance of their algorithms with respect to the same distribution as the training set [Vapnik, 1998, van de Geer, 2000, Shalev-Shwartz and Ben-David, 2014, Wainwright, 2019]. However, in many practical applications, such as weather forecast, finance prediction or consumer preference analysis, the training data is drawn from a non-stationary distribution that drifts in time. In this work, we consider a more general setting where the samples are still independent, but their distribution can change over time. To obtain accurate results, the learning algorithm needs to adjust to the distribution drift occurring in the input.

This framework has been extensively studied in the literature [Helmbold and Long, 1991, Bartlett, 1992, Helmbold and Long, 1994, Barve and Long, 1996, 1997]. This line of research culminated in showing that as long as the total variation distance of two consecutive distributions is bounded by ∆ , there exists an algorithm that agnostically learns a family of binary classifiers with VC dimension ν with expected error O (( ν ∆) 1 / 3 ) [Long, 1998], which can be shown to be tight. These results were generalized in the work of Mohri and Mu˜ noz Medina [2012] to address any family of functions with bounded Rademacher complexity and to use a finer problem-dependent distance between distributions called discrepancy, originally introduced in the context of domain adaptation [Mansour et al., 2009].

The core idea of the aforementioned work is to learn by using a number of previous samples that minimizes the trade-off between the error due to the variance of the estimation ( statistical error ), and the error due to the drifting of the samples with respect to the current distribution ( drift error ). If the algorithm trains using only a few recent observations, the statistical error will be large. If the algorithm uses a larger training set, including not very recent observations, the drift error will be large. For example, if the algorithm uses the most recent r training point, the hypothesis class has VC-dimension ν , and the distribution drift in each step is bounded by ∆ , then the statistical error is O ( √ ν/r ) and the drift error is O ( r ∆) . The trade-off with respect to r is optimized for r = Θ((∆ 2 /ν ) -1 / 3 ) giving O (( ν ∆) 1 / 3 ) error, as mentioned before. This, as well as similar approaches

in the literature, requires an upper bound to the magnitude of the drift and resolves the trade-off between the statistical and drift errors based on this knowledge. As noted in previous work [Hanneke and Yang, 2019], it is an open problem to develop an algorithm that adapts to the training set and does not rely on prior knowledge about the drift, whose solution would lead to the practical applicability of those ideas.

Our work resolves this open problem. Our algorithm does not require any prior knowledge of the magnitude of the drift, and it adapts based on the input data. Without explicitly estimating the drift (which is often impossible), the algorithm agnostically learns a family of functions with the same error guarantee as an algorithm that knows the exact magnitude of the drift in advance. Our approach has two advantages: it eliminates the, often unrealistic, requirement of having a bound on the drift, and it gives better results when the drift bounds are not tight. We showcase our algorithm in two important learning settings: binary classification and linear regression.

2 Preliminary

Let ( Z , A ) be a measurable space. Let Z 1 , . . . , Z T be a sequence of mutually independent random variables on Z distributed according to P 1 , . . . , P T respectively, i.e. Z t ∼ P t for t ≤ T . For r ≤ T , we denote by P r T the average distribution of the most recent r distributions P T -r +1 , . . . , P T :

We set P r T to be the corresponding empirical distribution over the random variables Z T -r +1 , . . . , Z T :

Let F be a class of measurable functions from Z to R . For f ∈ F , and any distributions P, Q on ( Z , A ) , we let

The norm ∥·∥ F is a notion of discrepancy introduced by the work of Mohri and Mu˜ noz Medina [2012] to quantify the error due to the distribution shift with respect to a family of functions F , and it is based on previous work on domain adaptation [Mansour et al., 2009].

The goal is to estimate

P

T

(

f

)

for all

f

∈ F

using the random variables

Z

1

, . . . , Z

T

. Let

1

r

T

,

and suppose that we do this estimate by considering the empirical values induced by the most recent

r

random variables. Then, by using triangle inequality, we have the following decomposition

The first term of the upper bound is the expected statistical error of the estimation, and quantifies how accurately the empirical values { P r T ( f ) : f ∈ F} approximate the expectation of each function f according to P r T . The second term ∥ P T -P r T ∥ F of the upper bound represents the drift error . Intuitively, the statistical error decreases by considering more samples, whereas the drift error can potentially increase since we are considering distributions that are further away from our current distribution. We are looking for the value of r that balances this trade-off between statistical error and drift error.

2.1 Statistical Error

Since our results revolve around learning a class of functions F , we first need to assume that F is 'learnable', i.e. the statistical error can be bounded as a function of r . For concreteness, we use the following standard assumption that the family of F satisfies the standard machine learning uniform convergence requirement with rate O (1 / √ r ) .

Assumption 1. (Uniform Convergence). There exists non-negative constants C F , 1 and C F , 2 such that for any fixed r ≤ T and δ ∈ (0 , 1) , it holds

and with probability at least 1 -δ , we have

The learnability of a family of functions F is an extensively studied topic in the statistical learning literature (e.g., [Bousquet et al., 2003, Wainwright, 2019]). For a family of binary functions, the above assumption is equivalent to F having a finite VC-dimension, in which case C F , 1 = O ( ν ) and C F , 2 = O (1) . For a general family of functions F , a sufficient requirement is that the Rademacher complexity of the first r samples is O ( r -1 / 2 ) and the range of any function in F is uniformly bounded. There is nothing special about the rate O ( r -1 / 2 ) . It is possible to adapt our analysis to any rate O ( r -α ) with α ∈ (0 , 1) by modifying the constants of our algorithm.

2.2 Quantifying the Drift Error

While for many classes F , it is possible to provide an upper bound to the statistical error by using standard statistical learning theory tools, the drift error is unknown and challenging to estimate. The literature used different approaches to quantify the drift error. By using triangle inequality, it is possible to show that for any r ≤ T , we have the following upper bounds to the drift error:

In a long line of research (e.g., [Bartlett, 1992, Long, 1998, Mohri and Mu˜ noz Medina, 2012, Hanneke and Yang, 2019]), it is assumed that an upper bound to the drift error is known apriori. One of the most used assumption is that there exists a known upper bound ∆ to the discrepancy between any two consecutive distributions, in which case we can upper bound ∑ r -1 t =1 ∥ P T -t -P T -t +1 ∥ F with r ∆ . In this case, for a binary family F with VC-dimension ν , we obtain that:

and we can choose the value of r that minimizes this upper bound. Since these algorithms rely on an unrealistic assumption that an upper bound to the drift is known a priori, they are not usable in practice. It is an open problem to provide a competitive algorithm that can choose r adaptively and it is oblivious to the magnitude of the drift.

Another sequence of work [Mohri and Mu˜ noz Medina, 2012, Awasthi et al., 2023] relaxes the problem setting assuming that the algorithm can observe multiple samples from each distribution P 1 , . . . , P T . In this case, they can provably estimate the discrepancies between different distributions, and compute a weighting of the samples that minimizes a trade-off between the statistical error and the estimated discrepancies. This strategy does not apply to our more general setting, as we can have access to at most one sample from each distribution.

Surprisingly, we show that we can adaptively choose the value of r to minimize the trade-off between statistical error and drift error without explicitly estimating the discrepancy. Noticeably, our method does not require any additional assumption on the drift. The only requirement for our algorithm is that we can compute the norm ∥·∥ F from a set of samples. This is formalized as follows.

Assumption 2. (Computability). There exists a procedure that computes ∥ P r T -P 2 r T ∥ F for any r ≤ T/ 2 .

In general, the hardness of computing the norm ∥·∥ F depends on the family F . This challenge is also common in previous work that uses this norm to quantify the distribution drift [Mansour et al., 2009, Awasthi et al., 2023]. In this paper, we provide two examples of important learning settings where this assumption is satisfied: binary classification with zero-one loss and linear regression with squared loss.

3 Main Result

Under those assumption, we prove the following theorem, which is our main result.

Theorem 1. Let δ ∈ (0 , 1) . Let Assumptions 1 and 2 hold. Given Z 1 , . . . , Z T , Algorithm 1 outputs a value ˆ r ≤ T such that with probability at least 1 -δ , it holds that

In order to appreciate this theorem, we can observe the following. If we learn using the most recent r samples, similarly to (1) we have the following error decomposition

By using Assumption 1 and (2), we have that with probability at least 1 -δ it holds that

Theorem 1 guarantees a learning error that is essentially within a multiplicative constant factor as good as the upper bound obtained by selecting the optimal choice of r in (3). This result provides an affirmative answer to the open problem posed by Hanneke and Yang [2019], that asked if it was possible to adaptively choose the value of r that minimizes (3) 1 . The upper bound of the theorem contains a negligible additional factor log( r +1) within the logarithm due to the union bound required to consider multiple candidate values of r . As further evidence of the efficiency of our algorithm, assume that there is no drift, and we are in the usual i.i.d. setting where P 1 = . . . = P T . In this setting, the following corollary immediately follows from Theorem 1 by setting the drift error equal to 0 .

Corollary 2. Consider the setting of Theorem 1, and also assume that P 1 = . . . = P T . With probability at least 1 -δ , the window size ˆ r computed by Algorithm 1 satisfies

Observe that in the i.i.d. case, Assumption 1 implies that with probability at least 1 -δ , it holds ∥ P T -P T T ∥ F ≤ C F , 1 / √ T + C F , 2 √ log(1 /δ ) /T . Corollary 2 shows that with our algorithm we obtain a result that is competitive except for a negligible extra factor O (log T ) within the logarithm. We want to emphasize that our algorithm does not know in advance whether the data is i.i.d. or drifting, and this factor is the small cost of our algorithm to adapt between those two cases.

4 Algorithm

We describe an algorithm that attains the results of Theorem 1. Throughout the remaining of this section, we let Assumptions 1 hold. In particular, the algorithm has access to constants C F , 1 and C F , 2 that satisfy this assumption. The main challenge is the fact that the drift error ∥ P T -P r T ∥ F is unknown for any r > 1 , and it is challenging to quantify since we have only a single sample Z T ∼ P T .

We first provide an informal description of the algorithm. Our algorithm revolves around the following strategy. We do not try to estimate the drift. Instead, we try to assess whether increasing the sample size can yield a better upper bound on the error. Recall that given the most recent r samples, we have the following upper bound on the error,

The algorithm cannot evaluate (4) since the drift component, ∥ P T -P r T ∥ F , is unknown. Our key idea is to compare the difference in the upper bound on the error when using the latest 2 r or r samples:

The last step follows from the triangle inequality: ∥ P T -P 2 r T ∥ F ≤ ∥ P T -P r T ∥ F + ∥ P r T -P 2 r T ∥ F . By considering the latest 2 r samples rather than r samples, we can see from (5) that the statistical error decreases, and the difference in drift error can be upper bounded by ∥ P 2 r T -P r T ∥ F . We can estimate ∥ P r T -P 2 r T ∥ F using ∥ P r T -P 2 r T ∥ F within expected error O ( C F , 1 / √ r ) as it depends on ≥ r samples. This suggests the following algorithm. Let r be the current sample size considered by the algorithm. The algorithm starts from r equal to 1 , and doubles the sample size as long as ∥ P r T -P 2 r T ∥ F is small. If ∥ P r T -P 2 r T ∥ F is big enough, a substantial drift must have occurred in the distributions P T -2 r +1 , . . . , P T , and we can show that this implies that max t< 2 r -1 ∥ P T -P T -t ∥ F is also large. When this happens, we can stop our algorithm and return the current sample size.

For a formal description of the algorithm, let δ ∈ (0 , 1) denote the probability of failure of our algorithm, and let r i = 2 i , for i ≥ 0 , be the size of the training set used by the algorithm at iteration i +1 . For ease of notation, we set

The proofs of the following propositions appear in the Appendix A.

Proposition 3. With probability at least 1 -δ , the following event holds:

We assume that the event of Proposition 3 holds, otherwise our algorithm fails (with probability ≤ δ ). Our algorithm considers the following inflated upper bound U ( r, δ ) to ∥ P r T -P T ∥ F defined as follows

Proposition 3 implies that with probability at least 1 -δ for any i ≥ 0 , we have

We use this value as an upper bound that our algorithm guarantees if we choose a sample size r i . We observe that with respect to (4), the upper bound of the algorithm also contains an additional term that is proportional to √ log(1 /δ ) for the high-probability guarantee, and a term proportional to √ log log r i that is necessary for the union bound across all possible window sizes r i for i ≥ 0 . This union bound is required as we need to have a correct estimation for all possible sample sizes r i in order to assure that the algorithm takes a correct decision at each step. The constant factor in front of the upper bound on the statistical error S ( r i , δ ) is a technical detail that allows taking into account the error of the estimation of the difference in drift error.

Wewant to compare the upper bound U ( r i , δ ) with the upper bound U ( r i +1 , δ ) obtained by doubling the current sample size r i . As we previously discussed, it is possible to show that if ∥ P r i T -P r i +1 T ∥ F is sufficiently small, then U ( r i +1 , δ ) ≤ U ( r i , δ ) , and this intuition is formalized in the following proposition.

Proposition 4. Assume that the event of Proposition 3 holds and let i ≥ 0 .

The algorithm works as follows. Starting from i = 0 , we iteratively increase i by one while ∥ P r i T -P r i +1 T ∥ F ≤ 4 S ( r i , δ ) . There are two cases. In the first case, we reach r i = ⌊ log 2 T ⌋ , and this implies that r i +1 > T . In this case, the algorithm returns r i , and Proposition 4 guarantees that the sample size returned by the algorithm is as good as any previously considered sample size. In the second case, we reach a value of i such that ∥ P r i T -P r i +1 T ∥ F > 4 S ( r i , δ ) . In this case, we return r i , and we can still prove that this is a good choice. In fact, as shown in the next proposition, this terminating condition implies a lower bound max t<r i +1 ∥ P T -P T -i ∥ F , thus any estimation with a number of recent samples greater or equal to r i +1 could have a non-negligible drift error.

and

Algorithm 1: Adaptive Learning Algorithm under Drift

Proposition 5. Assume that the event of Proposition 3 holds, and assume that there exists i ≥ 0 such that ∥ P r i T -P r i +1 T ∥ F > 4 S ( r i , δ ) , then max t<r i +1 ∥ P T -P T -t ∥ F > S ( r i , δ ) .

The pseudo-code of the algorithm is reported in Algorithm 1. The algorithm receives in input δ , the samples Z 1 , . . . , Z T , and returns an integer ˆ r ∈ { 1 , . . . , T } that satisfies Theorem 1.

Proof of Theorem 1. We assume that the event of Proposition 3 holds. If it doesn't, we say that our algorithm fails, and this happens with probability ≤ δ . Let ˆ r = r j = 2 j for j ≥ 0 be the value returned by the algorithm when it terminates. We remind that our algorithm guarantees an upper bound U ( r j , δ ) ≥ ∥ P T -P r j T ∥ F to the learning error by using r j samples. Let r ∗ be the value of r that minimizes this expression

and let B ∗ be the minimum value of this expression, i.e.

be any fixed optimal sample size, where we remind the definition of U from (7). The first observation is that the right-hand side of the inequality in Theorem 1 is O ( B ∗ ) . Therefore, in order to prove the theorem, it is sufficient to show that U ( r j , δ ) /B ∗ = O (1) .

We distinguish two cases: ( a ) r ∗ < 2 r j , and ( b ) r ∗ ≥ 2 r j . We first consider case ( a ) . We let k ≥ 0 be the largest integer such that r k ≤ r ∗ . Since r ∗ < 2 r j , it holds that k ≤ j . Since our algorithm returned r j , Proposition 4 applies for i = 0 , . . . , j -1 , thus U ( r j , δ ) ≤ U ( r k , δ ) . We have that:

We observe that

where the first inequality is due to (2), and the last inequality is due to the fact that r k ≤ r ∗ < 2 r k by definition of r k . By using the above inequality in (8), we obtain that U ( r j , δ ) /B ∗ ≤ 3 .

We consider case ( b ) . Since T ≥ r ∗ ≥ 2 r j = r j +1 , the algorithm returned r j because ∥ P r j T -P r j +1 T ∥ F > 4 S ( r j , δ ) . Using Proposition 5, we have

Therefore, we have that

This concludes the proof.

5 Binary Classification with Distribution Drift

In this section, we show an application of Theorem 1 for the fundamental statistical learning problem of agnostic learning a family of binary classifiers. Let Z = X × Y , where X is the feature space, and Y = { 0 , 1 } is the label space, i.e. Z t = ( X t , Y t ) . A hypothesis class H is a class of functions h : X ↦→ Y that classify the feature space X . Given a point ( x, y ) ∈ X × Y and a function h ∈ H , the risk of h on ( x, y ) is defined through the following function L h ( x, y ) = 1 { y = h ( x ) } . We work with the class of functions F = { L h : h ∈ H} .

̸

Let h ∗ = argmin h ∈H P T ( L h ) be a function with minimum expected risk with respect to the current distribution P T . We want to use Theorem 1 to find a function h ∈ H such that P T ( L h ) is close to P T ( L h ∗ ) . Let ν be the VC dimension of H . The VC dimension describes the complexity of the family H , and it is used to quantify the statistical error. In particular, using standard learning tools, it is possible to show that the family F satisfies Assumption 1 on the sample complexity with constants C F , 1 = O ( √ ν ) and C F , 2 = O (1) (e.g., [Mohri et al., 2018]).

Finally, to satisfy Assumption 2, we need to exhibit a procedure that for r ≤ T/ 2 , outputs the quantity ∥ P r T -P 2 r T ∥ F . This quantity is also referred to as Y -discrepancy between the empirical distributions P r T and P 2 r T in previous work on transfer learning [Mohri and Mu˜ noz Medina, 2012]. We can adapt a strategy from Ben-David et al. [2010] to our setting and show that it is possible to compute it by solving an empirical risk minimization problem. We say that a hypothesis class H is computationally tractable if given a finite set of points from X × Y , there exists an algorithm that returns a hypothesis h that achieves the minimum risk over this set of points.

Lemma 6. Assume that H is symmetric, i.e. h ∈ H ⇐⇒ 1 -h ∈ H . For r ≤ T/ 2 , it holds

Given r , the minimum in Lemma 6 can be computed by solving an empirical risk minimization over the most recent 2 r points, where we flip the label of half of those points, i.e. we use the points ( X T -2 r +1 , Y T -2 r +1 ) , . . . , ( X T -r , Y T -r ) , ( X T -r +1 , 1 -Y T -r +1 ) , . . . , ( X T , 1 -Y T ) . Thus, if H is computationally tractable and symmetric, Assumption 2 holds. This is indeed true for many hypothesis class, e.g., hyperplanes, axis-aligned rectangles, or threshold functions. However, the empirical risk minimization problem could be expensive to solve exactly, but as we discuss in Section 8, it is possible to modify the algorithm to allow for an approximation of ∥ P r T -P 2 r T ∥ F .

Our main result for binary classification is given in the following theorem:

Theorem 7. Let H be a computationally tractable and symmetric binary class with VC dimension ν . Let F = { L h : h ∈ H} . Let ˆ r ≤ T be output of Algorithm 1 with input Z 1 , . . . , Z T using the family F . Let ˆ h = argmin h ∈H P ˆ r T ( L h ) be an empirical risk minimizer over the most recent ˆ r samples. With probability at least 1 -δ , the following holds:

The symmetry assumption is not necessary, and in the appendix we show how to remove it at the cost of a more expensive computation of the discrepancy. It is instructive to compare this upper bound with the results of previous work. An often used assumption in the literature, originally introduced in Bartlett [1992], is that there is a known value ∆ > 0 , such that drift in each step is bounded by ∆ , i.e. for all t < T , ∥ P t +1 -P t ∥ F ≤ ∆ . Assume that T is sufficiently large, i.e. T = Ω(( ν/ ∆ 2 ) 1 / 3 ) . By using this assumption on the drift, previous work showed that with high-probability, they can find a classifier ˆ h such that P T ( L ˆ h ) -P T ( L h ∗ ) = O ( 3 √ ∆ ν ) , and it can be shown that this upper bound is tight up to constants within those assumptions [Barve and Long, 1996]. These previous works assumed they had access a priori to the value of ∆ , since those algorithms compute ˆ h by solving a empirical risk minimization over a number of previous samples Θ(( ν/ ∆ 2 ) 1 / 3 ) that is decided before observing the data. On the other hand, this assumption on the drift together with (2) implies that max t<r ∥ P T -P T -t ∥ F ≤ ( r -1)∆ . Our algorithm (Theorem 7) guarantees an error that depends on a minimum choice over r ≤ T , hence it is always smaller than

the one obtained with a specific choice of r . If we choose r = ( ν/ ∆ 2 ) 1 / 3 in the upper bound of Theorem 7, we can show that with high-probability, our algorithm returns a classifier ˆ h such that P T ( L ˆ h ) -P T ( L h ∗ ) = O ( 3 √ ∆ ν + 6 √ ∆ 2 /ν √ log log( ν/ ∆ 2 ) ) . Our algorithm achieves this guarantee while being adaptive with respect to the upper bound ∆ , and it can indeed guarantee a better result when this upper bound is loose.

For example, assume an extreme case in which the algorithm is given a ∆ > 0 bound on the drift in each step, but the training set has actually no drift, it was all drawn from the distribution P T . Previous methods are oblivious to the actual data, and they guarantee an upper bound O ( 3 √ ∆ ν ) with high-probability, since they decide the sample size Θ(( ν/ ∆ 2 ) 1 / 3 ) a priory without observing the input. In contrast, our algorithm adapts to this scenario, and Theorem 7 guarantees that we obtain an O ( √ ν/T + √ (log log T ) /T ) error, with high-probability, essentially retrieving the error guarantee for learning with independent and identically distributed samples. Observe that our upper bound depends on T , and it goes to 0 when T →∞ .

It is possible to show that our algorithm can obtain asymptotically better guarantee even if ∥ P t -P t -1 ∥ F = ∆ ≪ 1 for all t < T , i.e. there is an exact drift of ∆ at each step. This is because our algorithm provides a guarantee as a function of max t<r ∥ P T -P T -t ∥ F rather than the looser quantity ∑ r -1 t =1 ∥ P T -t -P T -t +1 ∥ F , as shown in the following example. Let X = [0 , 1] , and let H be a class of threshold functions over [0 , 1] i.e. for any c ∈ [0 , 1] , there exists classifiers h c , h ′ c ∈ H such that h c ( x ) = 1 if and only if x ≥ c and h ′ c ( x ) = 1 if and only if x < c . The class H has VC-dimension equal to 2 . We construct a sequence of distributions P 1 , . . . , P T as follows. The marginal distribution over X is uniform for each distribution P t with t = 1 , . . . , T . At time t ≤ T , the classification of x ∈ [0 , 1] is given by a function ℓ t : [0 , 1] ↦→ { 0 , 1 } . Assume that there exists two disjoint intervals I 0 = [ p 0 , p 0 + ∆ / 2) and I 1 = [ p 1 , p 1 + ∆ / 2) such that ℓ T ( x ) = 0 for all x ∈ I 0 and ℓ T ( x ) = 1 for all x ∈ I 1 . We let

In particular, ℓ T -t ( x ) differs from ℓ T ( x ) for x ∈ I 0 if t is odd, and for x ∈ I 1 if t is even. By construction, ∥ P t -P t -1 ∥ F = ∆ for all t < T , i.e. there is an exact drift of ∆ at each step (to be precise, in the last step ∥ P T -P T -1 ∥ F = ∆ / 2 ). As discussed before, with the assumption of a bounded drift ∆ at each step, previous methods guarantee an upper bound O ( 3 √ ∆) with highprobability. However, we also have that for any 1 ≤ r ≤ T , it holds by construction that

Hence, our algorithm (Theorem 7) guarantees with high-probability an upper bound

Since our algorithm is adaptive with respect to the drift, it can correctly use more samples. In contrast, previous non-adaptive algorithms that rely on the assumption of bounded drift ∆ at each step choose a sample size of Θ(∆ -2 / 3 ) based on this assumption, thus they can only guarantee a looser bound of O (∆ 1 / 3 ) , even when this assumption is satisfied with equality.

It is possible to use the recent lower bound strategy of Mazzetto and Upfal [2023] in order to show that the upper bound of Theorem 7 is essentially tight in a minimax sense.

Theorem 8. Let H be a binary class with VC dimension ν , and consider an arbitrary nondecreasing sequence ∆ 1 = 0 , ∆ 2 , . . . , ∆ T of non-negative real numbers. Let F = { L h : h ∈ H} . Let A be any algorithm that observes Z 1 , . . . , Z T , and it outputs a classifier h A ∈ H . If

then, for any algorithm A , there exists a sequence of distributions P 1 , . . . , P T such that max t<r ∥ P T -P T -t ∥ F ≤ ∆ r for any r ≤ T , and with probability at least 1 / 8 it holds that:

6 Linear Regression with Squared Loss

In the previous section, we showed an application of Theorem 1 for the problem of binary classification with zero-one loss. In this section, we show that our main result can also be applied to a linear regression problem. Similarly to the previous section, we let Z = X × Y , where X is the feature space and Y = [ -1 , 1] is the label space, i.e. Z t = ( X t , Y t ) . In this section, we constrain the feature space X = { x ∈ R d : ∥ x ∥ 2 ≤ 1 } to be the unit ball centered at the origin in R d .

We consider the ℓ 2 regularized linear prediction class H = { x ↦→ ⟨ x, w ⟩ : w ∈ R d ∧ ∥ w ∥ 2 ≤ 1 } . We denote each predictor h w ∈ H with its weight w . To evaluate the quality of a prediction y = h w ( x ) , we use the squared loss L : R × R ↦→ R defined as L ( y, y ′ ) = ( y -y ′ ) 2 . For each h w ∈ H , we let L w ( x, y ) = L ( h w ( x ) , y ) be the function that evaluates the loss incurred by h w for any z = ( x, y ) ∈ Z . We work with the family of functions F = { L w : h w ∈ H} . By a standard uniform convergence argument based on the Rademacher complexity of F (e.g., see [Kakade et al., 2008, Shamir, 2015, Awasthi et al., 2020]), we have that for any 1 ≤ r ≤ T , it holds:

thus we satisfy Assumption 1 with C F , 1 = O (1) and C F , 2 = O (1) .

The computation of the discrepancy is more challenging. Let 1 ≤ r ≤ T/ 2 . Using the definition of F , we have that:

Let c r ∈ R , b r ∈ R d , and A r ∈ R d × d be defined as follows:

and observe that the matrix A r is symmetric. Using those definition, we can manipulate the righthand side of (9) to show that it is equivalent to:

In order to compute (10), it is sufficient to be able to solve the minimization problem

where A ∈ R d × d is a symmetric matrix, and b ∈ R d . This minimization problem has been extensively studied for the trust-region method [Conn et al., 2000], and Hager [2001, Section 2] provides a way to compute the solution of (11) in terms of a diagonalization of A . Thus, we also satisfy Assumption 2, and we can obtain the following result as an immediate corollary of Theorem 1.

Theorem 9. Let δ ∈ (0 , 1) . Let 1 ≤ ˆ r ≤ T be the output of Algorithm 1 with input Z 1 , . . . , Z T and using the family F described in this section. Let ˆ w = argmin w : ∥ w ∥≤ 1 P ˆ r T ( L w ) be the linear classifier with minimum loss over the most recent ˆ r samples. Then, with probability at least 1 -δ :

where w ∗ = argmin w : ∥ w ∥≤ 1 P T ( L w ) is the linear predictor with minimum loss with respect to the current distribution P T .

7 Related Work

Additional variants of learning with distribution drift have been studied in the literature. Freund and Mansour [1997] provide a refined learning algorithm in the special case of rapid distribution shift with a constant direction of change. In the work of Bartlett et al. [2000], they show specialized bounds in the case of infrequent changes and other different restrictions on the distribution drift. The work of Crammer et al. [2010] provides regret bound for online learning with an adversarial bounded drift. Yang [2011] studies the problem of active learning in a distribution drift setting. Hanneke et al. [2015] provide an efficient polynomial time algorithm to learn a class of linear separators with a drifting target concept under the uniform distribution in the realizable setting. Interestingly, they also show how to adapt their algorithm with respect to an unknown drift, although their technique relies on the realizability of the learning problem. In the more recent work of Hanneke and Yang [2019], they relax the independence assumption and provide learning guarantees for a sequence of random variables that is both drifting and mixing.

8 Conclusion, Limitations and Future Directions

We present a general learning algorithm that adapts to an unknown distribution drift in the training set. Unlike previous work, our technique does not require prior knowledge about the magnitude of the drift. For the problem of binary classification, we show that without explicitly estimating the drift, there exists an algorithm that learns a binary classifier with the same or better error bounds compared to the state-of-the-art results that rely on prior information about the magnitude of the drift. This is a major step toward practical solutions to the problem since prior knowledge about the distribution drift in the training set is often hard to obtain.

We presented concrete results for binary classification and linear regression, but our technique can be applied for learning any family of functions F as long as it is possible to compute the distance ∥ P r T -P 2 r T ∥ F between the empirical distributions with r and 2 r samples according to the ∥·∥ F norm (Assumption 2). This is often a challenging problem, and it is related to the computation of the discrepancy between distributions, which was studied in previous work on transfer learning [Mansour et al., 2009, Ben-David et al., 2010]. For binary classification, we assume that the empirical risk minimization problem is tractable. However, an exact solution to this problem is computationally hard for many hypothesis classes of interest, and this is a limitation of our algorithm and previous work on transfer learning. In those cases, we can modify our analysis to use the best-known approximation for the distance ∥ P r T -P 2 r T ∥ F as long as there is an approximation guarantee with respect to its exact value. As an illustrative example, if we have a procedure that returns an approximation E r such that 1 ≤ ∥ P r T -P 2 r T ∥ F /E r ≤ α for any r ≥ 1 , then it is possible to change the algorithm to obtain a guarantee that is at the most a factor O ( α 2 ) worse than the one achieved by Algorithm 1 with the exact computation of the distance (we refer to Appendix B for additional details).

The method presented here uses a distribution-independent upper bound for the statistical error. While this upper bound can be tight in the worst-case, as shown in our lower bound for binary classification (Theorem 8), it can be loose for some other sequence of distributions. As a future direction, it is an interesting problem to provide an adaptive algorithm with respect to the drift that uses distribution-dependent upper bounds, for example, based on the Rademacher complexity, which can be possibly computed from the input data. Our algorithm does not naturally extend to this setting, as it requires knowing the rate at which the upper bound on the statistical error is decreasing (see proof of Proposition 4).

Acknowledgements. This material is based on research sponsored by Defense Advanced Research Projects Agency (DARPA) and Air Force Research Laboratory (AFRL) under agreement number FA8750-19-2-1006 and by the National Science Foundation (NSF) under award IIS-1813444. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of Defense Advanced Research Projects Agency (DARPA) and Air Force Research Laboratory (AFRL) or the U.S. Government.

References

Pranjal Awasthi, Natalie Frank, and Mehryar Mohri. On the rademacher complexity of linear hypothesis sets. arXiv preprint arXiv:2007.11045 , 2020.

Pranjal Awasthi, Corinna Cortes, and Christopher Mohri. Theory and algorithm for batch distribution drift problems. In International Conference on Artificial Intelligence and Statistics (AISTATS) , 2023.

Peter L Bartlett. Learning with a slowly changing distribution. In Proceedings of the fifth annual workshop on Computational Learning Theory (COLT) , 1992.

Peter L Bartlett, Shai Ben-David, and Sanjeev R Kulkarni. Learning changing concepts by exploiting the structure of change. Machine Learning , 41(2):153-174, 2000.

Rakesh D Barve and Philip M Long. On the complexity of learning from drifting distributions. In Conference on Computational Learning Theory (COLT) , 1996.

Rakesh D Barve and Philip M Long. On the complexity of learning from drifting distributions. Information and Computation , 138(2):170-193, 1997.

Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. A theory of learning from different domains. Machine learning , 79(1):151-175, 2010.

Olivier Bousquet, St´ ephane Boucheron, and G´ abor Lugosi. Introduction to statistical learning theory. In Summer School on Machine Learning . Springer, 2003.

Andrew R Conn, Nicholas IM Gould, and Philippe L Toint. Trust region methods . SIAM, 2000.

Koby Crammer, Eyal Even-Dar, Yishay Mansour, and Jennifer Wortman Vaughan. Regret minimization with concept drift. Conference on Learning Theory (COLT) , 2010.

Yoav Freund and Yishay Mansour. Learning under persistent drift. In European Conference on Computational Learning Theory , pages 109-118, 1997.

William W Hager. Minimizing a quadratic over a sphere. SIAM Journal on Optimization , 12(1): 188-208, 2001.

Steve Hanneke and Liu Yang. Statistical learning under nonstationary mixing processes. In International Conference on Artificial Intelligence and Statistics (AISTATS) , 2019.

Steve Hanneke, Varun Kanade, and Liu Yang. Learning with a drifting target concept. In Conference on Algorithmic Learning Theory (ALT) , 2015.

David P Helmbold and Philip M Long. Tracking drifting concepts using random examples. In Proceedings of the fourth annual workshop on Computational Learning Theory (COLT) , 1991.

David P Helmbold and Philip M Long. Tracking drifting concepts by minimizing disagreements. Machine learning , 14(1):27-45, 1994.

Sham M Kakade, Karthik Sridharan, and Ambuj Tewari. On the complexity of linear prediction: Risk bounds, margin bounds, and regularization. Advances in Neural Information Processing Systems (NeurIPS) , 2008.

Philip M Long. The complexity of learning according to two models of a drifting environment. In Conference on Computational Learning Theory (COLT) , 1998.

Yishay Mansour, Mehryar Mohri, and Afshin Rostamizadeh. Domain adaptation: Learning bounds and algorithms. arXiv preprint arXiv:0902.3430 , 2009.

Alessio Mazzetto and Eli Upfal. Nonparametric density estimation under distribution drift. International Conference on Machine Learning (ICML) , 2023.

Mehryar Mohri and Andres Mu˜ noz Medina. New analysis and algorithm for learning with drifting distributions. In International Conference on Algorithmic Learning Theory (ALT) , 2012.

Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of machine learning . MIT press, 2018.

Shai Shalev-Shwartz and Shai Ben-David. Understanding machine learning: From theory to algorithms . Cambridge university press, 2014.

Ohad Shamir. The sample complexity of learning linear predictors with the squared loss. Journal of Machine Learning Research , 16:3475-3486, 2015.

Alexander B Tsybakov. Introduction to nonparametric estimation. In Springer Series in Statistics , 2008.

Sara van de Geer. Empirical Processes in M-estimation , volume 6. Cambridge University Press, 2000.

Vladimir N. Vapnik. Statistical Learning Theory . John Wiley & Sons, Inc., 1998.

Martin J Wainwright. High-dimensional statistics: A non-asymptotic viewpoint , volume 48. Cambridge University Press, 2019.

Liu Yang. Active learning with a drifting distribution. Advances in Neural Information Processing Systems (NeurIPS) , 2011.

Bin Yu. Assouad, fano, and le cam. Festschrift for Lucien Le Cam: research papers in probability and statistics , 1997.

A Deferred Proofs

Proof of Proposition 3. For any i ≥ 0 , let δ i = 6 δ/ [ π 2 ( i +10) 2 ] . By using Assumption 1, we have that with probability at least 1 -δ i it holds that

where in the second inequality we used the definition of C F ,δ and the fact that √ x + y ≤ √ x + √ y . Additionaly, the following equality holds

where in the last equality, we used the known fact that ∑ ∞ i =1 1 /i 2 = π 2 / 6 . Thus, if we take an union bound over all i ≥ 0 , we have with probability at least 1 -δ it holds that

The statement follows by observing that the right-hand side of the above inequality is equal to S ( r i , δ ) for any i ≥ 0 .

Proof of Proposition 4. . We have

By using the triangle inequality, we obtain

thus we have

We use the triangle inequality and Proposition 3 to show that

If we plug the above inequality in (12) and use the assumption that ∥ P r i T -P r i +1 T ∥ F ≤ 4 S ( r i , δ ) , we obtain

If we expand the above upper bound by using the definition of the function S , we have

where the last inequality follows since 22 -16 √ 2 ≤ 0 , and 22 -16 √ 2 √ ln( i +10) / ln( i +11) ≤ 0 . Thus, it holds that

and we can conclude that U ( r i +1 , δ ) ≤ U ( r i , δ ) .

Proof of Proposition 5. . Observe that by using the triangle inequality, it holds that

where in the second inequality we used Proposition 3. By assumption, we have that ∥ P r i T -P r i +1 T ∥ F ≥ 4 S ( r i +1 , δ ) , hence

Observe that by triangle inequality, we have that

We use (2) and obtain that

By combining the above inequality with (13), we finally obtain that max t<r i +1 ∥ P T -P T -t ∥ F ≥ S ( r i , δ ) .

Proof of Lemma 6. We remind that F = { L h : h ∈ H} . By using the definition of ∥·∥ F , we have

We can remove the absolute value by re-writing this expression as

Consider the first argument of the above maximum. We can observe that for any ( X,Y ) ∈ X × Y , it holds that L h ( X,Y ) + L h ( X, 1 -Y ) = 1 , thus we obtain that

Similarly, we can demonstrate that the second term of the maximum is equal to:

Therefore, to compute the discrepancy (14), it is sufficient to solve the two empirical risk minimization problems in (15) and (16). To obtain the final statement, we can observe that for any ( X,Y ) and h ∈ H , it holds that L h ( X,Y ) = L 1 -h ( X, 1 -Y ) . Thus, we can show that (16) is equivalent to:

Since H is symmetric, i.e. 1 -h ∈ H ⇐⇒ h ∈ H , we can conclude that (15) and (16) have the same value. This concludes the proof.

Proof of Theorem 7. Since H has VC-dimension ν , by a standard argument we have that the family F = { L h : h ∈ H} has VC-dimension upper bounded by 2 ν , thus it satisfies Assumption 1 on the sample complexity for uniform convergence with C F , 1 = O ( √ ν ) and C F , 2 = O (1) .

We can observe that since H is symmetric, Lemma 6 shows that we can compute ∥ P r T -P 2 r T ∥ F for any r ≥ 1 by solving an empirical risk minimization problem. Since H is computationally tractable, there exists a procedure that solves this problem, thus we also satisfy Assumption 2.

Remark: If the symmetry assumption does not hold, in the proof of Lemma 6 we show that we can still compute the discrepancy ∥ P r T -P 2 r T ∥ F by solving the two empirical risk minimization problems in (15) and (16).

Hence, we can use Algorithm 1 with the family F , and let ˆ r be the value returned by the algorithm. Theorem 1 guarantees that with probability at least 1 -δ , we have that

Now, we have that

where the first inequality is due to the definition of ˆ h . Therefore, using (17), we have that with probability at least 1 -δ , it holds

Proof of Theorem 9. The theorem is proven following the same structure as the proof of Theorem 7.

A.1 Lower Bound.

In this section, we prove the lower bound of Theorem 8. The proof structure is based on the work of Mazzetto and Upfal [2023]. We provide a simpler statement of their proof in our setting, and we remove the additional regularity assumption used in that work to characterize the drift error. We introduce the following notation.

We say that a distribution Q over Z n is a product distribution if it can be written as the product of n distributions over Z , i.e. Q = Q 1 × . . . × Q n . For any n ≥ 1 , we can observe that since the random

variables Z 1 , . . . , Z n are independent, their distribution can be described as a product distribution over Z n . Given two strings τ, τ ′ ∈ {-1 , 1 } n , we let hd( τ, τ ′ ) = 1 2 ∥ τ -τ ′ ∥ 1 be the Hamming distance between the two strings, i.e. the number of positions in which the two strings differ.

Let ν be the VC dimension of the hypothesis class H . We will construct a (later defined) family of product distributions Q = { Q ( τ ) = Q ( τ ) 1 × . . . × Q ( τ ) T : τ ∈ {-1 , 1 } ν } over Z T that are indexed by a string τ ∈ {-1 , 1 } ν . Intuitively, each distribution Q ( τ ) is a possible candidate for the distribution of the random variables Z 1 , . . . , Z T . We will show that for any algorithm A , there exists a product distribution Q ( τ ) such that the classifier h A computed by A using the samples Z 1 , . . . , Z T from Q ( τ ) has large expected error. The proof is based on Assouad's Lemma. We provide a statement of this lemma that is an adaptation of its classical statement to our setting [Yu, 1997].

Lemma 10 (Assouad's Lemma) . Let Q be defined as above. For any function g : Z T ↦→{-1 , 1 } ν , there exists τ ∈ {-1 , 1 } ν such that

Let ∆ 1 = 0 , ∆ 2 , . . . , ∆ T be the sequence defined in the statement of Theorem 8. We let Φ : { 1 , . . . , T } ↦→ R be the function

We let Φ ∗ = min r Φ( r ) , and we remind that we assume Φ ∗ < 1 / 3 in the statement of the Theorem. We build the family Q based on the following value:

Proposition 11. The following holds:

Proof. Let r ∗ ∈ { 1 , . . . , r } be a value such that Φ ∗ = Φ( r ∗ ) . The statement follows by exploiting the definition of ˜ r . We distinguish two cases. If ˜ r ≥ r ∗ , we have that

Conversely, if r ∗ > ˜ r , we have that

In the first inequality we used the fact that the sequence ∆ 1 , . . . , ∆ T is non-decreasing, and the inequality √ ν/ (˜ r +1) ≤ ∆ ˜ r +1 due to the definition ot ˜ r .

We define the family of product distributions Q as follows. Let X 1 , . . . , X ν be a shatter set for the hypothesis class H . We build the following family Q = { Q ( τ ) = Q [ τ ] 1 × . . . × Q [ τ ] T : τ ∈ {-1 , 1 } ν } of product distributions over Z T = ( X × Y ) T that are indexed by τ ∈ {-1 , 1 } ν . They are defined as follows:

Those distributions are well-defined. In fact, we have that for all t > T -˜ r :

where we used Proposition 11. Given a classifier h ∈ H and τ ∈ {-1 , 1 } ν , we remind that

̸

We can observe that for any classifier h ∈ H and t > T -˜ r , it holds by construction that:

and for any t ≤ T -˜ r , we have that

where the first and the second inequality are due to the definition of ˜ r , and the last inequality follows from the fact that the sequence ∆ 1 , . . . , ∆ T is non-decreasing. Hence, if we let F = { L h : h ∈ H} , it results that for τ and for any 1 ≤ r ≤ T , it holds that:

We also let L h ∗ be the minimum loss that is achieved by a function h ∈ H with respect to Q ( τ ) T , i.e. L τ h ∗ = argmin L h : h ∈H Q ( τ ) T ( L h ) . By using the family Q together with Assouad's Lemma, we can show the following.

Lemma 12. Let Q be defined as above. Let A : Z T ↦→ H be any algorithm that observes a sequence of elements T elements from Z , and it outputs a classifier h A . For any algorithm A , there exists τ ∈ {-1 , 1 } ν such that if the input Z 1 , . . . , Z n is sampled according to Q ( τ ) , then:

Proof. Let τ ∈ {-1 , 1 } ν . We can define τ A = ( 2 h A ( X 1 ) -1 , . . . , 2 · h A ( X ν ) -1 ) ∈ {-1 , 1 } ν . We have that:

̸

By construction of Q ( τ ) T , we can observe that

hence, we have the following relation

Observe that A can be seen as a map from Z T to {-1 , 1 } ν (i.e., the string τ A ) . Hence, we can apply Lemma 10: this implies that there exists τ ∈ {-1 , 1 } ν such that

We are left to evaluate the right-hand side of the above inequality. We can use the following known relations that hold for any two distributions P and Q over Z T [Tsybakov, 2008]:

where KL is the Kullback-Leibler divergence (we refer to the classic definition of those distances as in [Tsybakov, 2008]). Let τ ′ and τ ′′ be two strings in {-1 , 1 } ν that only differ in one coordinate. Let Ber( p ) be a Bernoulli distribution with parameter p ∈ [0 , 1] . We have that:

where the first equality is due to the factorization property of the KL-divergence, the second and third equality are due to the definition of the family Q . For any ϵ < 1 / 4 , it holds that

By plugging the above inequality in (21), and using the fact that ( x + y ) 2 ≤ 2 x 2 +2 y 2 , we obtain that

By using the definition of ˜ r , it holds that ∆ 2 ˜ r ˜ r ≤ ν . Thus, we we have that KL( Q ( τ ′ ) , Q ( τ ′′ ) ) ≤ 1 / 32 . If we use this inequality with (20), we have

Hence, we can conclude that there exists τ ∈ {-1 , 1 } ν such that

By using this Lemma, we can easily prove Theorem 8.

Proof of Theorem 8. . By Lemma 12, there exists τ such that if P t = Q ( τ ) t for all 1 ≤ t ≤ T , then the algorithm A with input Z 1 , . . . , Z T satisfies

Due to (18), we observe that the distributions P 1 , . . . , P T satisfy the assumption on the drift.

Let E = P T ( L h A ) -P T ( L h ∗ ) . Observe that due to the construction of Q ( τ ) and (19), we have that E ≤ Φ(˜ r ) / (8 √ 6) ≤ 3Φ ∗ / (8 √ 6) , where the last inequality is due to Proposition 11. Let α > 0 be a later defined value. We have that:

By using the above inequality together with (22), we finally obtain that:

We set α = 1 / (112 √ 6) and we finally obtain that

.

B Relaxing Assumption 2

Given a family F , it is possible that the exact computation of ∥ P r T -P 2 r T ∥ F is computationally hard. In this section, we show an example on how to relax Assumption 2 to allow an approximation of this quantity.

Let α ≥ 1 . We say that an algorithm A in an α -approximation procedure if given Z t -2 r +1 , . . . , Z T , it computes an estimate A ( Z T -2 r +1 , . . . , Z T ) such that

for any r ≤ T/ 2 . That is, the algorithm A does not compute the value of the supremum of the norm ∥·∥ F exactly, but it guarantees a constant factor approximation α . In this case, we can modify the algorithm as follows.

Algorithm 2: Adaptive Learning Algorithm under Drift with Approximation Procedure

Theorem 13. Let δ ∈ (0 , 1) . Let Assumptions 1 hold, and assume that there exists an α -approximation procedure for estimating ∥ P r T -P 2 r T ∥ F for any r ≤ T/ 2 . Given Z 1 , . . . , Z T , there exists an algorithm that outputs a value ˆ r ≤ T such that with high-probability, it holds that

Proof. The proof follows the same strategy as the one of the main theorem with slight modifications, and we discuss those changes. We replace the definition of U ( r, δ ) with

in order to take into account the additional error due to the approximation procedure. We can observe that

since A is an α -approximation procedure. We can follow the steps of Proposition 4 with those different constants to show an equivalent statement of this Proposition. On the other hand, we have that if

and Proposition 5 applies. Therefore, we can use the same proof strategy of Theorem 1 to prove Theorem 13

implies then

Peer reviews:
comments: SUMMARY:
The paper proposes an algorithm for environments with changing distributions without assuming a priori knowledge about the change in distributions. The proposed algorithm provides error bounds that decrease with the number of time steps.

STRENGTHS:
The idea of considering independent distributions with distribution shift is unexplored.

The algorithm itself is a solid technical contribution, and the theoretical results are strong

WEAKNESSES:
Numerical experiments are not included.

Assumptions need further justification.

QUESTIONS:
How strong is the assumption on the distributions? What scenarios satisfy such assumptions, are there any real examples?

The reviewer believes that the problem formulation needs further explanation (e.g., what is to be learned at each instant of time).

Numerical results could help to see the superiority of the presented method.

The references are a bit old. There are more recent related works: 

“Minimax Classification under Concept Drift with Multidimensional Adaptation and Performance Guarantees.”

“Adaptive online learning in dynamic environment.”

“Random feature based online multi-kernel learning in environments with unknown dynamics.”

LIMITATIONS:
Numerical experiments are not included.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-06-26

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper under considers the theoretical problem of determining the sliding window size for empirical risk minimization in the presence of unknown distribution changes. The proposed method aims to enable the learning of a classifier comparable to approaches that have prior knowledge of the distribution change magnitude.\n\nSTRENGTHS:\n+ The paper's motivation is commendable, as it aims to tackle the challenge of non-stationary sequential learning, specifically with unknown distribution shifts. This is an important problem, and the proposed technique could potentially contribute to its advancement.\n\n+ Although not evaluated extensively, the results appear reasonable. While I haven't reviewed the proofs in detail, they seem sound.\n\nWEAKNESSES:\n- The paper relies on computability assumptions for key variables, such as the distribution discrepancy (e.g., $\\|P^r_T - P^r_T\\|_{\\mathcal{F}}$) and the constants ($C_{\\mathcal{F},1}$ and $C_{\\mathcal{F},2}$). However, the paper lacks a discussion on when these assumptions hold, how solvable they are, or how accurately they can be estimated or upper bounded. In comparison to previous methods that estimate distribution shifts, it is unclear whether estimating these parameters would lead to a more practical algorithm. This aspect requires further analysis and discussion.\n\n- The authors claim that their proposed algorithm aims to provide a practical solution to handling unknown sequential distribution shifts without prior knowledge. However, the proposed algorithm introduces new parameters that are challenging to estimate or even tightly upper bound, such as the distribution discrepancy (assumption 2). While the new analysis techniques presented may contribute to addressing the problem, the overall contribution seems limited, as the proposed method essentially shifts the difficulty from estimating the distribution shift magnitude to estimating other parameters.\n\n- The paper lacks a thorough discussion on when Assumption 2 holds, especially in practical scenarios. This assumption is crucial, as the proposed method relies on estimating the distribution discrepancy and other constants that evolve the hypothesis class $\\mathcal{F}$. Without addressing the practicality of these estimations, the proposed method's feasibility remains questionable. It would be beneficial to provide a detailed comparison of the estimation difficulty for the parameters required in the proposed method compared to previous works.\n\nQUESTIONS:\nIt is important to explicitly discuss the computability assumptions made in the paper and discuss when and how they hold. This discussion should include considerations of solvability, accurate estimation, or upper bounding of variables such as the distribution discrepancy and the constants. So it is suggested to provide an in-depth analysis of Assumption 2 and other computability assumptions on the constants mentioned above and its applicability in practical scenarios.\n\nLIMITATIONS:\nLimitations are discussed in this work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

comments: SUMMARY:
The author propose a general algorithm to learn a family of functions with respect to the current distribution at time T. This algorithm achieve a drifting-instance-dependent bound without any prior knowledge of the drift. Based on this, the author further analyze a tractable algorithm on binary classifier.

STRENGTHS:
1. The paper solves an open problem of drifting distribution with unknown prior. The proof technique is clear and solid.
2. It's further analysis on binary classification is inspiring.

WEAKNESSES:
1. The error bounded is only measured on windows that end at T. Instead, a better goal would be to select a window from t1 to t2. As an example of why this problem could be more relevant, there might be large drifts nearby T but early distributions are close to P_T.

2. The techniques used in this paper is not very novel. Specifically, although the authors claim they do not directly estimate the drift error, it is still drift detection only with tolerance towards estimation error. The doubling trick is intuitive and not surprising here.

3. As the authors admitted in our paper, this algorithm is very general and might be computationally intractable in more complicated cases. I think the author could discuss the benefit of their algorithm versus existing work on cases beyond binary classification. They may do this by showing either further proofs or empirical evidence. Currently, it is not immediate to me the value of an algorithm that ignores assumptions on the magnitude of the drifts.

QUESTIONS:
Why is empirical estimate based on a window up to T better than empirical estimates based on any consecutive window?
What's the value in getting rid of prior knowledge on drift magnitude in practice or in theory?

LIMITATIONS:
Sufficient.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-06

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis research paper presents a straightforward algorithm designed to facilitate adaptive learning of models in the presence of distribution drift. The algorithm is specifically designed to adapt to changing data patterns without requiring any prior knowledge of the drift. Moreover, the paper provides a proven bound that guarantees the learning error of the algorithm. Overall, the paper is well-written. The simplicity of the algorithm contributes to its comprehensibility, making the results easily understandable. Additionally, the inclusion of a detailed illustration showcasing the algorithm's effectiveness in handling binary classification with distribution drift serves as a strong validation of the proposed methodology.\n\nI think this is a breakthrough work for learning under non i.i.d data as it does not require prior knowledge of the potential drift. However, this could also be one of the limits of this paper. The algorithm proposed is to adaptively find r that can achieve the best trade-off between the statistical and drift error. Drift sometimes might represent an upcoming arrival of a new distribution. The proposed algorithm might get a competitive average accuracy compared to those algorithms that know the magnitude of drift in advance. However, I am not sure if this proposed algorithm will perform equivalent good if we test it on every sequential portion of the data as well. For example, a classifier of labeling every instance as 0 could also achieve an accuracy of 90% on the data where 90% of instances are 0 class. In this case, I get very limited information from its good results.\n\nOverall, I think the proposed methodology provides a new way of designing the learning algorithms under drift.\n\nSTRENGTHS:\nOverall, I think the proposed methodology provides a new way of designing the learning algorithms under drift.\nI also agree that r would be an important factor during learning under drift. Therefore, adding this to the learning boundary, from my perspective, is a good start of conducting the following theoretical learning works that discuss the non-identically distributed problem. Theorem 1 could be a quite useful guidance for the following studies in this area.\n\nWEAKNESSES:\nAs I have mentioned in the summary, Theorem 1 only provides a very general bound. The r is considered as sequentially increased. However, I think the usage of Theorem 1 is very limited if we apply it to learning sequential data. As for the sequential data, there should be always new data coming and thus very possibly new distribution in the upcoming instances. It is less practical to design the r as monotonically increasing if you don't consider the magnitude of the drift. The Inequality in Algorithm 1 is always comparing P_t between it is at r_i and r_{i+1}. This means the i is only updated when the distribution drift is large enough among two consecutive i. However, this is not true. Therefore, this algorithm removes the assumption of knowing the magnitude of distribution drift in advance, but for me, it actually adds a hidden assumption that the magnitude of distribution drift should be enough big between at least one of two consecutive i in [1, T]. \n\nLet's think of the situation that the magnitude of distribution drift is incremental increasing by i. I am not sure if your algorithm can output the expected r in such a case. But for those algorithms that assume a known magnitude of drift in advance. They can complete this task. \n\nAs is also mentioned in this paper that an adaptive algorithm with respect to the drift that uses distribution-dependent upper bounds\n\nQUESTIONS:\nIn Theorem 1, r is the output of Algorithm 1. What will be the limits if using other algorithms?\n\nLIMITATIONS:\nSome of the limitations have been listed in section 7. As is discussed, satisfying Assumption 2 is challenging and authors already discuss possible solutions.\n\nFrom my perspective, the author claims their contributions as removing the prior knowledge of distribution drift. I appreciate their contribution. But as a reader, I am more curious if we do get some prior knowledge of the distribution drift, how can we use that knowledge to improve what you have presented in this work? I think this will provide more insights of this work.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}