Original: Adapting to Online Label Shift with Provable Guarantees
Revised: Adapting to Online Label Shift with Provable Guarantees

Original: Abstract
Revised: Abstract

Original: The standard supervised learning paradigm works effectively when training data shares the same distribution as the upcoming testing samples.
Revised: The standard supervised learning paradigm works effectively when training data shares the same distribution as the upcoming testing samples.

Original: However, this stationary assumption is often violated in real-world applications, especially when testing data appear in an online fashion.
Revised: However, this stationary assumption is often violated in real-world applications, especially when testing data appear in an online fashion.

Original: In this paper, we formulate and investigate the problem of online label shift (OLaS): the learner trains an initial model from the labeled offline data and then deploys it to an unlabeled online environment where the underlying label distribution changes over time but the label-conditional density does not.
Revised: In this paper, we formulate and investigate the problem of online label shift (OLaS): the learner trains an initial model from the labeled ofﬂine data and then deploys it to an unlabeled online environment where the underlying label distribution changes over time but the label-conditional density does not.

Original: The non-stationarity nature and the lack of supervision make the problem challenging to be tackled.
Revised: The non-stationarity nature and the lack of supervision make the problem challenging to be tackled.

Original: To address the difficulty, we construct a new unbiased risk estimator that utilizes the unlabeled data, which exhibits many benign properties albeit with potential non-convexity.
Revised: To address the difﬁculty, we construct a new unbiased risk estimator that utilizes the unlabeled data, which exhibits many benign properties albeit with potential non-convexity.

Original: Building upon that, we propose novel online ensemble algorithms to deal with the non-stationarity of the environments.
Revised: Building upon that, we propose novel online ensemble algorithms to deal with the non-stationarity of the environments.

Original: Our approach enjoys optimal dynamic regret , indicating that the performance is competitive with a clairvoyant who knows the online environments in hindsight and then chooses the best decision for each round.
Revised: Our approach enjoys optimal dynamic regret , indicating that the performance is competitive with a clairvoyant who knows the online environments in hindsight and then chooses the best decision for each round.

Original: The obtained dynamic regret bound scales with the intensity and pattern of label distribution shift, hence exhibiting the adaptivity in the OLaS problem.
Revised: The obtained dynamic regret bound scales with the intensity and pattern of label distribution shift, hence exhibiting the adaptivity in the OLaS problem.

Original: Extensive experiments are conducted to validate the effectiveness and support our theoretical findings.
Revised: Extensive experiments are conducted to validate the effectiveness and support our theoretical ﬁndings.

Original:  Introduction
Revised:  Introduction

Original: One of the fundamental challenges for modern machine learning is the distribution shift [1, 2, 3, 4].
Revised: One of the fundamental challenges for modern machine learning is the distribution shift [1, 2, 3, 4].

Original: The learned model’s testing performance would significantly drop when the distribution is different from the initial training distribution.
Revised: The learned model’s testing performance would signiﬁcantly drop when the distribution is different from the initial training distribution.

Original: More severely, in many real-world applications, testing data often come in an online fashion after deploying the trained model such that the underlying distribution might continuously change over time.
Revised: More severely, in many real-world applications, testing data often come in an online fashion after deploying the trained model such that the underlying distribution might continuously change over time.

Original: Hence, it is necessary to develop learning methods to handle distribution shift in online and open environments [5].
Revised: Hence, it is necessary to develop learning methods to handle distribution shift in online and open environments [5].

Original: Another practical concern is the label scarcity issue in real tasks, particularly those tasks emerging in online scenarios.
Revised: Another practical concern is the label scarcity issue in real tasks, particularly those tasks emerging in online scenarios.

Original: For example, in the species monitoring task [6, 7], a learned model is deployed to detect species of wild animals.
Revised: For example, in the species monitoring task [6, 7], a learned model is deployed to detect species of wild animals.

Original: The data consist of received signals from sensors and hence are naturally in the streaming form.
Revised: The data consist of received signals from sensors and hence are naturally in the streaming form.

Original: The data distribution of upcoming animals will change due to the variety of species across different geographic locations and seasons, and moreover, it is hard to gather the labels of streaming data in time.
Revised: The data distribution of upcoming animals will change due to the variety of species across different geographic locations and seasons, and moreover, it is hard to gather the labels of streaming data in time.

Original: Motivated by the above real demand, this paper is concerned with the following problem: how to design an algorithm that can adapt to non-stationary environments with a few labeled data or even unlabeled data observed at every time?
Revised: Motivated by the above real demand, this paper is concerned with the following problem: how to design an algorithm that can adapt to non-stationary environments with a few labeled data or even unlabeled data observed at every time?

Original: In addition to getting empirical performance gain, the overall method is desired to have clear and strong theoretical guarantees.
Revised: In addition to getting empirical performance gain, the overall method is desired to have clear and strong theoretical guarantees.

Original: The problem is generally hard due to the non-stationarity of online environments and the lack of supervision.
Revised: The problem is generally hard due to the non-stationarity of online environments and the lack of supervision.

Original: As such, we investigate a simplified case with a focus on the specific change on the label distribution .
Revised: As such, we investigate a simpliﬁed case with a focus on the speciﬁc change on the label distribution .

Original: We formalize the online label shift (OLaS) problem, 3 which consists of two stages, including offline initialization and online adaptation.
Revised: We formalize the online label shift (OLaS) problem, 3 which consists of two stages, including ofﬂine initialization and online adaptation.

Original: Specifically, the learner collects labeled samples drawn independently from the initial distribution D 0 ( x , y ) where x and y denote the feature vector and its associated label, and trains an initial model following standard supervised learning methods.
Revised: Speciﬁcally, the learner collects labeled samples drawn independently from the initial distribution D 0 ( x , y ) where x and y denote the feature vector and its associated label, and trains an initial model following standard supervised learning methods.

Original: Subsequently, she needs to adapt it to an unknown non-stationary environment where the underlying label distributions change over time.
Revised: Subsequently, she needs to adapt it to an unknown non-stationary environment where the underlying label distributions change over time.

Original: Specifically, at time t , she receives a few unlabeled samples drawn from the current distribution
Revised: Speciﬁcally, at time t , she receives a few unlabeled samples drawn from the current distribution

Original: D t ( x ) and uses them to update the model.
Revised: D t ( x ) and uses them to update the model.

Original: In OLaS problems, the essential environment change happens on the label distribution D t ( y ) with the conditional D ( x | y ) always remaining the same.
Revised: In OLaS problems, the essential environment change happens on the label distribution D t ( y ) with the conditional D ( x | y ) always remaining the same.

Original: The label shift problem has been widely studied in the offline setting [8, 9, 10, 11, 12, 13, 14], but this is less explored in the more challenging online setup.
Revised: The label shift problem has been widely studied in the ofﬂine setting [8, 9, 10, 11, 12, 13, 14], but this is less explored in the more challenging online setup.

Original: One natural impulse is to handle OLaS by online learning techniques [15], but it is generally non-trivial due to the lack of supervision in the adaptation stage and also the non-stationarity issue.
Revised: One natural impulse is to handle OLaS by online learning techniques [15], but it is generally non-trivial due to the lack of supervision in the adaptation stage and also the non-stationarity issue.

Original: Wu et al.
Revised: Wu et al.

Original: [16] made the first such attempt for OLaS, where they constructed an unbiased risk estimator with the unlabeled data for model assessment and used online gradient descent for model updating.
Revised: [16] made the ﬁrst such attempt for OLaS, where they constructed an unbiased risk estimator with the unlabeled data for model assessment and used online gradient descent for model updating.

Original: Let T denote the number of rounds.
Revised: Let T denote the number of rounds.

Original: They proved an O ( √ T ) regret bound, which measures the gap between the learner’s decision and the best fixed decision in hindsight.
Revised: They proved an O ( √ T ) regret bound, which measures the gap between the learner’s decision and the best ﬁxed decision in hindsight.

Original: However, in non-stationary environments, a single decision can hardly perform well all the time, which makes the guarantee less attractive for OLaS problems.
Revised: However, in non-stationary environments, a single decision can hardly perform well all the time, which makes the guarantee less attractive for OLaS problems.

Original: Another technical caveat is that their theory relies on a vital assumption of the convexity of risk functions, which was not verified strictly.
Revised: Another technical caveat is that their theory relies on a vital assumption of the convexity of risk functions, which was not veriﬁed strictly.

Original: In fact, this assumption can hardly be satisfied as an operation to take the argument of the maximum is involved in its formulation of the risk estimator.
Revised: In fact, this assumption can hardly be satisﬁed as an operation to take the argument of the maximum is involved in its formulation of the risk estimator.

Original: In this paper, we aim to develop an algorithm for adapting to online label shift with provable guarantees.
Revised: In this paper, we aim to develop an algorithm for adapting to online label shift with provable guarantees.

Original: To this end, we first reframe the construction of the unbiased risk estimator via risk rewriting techniques and prove that the estimator still enjoys benign theoretical properties, albeit with a potential non-convex behavior.
Revised: To this end, we ﬁrst reframe the construction of the unbiased risk estimator via risk rewriting techniques and prove that the estimator still enjoys benign theoretical properties, albeit with a potential non-convex behavior.

Original: Second, to handle the non-stationarity of the online stream, instead of using traditional regret as the performance measure, we employ dynamic regret to guide the algorithm design, which ensures the online algorithm is competitive with a clairvoyant who knows the online functions in hindsight and hence chooses the best decision of each round.
Revised: Second, to handle the non-stationarity of the online stream, instead of using traditional regret as the performance measure, we employ dynamic regret to guide the algorithm design, which ensures the online algorithm is competitive with a clairvoyant who knows the online functions in hindsight and hence chooses the best decision of each round.

Original: To optimize such a strengthened measure, we propose a novel online ensemble algorithm building upon the risk estimator, consisting of a meta-algorithm running over a group of base-learners, each associated with a customized configuration.
Revised: To optimize such a strengthened measure, we propose a novel online ensemble algorithm building upon the risk estimator, consisting of a meta-algorithm running over a group of base-learners, each associated with a customized conﬁguration.

Original: Our algorithm enjoys an O ( V 1 / 3 T
Revised: Our algorithm enjoys an O ( V 1 / 3 T

Original: T 2 / 3 ) dynamic regret, where
Revised: T 2 / 3 ) dynamic regret, where

Original: V T = (cid:80)
Revised: V T = (cid:80)

Original: Tt [-=
 ∥-] µ y t − µ y t − 1 ∥ 1 measures the variation of the label distributions, with µ y t denoting the vector consisting of the class-prior probabilities at time t .
Revised: Tt [-=
 ∥-]=2 (cid:107) µ y t − µ y t − 1 (cid:107) 1 measures the variation of the label distributions, with µ y t denoting the vector consisting of the class-prior probabilities at time t .

Original: Notably, the obtained dynamic regret upper bound is minimax optimal in terms of both T and V T , and importantly, our algorithm does not require the unknown non-stationarity measure V T as the input.
Revised: Notably, the obtained dynamic regret upper bound is minimax optimal in terms of both T and V T , and importantly, our algorithm does not require the unknown non-stationarity measure V T as the input.

Original: Furthermore, for many situations where online label shift contains some patterns such as periodicity or gradual change behavior, we present an improved algorithm to exploit such structures and achieve provably better guarantees.
Revised: Furthermore, for many situations where online label shift contains some patterns such as periodicity or gradual change behavior, we present an improved algorithm to exploit such structures and achieve provably better guarantees.

Original: The key idea is to leverage historical information to serve as a hint for online updates.
Revised: The key idea is to leverage historical information to serve as a hint for online updates.

Original: We prove an O ( V 1 / 3 T
Revised: We prove an O ( V 1 / 3 T

Original: G 1 / 3 T T 1 / 3 ) dynamic regret, where G T measures the reusability of historical information that is at most O ( T ) while could be much smaller in benign environments.
Revised: G 1 / 3 T T 1 / 3 ) dynamic regret, where G T measures the reusability of historical information that is at most O ( T ) while could be much smaller in benign environments.

Original: As a benefit, the improved algorithm safeguards the O ( V 1 / 3 T
Revised: As a beneﬁt, the improved algorithm safeguards the O ( V 1 / 3 T

Original: T 2 / 3 ) worst-case optimality and meanwhile achieves great improvement in easier environments.
Revised: T 2 / 3 ) worst-case optimality and meanwhile achieves great improvement in easier environments.

Original: Extensive experiments are conducted to evaluate our approach, which show the usefulness of meta-base structure in tackling non-stationarity and validate the effectiveness of other adaptive components.
Revised: Extensive experiments are conducted to evaluate our approach, which show the usefulness of meta-base structure in tackling non-stationarity and validate the effectiveness of other adaptive components.

Original: Technical Contribution.
Revised: Contribution.

Original: Our method is not a direct application of existing non-stationary online learning methods [17, 18, 19], but rather requires in-depth technical innovations.
Revised: Our method is not a direct application of existing non-stationary online learning methods [17, 18, 19], but rather requires in-depth technical innovations.

Original: First, the potentially non-convex risk estimator makes it hard to apply existing techniques of online convex optimization (OCO) [15], but fortunately, we prove the convexity of its expectation such that the OCO framework can still be used (see Remark 2).
Revised: First, the potentially non-convex risk estimator makes it hard to apply existing techniques of online convex optimization (OCO) [15], but fortunately, we prove the convexity of its expectation such that the OCO framework can still be used (see Remark 2).

Original: Second, to optimize the dynamic regret, we employ a meta-base structure to hedge the uncertainty of the unknown minimizer of the expected risk function at each round and convert the variation of expected risk minimizers to the intensity of label distribution drifts, a natural non-stationarity measure for the OLaS problem (see Remark 3).
Revised: Second, to optimize the dynamic regret, we employ a meta-base structure to hedge the uncertainty of the unknown minimizer of the expected risk function at each round and convert the variation of expected risk minimizers to the intensity of label distribution drifts, a natural non-stationarity measure for the OLaS problem (see Remark 3).

Original: Third, earlier study showed adaptive dynamic regret bounds for convex and smooth functions [19], while the smoothness assumption can hardly be satisfied in our case.
Revised: Third, earlier study showed adaptive dynamic regret bounds for convex and smooth functions [19], while the smoothness assumption can hardly be satisﬁed in our case.

Original: We remove such a constraint by introducing an implicit update , which could be of independent interest for general OCO purposes (see Remark 4).
Revised: We remove such a constraint by introducing an implicit update , which could be of independent interest for general OCO purposes (see Remark 4).

Original:  Problem Formulation
Revised:  Problem Formulation

Original: We focus on OLaS of multi-class classification with feature space X ⊆ R d and label space Y =
Revised: We focus on OLaS of multi-class classiﬁcation with feature space X ⊆ R d and label space Y =

Original: [ K ] ≜ { 1 , . . 
Revised: [ K ] (cid:44) { 1 , . . .

Original: , K } , where d is the dimension and K is the number of classes.
Revised: , K } , where d is the dimension and K is the number of classes.

Original: Below we formulate setups of two stages of OLaS (offline initialization and online adaptation).
Revised: Below we formulate setups of two stages of OLaS (ofﬂine initialization and online adaptation).

Original: Problem Setup.
Revised: Problem Setup.

Original: In the offline initialization stage , the learner collects a number of labeled samples denoted by S 

Revised: In the ofﬂine initialization stage , the learner collects a number of labeled samples denoted by S

Original: = { ( x n , y n ) } N 0 n =1 drawn from the distribution D 0 ( x , y ) and then obtains a wellperformed initial model f 

Revised: = { ( x n , y n ) } N 0 n =1 drawn from the distribution D 0 ( x , y ) and then obtains a wellperformed initial model f

Original: : X (cid:55)→ Y .
Revised: : X (cid:55)→ Y .

Original: In the online adaptation stage , data come in the streaming form without labels.
Revised: In the online adaptation stage , data come in the streaming form without labels.

Original: The current model is deployed to predict the labels of online data and also to evolve adaptively.
Revised: The current model is deployed to predict the labels of online data and also to evolve adaptively.

Original: Specifically, at each round t ∈ [ T ] , the learner receives a small number of unlabeled data
Revised: Speciﬁcally, at each round t ∈ [ T ] , the learner receives a small number of unlabeled data

Original: S t = { x n } N t n =1 drawn from the distribution D t ( x ) .
Revised: S t = { x n } N t n =1 drawn from the distribution D t ( x ) .

Original: The non-stationary nature indicates D t ̸ = D t ′ in general for different t, t ′ ∈
Revised: The non-stationary nature indicates D t (cid:54) = D t (cid:48) in general for different t, t (cid:48) ∈

Original: [ T ] .
Revised: [ T ] .

Original: OLaS considers the simplified case that essential changes come from label distributions and there are no new classes, formally described in the following condition.
Revised: OLaS considers the simpliﬁed case that essential changes come from label distributions and there are no new classes, formally described in the following condition.

Original: Assumption 1 (Online Label Shift) .
Revised: Assumption 1 (Online Label Shift) .

Original: In the online label shift problem, the label distribution D t ( y ) changes over time while the class-conditional distribution D t ( x | y ) is identical throughout the process for t ∈ { 0 , 1 , . .
Revised: In the online label shift problem, the label distribution D t ( y ) changes over time while the class-conditional distribution D t ( x | y ) is identical throughout the process for t ∈ { 0 , 1 , . .

Original: , T } .
Revised: , T } .

Original: Moreover, it holds that D 0 ( y ) > 0 for any y ∈ Y .
Revised: Moreover, it holds that D 0 ( y ) > 0 for any y ∈ Y .

Original: Performance Measure.
Revised: Performance Measure.

Original: At round t ∈ [ T ] , the learner uses the information observed so far to make the prediction and also update the model w t ∈ W , where W is a convex decision set with diameter Γ ≜ sup w , w ′ ∈W ∥ w − w ′ ∥ 2 .
Revised: At round t ∈ [ T ] , the learner uses the information observed so far to make the prediction and also update the model w t ∈ W , where W is a convex decision set with diameter Γ (cid:44) sup w , w (cid:48) ∈W (cid:107) w − w (cid:48) (cid:107) 2 .

Original: The goal is to ensure that the t -round model w t generalizes well on the underlying distribution D t .
Revised: The goal is to ensure that the t -round model w t generalizes well on the underlying distribution D t .

Original: Thus, the model’s quality is evaluated by its risk defined as R t ( w ) =
Revised: Thus, the model’s quality is evaluated by its risk deﬁned as R t ( w ) =

Original: E ( x ,y )
Revised: E ( x ,y )

Original: ∼D t [ ℓ ( f ( w , x ) , y )] , where f : W × X (cid:55)→ R K is the predictive function and ℓ : R K × Y (cid:55)→ R is any convex surrogate loss for classification such that ℓ ( f ( w , x ) , y ) is convex in w .
Revised: ∼D t [ (cid:96) ( f ( w , x ) , y )] , where f : W × X (cid:55)→ R K is the predictive function and (cid:96) : R K × Y (cid:55)→ R is any convex surrogate loss for classiﬁcation such that (cid:96) ( f ( w , x ) , y ) is convex in w .

Original: We introduce two constants, G ≜ sup ( x ,y ) ∈X×Y , w ∈W ∥∇ w ℓ ( f ( w , x ) , y ) ∥ 2 and
Revised: We introduce two constants, G (cid:44) sup ( x ,y ) ∈X×Y , w ∈W (cid:107)∇ w (cid:96) ( f ( w , x ) , y ) (cid:107) 2 and

Original: B ≜ sup ( x ,y ) ∈X×Y , w ∈W | ℓ ( f ( w , x ) , y ) | , as upper bounds of gradient norm and loss function value.
Revised: B (cid:44) sup ( x ,y ) ∈X×Y , w ∈W | (cid:96) ( f ( w , x ) , y ) | , as upper bounds of gradient norm and loss function value.

Original: We use regret to examine the performance of online algorithms.
Revised: We use regret to examine the performance of online algorithms.

Original: In particular, dynamic regret [20, 17] is employed to compete the algorithm’s performance with the best response at each round, defined as
Revised: In particular, dynamic regret [20, 17] is employed to compete the algorithm’s performance with the best response at each round, deﬁned as

Original: [Equation]
Revised: [Equation]

Original: where w ∗ t ∈ arg min w ∈W R t ( w ) is the model (or one of the models) with the best generalization ability on the distribution D t .
Revised: where w ∗ t ∈ arg min w ∈W R t ( w ) is the model (or one of the models) with the best generalization ability on the distribution D t .

Original: Notably, it is known that a sublinear dynamic regret is impossible in the worst case [17], so an upper bound of dynamic regret is desired to scale with a certain non-stationarity measure.
Revised: Notably, it is known that a sublinear dynamic regret is impossible in the worst case [17], so an upper bound of dynamic regret is desired to scale with a certain non-stationarity measure.

Original: A natural measure for OLaS would be the variation intensity of label distributions.
Revised: A natural measure for OLaS would be the variation intensity of label distributions.

Original: Remark 1 (Static regret vs. dynamic regret) .
Revised: Remark 1 (Static regret vs. dynamic regret) .

Original: The classic measure for online learning is static regret, defined as Reg s T=
Revised: The classic measure for online learning is static regret, deﬁned as Reg s T

Original:  (cid:80) Tt =1 R t ( w t ) − (cid:80)
Revised: = (cid:80) Tt =1 R t ( w t ) − (cid:80)

Original: Tt =
Revised: Tt =

Original: 1 R t ( w ∗ ) , where w ∗ ∈ arg min w ∈W (cid:80) Tt =1 R t ( w ) is the best fixed model in hindsight.
Revised: 1 R t ( w ∗ ) , where w ∗ ∈ arg min w ∈W (cid:80) Tt =1 R t ( w ) is the best ﬁxed model in hindsight.

Original: The measure was adopted in the prior work of OLaS [16].
Revised: The measure was adopted in the prior work of OLaS [16].

Original: However, the measure is not suitable for OLaS, because it is too optimistic to expect a single fixed model to behave well over the whole process in changing environments.
Revised: However, the measure is not suitable for OLaS, because it is too optimistic to expect a single ﬁxed model to behave well over the whole process in changing environments.

Original: By contrast, minimizing dynamic regret facilitates the online algorithm with more adaptivity and robustness to non-stationary environments.
Revised: By contrast, minimizing dynamic regret facilitates the online algorithm with more adaptivity and robustness to non-stationary environments.

Original:  Proposed Approach
Revised:  Proposed Approach

Original: This section presents our approach for the OLaS problem, including the algorithms and theoretical guarantees.
Revised: This section presents our approach for the OLaS problem, including the algorithms and theoretical guarantees.

Original: In the following, we respectively address the two central challenges of OLaS: the lack of supervision and the non-stationarity of online environments.
Revised: In the following, we respectively address the two central challenges of OLaS: the lack of supervision and the non-stationarity of online environments.

Original: 3.1 Unbiased Risk Estimator for Online Convex Optimization
Revised: 3.1 Unbiased Risk Estimator for Online Convex Optimization

Original: OCO is a powerful and versatile framework for online learning problems, which enjoys both practical and theoretical appeals [21, 15].
Revised: OCO is a powerful and versatile framework for online learning problems, which enjoys both practical and theoretical appeals [21, 15].

Original: Online Gradient Descent (OGD) [20] is one of the most fundamental and powerful algorithms due to its light computational cost and sound regret guarantees.
Revised: Online Gradient Descent (OGD) [20] is one of the most fundamental and powerful algorithms due to its light computational cost and sound regret guarantees.

Original: In the
Revised: In the

Original: OLaS problem, recall that the learner’s goal is to obtain a model sequence { w t } Tt =1 enjoying low cumulative risk (cid:80)
Revised: OLaS problem, recall that the learner’s goal is to obtain a model sequence { w t } Tt =1 enjoying low cumulative risk (cid:80)

Original: Tt =1 R t ( w t ) .
Revised: Tt =1 R t ( w t ) .

Original: Thus, suppose the model’s risk R t ( w t ) is known at each round; then OGD simply updates the model by w t
Revised: Thus, suppose the model’s risk R t ( w t ) is known at each round; then OGD simply updates the model by w t

Original: +1 =
Revised: +1 =

Original: Π W [ w t − η ∇ R t ( w t )] , where Π W [ · ] denotes the projection onto W and η > 0 is the step size.
Revised: Π W [ w t − η ∇ R t ( w t )] , where Π W [ · ] denotes the projection onto W and η > 0 is the step size.

Original: It is well-known that OGD guarantees the regret bound (cid:80)
Revised: It is well-known that OGD guarantees the regret bound (cid:80)

Original: Tt =1 R t ( w t ) − min w ∈W (cid:80) Tt =
Revised: Tt =1 R t ( w t ) − min w ∈W (cid:80) Tt =

Original: 1 R t ( w ) ≤ O ( √ T ) when risk function R t ( w ) is convex and step size is set as η = Θ( T − 1 / 2 ) [15] (see Appendix B.1 for more details).
Revised: 1 R t ( w ) ≤ O ( √ T ) when risk function R t ( w ) is convex and step size is set as η = Θ( T − 1 / 2 ) [15] (see Appendix B.1 for more details).

Original: However, the expected risk function R t ( w ) is unknown in the current OLaS setup as it is defined over the underlying joint distribution D t ( x , y ) .
Revised: However, the expected risk function R t ( w ) is unknown in the current OLaS setup as it is deﬁned over the underlying joint distribution D t ( x , y ) .

Original: More severely, the online environments in the OLaS problem are fully unlabeled , which poses great challenges to apply the OCO framework.
Revised: More severely, the online environments in the OLaS problem are fully unlabeled , which poses great challenges to apply the OCO framework.

Original: Indeed, the lack of supervision makes it hard to empirically assess the expected risk, not to mention ensuring the convexity.
Revised: Indeed, the lack of supervision makes it hard to empirically assess the expected risk, not to mention ensuring the convexity.

Original: In the following, we establish an unbiased estimator (cid:98) R t ( w ) with unlabeled data S t , which exhibits nice properties such that the OCO framework is still applicable for our purpose.
Revised: In the following, we establish an unbiased estimator (cid:98) R t ( w ) with unlabeled data S t , which exhibits nice properties such that the OCO framework is still applicable for our purpose.

Original: Unbiased Estimator under Label Shift.
Revised: Unbiased Estimator under Label Shift.

Original: Inspired by the progress in offline label shift [9, 12, 13], we establish an unbiased risk estimator in OLaS for R t ( w ) with unlabeled data S t and offline data
Revised: Inspired by the progress in ofﬂine label shift [9, 12, 13], we establish an unbiased risk estimator in OLaS for R t ( w ) with unlabeled data S t and ofﬂine data

Original: S 0 by the risk rewriting technique.
Revised: S 0 by the risk rewriting technique.

Original: To this end, let µ y t ∈
Revised: To this end, let µ y t ∈

Original: ∆ K denote the label distribution vector with the k -th entry [ µ y t ] k ≜ D t ( y = k ) , then we have the following decomposition for the true risk: 
Revised: ∆ K denote the label distribution vector with the k -th entry [ µ y t ] k (cid:44) D t ( y = k ) , then we have the following decomposition for the true risk: [Equation]

Original: [Equation]
Revised: [Equation]

Revised: w , x

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: ] =
Revised: =

Original: [Equation]
Revised: [Equation]

Original: where R kt ( w ) ≜
Revised: where R kt ( w ) (cid:44) E x

Original: E x ∼D t ( x | y = k ) [ ℓ ( f ( w , x ) , k )] is the risk of the model over the k -th label at round t .
Revised: ∼D t ( x | y = k ) [ (cid:96) ( f ( w , x ) , k )] is the risk of the model over the k -th label at round t .

Original: The second equality holds due to the law of total probability, and the third equality is by the label shift assumption that D t ( x | y )
Revised: The second equality holds due to the law of total probability, and the third equality is by the label shift assumption that D t ( x | y )

Original: =
Revised: =

Original: D 0 ( x | y ) for any t ∈
Revised: D 0 ( x | y ) for any t ∈

Original: [ T ] .
Revised: [ T ] .

Original: Since the labeled offline data S 0 is always available, one can approximate R k 0 ( w ) by its empirical version (cid:98) R k 0 ( w ) with offline data S 0 , where (cid:98) R k 0 ( w ) ≜ 1 | S k 0 | (cid:80) x n ∈ S k 0 ℓ ( f ( w , x n ) , k ) and S k 0 is a subset of S 0 containing all samples with label k .
Revised: Since the labeled ofﬂine data S 0 is always available, one can approximate R k 0 ( w ) by its empirical version (cid:98) R k 0 ( w ) with ofﬂine data S 0 , where (cid:98) R k 0 ( w ) (cid:44) 1 | S k 0 | (cid:80) x n ∈ S k 0 (cid:96) ( f ( w , x n ) , k ) and S k 0 is a subset of S 0 containing all samples with label k .

Original: Therefore, the task is now to estimate the label distribution vector µ y t .
Revised: Therefore, the task is now to estimate the label distribution vector µ y t .

Original: To this end, we employ the Black Box Shift Estimation (BBSE) method [12] to construct an estimator via only offline data
Revised: To this end, we employ the Black Box Shift Estimation (BBSE) method [12] to construct an estimator via only ofﬂine data

Original: S 0 and unlabeled data S t .
Revised: S 0 and unlabeled data S t .

Original: Specifically, we first use the initial offline model f 0 to predict over the unlabeled data S t and get predictive labels (cid:98) y t , and next estimate the label distribution via solving the crucial equation µ y t =
Revised: Speciﬁcally, we ﬁrst use the initial ofﬂine model f 0 to predict over the unlabeled data S t and get predictive labels (cid:98) y t , and next estimate the label distribution via solving the crucial equation µ y t =

Original: C − 1 f 0 µ (cid:98) y t , where µ (cid:98)
Revised: C − 1 f 0 µ (cid:98) y t , where µ (cid:98)

Original: y t ∈ ∆ K is the distribution vector of the predictive labels (cid:98) y t and C f 0 ∈ R K × K is the confusion matrix with [ C f 0 ] ij =
Revised: y t ∈ ∆ K is the distribution vector of the predictive labels (cid:98) y t and C f 0 ∈ R K × K is the confusion matrix with [ C f 0 ] ij =

Original: E x ∼D 0 ( x | y =
Revised: E x ∼D 0 ( x | y =

Original: j ) [ 1 { f 0 ( x ) = i } ] being the classification rate that the initial model f 0 predicts samples from class i as class j .
Revised: j ) [ 1 { f 0 ( x ) = i } ] being the classiﬁcation rate that the initial model f 0 predicts samples from class i as class j .

Original: We defer more details to Appendix B.2.
Revised: We defer more details to Appendix B.2.

Original: As a result, through risk rewriting and prior estimation, we can construct the following estimator for the true risk R t ( w ) :
Revised: As a result, through risk rewriting and prior estimation, we can construct the following estimator for the true risk R t ( w ) :

Original: [Equation]
Revised: [Equation]

Original: where (cid:98) C f 0 and (cid:98) µ (cid:98) y t are empirical estimators of the confusion matrix and predictive label distribution vector using offline data S 0 and unlabeled data S t only.
Revised: where (cid:98) C f 0 and (cid:98) µ (cid:98) y t are empirical estimators of the confusion matrix and predictive label distribution vector using ofﬂine data S 0 and unlabeled data S t only.

Original: Our constructed risk estimator enjoys the unbiasedness property, which plays a crucial role in the later algorithm design and theoretical analysis.
Revised: Our constructed risk estimator enjoys the unbiasedness property, which plays a crucial role in the later algorithm design and theoretical analysis.

Original: Lemma 1.
Revised: Lemma 1.

Original: The estimator (cid:98) R t ( w ) in Eq.
Revised: The estimator (cid:98) R t ( w ) in Eq.

Original: (3) is unbiased to R t ( w )
Revised: (3) is unbiased to R t ( w )

Original: =
Revised: =

Original: E ( x ,y )
Revised: E ( x ,y )

Original: ∼D t [ ℓ ( f ( w , x ) , y )] , i.e.,
Revised: ∼D t [ (cid:96) ( f ( w , x ) , y )] , i.e.,

Original: E S t ∼D t
Revised: E S t ∼D t

Original: [ (cid:98) R t ( w )]
Revised: [ (cid:98) R t ( w )]

Original: =
Revised: =

Original: R t ( w ) , for any w ∈ W independent of the dataset S t , provided C f 0 is invertible and the offline dataset S 0 has sufficient samples such that (cid:98) C f 
 
Revised: R t ( w ) , for any w ∈ W independent of the dataset S t , provided C f 0 is invertible and the ofﬂine dataset S 0 has sufﬁcient samples such that (cid:98) C f 
 = C f 0 and (cid:98) R k 0 ( w )

Original: =
Revised: =

Original: C f 0 and (cid:98) R k 0 ( w ) =

Original: R k 0 ( w ) , ∀ k ∈ Y .
Revised: R k 0 ( w ) , ∀ k ∈ Y .

Original: The proof of Lemma 1 can be found in Appendix D.1.
Revised: The proof of Lemma 1 can be found in Appendix D.1.

Original: Note that the sufficient sample assumption is introduced on the offline data S 0 to simplify the presentation.
Revised: Note that the sufﬁcient sample assumption is introduced on the ofﬂine data S 0 to simplify the presentation.

Original: Indeed, we can further show | E S t ∼D t [ (cid:98) R t ( w )] − R t ( w ) | ≤ O ( (cid:112) 1 / | S 0 | ) with high probability in Appendix D.2.
Revised: Indeed, we can further show | E S t ∼D t [ (cid:98) R t ( w )] − R t ( w ) | ≤ O ( (cid:112) 1 / | S 0 | ) with high probability in Appendix D.2.

Original: Thus, the additional term is negligible when a large number of offline data is collected at the initial stage, which is easy to realize and also will not trivialize the online adaptation.
Revised: Thus, the additional term is negligible when a large number of ofﬂine data is collected at the initial stage, which is easy to realize and also will not trivialize the online adaptation.

Original: Remark 2 (Non-convexity issue) .
Revised: Remark 2 (Non-convexity issue) .

Original: Our risk estimator (cid:98) R t ( w ) can be non-convex as the estimated label distribution [ (cid:98) [Table]
Revised: Our risk estimator (cid:98) R t ( w ) can be non-convex as the estimated label distribution [ (cid:98) C − 1 f 0 (cid:98) µ (cid:98)

Original:  ] k might be negative in order to ensure the unbiasedness.
Revised: y t ] k might be negative in order to ensure the unbiasedness.

Original: Such a non-convex behavior introduces a great challenge for applying the OCO framework.
Revised: Such a non-convex behavior introduces a great challenge for applying the OCO framework.

Original: Fortunately, owing to its unbiasedness and the fact that the expected risk R t ( w ) is indeed convex, we can continue the following algorithm design and theoretical analysis building upon the constructed unbiased estimator.
Revised: Fortunately, owing to its unbiasedness and the fact that the expected risk R t ( w ) is indeed convex, we can continue the following algorithm design and theoretical analysis building upon the constructed unbiased estimator.

Original: OGD with Unbiased Estimator.
Revised: OGD with Unbiased Estimator.

Original: Building upon the risk estimator in Eq.
Revised: Building upon the risk estimator in Eq.

Original: (3), we then deploy OGD and obtain our UOGD algorithm (abbreviated for “OGD with unbiased risk estimator”), namely,
Revised: (3), we then deploy OGD and obtain our UOGD algorithm (abbreviated for “OGD with unbiased risk estimator”), namely,

Original: [Equation]
Revised: [Equation]

Original: Despite the potential non-convexity of the risk estimator itself, we can still establish solid regret guarantees via the OCO framework due to the benign property that the risk estimator is unbiased and the expected risk is indeed convex.
Revised: Despite the potential non-convexity of the risk estimator itself, we can still establish solid regret guarantees via the OCO framework due to the benign property that the risk estimator is unbiased and the expected risk is indeed convex.

Original: For example, UOGD provably enjoys an O ( √ T ) static regret.
Revised: For example, UOGD provably enjoys an O ( √ T ) static regret.

Original: See the formal statement in Appendix D.3.
Revised: See the formal statement in Appendix D.3.

Original: We remark that our attained static regret already achieves the state-of-the-art theoretical understanding of OLaS, in the sense that previously the same bound can be only achieved with an additional unrealistic convexity assumption imposed over the algorithm [16], which UOGD does not require.
Revised: We remark that our attained static regret already achieves the state-of-the-art theoretical understanding of OLaS, in the sense that previously the same bound can be only achieved with an additional unrealistic convexity assumption imposed over the algorithm [16], which UOGD does not require.

Original: Concretely, Wu et al.
Revised: Concretely, Wu et al.

Original: [16] assumed that the risk estimator is convex (in expectation), which is hard to theoretically verify since the estimator approximates the 0 / 1 -loss and involves an indicator function and an argmax operation due to the employed reweighting mechanism.
Revised: [16] assumed that the risk estimator is convex (in expectation), which is hard to theoretically verify since the estimator approximates the 0 / 1 -loss and involves an indicator function and an argmax operation due to the employed reweighting mechanism.

Original: By contrast, our estimator directly approximates the surrogate loss without reweighting and thus does not suffer from such limitations.
Revised: By contrast, our estimator directly approximates the surrogate loss without reweighting and thus does not suffer from such limitations.

Original: Even if modifying their estimator to optimize a surrogate loss, the reweighting mechanism makes their method still hardly suitable for the OCO framework.
Revised: Even if modifying their estimator to optimize a surrogate loss, the reweighting mechanism makes their method still hardly suitable for the OCO framework.

Original: More details are in Appendix C.3.
Revised: More details are in Appendix C.3.

Original: In a nutshell, our constructed risk estimator enjoys nice properties, which are indispensable for the algorithm design and theoretical analysis.
Revised: In a nutshell, our constructed risk estimator enjoys nice properties, which are indispensable for the algorithm design and theoretical analysis.

Original: 3.2 Adapting to Non-stationarity of Online Label Shift
Revised: 3.2 Adapting to Non-stationarity of Online Label Shift

Original: So far, an O ( √ T ) static regret has been established for OLaS; however, the guarantee is not appealing because static regret is not suitable for non-stationary online problems as discussed in Remark 1.
Revised: So far, an O ( √ T ) static regret has been established for OLaS; however, the guarantee is not appealing because static regret is not suitable for non-stationary online problems as discussed in Remark 1.

Original: We now introduce our method adapting to the non-stationarity with provable dynamic regret guarantees.
Revised: We now introduce our method adapting to the non-stationarity with provable dynamic regret guarantees.

Original: First, benefiting from the unbiasedness and expected convexity of our risk estimator, we prove that
Revised: First, beneﬁting from the unbiasedness and expected convexity of our risk estimator, we prove that

Original: UOGD achieves a dynamic regret scaling with the label distribution drift V T .
Revised: UOGD achieves a dynamic regret scaling with the label distribution drift V T .

Original: Theorem 1.
Revised: Theorem 1.

Original: Under the same assumptions as Lemma 1, UOGD in Eq.
Revised: Under the same assumptions as Lemma 1, UOGD in Eq.

Original: (4) with step size η satisfies
Revised: (4) with step size η satisﬁes

Original: [Equation]
Revised: [Equation]

Original: where the constant σ > 0 denotes the minimum singular value of the invertible confusion matrix C f 0 .
Revised: where the constant σ > 0 denotes the minimum singular value of the invertible confusion matrix C f 0 .

Original: Moreover, V T = (cid:80) Tt [-=
 ∥-] µ y t − µ y t − 1 ∥ 1 measures the intensity of the label distribution shift.
Revised: Moreover, V T = (cid:80) Tt [-=
 ∥-]=2 (cid:107) µ y t − µ y t − 1 (cid:107) 1 measures the intensity of the label distribution shift.

Original: The proof of Theorem 1 can be found in Appendix E.1.
Revised: The proof of Theorem 1 can be found in Appendix E.1.

Original: The dynamic regret guarantee is obtained in a non-trivial way, and below we expand the technical innovations.
Revised: The dynamic regret guarantee is obtained in a non-trivial way, and below we expand the technical innovations.

Original: Remark 3 (Non-stationarity measure for OLaS) .
Revised: Remark 3 (Non-stationarity measure for OLaS) .

Original: For readers who are familiar with the literature, our result is reminiscent of the existing dynamic regret bound in the OCO studies [17, 22] on the surface; however, our result exhibits fundamental differences.
Revised: For readers who are familiar with the literature, our result is reminiscent of the existing dynamic regret bound in the OCO studies [17, 22] on the surface; however, our result exhibits fundamental differences.

Original: The key caveat is that in our case the comparator w ∗ t at each round is not the minimizer of the online function.
Revised: The key caveat is that in our case the comparator w ∗ t at each round is not the minimizer of the online function.

Original: Specifically, as the expected risk R t is inaccessible, one has to work on the unbiased risk estimator (cid:98) R t and requires optimizing the empirical dynamic regret (cid:80)
Revised: Speciﬁcally, as the expected risk R t is inaccessible, one has to work on the unbiased risk estimator (cid:98) R t and requires optimizing the empirical dynamic regret (cid:80)

Original: Tt =1 (cid:98) R t ( w t ) − (cid:80)
Revised: Tt =1 (cid:98) R t ( w t ) − (cid:80)

Original: Tt =1 (cid:98) R t ( w ∗ t ) .
Revised: Tt =1 (cid:98) R t ( w ∗ t ) .

Original: Importantly, w ∗ t ∈ arg min w ∈W R t ( w ) but w ∗ t / ∈ arg min w ∈W (cid:98) R t ( w ) in general.
Revised: Importantly, w ∗ t ∈ arg min w ∈W R t ( w ) but w ∗ t / ∈ arg min w ∈W (cid:98) R t ( w ) in general.

Original: Although the empirical dynamic regret can be trivially bounded by (cid:80) Tt =1 (cid:98) R t ( w t ) − (cid:80) Tt =1 min w ∈W (cid:98) R t ( w ) , the bound will then be loose and related to temporal variation of risk estimators [17, 22], making it hard to establish relationship to the natural non-stationarity measure of OLaS: V T .
Revised: Although the empirical dynamic regret can be trivially bounded by (cid:80) Tt =1 (cid:98) R t ( w t ) − (cid:80) Tt =1 min w ∈W (cid:98) R t ( w ) , the bound will then be loose and related to temporal variation of risk estimators [17, 22], making it hard to establish relationship to the natural non-stationarity measure of OLaS: V T .

Original: On the other hand, there exist studies benchmarking dynamic regret with other choices of comparators [20, 18], but the bounds scale with the consecutive variation of comparators (cid:80)
Revised: On the other hand, there exist studies benchmarking dynamic regret with other choices of comparators [20, 18], but the bounds scale with the consecutive variation of comparators (cid:80)

Original: Tt [-=
 ∥-] w ∗ t − w ∗ t − 1 ∥ 2 , whose relation to V T is also unclear.
Revised: Tt [-=
 ∥-]=2 (cid:107) w ∗ t − w ∗ t − 1 (cid:107) 2 , whose relation to V T is also unclear.

Original: To address the difficulty, drawing inspiration from [23], we decompose the expected dynamic regret into two parts: (i) the analysis of the first part relies on an in-depth analysis of the empirical dynamic regret of UOGD
Revised: To address the difﬁculty, drawing inspiration from [23], we decompose the expected dynamic regret into two parts: (i) the analysis of the ﬁrst part relies on an in-depth analysis of the empirical dynamic regret of UOGD

Original: Input: step size η i ∈ H 1: let w 1 ,i be any point in W 2: for t = 2 to T do 3: construct the risk estimator (cid:98) R t − 1 as (3) 4: update the model of base-learner by w t,i =
Revised: Input: step size η i ∈ H 1: let w 1 ,i be any point in W 2: for t = 2 to T do 3: construct the risk estimator (cid:98) R t − 1 as (3) 4: update the model of base-learner by w t,i =

Original: Π W [ w t − 1 ,i − η i ∇ (cid:98) R t − 1 ( w t − 1 ,i )] 5: send w t,i to the meta-algorithm 6: end for
Revised: Π W [ w t − 1 ,i − η i ∇ (cid:98) R t − 1 ( w t − 1 ,i )] 5: send w t,i to the meta-algorithm 6: end for

Original: Input: step size pool H ; learning rate ε 1: initialization: ∀ i ∈ [ N ] , p 1 ,i = 1 /N 2: for t = 2 to T do 3: receive { w t,i } Ni =1 from base-learners 4: update weight p t ∈ ∆ N according to p t,i ∝ exp( − ε (cid:80) t − 1 s =1 (cid:98) R s ( w s,i )) , i ∈
Revised: Input: step size pool H ; learning rate ε 1: initialization: ∀ i ∈ [ N ] , p 1 ,i = 1 /N 2: for t = 2 to T do 3: receive { w t,i } Ni =1 from base-learners 4: update weight p t ∈ ∆ N according to p t,i ∝ exp( − ε (cid:80) t − 1 s =1 (cid:98) R s ( w s,i )) , i ∈

Original: [ N ] 5: predict final output w t
Revised: [ N ] 5: predict ﬁnal output w t

Original: = (cid:80) Ni =1 p t,i w t,i 6: end for to track a specific sequence of piecewise-stationary comparators to avoid undesired variation of comparators; and (ii) the analysis of the second part is directly conducted on the original risk functions to attain a non-stationarity measure only related to the underlying label distribution shift.
Revised: = (cid:80) Ni =1 p t,i w t,i 6: end for to track a speciﬁc sequence of piecewise-stationary comparators to avoid undesired variation of comparators; and (ii) the analysis of the second part is directly conducted on the original risk functions to attain a non-stationarity measure only related to the underlying label distribution shift.

Original: From the upper bound in Eq.
Revised: From the upper bound in Eq.

Original: (5), we can observe that a proper step size tuning is crucial.
Revised: (5), we can observe that a proper step size tuning is crucial.

Original: Specifically, it can be verified that when the environment is near-stationary (more precisely, V T ≤ Θ( T − 12 ) ), simply choosing η = Θ(1 / √ T ) ensures an O ( √ T ) dynamic regret, which is known to be minimax optimal even for the weaker measure of static regret [24].
Revised: Speciﬁcally, it can be veriﬁed that when the environment is near-stationary (more precisely, V T ≤ Θ( T − 12 ) ), simply choosing η = Θ(1 / √ T ) ensures an O ( √ T ) dynamic regret, which is known to be minimax optimal even for the weaker measure of static regret [24].

Original: Thus, in the following, we focus on the non-degenerated OLaS situation where V T ≥ Θ( T − 12 ) , and then the dynamic regret upper bound can be further simplified as O ( ηT + 1 /η + (cid:112) ( V T T ) /η ) .
Revised: Thus, in the following, we focus on the non-degenerated OLaS situation where V T ≥ Θ( T − 12 ) , and then the dynamic regret upper bound can be further simpliﬁed as O ( ηT + 1 /η + (cid:112) ( V T T ) /η ) .

Original: As a result, it is clear that UOGD can attain an
Revised: As a result, it is clear that UOGD can attain an

Original: O ( V 13 T T 23 ) dynamic regret by setting the step size optimally as η = Θ( T − 13 V 13 T ) , which is known to be minimax optimal due to the Ω( V 13 T T 23 ) lower bound in [25, Theorem 6].
Revised: O ( V 13 T T 23 ) dynamic regret by setting the step size optimally as η = Θ( T − 13 V 13 T ) , which is known to be minimax optimal due to the Ω( V 13 T T 23 ) lower bound in [25, Theorem 6].

Original: However, the optimal step size η ∗ =
Revised: However, the optimal step size η ∗ =

Original: Θ( T − 13 V 13 T ) crucially depends on the label distribution drift
Revised: Θ( T − 13 V 13 T ) crucially depends on the label distribution drift

Original: V T = (cid:80)
Revised: V T = (cid:80)

Original: Tt [-=
 ∥-] µ y t − µ y t − 1 ∥ 1 , which measures the non-stationarity and is unfortunately unknown to the learner.
Revised: Tt [-=
 ∥-]=2 (cid:107) µ y t − µ y t − 1 (cid:107) 1 , which measures the non-stationarity and is unfortunately unknown to the learner.

Original: It is worth emphasizing that the problem cannot be addressed by standard adaptive step size tuning mechanisms in online learning literature, such as the doubling trick [26] or self-confident tuning [27], in that the variation quantity V T cannot be empirically evaluated as it is defined over the underlying label distribution D t ( y ) inaccessible to the learner.
Revised: It is worth emphasizing that the problem cannot be addressed by standard adaptive step size tuning mechanisms in online learning literature, such as the doubling trick [26] or self-conﬁdent tuning [27], in that the variation quantity V T cannot be empirically evaluated as it is deﬁned over the underlying label distribution D t ( y ) inaccessible to the learner.

Original: Even diving into the analysis of
Revised: Even diving into the analysis of

Original: Theorem 1, the adaptive tuning requires the knowledge of original risk functions { R t } Tt =1 , which are also unavailable.
Revised: Theorem 1, the adaptive tuning requires the knowledge of original risk functions { R t } Tt =1 , which are also unavailable.

Original: Intuitively, the hardness of such optimal step size tuning essentially comes from the uncertainty of the non-stationary label shifts.
Revised: Intuitively, the hardness of such optimal step size tuning essentially comes from the uncertainty of the non-stationary label shifts.

Original: To overcome the difficulty, inspired by recent advances in non-stationary online learning [18, 19], we propose an online ensemble algorithm for the OLaS problem called A TLAS (Adapting To LAbel
Revised: To overcome the difﬁculty, inspired by recent advances in non-stationary online learning [18, 19], we propose an online ensemble algorithm for the OLaS problem called A TLAS (Adapting To LAbel

Original: Shift).
Revised: Shift).

Original: Specifically, to cope with the uncertainty in the optimal step size tuning, we first design a pool of candidate step sizes denoted by H = { η 1 , . . .
Revised: Speciﬁcally, to cope with the uncertainty in the optimal step size tuning, we ﬁrst design a pool of candidate step sizes denoted by H = { η 1 , . . .

Original: , η N } such that η ∗ can be well approximated by at least one of those candidates, which will be configured later and here N is the number of candidate step sizes.
Revised: , η N } such that η ∗ can be well approximated by at least one of those candidates, which will be conﬁgured later and here N is the number of candidate step sizes.

Original: Then, A TLAS deploys a two-layer structure by maintaining a group of base-learners, each associated with a candidate step size from the pool H and then employs a meta-algorithm to track the best base-learner.
Revised: Then, A TLAS deploys a two-layer structure by maintaining a group of base-learners, each associated with a candidate step size from the pool H and then employs a meta-algorithm to track the best base-learner.

Original: The main procedures are presented in Algorithm 1 (base-algorithm) and
Revised: The main procedures are presented in Algorithm 1 (base-algorithm) and

Original: Algorithm 2 (meta-algorithm), and we describe the details below.
Revised: Algorithm 2 (meta-algorithm), and we describe the details below.

Original: Base-algorithm.
Revised: Base-algorithm.

Original: We parallelly run a group of instances of UOGD, each one is associated with a candidate step size in the step size pool H .
Revised: We parallelly run a group of instances of UOGD, each one is associated with a candidate step size in the step size pool H .

Original: Formally, the i -th base-learner yields a sequence of base models { w t,i } Tt =1 , which is updated by w t,i =
Revised: Formally, the i -th base-learner yields a sequence of base models { w t,i } Tt =1 , which is updated by w t,i =

Original: Π W [ w t − 1 ,i − η i ∇ (cid:98) R t − 1 ( w t − 1 ,i )] with η i ∈ H .
Revised: Π W [ w t − 1 ,i − η i ∇ (cid:98) R t − 1 ( w t − 1 ,i )] with η i ∈ H .

Original: Meta-algorithm.
Revised: Meta-algorithm.

Original: The meta-algorithm aims to combine all the base-learners’ decisions such that the final output is competitive with the decisions returned by the (unknown) base-learner associated with the best step size.
Revised: The meta-algorithm aims to combine all the base-learners’ decisions such that the ﬁnal output is competitive with the decisions returned by the (unknown) base-learner associated with the best step size.

Original: To achieve this, we employ a weighted combination mechanism with the final model as w t = (cid:80) N i =1 p t,i w t,i , where p t ∈ ∆ N is the weight vector with p t,i denoting the weight of the i -th base-learner.
Revised: To achieve this, we employ a weighted combination mechanism with the ﬁnal model as w t = (cid:80) N i =1 p t,i w t,i , where p t ∈ ∆ N is the weight vector with p t,i denoting the weight of the i -th base-learner.

Original: We use the classic Hedge algorithm [28] to update the weight vector, namely, p t,i ∝ exp (cid:0) − ε (cid:80) t − 1 s =1 (cid:98) R s ( w s,i ) (cid:1) , where ε > 0 is the learning rate of the meta-algorithm that can be simply set as Θ( (cid:112) (ln N ) /T ) without dependence on label shift quantity.
Revised: We use the classic Hedge algorithm [28] to update the weight vector, namely, p t,i ∝ exp (cid:0) − ε (cid:80) t − 1 s =1 (cid:98) R s ( w s,i ) (cid:1) , where ε > 0 is the learning rate of the meta-algorithm that can be simply set as Θ( (cid:112) (ln N ) /T ) without dependence on label shift quantity.

Original: Intuitively, the meta-algorithm puts larger weights on base-learners with a smaller cumulative estimated risk so that the overall models { w t } Tt =1 can be competitive to the base-learner with the best performance.
Revised: Intuitively, the meta-algorithm puts larger weights on base-learners with a smaller cumulative estimated risk so that the overall models { w t } Tt =1 can be competitive to the base-learner with the best performance.

Original: A TLAS enjoys the following dynamic regret guarantee with the proof in Appendix E.2.
Revised: A TLAS enjoys the following dynamic regret guarantee with the proof in Appendix E.2.

Original: Theorem 2.
Revised: Theorem 2.

Original: Set the step size pool as H =
Revised: Set the step size pool as H =

Original: { η i = Γ σ 2 G √ KT · 2 i − 1 | i ∈
Revised: { η i = Γ σ 2 G √ KT · 2 i − 1 | i ∈

Original: [ N ] } , where
Revised: [ N ] } , where

Original: N = 1 + ⌈ 12 log 2 (1 + 2 T ) ⌉ is the number of base-learners.
Revised: N = 1 + (cid:100) 12 log 2 (1 + 2 T ) (cid:101) is the number of base-learners.

Original: A TLAS ensures that E [ Reg d T ] ≤
Revised: A TLAS ensures that E [ Reg d T ] ≤

Original: O (max { V 13 T T 23 , √ T } ) , or simplified as O ( V 13 T T 23 ) for non-degenerated cases of V T ≥ Θ( T − 12 ) .
Revised: O (max { V 13 T T 23 , √ T } ) , or simpliﬁed as O ( V 13 T T 23 ) for non-degenerated cases of V T ≥ Θ( T − 12 ) .

Original: Theorem 2 shows that A TLAS enjoys the same dynamic regret as UOGD with the (unknown) optimal step size, but unknown non-stationarity V T is no more required in advance.
Revised: Theorem 2 shows that A TLAS enjoys the same dynamic regret as UOGD with the (unknown) optimal step size, but unknown non-stationarity V T is no more required in advance.

Original: Algorithmically, our online method maintains N = Θ(log T ) base-learners to achieve an optimal dynamic regret, which would be computationally acceptable given a logarithmic dependence on T .
Revised: Algorithmically, our online method maintains N = Θ(log T ) base-learners to achieve an optimal dynamic regret, which would be computationally acceptable given a logarithmic dependence on T .

Original: 3.3 More Adaptive Algorithm by Exploiting Label Shift Structures
Revised: 3.3 More Adaptive Algorithm by Exploiting Label Shift Structures

Original: A TLAS is equipped with a minimax optimal dynamic regret, which indicates that it can safeguard the optimal theoretical property in the worst case.
Revised: A TLAS is equipped with a minimax optimal dynamic regret, which indicates that it can safeguard the optimal theoretical property in the worst case.

Original: Worst-case optimality serves as the “stress-testing” for the robustness to non-stationarity environments [29].
Revised: Worst-case optimality serves as the “stress-testing” for the robustness to non-stationarity environments [29].

Original: At the same time, more adaptive results beyond the worst-case analysis are also urgently desired, as improved performance is naturally expected in many easier situations when the shift admits specific patterns such as periodicity or gradual evolution.
Revised: At the same time, more adaptive results beyond the worst-case analysis are also urgently desired, as improved performance is naturally expected in many easier situations when the shift admits speciﬁc patterns such as periodicity or gradual evolution.

Original: To this end, we propose an improved algorithm called A TLAS - ADA with provably more adaptive guarantees.
Revised: To this end, we propose an improved algorithm called A TLAS - ADA with provably more adaptive guarantees.

Original: The key idea is to exploit the label shift patterns and reuse historical information to help the current online update [30].
Revised: The key idea is to exploit the label shift patterns and reuse historical information to help the current online update [30].

Original: We build on the framework of optimistic online learning [31, 32] by introducing a hint function H t : W (cid:55)→ R to encode shift patterns from historical data, which serves as an estimation of the expected risk R t ( w t ) .
Revised: We build on the framework of optimistic online learning [31, 32] by introducing a hint function H t : W (cid:55)→ R to encode shift patterns from historical data, which serves as an estimation of the expected risk R t ( w t ) .

Original: Below, we start with a given hint function and describe the usage, and finally elaborate on how to design H t ( · ) guided by the theory.
Revised: Below, we start with a given hint function and describe the usage, and ﬁnally elaborate on how to design H t ( · ) guided by the theory.

Original: Similar to A TLAS , the improved A TLAS - ADA also deploys a two-layer meta-base structure.
Revised: Similar to A TLAS , the improved A TLAS - ADA also deploys a two-layer meta-base structure.

Original: The key difference lies in the usage of the hint function at both the base-level and meta-level.
Revised: The key difference lies in the usage of the hint function at both the base-level and meta-level.

Original: Base-algorithm.
Revised: Base-algorithm.

Original: Besides the gradient descent step as did in A TLAS , another update step related to the hint function H t ( · ) is performed.
Revised: Besides the gradient descent step as did in A TLAS , another update step related to the hint function H t ( · ) is performed.

Original: Concretely, the i -th base-learner updates the parameters by (cid:98)w t,i = Π W [ (cid:98) w t − 1 ,i − [Equation] i
Revised: Concretely, the i -th base-learner updates the parameters by (cid:98)

Original: [Equation]
Revised: [Equation]

Original: (cid:98)

Original: [Equation]

Original: [Equation]

Original: [Equation] i )] , [Equation] where (cid:98) w t,i is an intermediate output and w t,i is the final returned model.
Revised: where (cid:98) w t,i is an intermediate output and w t,i is the ﬁnal returned model.

Original: When H t ( w ) = 

Revised: When H t ( w ) =

Original: (i.e., without a hint function), the above two-step update simply degenerates to the same UOGD update in the base-learner of A TLAS by noting that now w t,i = (cid:98) w t,i .
Revised: (i.e., without a hint function), the above two-step update simply degenerates to the same UOGD update in the base-learner of A TLAS by noting that now w t,i = (cid:98) w t,i .

Original: In the general H t ( · ) case, the second step in (6) is crucial and can be regarded as another descent towards the direction specified by the hint function.
Revised: In the general H t ( · ) case, the second step in (6) is crucial and can be regarded as another descent towards the direction speciﬁed by the hint function.

Original: As a result, this will reduce the regret whenever the hint function is set appropriately to approximate well the next-round risk function, which will be clear in the regret bound presented later.
Revised: As a result, this will reduce the regret whenever the hint function is set appropriately to approximate well the next-round risk function, which will be clear in the regret bound presented later.

Original: Meta-algorithm.
Revised: Meta-algorithm.

Original: The meta-algorithm is used to track the best base-learner, and the hint function
Revised: The meta-algorithm is used to track the best base-learner, and the hint function

Original: H t ( · ) is also necessary to be considered in the update to achieve the adaptivity.
Revised: H t ( · ) is also necessary to be considered in the update to achieve the adaptivity.

Original: To this end, we inject the hint function as the loss evaluation of the meta-algorithm, and then the weight is updated by p t,i ∝ exp (cid:0) − ε (cid:0) (cid:80) t − 1 s =1 (cid:98) R s ( w s,i )
Revised: To this end, we inject the hint function as the loss evaluation of the meta-algorithm, and then the weight is updated by p t,i ∝ exp (cid:0) − ε (cid:0) (cid:80) t − 1 s =1 (cid:98) R s ( w s,i )

Original: + H t ( w t,i ) (cid:1)(cid:1) for all i ∈ [ N ] , where ε is the learning rate that can be set properly without dependence on V T .
Revised: + H t ( w t,i ) (cid:1)(cid:1) for all i ∈ [ N ] , where ε is the learning rate that can be set properly without dependence on V T .

Original: The key distinction to the meta-algorithm of A TLAS is the additional loss H t ( w t,i ) evaluated over the current local models { w t,i } Ni =1 by the hint function.
Revised: The key distinction to the meta-algorithm of A TLAS is the additional loss H t ( w t,i ) evaluated over the current local models { w t,i } Ni =1 by the hint function.

Original: The main procedures are presented in Algorithm 4 (base-algorithm) and Algorithm 5 (meta-algorithm).
Revised: The main procedures are presented in Algorithm 4 (base-algorithm) and Algorithm 5 (meta-algorithm).

Original: We have the following dynamic regret bound for A TLAS - ADA (proof in Appendix F.2).
Revised: We have the following dynamic regret bound for A TLAS - ADA (proof in Appendix F.2).

Original: Theorem 3.
Revised: Theorem 3.

Original: Suppose the hint function H t : W (cid:55)→ R is convex, satisfies max w ∈W ∥∇ H t ( w ) ∥ 2 ≤ max w ∈W ∥∇ (cid:98) R t ( w ) ∥ 2 , and is independent of current data S t .
Revised: Suppose the hint function H t : W (cid:55)→ R is convex, satisﬁes max w ∈W (cid:107)∇ H t ( w ) (cid:107) 2 ≤ max w ∈W (cid:107)∇ (cid:98) R t ( w ) (cid:107) 2 , and is independent of current data S t .

Original: Set the step size pool as H =
Revised: Set the step size pool as H =

Original: { η i = Γ σ √ σ 2 +4 G 2 KT · 2 i − 1 | i ∈
Revised: { η i = Γ σ √ σ 2 +4 G 2 KT · 2 i − 1 | i ∈

Original: [ N ] } with N = 2{+
+} + ⌈ 12 log 2 (3 T (1 + 4 G 2 KT/σ )) 
Revised: [ N ] } with N = {+
+} + (cid:100) 12 log 2 (3 T (1 + 4 G 2 KT/σ )) (cid:101) .

Original: A TLAS - ADA ensures
Revised: A TLAS - ADA ensures

Original: E [ Reg d T ] ≤ O ( V 1 / 3 T
Revised: E [ Reg d T ] ≤ O ( V 1 / 3 T

Original: G 1 / 3 T T 1 / 3 ) , where G T =
Revised: G 1 / 3 T T 1 / 3 ) , where G T =

Original: (cid:80) Tt =1 E [sup w ∈W ∥∇ (cid:98) R t ( w ) − ∇ H t ( w ) ∥ 22 ] measures the reusability of historical information, depending on label shift patterns and hint function designs.
Revised: (cid:80) Tt =1 E [sup w ∈W (cid:107)∇ (cid:98) R t ( w ) − ∇ H t ( w ) (cid:107) 22 ] measures the reusability of historical information, depending on label shift patterns and hint function designs.

Original: In the worst case, G T is at most O ( T ) given a bounded gradient norm ∥∇ H t ( w ) ∥ 2 , and thus the bound presented in Theorem 3 safeguards the same O
Revised: In the worst case, G T is at most O ( T ) given a bounded gradient norm (cid:107)∇ H t ( w ) (cid:107) 2 , and thus the bound presented in Theorem 3 safeguards the same O

Original: ( V 1 / 3 T
Revised: ( V 1 / 3 T

Original: T 2 / 3 ) minimax bound as A TLAS .
Revised: T 2 / 3 ) minimax bound as A TLAS .

Original: More importantly, when the hint function H t ( · ) encodes beneficial information and is close to the risk function, the obtained bound can be substantially better than the minimax rate.
Revised: More importantly, when the hint function H t ( · ) encodes beneﬁcial information and is close to the risk function, the obtained bound can be substantially better than the minimax rate.

Original: Remark 4 (Implicit update) .
Revised: Remark 4 (Implicit update) .

Original: Problem-dependent dynamic regret was first presented in [19] for convex and smooth functions.
Revised: Problem-dependent dynamic regret was ﬁrst presented in [19] for convex and smooth functions.

Original: However, their result critically relied on the smoothness condition, which is not satisfied in our OLaS case.
Revised: However, their result critically relied on the smoothness condition, which is not satisﬁed in our OLaS case.

Original: Our key technical innovation is the implicit update in the second step of (6).
Revised: Our key technical innovation is the implicit update in the second step of (6).

Original: The previous method required the gradient-descent type update w t,i =
Revised: The previous method required the gradient-descent type update w t,i =

Original: Π w ∈W [ (cid:98) w t,i − η i ∇ H t ( w t − 1 ,i )] , which can be deemed as an approximated optimization over the linearized loss ⟨∇ H t ( w t − 1 ,i ) , w ⟩ .
Revised: Π w ∈W [ (cid:98) w t,i − η i ∇ H t ( w t − 1 ,i )] , which can be deemed as an approximated optimization over the linearized loss (cid:104)∇ H t ( w t − 1 ,i ) , w (cid:105) .

Original: By contrast, we directly updates over the original function H t ( w ) , hence called the “implicit” update [33, 34].
Revised: By contrast, we directly updates over the original function H t ( w ) , hence called the “implicit” update [33, 34].

Original: Albeit with slightly larger computational complexity (which will not be a barrier given a proper design of hint functions), our method enjoys the same dynamic regret without smoothness, which could be of independent interest for general OCO purposes.
Revised: Albeit with slightly larger computational complexity (which will not be a barrier given a proper design of hint functions), our method enjoys the same dynamic regret without smoothness, which could be of independent interest for general OCO purposes.

Original: Design of Hint Functions.
Revised: Design of Hint Functions.

Original: The hint functions should minimize the reusability measure G T to sharpen the dynamic regret as suggested by Theorem 3.
Revised: The hint functions should minimize the reusability measure G T to sharpen the dynamic regret as suggested by Theorem 3.

Original: Recall that (cid:98) R t ( w ) =
Revised: Recall that (cid:98) R t ( w ) =

Original: (cid:80)
Revised: (cid:80)

Original: Kk =

Revised: Kk =

Original: [ (cid:98) µ y t ] k · (cid:98) R k 0 ( w ) .
Revised: [ (cid:98) µ y t ] k · (cid:98) R k 0 ( w ) .

Original: Thus, a natural construction is H t ( w ) =
Revised: Thus, a natural construction is H t ( w ) =

Original: (cid:80)
Revised: (cid:80)

Original: Kk =

Revised: Kk =

Original: [ h y t ] k · (cid:98) R k 0 ( w ) parametrized by hint priors h y t ∈ R K , which is used to estimate the class prior based on the past observed data { S τ } t − 1 τ =1 .
Revised: [ h y t ] k · (cid:98) R k 0 ( w ) parametrized by hint priors h y t ∈ R K , which is used to estimate the class prior based on the past observed data { S τ } t − 1 τ =1 .

Original: Then, G T satisfies the bias-variance decomposition (with bias term E [ ∥ h y t − µ y t ∥ 22 ] and variance term E
Revised: Then, G T satisﬁes the bias-variance decomposition (with bias term E [ (cid:107)

Original:  [ ∥ µ y t − (cid:98) µ y t ∥ 22 ] ):
Revised: h y t − µ y t (cid:107) 22 ] and variance term E [ (cid:107) µ y t − (cid:98) µ y t (cid:107) 22 ] ):

Original: G T ≤ KG
Revised: G T ≤ KG

Original: T (cid:88) t =
Revised: T (cid:88) t =

Original: E (cid:2) ∥
Revised: E (cid:2) (cid:107) h

Original: h y t − (cid:98) µ y t ∥ 22 (cid:3)≤ 2 KG
Revised: y t − (cid:98) µ y t (cid:107) 22 (cid:3)

Original: T (cid:88) t = (cid:16) E (cid:2) ∥ h y t − µ y t ∥ 22 (cid:3) + E (cid:2) ∥ µ y t − (cid:98) µ y t ∥ 22 (cid:3) (cid:17) .
Revised: [Table]

Original: Setting h y t = µ y t will make the upper bound tightest possible, though the underlying class prior µ y t is not accessible in practice.
Revised: Setting h y t = µ y t will make the upper bound tightest possible, though the underlying class prior µ y t is not accessible in practice.

Original: So the design of the hint function is actually a task of approximating it with different parts of previous data guided by prior patterns.
Revised: So the design of the hint function is actually a task of approximating it with different parts of previous data guided by prior patterns.

Original: In the experiments, we design four hint functions by encoding different knowledge, including Forward Hint ( Fwd ), Window Hint ( Win ),
Revised: In the experiments, we design four hint functions by encoding different knowledge, including Forward Hint ( Fwd ), Window Hint ( Win ),

Original: Periodic Hint ( Peri ), and Online KMeans Hint ( OKM ).
Revised: Periodic Hint ( Peri ), and Online KMeans Hint ( OKM ).

Original: More details are in Appendix F.1.
Revised: More details are in Appendix F.1.

Original:  Experiments
Revised:  Experiments

Original: In this section, we conduct extensive experiments to validate the effectiveness of the proposed methods (A TLAS and A TLAS -A DA ) and justify the theoretical findings.
Revised: In this section, we conduct extensive experiments to validate the effectiveness of the proposed methods (A TLAS and A TLAS -A DA ) and justify the theoretical ﬁndings.

Original: We begin this section with a brief introduction to the experimental setups (more details are deferred to Appendix A) and then present empirical results on the synthetic and real-world data, respectively.
Revised: We begin this section with a brief introduction to the experimental setups (more details are deferred to Appendix A) and then present empirical results on the synthetic and real-world data, respectively.

Original: Experiments Setup.
Revised: Experiments Setup.

Original: We compare seven algorithms in various experimental configurations.
Revised: We compare seven algorithms in various experimental conﬁgurations.

Original: The contenders include a baseline that predicts with the initial model directly ( FIX ), three OLaS algorithms proposed by the previous work [16] ( ROGD , FTH , and FTFWH ), and our proposals ( UOGD , A TLAS , and A TLAS -A DA ) with the logistic regression model.
Revised: The contenders include a baseline that predicts with the initial model directly ( FIX ), three OLaS algorithms proposed by the previous work [16] ( ROGD , FTH , and FTFWH ), and our proposals ( UOGD , A TLAS , and A TLAS -A DA ) with the logistic regression model.

Original: Besides, we simulate four types of label shift on synthetic and benchmark data to capture different distribution change patterns.
Revised: Besides, we simulate four types of label shift on synthetic and benchmark data to capture different distribution change patterns.

Original: Two of them, including Sine Shift ( ) and Square Shift ( ) change in a periodic pattern.
Revised: Two of them, including Sine Shift ( ) and Square Shift ( ) change in a periodic pattern.

Original: The other two have no periodic structure but are introduced to capture different shift intensities.
Revised: The other two have no periodic structure but are introduced to capture different shift intensities.

Original: The underlying distribution changes slowly in the Linear Shift ( ) while changes fast in Bernoulli
Revised: The underlying distribution changes slowly in the Linear Shift ( ) while changes fast in Bernoulli

Original: Shift ( ).
Revised: Shift ( ).

Original: We repeat all experiments for five times and evaluate the contenders by the average error1 T (cid:80) T t =1 1 | S t | (cid:80) | S t | n =1 1 [ f ( w , x n ) ̸
Revised: We repeat all experiments for ﬁve times and evaluate the contenders by the average error

Original: = y n ] over T = 10 , 000 rounds.
Revised: [Table] [Table] [Table]

Original: 4.1 Illustrations on Synthetic Data
Revised: 4.1 Illustrations on Synthetic Data

Original: This subsection first compares all contenders on the synthetic data.
Revised: This subsection ﬁrst compares all contenders on the synthetic data.

Original: Then, we further illustrate the effectiveness of our proposal by a closer look at the two key components, including a meta-base structure for step size search and a hint function for historical information reuse.
Revised: Then, we further illustrate the effectiveness of our proposal by a closer look at the two key components, including a meta-base structure for step size search and a hint function for historical information reuse.

Original: Overall Performance Comparison.
Revised: Overall Performance Comparison.

Original: Table 1 compares A TLAS with other methods when N t =samples are received at every iteration.
Revised: Table 1 compares A TLAS with other methods when N t =samples are received at every iteration.

Original: Basically, FIX is inferior to the online methods, which shows the necessity of designing online algorithms for the OLaS problem.
Revised: Basically, FIX is inferior to the online methods, which shows the necessity of designing online algorithms for the OLaS problem.

Original: Besides, UOGD outperforms
Revised: Besides, UOGD outperforms

Original: ROGD , the OGD algorithm running with the risk estimator proposed by [16].
Revised: ROGD , the OGD algorithm running with the risk estimator proposed by [16].

Original: The comparison demonstrates the empirical superiority of our estimator besides its benign theoretical properties.
Revised: The comparison demonstrates the empirical superiority of our estimator besides its benign theoretical properties.

Original: Moreover, the A TLAS surpasses almost all other methods in the four shift patterns.
Revised: Moreover, the A TLAS surpasses almost all other methods in the four shift patterns.

Original: In particular, it achieves a significant advantage over UOGD (with step size η = Θ( T − 1 / 2 ) ) when the environments change relatively fast ( Squ , Sin , and Ber ).
Revised: In particular, it achieves a signiﬁcant advantage over UOGD (with step size η = Θ( T − 1 / 2 ) ) when the environments change relatively fast ( Squ , Sin , and Ber ).

Original: The results justify our theoretical finding that the small step size Θ( T − 1 / 2 ) suggested by the static regret analysis is unsuitable for the dynamic environments.
Revised: The results justify our theoretical ﬁnding that the small step size Θ( T − 1 / 2 ) suggested by the static regret analysis is unsuitable for the dynamic environments.

Original: Our method can better adapt to the changing environments by enjoying the dynamic regret guarantees.
Revised: Our method can better adapt to the changing environments by enjoying the dynamic regret guarantees.

Original: The best algorithms are emphasized in bold (paired t -test at a 5% confidence level).
Revised: The best algorithms are emphasized in bold (paired t -test at a 5% conﬁdence level).

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure] 
Revised: [Figure] [Table] [Figure] [Figure]

Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: Effectiveness of Meta-Base Structure.
Revised: Effectiveness of Meta-Base Structure.

Original: One key component of our method is the meta-base structure to address the non-stationarity.
Revised: One key component of our method is the meta-base structure to address the non-stationarity.

Original: To better illustrate its effectiveness, we visualize the weights p t,i assigned for each base-learner of A TLAS .
Revised: To better illustrate its effectiveness, we visualize the weights p t,i assigned for each base-learner of A TLAS .

Original: As shown in Figure 5, the meta-algorithm can quickly assign larger weights to appropriate base-learners along the learning process.
Revised: As shown in Figure 5, the meta-algorithm can quickly assign larger weights to appropriate base-learners along the learning process.

Original: Specifically, Figure 1(a) illustrates the case of slowly changing environments, where more weights are assigned to the baselearners with small step sizes.
Revised: Speciﬁcally, Figure 1(a) illustrates the case of slowly changing environments, where more weights are assigned to the baselearners with small step sizes.

Original: In the fast-changing case, see Figure 1(b), larger step sizes are preferred.
Revised: In the fast-changing case, see Figure 1(b), larger step sizes are preferred.

Original: The results show that our algorithm can adaptively track the suitable step sizes according to the shift intensity of environments.
Revised: The results show that our algorithm can adaptively track the suitable step sizes according to the shift intensity of environments.

Original: Additional results for other shifts can be found in Figure 5.
Revised: Additional results for other shifts can be found in Figure 5.

Original: Effectiveness of Using Hint Functions.
Revised: Effectiveness of Using Hint Functions.

Original: Table 2 reports the performance of A TLAS -A DA with four different hint functions under sample sizes are N t = 1 , 10 , and 100 .
Revised: Table 2 reports the performance of A TLAS -A DA with four different hint functions under sample sizes are N t = 1 , 10 , and 100 .

Original: All hint functions improve over vanilla A TLAS in most cases.
Revised: All hint functions improve over vanilla A TLAS in most cases.

Original: When N t is reasonably large, the Fwd hint, performing transductive learning with current unlabeled data, achieves the best performance.
Revised: When N t is reasonably large, the Fwd hint, performing transductive learning with current unlabeled data, achieves the best performance.

Original: While, the OKM hint, which learns previous patterns by online K-means, is the best choice for a small sample size case ( N t = 1 ).
Revised: While, the OKM hint, which learns previous patterns by online K-means, is the best choice for a small sample size case ( N t = 1 ).

Original: To illustrate how the hint function works, we further vary the buffer size of the Periodic Hint ( Peri ) on the Squ environment ( ), which shifts in a periodic length L = 40 .
Revised: To illustrate how the hint function works, we further vary the buffer size of the Periodic Hint ( Peri ) on the Squ environment ( ), which shifts in a periodic length L = 40 .

Original: As shown in Figure 2, when the buffer size matches the multiples of length L , Peri can significantly improve the vanilla A TLAS .
Revised: As shown in Figure 2, when the buffer size matches the multiples of length L , Peri can signiﬁcantly improve the vanilla A TLAS .

Original: Besides the improvement, our method is shown to achieve comparable performance with A TLAS even if the buffer size is misspecified.
Revised: Besides the improvement, our method is shown to achieve comparable performance with A TLAS even if the buffer size is misspeciﬁed.

Original: The result validates our theoretical guarantee of the safety of using hint functions (the readers can refer to the paragraph above Remark 4).
Revised: The result validates our theoretical guarantee of the safety of using hint functions (the readers can refer to the paragraph above Remark 4).

Original: 4.2 Comparisons on Real-world Data
Revised: 4.2 Comparisons on Real-world Data

Original: We conduct experiments on real-world data, including six benchmark datasets (ArXiv, EuroSAT,
Revised: We conduct experiments on real-world data, including six benchmark datasets (ArXiv, EuroSAT,

Original: MNIST, Fashion, CIFAR10, and CINIC10) and the SHL dataset [35] for the real-life locomotion detection task.
Revised: MNIST, Fashion, CIFAR10, and CINIC10) and the SHL dataset [35] for the real-life locomotion detection task.

Original: Table 3 reports the averaged error of different algorithms, which shows a similar tendency as the results in the synthetic experiments.
Revised: Table 3 reports the averaged error of different algorithms, which shows a similar tendency as the results in the synthetic experiments.

Original: When the distribution changes rapidly ( Ber ),
Revised: When the distribution changes rapidly ( Ber ),

Original: A TLAS and A TLAS -A DA outperform other contenders.
Revised: A TLAS and A TLAS -A DA outperform other contenders.

Original: Similar results are also observed in the Sine and Square shift, see Appendix A.1 for details.
Revised: Similar results are also observed in the Sine and Square shift, see Appendix A.1 for details.

Original: Even in a relatively stationary environment ( Lin ), our algorithms are comparable with the best algorithm (UOGD), which is specifically designed for stationary cases.
Revised: Even in a relatively stationary environment ( Lin ), our algorithms are comparable with the best algorithm (UOGD), which is speciﬁcally designed for stationary cases.

Original: The above results validate the adaptivity of the proposed algorithms.
Revised: The above results validate the adaptivity of the proposed algorithms.

Original: Further, we highlight the results on the locomotion detection task.
Revised: Further, we highlight the results on the locomotion detection task.

Original: The task aims to distinguish six types of locomotion with sensor data from mobile phones.
Revised: The task aims to distinguish six types of locomotion with sensor data from mobile phones.

Original: Figure 3(a) reports the averaged error of all contenders, which shows the superiority of our proposals A TLAS and A TLAS - ADA (with the OKM hint) over the entire time horizons.
Revised: Figure 3(a) reports the averaged error of all contenders, which shows the superiority of our proposals A TLAS and A TLAS - ADA (with the OKM hint) over the entire time horizons.

Original: In addition, to further validate the adaptivity of our algorithm to the underlying environment regardless of the fast or slow changes, we simulate various shift
Revised: In addition, to further validate the adaptivity of our algorithm to the underlying environment regardless of the fast or slow changes, we simulate various shift

Original: [Table]
Revised: [Table]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: intensities by sampling the original data with different frequencies.
Revised: intensities by sampling the original data with different frequencies.

Original: Figure 3(b) shows the weight assignment of A TLAS -A DA for each step size in the final round.
Revised: Figure 3(b) shows the weight assignment of A TLAS -A DA for each step size in the ﬁnal round.

Original: Our method automatically selects a larger step size for larger V T while tracking a small step size in a relatively static scenario.
Revised: Our method automatically selects a larger step size for larger V T while tracking a small step size in a relatively static scenario.

Original:  Conclusion
Revised:  Conclusion

Original: This paper proposed algorithms for online label shift with provable guarantees.
Revised: This paper proposed algorithms for online label shift with provable guarantees.

Original: We constructed an unbiased risk estimator without using any supervision at test time.
Revised: We constructed an unbiased risk estimator without using any supervision at test time.

Original: Then, we designed novel online ensemble algorithms that automatically adapt to the non-stationary online label shift and enjoy problem-dependent dynamic regret.
Revised: Then, we designed novel online ensemble algorithms that automatically adapt to the non-stationary online label shift and enjoy problem-dependent dynamic regret.

Original: Our proposed A TLAS algorithm employed a meta-base structure to handle the non-stationarity and obtained an O ( V 1 / 3 T
Revised: Our proposed A TLAS algorithm employed a meta-base structure to handle the non-stationarity and obtained an O ( V 1 / 3 T

Original: T 2 / 3 ) guarantee, and A TLAS - ADA further introduced hint functions to exploit the shift structure and obtained an improved O ( V 1 / 3 T
Revised: T 2 / 3 ) guarantee, and A TLAS - ADA further introduced hint functions to exploit the shift structure and obtained an improved O ( V 1 / 3 T

Original: G 1 / 3 T T 1 / 3 ) guarantee.
Revised: G 1 / 3 T T 1 / 3 ) guarantee.

Original: Extensive experiments validated the effectiveness of the proposed algorithms.
Revised: Extensive experiments validated the effectiveness of the proposed algorithms.

Original: Our study serves as a preliminary attempt to bridge the distribution change problem and online learning techniques by focusing on the label shift case.
Revised: Our study serves as a preliminary attempt to bridge the distribution change problem and online learning techniques by focusing on the label shift case.

Original: Considering a more general distribution change setting is an important future direction.
Revised: Considering a more general distribution change setting is an important future direction.

Original: Besides, our current algorithm is designed for the most challenging unlabeled scenario, and it is interesting to consider relaxed real-world demand where a few labels could be available in the learning process.
Revised: Besides, our current algorithm is designed for the most challenging unlabeled scenario, and it is interesting to consider relaxed real-world demand where a few labels could be available in the learning process.

Original: Moreover, our obtained regret guarantees hold in expectation, and we will take high-probability bounds as the future work.
Revised: Moreover, our obtained regret guarantees hold in expectation, and we will take high-probability bounds as the future work.

Original: Acknowledgments
Revised: Acknowledgments

Original: Yong Bai, Peng Zhao, and Zhi-Hua Zhou were supported by NSFC (61921006), JiangsuSF (BK20220776), and Collaborative Innovation Center of Novel Software Technology and Industrialization.
Revised: Yong Bai, Peng Zhao, and Zhi-Hua Zhou were supported by NSFC (61921006), JiangsuSF (BK20220776), and Collaborative Innovation Center of Novel Software Technology and Industrialization.

Original: Yu-Jie Zhang was supported by Todai Fellowship and the Institute for AI and Beyond,
Revised: Yu-Jie Zhang was supported by Todai Fellowship and the Institute for AI and Beyond,

Original: UTokyo.
Revised: UTokyo.

Original: Masashi Sugiyama was supported by JST AIP Acceleration Research Grant Number
Revised: Masashi Sugiyama was supported by JST AIP Acceleration Research Grant Number

Original: JPMJCR20U3 and the Institute for AI and Beyond, UTokyo.
Revised: JPMJCR20U3 and the Institute for AI and Beyond, UTokyo.

Original: Peng Zhao thanks Chen-Yu Wei for discussions on the conversation between function variation and gradient variation.
Revised: Peng Zhao thanks Chen-Yu Wei for discussions on the conversation between function variation and gradient variation.

Original: The authors also thank Ruihan Wu and Yu-Yang Qian for their helpful discussions.
Revised: The authors also thank Ruihan Wu and Yu-Yang Qian for their helpful discussions.

Original: Checklist
Revised: Checklist

Original: The checklist follows the references.
Revised: The checklist follows the references.

Original: Please read the checklist guidelines carefully for information on how to answer these questions.
Revised: Please read the checklist guidelines carefully for information on how to answer these questions.

Original: For each question, change the default [TODO] to [Yes] , [No] or [N/A] .
Revised: For each question, change the default [TODO] to [Yes] , [No] or [N/A] .

Original: You are strongly encouraged to include a justification to your answer, either by referencing the appropriate section of your paper or providing a brief inline description.
Revised: You are strongly encouraged to include a justiﬁcation to your answer either by referencing the appropriate section of your paper or providing a brief inline description.

Original: Please do not modify the questions and only use the provided macros for your answers.
Revised: Please do not modify the questions and only use the provided macros for your answers.

Original: Note that the Checklist section does not count towards the page limit.
Revised: Note that the Checklist section does not count towards the page limit.

Original: In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.
Revised: In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] See Remark 4 for a discussion on computational efficiency and Appendix 5 for a discussion on high probability bound.
Revised: [Yes] See Remark 4 for a discussion on computational efﬁciency and Section 5 for a discussion on high probability bound.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [No] We cannot foresee a potential negative societal impact of this work.
Revised: [No] We cannot foresee a potential negative societal impact of this work.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 2.
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] (b) Did you include complete proofs of all theoretical results?
Revised: [Yes] (b) Did you include complete proofs of all theoretical results?

Original: [Yes] See Appendix D- F. 3.
Revised: [Yes] See Appendix D- F. 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] See the supplemental material.
Revised: [Yes] See the supplemental material.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] See Appendix A.2.
Revised: [Yes] See Appendix A.2.

Original: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] 4.
Revised: [Yes] 4.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?
Revised: [Yes] (b) Did you mention the license of the assets?

Original: [Yes] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [Yes] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes] Our code is in the supplemental material.
Revised: [Yes] Our code is in the supplemental material.

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [Yes] All the data are open source.
Revised: [Yes] All the data are open source.

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
Revised: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?

Original: [Yes] None of the personally identifiable information or offensive content is involved.
Revised: [Yes] None of the personally identiﬁable information or offensive content is involved.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [No] We do not use crowdsourcing.
Revised: [No] We do not use crowdsourcing.

Original: (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [No] We do not use crowdsourcing.
Revised: [No] We do not use crowdsourcing.

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [No] We do not use crowdsourcing.
Revised: [No] We do not use crowdsourcing.

Original: Supplementary Material
Revised: Supplementary Material

Original: for “Adapting to Online Label Shift with Provable Guarantees”
Revised: for “Adapting to Online Label Shift with Provable Guarantees”

Original: This is the supplemental material for the paper “Adapting to Online Label Shift with Provable
Revised: This is the supplemental material for the paper “Adapting to Online Label Shift with Provable

Original: Guarantees”.
Revised: Guarantees”.

Original: The appendix is organized as follows.
Revised: The appendix is organized as follows.

Original: • Appendix A: details of the experiments.
Revised: • Appendix A: details of the experiments.

Original: – Appendix A.1: more numerical results on real-world datasets.
Revised: – Appendix A.1: more numerical results on real-world datasets.

Original: – Appendix A.2: details of experimental setup.
Revised: – Appendix A.2: details of experimental setup.

Original: – Appendix A.3: additional experimental results.
Revised: – Appendix A.3: additional experimental results.

Original: • Appendix B: a brief introduction to OCO and BBSE for label shift.
Revised: • Appendix B: a brief introduction to OCO and BBSE for label shift.

Original: – Appendix B.1: review of the OCO framework.
Revised: – Appendix B.1: review of the OCO framework.

Original: – Appendix B.2: details of the BBSE method.
Revised: – Appendix B.2: details of the BBSE method.

Original: • Appendix C: related work and discussion.
Revised: • Appendix C: related work and discussion.

Original: – Appendix C.1: related work of the offline label shift problem.
Revised: – Appendix C.1: related work of the ofﬂine label shift problem.

Original: – Appendix C.2: related work of non-stationary online convex optimization.
Revised: – Appendix C.2: related work of non-stationary online convex optimization.

Original: – Appendix C.3: comparison on the risk estimator of our method and the work [16].
Revised: – Appendix C.3: comparison on the risk estimator of our method and the work [16].

Original: • Appendix D: omitted proofs for Section 3.1.
Revised: • Appendix D: omitted proofs for Section 3.1.

Original: – Appendix D.1: proof of Lemma 1.
Revised: – Appendix D.1: proof of Lemma 1.

Original: – Appendix D.2: high-probability bound of the risk estimator.
Revised: – Appendix D.2: high-probability bound of the risk estimator.

Original: – Appendix D.3: static regret for UOGD algorithm.
Revised: – Appendix D.3: static regret for UOGD algorithm.

Original: • Appendix E: omitted proofs for Section 3.2.
Revised: • Appendix E: omitted proofs for Section 3.2.

Original: – Appendix E.1: proof of Theorem 1.
Revised: – Appendix E.1: proof of Theorem 1.

Original: – Appendix E.2: proof of Theorem 2.
Revised: – Appendix E.2: proof of Theorem 2.

Original: – Appendix E.3: useful lemmas in the analysis of Theorem 1 and Theorem 2.
Revised: – Appendix E.3: useful lemmas in the analysis of Theorem 1 and Theorem 2.

Original: • Appendix F: algorithm details for A TLAS - ADA and omitted proofs for Section 3.3.
Revised: • Appendix F: algorithm details for A TLAS - ADA and omitted proofs for Section 3.3.

Original: – Appendix F.1: algorithm details for A TLAS - ADA and the design of the hint function.
Revised: – Appendix F.1: algorithm details for A TLAS - ADA and the design of the hint function.

Original: – Appendix F.2: proof of Theorem 3.
Revised: – Appendix F.2: proof of Theorem 3.

Original: – Appendix F.3: useful lemmas in the analysis of Theorem 3.
Revised: – Appendix F.3: useful lemmas in the analysis of Theorem 3.

Original: • Appendix G: technical lemmas.
Revised: • Appendix G: technical lemmas.

Original: A Omitted Details for Experiments
Revised: A Omitted Details for Experiments

Original: In this section, we mainly supplement the omitted details in Section 4.
Revised: In this section, we mainly supplement the omitted details in Section 4.

Original: We first present the omitted numerical results on benchmark datasets in Appendix A.1.
Revised: We ﬁrst present the omitted numerical results on benchmark datasets in Appendix A.1.

Original: Then we list the experiment setups in
Revised: Then we list the experiment setups in

Original: Appendix A.2, followed by additional experimental results in Appendix A.3.
Revised: Appendix A.2, followed by additional experimental results in Appendix A.3.

Original: A.1 Omitted Numerical Results for Section 4. 
Revised: A.1 More Numerical Results for Section 4.

Original: Table 4 presents the numerical results omitted in Section 4.2, which reports the average error of different algorithms on various real-world datasets.
Revised: Table 4 presents the numerical results omitted in Section 4.2, which reports the average error of different algorithms on various real-world datasets.

Original: The results show that the A TLAS and A TLAS - ADA methods outperform other contenders on all tasks.
Revised: The results show that the A TLAS and A TLAS - ADA methods outperform other contenders on all tasks.

Original: The advantage is particularly significant when the underlying class prior changes fast.
Revised: The advantage is particularly signiﬁcant when the underlying class prior changes fast.

Original: The empirical results validate that our method can effectively adapt to the non-stationary label shift.
Revised: The empirical results validate that our method can effectively adapt to the non-stationary label shift.

Original: A.2 Experiment Setup
Revised: A.2 Experiment Setup

Original: This subsection describes details of experimental setups, including contenders, simulated shifts, and datasets.
Revised: This subsection describes details of experimental setups, including contenders, simulated shifts, and datasets.

Original: All our experiments are run on a machine with 2 CPUs (24 cores for each).
Revised: All our experiments are run on a machine with 2 CPUs (24 cores for each).

Original: [Table]
Revised: [Table]

Original: Contenders.
Revised: Contenders.

Original: In the experiments
Revised: In the experiments

Original: [Table]
Revised: [Table]

Original: we mainly compare seven online label shift algorithms, including:
Revised: we mainly compare seven online label shift algorithms, including:

Original: • FIX predicts with the fixed initialized classifier without any online updates.
Revised: • FIX predicts with the ﬁxed initialized classiﬁer without any online updates.

Original: • ROGD is a variant of OGD algorithm proposed by Wu et al.
Revised: • ROGD is a variant of OGD algorithm proposed by Wu et al.

Original: The algorithm constructs the risk estimator with the 0 / 1 loss and uses a re-weighting classifier for the model update.
Revised: The algorithm constructs the risk estimator with the 0 / 1 loss and uses a re-weighting classiﬁer for the model update.

Original: (Detailed description and the comparison with our risk estimator can be found in Appendix C.3).
Revised: (Detailed description and the comparison with our risk estimator can be found in Appendix C.3).

Original: • FTH is short for Following The History algorithm proposed by Wu et al.
Revised: • FTH is short for Following The History algorithm proposed by Wu et al.

Original: The method takes the class prior for every iteration t as the average of all previously estimated priors.
Revised: The method takes the class prior for every iteration t as the average of all previously estimated priors.

Original: • FTFWH is a variant of FTH proposed as a compared method in experiments of Wu et al.
Revised: • FTFWH is a variant of FTH proposed as a compared method in experiments of Wu et al.

Original: The method takes an average across previously estimated priors within a sliding window.
Revised: The method takes an average across previously estimated priors within a sliding window.

Original: In all experiments, the length of the sliding window is set as 

Revised: In all experiments, the length of the sliding window is set as

Original: . • UOGD /A
Revised: . • UOGD /A

Original: TLAS /A
Revised: TLAS /A

Original: TLAS - ADA :
Revised: TLAS - ADA :

Original: The OLaS algorithms proposed in Section 3.
Revised: The OLaS algorithms proposed in Section 3.

Original: Following our theoretical results, the step size of ROGD and UOGD is set to be Γ / ( G √ T ) , where G can be estimated during training the initial model and Γ is given by the decision set domain.
Revised: Following our theoretical results, the step size of ROGD and UOGD is set to be Γ / ( G √ T ) , where G can be estimated during training the initial model and Γ is given by the decision set domain.

Original: In the experiments, we choose the decision domain as a ball with a fixed diameter for each dataset.
Revised: In the experiments, we choose the decision domain as a ball with a ﬁxed diameter for each dataset.

Original: For a fair comparison, all contenders use the same decision domain W with the diameter Γ set according to the parameter norm of the initial offline model f 0 .
Revised: For a fair comparison, all contenders use the same decision domain W with the diameter Γ set according to the parameter norm of the initial ofﬂine model f 0 .

Original: The settings of step size pool of A TLAS and
Revised: The settings of step size pool of A TLAS and

Original: A TLAS - ADA are guided by Theorem 2 and Theorem 3, respectively.
Revised: A TLAS - ADA are guided by Theorem 2 and Theorem 3, respectively.

Original: We employ a multinomial logistic regression classifier for UOGD , A TLAS , and A TLAS - ADA .
Revised: We employ a multinomial logistic regression classiﬁer for UOGD , A TLAS , and A TLAS - ADA .

Original: Lin
Revised: Lin

Original: [Figure]
Revised: [Figure]

Original: Simulated Shifts.
Revised: Simulated Shifts.

Original: We simulate four kinds of label shift patterns to capture different kinds of non-stationary environments.
Revised: We simulate four kinds of label shift patterns to capture different kinds of nonstationary environments.

Original: For each shift, the priors are a mixture of two different constant priors µ 1 , µ 2 ∈ ∆ K with a time-varying coefficient α t : µ y t
Revised: For each shift, the priors are a mixture of two different constant priors µ 1 , µ 2 ∈ ∆ K with a time-varying coefﬁcient α t : µ y t

Original: = (1 − α t ) µ 1 + α t µ 2 , where µ y t denotes the label distribution at round t and α t controls the shift non-stationarity and patterns.
Revised: = (1 − α t ) µ 1 + α t µ 2 , where µ y t denotes the label distribution at round t and α t controls the shift non-stationarity and patterns.

Original: We list the details in the following.
Revised: We list the details in the following.

Original: • Linear Shift ( Lin ) : the parameter α t = tT , which represents the gradually changed environments.
Revised: • Linear Shift ( Lin ) : the parameter α t = tT , which represents the gradually changed environments.

Original: • Square Shift
Revised: • Square Shift

Original: ( Squ ) :
Revised: ( Squ ) :

Original: the parameter α t switches between 1 and 0 every L/ 2 rounds, where L is the periodic length.
Revised: the parameter α t switches between 1 and 0 every L/ 2 rounds, where L is the periodic length.

Original: In the experiments, we set L = Θ( √ T ) by default, which implies that the fluctuation of the class prior V T = (cid:80) Tt =2 ∥ µ y t − µ y t − 1 ∥ 1 is Θ( √ T ) .
Revised: In the experiments, we set L = Θ( √ T ) by default, which implies that the ﬂuctuation of the class prior V T = (cid:80) Tt =2 (cid:107) µ y t − µ y t − 1 (cid:107) 1 is Θ( √ T ) .

Original: The Square Shift simulates a fast-changing environment with periodic patterns.
Revised: The Square Shift simulates a fast-changing environment with periodic patterns.

Original: • Sine Shift ( Sin ) : α t = sin iπL , where i =
Revised: • Sine Shift ( Sin ) : α t = sin iπL , where i =

Original: t mod L and L is a given periodic length.
Revised: t mod L and L is a given periodic length.

Original: In the experiments, we set L = Θ( √ T ) by default.
Revised: In the experiments, we set L = Θ( √ T ) by default.

Original: The Sine Shift also simulates a fast-changing environment with periodic patterns.
Revised: The Sine Shift also simulates a fast-changing environment with periodic patterns.

Original: • Bernoulli Shift
Revised: • Bernoulli Shift

Original: ( Ber ) :
Revised: ( Ber ) :

Original: At every iteration, we keep the α t = α t − 1 with probability p ∈ [0 , 1] and otherwise set α t = 1 − α t − 1 .
Revised: At every iteration, we keep the α t = α t − 1 with probability p ∈ [0 , 1] and otherwise set α t = 1 − α t − 1 .

Original: In the experiments, the parameter is set as p = 1 / √ T by default, which implies V t =
Revised: In the experiments, the parameter is set as p = 1 / √ T by default, which implies V t =

Original: Θ( √ T ) .
Revised: Θ( √ T ) .

Original: The Bernoulli Shift simulates a fast-changing environment without periodic patterns.
Revised: The Bernoulli Shift simulates a fast-changing environment without periodic patterns.

Original: Figure 4 demonstrates how α t changes over time.
Revised: Figure 4 demonstrates how α t changes over time.

Original: We can observe that Square Shift and
Revised: We can observe that Square Shift and

Original: Sine Shift change in a periodic pattern while the others do not.
Revised: Sine Shift change in a periodic pattern while the others do not.

Original: Moreover, it can be seen that
Revised: Moreover, it can be seen that

Original: Linear Shift has simulated a much more tender class prior change than the other three shifts.
Revised: Linear Shift has simulated a much more tender class prior change than the other three shifts.

Original: Datasets.
Revised: Datasets.

Original: We conduct experiments on synthetic data, six real-world datasets, and a real-life application of locomotion detection.
Revised: We conduct experiments on synthetic data, six real-world datasets, and a real-life application of locomotion detection.

Original: Synthetic data.
Revised: Synthetic data.

Original: There are three classes in the synthetic data, where the feature distribution of each class follows a Gaussian distribution.
Revised: There are three classes in the synthetic data, where the feature distribution of each class follows a Gaussian distribution.

Original: More specifically, for each instance ( x i , y i ) in the dataset S t at round t , the label is generated from the discrete distribution defined by the given priors D t ( y ) , and the feature x ∈ R 12 is generated from the corresponding multivariate normal distributions N ( µ y , Σ) .
Revised: More speciﬁcally, for each instance ( x i , y i ) in the dataset S t at round t , the label is generated from the discrete distribution deﬁned by the given priors D t ( y ) , and the feature x ∈ R 12 is generated from the corresponding multivariate normal distributions N ( µ y , Σ) .

Original: We set µ 1 =
Revised: We set µ 1 =

Original: [1 / 3 , 1 / 3 , 1 / 3] and µ 2 =
Revised: [1 / 3 , 1 / 3 , 1 / 3] and µ 2 =

Original: [1 , 0 , 0] for simulated shifts.
Revised: [1 , 0 , 0] for simulated shifts.

Original: Real-world datasets .
Revised: Real-world datasets .

Original: We conduct experiments on six real-world datasets.
Revised: We conduct experiments on six real-world datasets.

Original: For each dataset, we set µ 

Revised: For each dataset, we set µ

Original: =
Revised: =

Original: [1 /K, 1 /K, ..., 1 /K ] and µ 2 =
Revised: [1 /K, 1 /K, ..., 1 /K ] and µ 2 =

Original: [1 , 0 , ..., 0] to generate the simulated shifts.
Revised: [1 , 0 , ..., 0] to generate the simulated shifts.

Original: Specifically, we include the following datasets
Revised: Speciﬁcally, we include the following datasets

Original: • ArXiv 5 : A paper classification dataset which contains metadata of the scholarly articles.
Revised: • ArXiv 5 : A paper classiﬁcation dataset which contains metadata of the scholarly articles.

Original: We select 296, 708 papers from the computer science domain, which cover 23 classes, including cs.
Revised: We select 296, 708 papers from the computer science domain, which cover 23 classes, including cs.

Original: AI, cs.CC, cs.
Revised: AI, cs.CC, cs.

Original: CL, cs.
Revised: CL, cs.

Original: CR, cs.
Revised: CR, cs.

Original: CV, cs.CY, cs.
Revised: CV, cs.CY, cs.

Original: DB, cs.
Revised: DB, cs.

Original: DC, cs.
Revised: DC, cs.

Original: DM, cs.
Revised: DM, cs.

Original: DS, cs.
Revised: DS, cs.

Original: GT, cs.
Revised: GT, cs.

Original: HC, cs.
Revised: HC, cs.

Original: IR, cs.
Revised: IR, cs.

Original: IT, cs.
Revised: IT, cs.

Original: LG, cs.
Revised: LG, cs.

Original: LO, cs.
Revised: LO, cs.

Original: NE, cs. NI, cs.
Revised: NE, cs. NI, cs.

Original: PL, cs.
Revised: PL, cs.

Original: RO, cs.
Revised: RO, cs.

Original: SE, cs.
Revised: SE, cs.

Original: SI, and cs.
Revised: SI, and cs.

Original: SY.
Revised: SY.

Original: 6 We use a finetuned DistilBERT [36] to extract features from authors, titles and abstracts.
Revised: 6 We use a ﬁnetuned DistilBERT [36] to extract features from authors, titles and abstracts.

Original: The papers selected additionally to finetune the DistilBERT do not overlap with the offline and online datasets.
Revised: The papers selected additionally to ﬁnetune the DistilBERT do not overlap with the ofﬂine and online datasets.

Original: • EuroSAT [37]: A land cover classification dataset, which includes satellite images with the purpose of identifying the visible land use or land cover class.
Revised: • EuroSAT [37]: A land cover classiﬁcation dataset, which includes satellite images with the purpose of identifying the visible land use or land cover class.

Original: The dataset consists of 27, 000 satellite images from over 30 different European countries.
Revised: The dataset consists of 27, 000 satellite images from over 30 different European countries.

Original: It contains ten different classes, including industrial, residential, annual crop, permanent crop, river, sea and lake, herbaceous vegetation, highway, pasture, and forest.
Revised: It contains ten different classes, including industrial, residential, annual crop, permanent crop, river, sea and lake, herbaceous vegetation, highway, pasture, and forest.

Original: We use a finetuned ResNet [38] to extract features from the images of EuroSAT and the following four datasets.
Revised: We use a ﬁnetuned ResNet [38] to extract features from the images of EuroSAT and the following four datasets.

Original: The images selected to train the ResNet also do not overlap with both the offline or online datasets.
Revised: The images selected to train the ResNet also do not overlap with both the ofﬂine or online datasets.

Original: • MNIST [39]: A widely-used image dataset of handwritten digits, which consists of 70,grayscale images with 10 different classes.
Revised: • MNIST [39]: A widely-used image dataset of handwritten digits, which consists of 70,grayscale images with 10 different classes.

Original: • Fashion [40]: A dataset of 70, 000 grayscale fashion images, consisting of 10 different classes: T-shirt, trouser, shirt and sneaker, pullover, dress, coat, sandal, bag, and ankle boot.
Revised: • Fashion [40]: A dataset of 70, 000 grayscale fashion images, consisting of 10 different classes: T-shirt, trouser, shirt and sneaker, pullover, dress, coat, sandal, bag, and ankle boot.

Original: • CIFAR

Revised: • CIFAR

Original: [41]: A dataset consists of 60, 000 color images in 10 classes, including airplane, automobile, ship, truck, bird, cat, deer, dog, frog, and horse.
Revised: [41]: A dataset consists of 60, 000 color images in 10 classes, including airplane, automobile, ship, truck, bird, cat, deer, dog, frog, and horse.

Original: • CINIC

Revised: • CINIC

Original: [42]: A tiny ImageNet [43] dataset, which consists of images from CIFAR10 and ImageNet, and has the same ten classes as CIFAR10.
Revised: [42]: A tiny ImageNet [43] dataset, which consists of images from CIFAR10 and ImageNet, and has the same ten classes as CIFAR10.

Original: Real-life application.
Revised: Real-life application.

Original: The real-life application is to distinguish human locomotion through the sensor data collected by the carry-on mobile phones 7 .
Revised: The real-life application is to distinguish human locomotion through the sensor data collected by the carry-on mobile phones 7 .

Original: The tabular data covers the sensor data (e.g., acceleration, gyroscope, magnetometer, orientation, gravity, pressure, altitude, and temperature) and
Revised: The tabular data covers the sensor data (e.g., acceleration, gyroscope, magnetometer, orientation, gravity, pressure, altitude, and temperature) and

Original: Step Size ( ): 0.
Revised: Step Size ( ): 0.

Original: [Figure] Iterations 0.0 0.
Revised: 10101010 3 Iterations 0.0 0.

Original: W e i g h t
Revised: W e i g h t

Original: A ss i g n m e n t (a) Linear shift 10101010 3 Iterations 0.0 0.
Revised: A ss i g n m e n t (a) Linear shift 10101010 3 Iterations 0.0 0.

Original: W e i g h t
Revised: W e i g h t

Revised: A ss i g n m e n t (b) Square shift 10101010 3 Iterations 0.0 0.

Revised: W e i g h t

Original: A ss i g n m e n t ({+(c) Sine shift 
+}
Revised: A ss i g n m e n t {+(c) Sine shift 
+}

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original:  the corresponding human motion and timestamp.
Revised: W e i g h t A ss i g n m e n t (d) Bernoulli shift the corresponding human motion and timestamp.

Original: We sample 30, 000 offline data and 77, 000 online data from 11 days, covering six classes, including still, walking, run, bike, car, and bus.
Revised: We sample 30, 000 ofﬂine data and 77, 000 online data from 11 days, covering six classes, including still, walking, run, bike, car, and bus.

Original: During the online update, the online samples arrive in real chronological order based on the timestamp.
Revised: During the online update, the online samples arrive in real chronological order based on the timestamp.

Original: A.3 Additional Experimental Results
Revised: A.3 Additional Experimental Results

Original: This part further reports the additional experimental results omitted in the main paper.
Revised: This part further reports the additional experimental results omitted in the main paper.

Original: Specifically, we show a supplement for the assigned weight visualization of A TLAS and a skyline to illustrate how the meta-algorithm tracks the proper base-algorithms.
Revised: Speciﬁcally, we show a supplement for the assigned weight visualization of A TLAS and a skyline to illustrate how the meta-algorithm tracks the proper base-algorithms.

Original: Weight Assignment of A TLAS .
Revised: Weight Assignment of A TLAS .

Original: In Figure 1, we show that slower Linear Shift is assigned relatively small step sizes, while faster Bernoulli Shift is assigned relatively large step sizes.
Revised: In Figure 1, we show that slower Linear Shift is assigned relatively small step sizes, while faster Bernoulli Shift is assigned relatively large step sizes.

Original: Here, we replenish the experimental results for Square Shift and Sine Shift .
Revised: Here, we replenish the experimental results for Square Shift and Sine Shift .

Original: As these two simulated shifts also change relatively fast like Bernoulli Shift , the assigned weights are also expected to be relatively large, which is in line with the experimental results in Figure 5(b) and 5(c).
Revised: As these two simulated shifts also change relatively fast like Bernoulli Shift , the assigned weights are also expected to be relatively large, which is in line with the experimental results in Figure 5(b) and 5(c).

Original: Skyline of A TLAS - ADA .
Revised: Skyline of A TLAS - ADA .

Original: The purpose of this part is to answer the question of whether our proposed
Revised: The purpose of this part is to answer the question of whether our proposed

Original: A TLAS - ADA can cope with online label shifts, regardless of how fast or slow the change is.
Revised: A TLAS - ADA can cope with online label shifts, regardless of how fast or slow the change is.

Original: To see this, we use the SHL dataset to simulate shifts with different non-stationarities, increasing from left to right side.
Revised: To see this, we use the SHL dataset to simulate shifts with different non-stationarities, increasing from left to right side.

Original: We plot the final average errors of the major base-learners and A TLAS - ADA in Figure 6.
Revised: We plot the ﬁnal average errors of the major base-learners and A TLAS - ADA in Figure 6.

Original: Note that base-learners with much smaller or larger step sizes lead to worse results for all situations and are omitted for clarity.
Revised: Note that base-learners with much smaller or larger step sizes lead to worse results for all situations and are omitted for clarity.

Original: As shown in Figure 6, small step sizes achieve the best average errors for tender nonstationarity cases (left side) but fail for dramatic cases (right side), which is consistent with the theoretical results that the optimal step size scales with the nonstationarity measure V T .
Revised: As shown in Figure 6, small step sizes achieve the best average errors for tender nonstationarity cases (left side) but fail for dramatic cases (right side), which is consistent with the theoretical results that the optimal step size scales with the nonstationarity measure V T .

Original: Moreover, we find that the proposed A TLAS - ADA can always track the best base-learners, no matter tender cases or dramatic cases, which implies that we can safely employ it without knowing the change of the unknown environment in advance.
Revised: Moreover, we ﬁnd that the proposed A TLAS - ADA can always track the best base-learners, no matter tender cases or dramatic cases, which implies that we can safely employ it without knowing the change of the unknown environment in advance.

Original: B Preliminaries
Revised: B Preliminaries

Original: In this section, we first provide a brief introduction to the online convex optimization framework in
Revised: In this section, we ﬁrst provide a brief introduction to the online convex optimization framework in

Original: Appendix B.1, and then in Appendix B.2 we present a detailed description of the BBSE method that is designed for the offline label shift problem, as well as its usage in constructing our risk estimator for the online label shift problem.
Revised: Appendix B.1, and then in Appendix B.2 we present a detailed description of the BBSE method that is designed for the ofﬂine label shift problem, as well as its usage in constructing our risk estimator for the online label shift problem.

Original: B.1 Brief Introduction to Online Convex Optimization
Revised: B.1 Brief Introduction to Online Convex Optimization

Original: This section briefly introduces the online convex optimization framework.
Revised: This section brieﬂy introduces the online convex optimization framework.

Original: In many real-world tasks, data are often accumulated in a sequential way, which requires a learning paradigm to update the model in an online fashion.
Revised: In many real-world tasks, data are often accumulated in a sequential way, which requires a learning paradigm to update the model in an online fashion.

Original: Taking the locomotion detection task as an example, each time, wea few data collected by smartphone sensors need to predict the motion type immediately
Revised: Taking the locomotion detection task as an example, each time, we

Original: The sequential nature of the data makes it challenging to train a model offline using the standard machine learning methods.
Revised: The sequential nature of the data makes it challenging to train a model ofﬂine using the standard machine learning methods.

Original: These circumstances can be modeled and handled with the Online Convex
Revised: These circumstances can be modeled and handled with the Online Convex

Original: Optimization (OCO) framework [20, 15, 44].
Revised: Optimization (OCO) framework [20, 15, 44].

Original: The OCO framework can be viewed as a structured repeated game between a learner and the environment.
Revised: The OCO framework can be viewed as a structured repeated game between a learner and the environment.

Original: During the learning process, the learner iteratively chooses decisions from a fixed convex decision set W based on the feedback from the environments.
Revised: During the learning process, the learner iteratively chooses decisions from a ﬁxed convex decision set W based on the feedback from the environments.

Original: Let T denote the total number of game iterations, and the protocol of OCO is given as follows.
Revised: Let T denote the total number of game iterations, and the protocol of OCO is given as follows.

Original: [Equation]

Original: decision w t W , simultaneously choose a convex loss function R t W
Revised: O NLINE C ONVEX O PTIMIZATION F RAMEWORK

Original: cost

Original: t information about t The classic performance measure for the OCO algorithms is the (static) regret defined as
Revised: The classic performance measure for the OCO algorithms is the (static) regret deﬁned as

Original: Reg s T =
Revised: Reg s T =

Original: T (cid:88) t =
Revised: T (cid:88) t =

Original: R t ( w t ) −
Revised: R t ( w t ) −

Original: T (cid:88) t =
Revised: T (cid:88) t =

Original: R t ( w ∗ ) , which compares the learner’s prediction with the best single decision in hindsight w ∗ = arg min w ∈W (cid:80) Tt =
Revised: R t ( w ∗ ) , which compares the learner’s prediction with the best single decision in hindsight w ∗ = arg min w ∈W (cid:80) Tt =

Original: 1 R t ( w ) .
Revised: 1 R t ( w ) .

Original: The static regret is a reasonable measure for stationary environments.
Revised: The static regret is a reasonable measure for stationary environments.

Original: But the performance measure could be too optimistic in non-stationary environments, where the underlying distribution changes over time, and the single best model could perform badly.
Revised: But the performance measure could be too optimistic in non-stationary environments, where the underlying distribution changes over time, and the single best model could perform badly.

Original: Under such a case, a more suitable performance is the dynamic regret that competes with the online learner’s performance with the best decisions for each round.
Revised: Under such a case, a more suitable performance is the dynamic regret that competes with the online learner’s performance with the best decisions for each round.

Original: A more detailed discussion and literature review for the dynamic regret can be found in Section C.2.
Revised: A more detailed discussion and literature review for the dynamic regret can be found in Appendix C.2.

Original: The OCO framework has received extensive studies over the decades.
Revised: The OCO framework has received extensive studies over the decades.

Original: Among them, one of the most prominent algorithms is the Online Gradient Descent (OGD) algorithm [20], which takes a step in the opposite direction to the gradient of the previous risk at each iteration (See Algorithmfor the procedures).
Revised: Among them, one of the most prominent algorithms is the Online Gradient Descent (OGD) algorithm [20], which takes a step in the opposite direction to the gradient of the previous risk at each iteration (See Algorithmfor the procedures).

Original: Despite its simplicity, OGD algorithm is powerful enough to handle a large family of online problems.
Revised: Despite its simplicity, OGD algorithm is powerful enough to handle a large family of online problems.

Original: Specifically, it enjoys a static regret of O ( √ T ) by setting the step size to be Θ(1 / √ T ) , which was proven to be minimax optimal [45] for convex functions.
Revised: Speciﬁcally, it enjoys a static regret of O ( √ T ) by setting the step size to be Θ(1 / √ T ) , which was proven to be minimax optimal [45] for convex functions.

Original: Latter, tighter static bounds for loss functions with stronger curvature [46] and that can adapt to benign environments [47, 31, 32] were proposed.
Revised: Latter, tighter static bounds for loss functions with stronger curvature [46] and that can adapt to benign environments [47, 31, 32] were proposed.

Original: For more detailed reviews of the OCO framework, we refer the readers to the seminal books [44, 15].
Revised: For more detailed reviews of the OCO framework, we refer the readers to the seminal books [44, 15].

Original: B.2 BBSE Method for Online Label Shift
Revised: B.2 BBSE Method for Online Label Shift

Original: The Black Box Shift Estimation (BBSE) method [12] is a family of label shift algorithms that uses the confusion matrix of a given black-box classifier to estimate the label distributions from unlabeled samples.
Revised: The Black Box Shift Estimation (BBSE) method [12] is a family of label shift algorithms that uses the confusion matrix of a given black-box classiﬁer to estimate the label distributions from unlabeled samples.

Original: This part illustrates how to employ BBSE to the online label distribution estimation.
Revised: This part illustrates how to employ BBSE to the online label distribution estimation.

Original: Recall that we have the labeled offline data S 

Revised: Recall that we have the labeled ofﬂine data S

Original: and the unlabeled data S t , and we need to estimate the label distribution µ y t .
Revised: and the unlabeled data S t , and we need to estimate the label distribution µ y t .

Original: Note that µ y t cannot be estimated directly due to labels of S t being unavailable.
Revised: Note that µ y t cannot be estimated directly due to labels of S t being unavailable.

Original: Since S 0 is labeled, a possible way is to employ the labels of S 0 .
Revised: Since S 0 is labeled, a possible way is to employ the labels of S 0 .

Original: To this end, we introduce an initial black-box model f 0 and have the following equation:
Revised: To this end, we introduce an initial black-box model f 0 and have the following equation:

Original: [Equation]
Revised: [Equation]

Original: which holds for all i ∈
Revised: which holds for all i ∈

Original: [ K ] .
Revised: [ K ] .

Original: The first equality holds due to the law of total probability and the second equality is based on Assumption 1 that D t ( x | y )
Revised: The ﬁrst equality holds due to the law of total probability and the second equality is based on Assumption 1 that D t ( x | y )

Original: =
Revised: =

Original: D 0 ( x | y ) for any t ∈
Revised: D 0 ( x | y ) for any t ∈

Original: [ T ] .
Revised: [ T ] .

Original: (7) can be equivalently expressed in matrix notation, which is given by:
Revised: (7) can be equivalently expressed in matrix notation, which is given by:

Original: [Equation]
Revised: [Equation]

Original: where µ (cid:98)
Revised: where µ (cid:98)

Original: y t ∈ ∆ K is the distribution vector of the black-box model’s prediction under the distribution
Revised: y t ∈ ∆ K is the distribution vector of the black-box model’s prediction under the distribution

Original: D t ( x ) , with the i -th entry [ µ (cid:98) y t
Revised: D t ( x ) , with the i -th entry [ µ (cid:98) y t

Original: i
Revised: i

Original: =
Revised: =

Original: E x
Revised: E x

Original: ∼D t ( x ) [ 1 { f 0 ( x ) = i } ] .
Revised: ∼D t ( x ) [ 1 { f 0 ( x ) = i } ] .

Original: C f 0 ∈ R K × K is the confusion matrix, whose ( i, j ) -th entry [ C f 0 ] ij =
Revised: C f 0 ∈ R K × K is the confusion matrix, whose ( i, j ) -th entry [ C f 0 ] ij =

Original: E x ∼D 0 ( x | y =
Revised: E x ∼D 0 ( x | y =

Original: j ) [ 1 { f 0 ( x ) = i } ] is the classification rate that the initial model f 0 predicts samples from class i as class j .
Revised: j ) [ 1 { f 0 ( x ) = i } ] is the classiﬁcation rate that the initial model f 0 predicts samples from class i as class j .

Original: Here we assume the confusion matrix C f 0 is invertible.
Revised: Here we assume the confusion matrix C f 0 is invertible.

Original: By solving (8), we have
Revised: By solving (8), we have

Original: [Equation]
Revised: [Equation]

Original: The LHS of (9) µ y t is the label distribution we want.
Revised: The LHS of (9) µ y t is the label distribution we want.

Original: In the RHS, we note that the distribution D t ( x ) and D 0 ( x | y ) is empirically accessible via the unlabeled data S t and the labeled data S 0 , so both C fand µ (cid:98) y t can be unbiasedly estimated with its empirical version.
Revised: In the RHS, we note that the distribution D t ( x ) and D 0 ( x | y ) is empirically accessible via the unlabeled data S t and the labeled data S 0 , so both C fand µ (cid:98) y t can be unbiasedly estimated with its empirical version.

Original: Specifically, C f 0 can be estimated with the offline labeled data S 0 :
Revised: Speciﬁcally, C f 0 can be estimated with the ofﬂine labeled data S 0 :

Original: [Equation]
Revised: [Equation]

Original: And µ (cid:98) y t in the RHS can be estimated with the online data S t , which is given by
Revised: And µ (cid:98) y t in the RHS can be estimated with the online data S t , which is given by

Original: [Equation]
Revised: [Equation]

Original: With the above estimation, we construct the final estimator for the label distribution vector:
Revised: With the above estimation, we construct the ﬁnal estimator for the label distribution vector:

Original: [Equation]
Revised: [Equation]

Original: Note that although BBSE has benign unbiasedness, it has limitations arising from the inverse operation, which might amplify slight errors in the confusion matrix.
Revised: Note that although BBSE has benign unbiasedness, it has limitations arising from the inverse operation, which might amplify slight errors in the confusion matrix.

Original: Other powerful unbiased estimators can also be applied to our OLaS method directly.
Revised: Other powerful unbiased estimators can also be applied to our OLaS method directly.

Original: C Related Work and Discussion
Revised: C Related Work and Discussion

Original: This section introduces related works to our paper, including supervised learning with label shift in Appendix C.1, and online learning measures for non-stationary environments in Appendix C.2.
Revised: This section introduces related works to our paper, including supervised learning with label shift in Appendix C.1, and online learning measures for non-stationary environments in Appendix C.2.

Original: Furthermore, in Appendix C.3, we clarify the main difference between Wu et al.
Revised: Furthermore, in Appendix C.3, we clarify the main difference between Wu et al.

Original: [16] and our work.
Revised: [16] and our work.

Original: C.1 Supervised Learning with Label Shift
Revised: C.1 Supervised Learning with Label Shift

Original: Label shift is a typical scenario of learning with distribution change problem [1].
Revised: Label shift is a typical scenario of learning with distribution change problem [1].

Original: Most existing works have focused on the offline setting, where the label distribution varies from the source to target stages, but the label-conditional density remains the same.
Revised: Most existing works have focused on the ofﬂine setting, where the label distribution varies from the source to target stages, but the label-conditional density remains the same.

Original: The main challenge of offline label shift problem lies in the estimation of the target label distribution.
Revised: The main challenge of ofﬂine label shift problem lies in the estimation of the target label distribution.

Original: With such knowledge, the learner can train classifiers guaranteed to perform well over the target distribution.
Revised: With such knowledge, the learner can train classiﬁers guaranteed to perform well over the target distribution.

Original: Saerens et al.
Revised: Saerens et al.

Original: [8] first introduced two kinds of solutions for label distribution estimation, including the confusion matrixbased method and the maximum likelihood estimation.
Revised: [8] ﬁrst introduced two kinds of solutions for label distribution estimation, including the confusion matrixbased method and the maximum likelihood estimation.

Original: Lipton et al.
Revised: Lipton et al.

Original: [12] provided a convergence analysis of the matrix-based method with black box models and Azizzadenesheli et al.
Revised: [12] provided a convergence analysis of the matrix-based method with black box models and Azizzadenesheli et al.

Original: [13] enhanced the matrix-based method by regularization.
Revised: [13] enhanced the matrix-based method by regularization.

Original: Another line of studies investigates the label distribution estimation problem via distribution matching [9, 10, 11, 14].
Revised: Another line of studies investigates the label distribution estimation problem via distribution matching [9, 10, 11, 14].

Original: Garg et al.
Revised: Garg et al.

Original: [14] further showed that the matrix-based method can be interpreted from the distribution matching view.
Revised: [14] further showed that the matrix-based method can be interpreted from the distribution matching view.

Original: Beyond the label shift, the label distribution estimation with labeled data from the source domain and unlabeled data from the target domain is also known as the mixture proportion estimation problem [48, 49, 50], which has been wildly applied in learning with noisy label [51], positive and unlabeled learning [52, 53], and learning with unknown classes [54], etc.
Revised: Beyond the label shift, the label distribution estimation with labeled data from the source domain and unlabeled data from the target domain is also known as the mixture proportion estimation problem [48, 49, 50], which has been wildly applied in learning with noisy label [51], positive and unlabeled learning [52, 53], and learning with unknown classes [54], etc.

Original: The above methods only considered the scenario where the label distribution changes once, and sufficient unlabeled samples from the target distribution can be collected in advance.
Revised: The above methods only considered the scenario where the label distribution changes once, and sufﬁcient unlabeled samples from the target distribution can be collected in advance.

Original: In many realworld applications, data are accumulated with time, and thus it is important to consider the online version of the label shift problem.
Revised: In many realworld applications, data are accumulated with time, and thus it is important to consider the online version of the label shift problem.

Original: Wu et al.
Revised: Wu et al.

Original: [16] first introduced online label shift and considered the scenario where test samples come in sequence and label distribution changes with time.
Revised: [16] ﬁrst introduced online label shift and considered the scenario where test samples come in sequence and label distribution changes with time.

Original: To address the OLaS problem, Wu et al.
Revised: To address the OLaS problem, Wu et al.

Original: [16] proposed a risk estimator and used the vanilla OGD algorithm to learn over the online streams with label shift.
Revised: [16] proposed a risk estimator and used the vanilla OGD algorithm to learn over the online streams with label shift.

Original: They derived a static regret of order O ( √ T ) , but the analysis crucially relied on the assumption of the convexity of the (expected) risk function, which is hard to be theoretically verified due to the use of 0/1-loss and the argmax operation.
Revised: They derived a static regret of order O ( √ T ) , but the analysis crucially relied on the assumption of the convexity of the (expected) risk function, which is hard to be theoretically veriﬁed due to the use of 0/1-loss and the argmax operation.

Original: In contrast, by constructing an unbiased risk estimator with nice properties, our methods can adapt to dynamic environments with provable grantees.
Revised: In contrast, by constructing an unbiased risk estimator with nice properties, our methods can adapt to dynamic environments with provable grantees.

Original: We provide more detailed discussions on the differences between the risk estimators in Appendix C.3.
Revised: We provide more detailed discussions on the differences between the risk estimators in Appendix C.3.

Original: C.2 Online Learning in Non-stationary Environments
Revised: C.2 Online Learning in Non-stationary Environments

Original: This section first reviews existing results for non-stationary online convex optimization, including the worst-case dynamic regret and the universal dynamic regret.
Revised: This section ﬁrst reviews existing results for non-stationary online convex optimization, including the worst-case dynamic regret and the universal dynamic regret.

Original: Then, we discuss the difference between our dynamic regret bounds and those in the previous works.
Revised: Then, we discuss the difference between our dynamic regret bounds and those in the previous works.

Original: We also present a brief introduction to readers unfamiliar with the OCO paradigm in Appendix B.1.
Revised: We also present a brief introduction to readers unfamiliar with the OCO paradigm in Appendix B.1.

Original: Worst-case Dynamic Regret.
Revised: Worst-case Dynamic Regret.

Original: The rationale behind the static regret is that the best single decision could perform well over the time horizon.
Revised: The rationale behind the static regret is that the best single decision could perform well over the time horizon.

Original: However, the underlying environments could change over the online learning process, making the static regret unsuitable.
Revised: However, the underlying environments could change over the online learning process, making the static regret unsuitable.

Original: A better measure for the non-stationary environments is the worst-case dynamic regret Reg d T = (cid:80) t =1 R t ( w t ) − (cid:80)
Revised: A better measure for the non-stationary environments is the worst-case dynamic regret Reg d T = (cid:80) t =1 R t ( w t ) − (cid:80)

Original: Tt =1 R ( w ∗ t ) , which compares the learner’s decision with the best decision at each iteration w ∗ t = arg min w ∈W R t ( w ) and has draw growing attention recently [20, 17, 22, 25, 55, 56, 57, 58, 23, 59].
Revised: Tt =1 R ( w ∗ t ) , which compares the learner’s decision with the best decision at each iteration w ∗ t = arg min w ∈W R t ( w ) and has draw growing attention recently [20, 17, 22, 25, 55, 56, 57, 58, 23, 59].

Original: It is well-known that a sublinear regret bound is impossible for the dynamic regret in the worst case unless imposing certain regularities on the non-stationarity of the environments.
Revised: It is well-known that a sublinear regret bound is impossible for the dynamic regret in the worst case unless imposing certain regularities on the non-stationarity of the environments.

Original: There are two commonly studied non-stationarity measures.
Revised: There are two commonly studied non-stationarity measures.

Original: The first one is the path length
Revised: The ﬁrst one is the path length

Original: P ∗ T = (cid:80) Tt [-=
 ∥-] w ∗ t − w ∗ t − 1 ∥ 2 , which reflects the fluctuation of the comparator sequence.
Revised: P ∗ T = (cid:80) Tt [-=
 ∥-]=2 (cid:107) w ∗ t − w ∗ t − 1 (cid:107) 2 , which reﬂects the ﬂuctuation of the comparator sequence.

Original: When the loss functions are convex, Yang et al.
Revised: When the loss functions are convex, Yang et al.

Original: [25] showed that OGD algorithm attains an O ( (cid:112) T (1 + P ∗ T )) dynamic regret.
Revised: [25] showed that OGD algorithm attains an O ( (cid:112) T (1 + P ∗ T )) dynamic regret.

Original: The bound can be further improved to O
Revised: The bound can be further improved to O

Original: ( P ∗ T ) when the functions are convex and smooth, and the minimizer w ∗ t lies in the interior of the feasible domain W [25], or the loss functions are strongly convex and smooth [60].
Revised: ( P ∗ T ) when the functions are convex and smooth, and the minimizer w ∗ t lies in the interior of the feasible domain W [25], or the loss functions are strongly convex and smooth [60].

Original: Another commonly studied non-stationarity measure is the temporal variability
Revised: Another commonly studied non-stationarity measure is the temporal variability

Original: V fT = (cid:80) Tt =2 sup w ∈W | f t ( w ) − f t − 1 ( w ) | , which quantifies the variation of the online functions.
Revised: V fT = (cid:80) Tt =2 sup w ∈W | f t ( w ) − f t − 1 ( w ) | , which quantiﬁes the variation of the online functions.

Original: When the online functions are convex, Besbes et al.
Revised: When the online functions are convex, Besbes et al.

Original: [17] showed an O ( T 2 / 3 V fT 1 / 3 ) dynamic regret by the restarted OGD algorithm, and the bound can be further improved to O ( T 1 / 2 V fT 1 / 2 ) for strongly convex functions.
Revised: [17] showed an O ( T 2 / 3 V fT 1 / 3 ) dynamic regret by the restarted OGD algorithm, and the bound can be further improved to O ( T 1 / 2 V fT 1 / 2 ) for strongly convex functions.

Original: Then, the temporal variability bound was generalized to high-order deviations [57] and the p, q -norm [58].
Revised: Then, the temporal variability bound was generalized to high-order deviations [57] and the p, q -norm [58].

Original: There are also studies on the best-of-both-worlds bounds [22, 23, 59], where the path-length bound and temporal variability bound are achieved simultaneously by a single algorithm.
Revised: There are also studies on the best-of-both-worlds bounds [22, 23, 59], where the path-length bound and temporal variability bound are achieved simultaneously by a single algorithm.

Original: For strongly convex and smooth functions, the best known result is O (min { P ∗ T , S ∗ T , V fT } ) due to Zhao and Zhang [59], where S ∗ T =
Revised: For strongly convex and smooth functions, the best known result is O (min { P ∗ T , S ∗ T , V fT } ) due to Zhao and Zhang [59], where S ∗ T =

Original: (cid:80) Tt [-=
 ∥-] w ∗ t − w ∗ t − 1 ∥ 22 is the squared path length.
Revised: (cid:80) Tt [-=
 ∥-]=2 (cid:107) w ∗ t − w ∗ t − 1 (cid:107) 22 is the squared path length.

Original: Universal Dynamic Regret.
Revised: Universal Dynamic Regret.

Original: The worst-case dynamic regret will sometimes be too pessimistic to guide the algorithm design as the function minimizer w ∗ t can easily overfit to the noise [18].
Revised: The worst-case dynamic regret will sometimes be too pessimistic to guide the algorithm design as the function minimizer w ∗ t can easily overﬁt to the noise [18].

Original: To this end, the universal dynamic regret is proposed and defined as Reg d T ( u 1 , . . .
Revised: To this end, the universal dynamic regret is proposed and deﬁned as Reg d T ( u 1 , . . .

Original: , u T ) =
Revised: , u T ) =

Original: (cid:80) Tt =1 R t ( w t ) − (cid:80)
Revised: (cid:80) Tt =1 R t ( w t ) − (cid:80)

Original: Tt =1 R t ( u t ) , which supports the comparison to any feasible comparator sequence { u t } Tt =1 with u t ∈ W , and the measure has gained more and more attention [20, 18, 19, 61, 62, 63, 64, 65, 66, 67, 68, 69].
Revised: Tt =1 R t ( u t ) , which supports the comparison to any feasible comparator sequence { u t } Tt =1 with u t ∈ W , and the measure has gained more and more attention [20, 18, 19, 61, 62, 63, 64, 65, 66, 67, 68, 69].

Original: Zinkevich [20] showed that OGD can achieve an O ((1 + P T ) √ T ) universal dynamic regret, where P T = (cid:80) Tt [-=
 ∥-] u t − u t − 1 ∥ 2 is the path length of the comparator sequence { u t } Tt =1 .
Revised: Zinkevich [20] showed that OGD can achieve an O ((1 + P T ) √ T ) universal dynamic regret, where P T = (cid:80) Tt [-=
 ∥-]=2 (cid:107) u t − u t − 1 (cid:107) 2 is the path length of the comparator sequence { u t } Tt =1 .

Original: Nevertheless, the guarantee is far from the Ω( (cid:112) T (1 + P T )) minimax lower bound for convex functions as shown by Zhang et al.
Revised: Nevertheless, the guarantee is far from the Ω( (cid:112) T (1 + P T )) minimax lower bound for convex functions as shown by Zhang et al.

Original: [18], who further developed an algorithm with an O ( (cid:112) T (1 + P T ))
Revised: [18], who further developed an algorithm with an O ( (cid:112) T (1 + P T ))

Original: universal dynamic regret and hence closed the gap [18].
Revised: universal dynamic regret and hence closed the gap [18].

Original: Later, many studies are devoted to developing algorithms with tighter guarantees that can benefit from stronger curvature of loss functions [65], or can adapt to the benign environments [19, 63], or to the more challenging setting with bandit feedback [64].
Revised: Later, many studies are devoted to developing algorithms with tighter guarantees that can beneﬁt from stronger curvature of loss functions [65], or can adapt to the benign environments [19, 63], or to the more challenging setting with bandit feedback [64].

Original: Discussion on Our Dynamic Regret Bounds.
Revised: Discussion on Our Dynamic Regret Bounds.

Original: Although there are many studies on non-stationary online learning, it is hard to apply existing methods to OLaS, and our regret bound is novel.
Revised: Although there are many studies on non-stationary online learning, it is hard to apply existing methods to OLaS, and our regret bound is novel.

Original: First of all, although our performance measure shares seemingly similarity with the worst-case dynamic regret, the algorithm with such bounds is not suitable for the OLaS problem.
Revised: First of all, although our performance measure shares seemingly similarity with the worst-case dynamic regret, the algorithm with such bounds is not suitable for the OLaS problem.

Original: The main reason is that existing methods require access to the loss function R t for the model update.
Revised: The main reason is that existing methods require access to the loss function R t for the model update.

Original: However, in the OLaS problem, the true loss function R t is established on the inaccessible data distribution D t and the learner can only approximate it with (cid:98) R t based on the empirical data.
Revised: However, in the OLaS problem, the true loss function R t is established on the inaccessible data distribution D t and the learner can only approximate it with (cid:98) R t based on the empirical data.

Original: A direct application of algorithms with a worst-case dynamic regret can only lead to regret bounds defined over (cid:98) R t such that the learned model would suffer from severe overfitting on the sample noise.
Revised: A direct application of algorithms with a worst-case dynamic regret can only lead to regret bounds deﬁned over (cid:98) R t such that the learned model would suffer from severe overﬁtting on the sample noise.

Original: A few previous works [17, 25, 57] have attempted to develop methods to learn with the noisy loss function, but nevertheless the methods in [17, 25] require knowing the non-stationarity of the underlying distribution D t ahead of time, which is generally unavailable.
Revised: A few previous works [17, 25, 57] have attempted to develop methods to learn with the noisy loss function, but nevertheless the methods in [17, 25] require knowing the non-stationarity of the underlying distribution D t ahead of time, which is generally unavailable.

Original: Given that the comparators in our regret measure are not the optimizers of the loss functions, our dynamic regret measure is essentially a kind of universal dynamic regret rather than the worst-case dynamic regret.
Revised: Given that the comparators in our regret measure are not the optimizers of the loss functions, our dynamic regret measure is essentially a kind of universal dynamic regret rather than the worst-case dynamic regret.

Original: Our results differ from the existing universal dynamic regret bounds appearing in the literature by carefully exploiting the structure of online label shift.
Revised: Our results differ from the existing universal dynamic regret bounds appearing in the literature by carefully exploiting the structure of online label shift.

Original: In particular, existing bounds always scale with the path length of compactors P T = (cid:80)
Revised: In particular, existing bounds always scale with the path length of compactors P T = (cid:80)

Original: Tt [-=
 ∥-]
Revised: Tt [-=
 ∥-]=2 (cid:107)

Original: u t − u t − 1 ∥ 2 , whereas our dynamic regret can adapt to the variation of label distribution V T = (cid:80) Tt =2 ∥ µ y t − µ y t − 1 ∥ 1 directly, which is a more interpretable and suitable measure for OLaS.
Revised: u t − u t − 1 (cid:107) 2 , whereas our dynamic regret can adapt to the variation of label distribution V T = (cid:80) Tt =2 (cid:107) µ y t − µ y t − 1 (cid:107) 1 directly, which is a more interpretable and suitable measure for OLaS.

Original: From the technical side, the most related works to us are [18, 19] as all algorithms use the meta-base structure to achieve the dynamic regret bound.
Revised: From the technical side, the most related works to us are [18, 19] as all algorithms use the meta-base structure to achieve the dynamic regret bound.

Original: Besides the difference in the non-stationarity measure, our method improves the regret bound of Zhang et al.
Revised: Besides the difference in the non-stationarity measure, our method improves the regret bound of Zhang et al.

Original: [18] for convex functions by replacing the dependence on T with the problem-dependent quantity G T .
Revised: [18] for convex functions by replacing the dependence on T with the problem-dependent quantity G T .

Original: The G T quantity can be much smaller than T in benign environments, which implies a better adaptivity of our algorithm to underlying environments.
Revised: The G T quantity can be much smaller than T in benign environments, which implies a better adaptivity of our algorithm to underlying environments.

Original: The first dynamic regret bound with such adaptivity was achieved by Zhao et al.
Revised: The ﬁrst dynamic regret bound with such adaptivity was achieved by Zhao et al.

Original: [19], but their work requires an additional smoothness assumption on the loss functions.
Revised: [19], but their work requires an additional smoothness assumption on the loss functions.

Original: We remove such a constraint via refined analyses for the meta-algorithm, which updates the weights with function values and an improved algorithm design for base-algorithm, which trains the model following an implicit update rule.
Revised: We remove such a constraint via reﬁned analyses for the meta-algorithm, which updates the weights with function values and an improved algorithm design for base-algorithm, which trains the model following an implicit update rule.

Original: The new finding might be of independent interest to the general online convex optimization.
Revised: The new ﬁnding might be of independent interest to the general online convex optimization.

Original: We note that a similar implicit update rule also appeared in a concurrent work [70, Section 5] for achieving problem-dependent static regret.
Revised: We note that a similar implicit update rule also appeared in a concurrent work [70, Section 5] for achieving problem-dependent static regret.

Original: Our dynamic regret is much more challenging, as we not only need an implicit update in base-algorithms but also need a refined analysis of the meta-algorithm, which is unique in online ensemble analysis of dynamic regret minimization.
Revised: Our dynamic regret is much more challenging, as we not only need an implicit update in base-algorithms but also need a reﬁned analysis of the meta-algorithm, which is unique in online ensemble analysis of dynamic regret minimization.

Original: C.3 Detailed Comparison with Wu et al. [16]
Revised: C.3 Detailed Comparison with Wu et al. [16]

Original: As mentioned in the main paper the work of Wu et al.
Revised: As mentioned in the main paper the work of Wu et al.

Original: [16] is the closest one to our proposal, who also proposed a risk estimator to cope with OLaS.
Revised: [16] is the closest one to our proposal, who also proposed a risk estimator to cope with OLaS.

Original: In this part, we first review their result by restating their proposed risk estimator and then elaborate on the salient differences between their method and ours.
Revised: In this part, we ﬁrst review their result by restating their proposed risk estimator and then elaborate on the salient differences between their method and ours.

Original: The comparison demonstrates both theoretical and practical appeals of our proposed risk estimator in that it makes the OCO framework still applicable for OLaS.
Revised: The comparison demonstrates both theoretical and practical appeals of our proposed risk estimator in that it makes the OCO framework still applicable for OLaS.

Original: Estimation of 0 / 1 risk.
Revised: Estimation of 0 / 1 risk.

Original: The performance measure of Wu et al.
Revised: The performance measure of Wu et al.

Original: [16] is slightly different from ours, where the authors aim to minimize the regret defined over the 0 / 1 risk
Revised: [16] is slightly different from ours, where the authors aim to minimize the regret deﬁned over the 0 / 1 risk

Original: [Equation]
Revised: [Equation]

Revised: t =

Revised: R 01 t ( g t )

Revised: − min g ∈G

Revised: T (cid:88) t =

Original:  01 t ( g ) .
Revised: R 01 t ( g ) .

Original: In above, the 0 / 1 risk is defined as R 01 t ( g ) =
Revised: In above, the 0 / 1 risk is deﬁned as R 01 t ( g ) =

Original: E ( x ,y )
Revised: E ( x ,y )

Original: ∼D t [ 1 { g ( x ) ̸ = y } ] and g :X
Revised: ∼D t [ 1 { g ( x ) (cid:54) = y } ] and g :

Original:  (cid:55)→ Y is the classifier whose definition will be clear latter.
Revised: X (cid:55)→ Y is the classiﬁer whose deﬁnition will be clear latter.

Original: Denoting by µ y t the class prior vector with
Revised: Denoting by µ y t the class prior vector with

Original: [ µ y t ] k = D t ( y = k ) , the expected 0 / 1 risk can be rewritten as
Revised: [ µ y t ] k = D t ( y = k ) , the expected 0 / 1 risk can be rewritten as

Original: [Equation]
Revised: [Equation]

Original: In decomposition (12), the first term E x ∼D 0 ( x | y = k ) [ 1 { g ( x ) ̸ = y } ] is established on the classconditional distribution D 0 ( x | y ) , which can be empirically approximated with the offline data S 0 .
Revised: In decomposition (12), the ﬁrst term E x ∼D 0 ( x | y = k ) [ 1 { g ( x ) (cid:54) = y } ] is established on the classconditional distribution D 0 ( x | y ) , which can be empirically approximated with the ofﬂine data S 0 .

Original: The second term is the class prior at each iteration.
Revised: The second term is the class prior at each iteration.

Original: One can estimate it with the BBSE estimator introduced in Section 3.
Revised: One can estimate it with the BBSE estimator introduced in Section 3.

Original: As a consequence, Wu et al.
Revised: As a consequence, Wu et al.

Original: [16] empirically approximated R 01 t ( g ) by
Revised: [16] empirically approximated R 01 t ( g ) by

Original: [Equation]
Revised: [Equation]

Original: where (cid:98) µ y t =
Revised: where (cid:98) µ y t =

Original: C − 1 f 0 · (cid:98) µ (cid:98) y t is the estimated class prior returned by the BBSE model and (cid:98) R 01 t,k =| S k 0 | (cid:80) x n ∈ S k 0 

Revised: C − 1 f 0 · (cid:98) µ (cid:98) y t is the estimated class prior returned by the BBSE model and (cid:98) R 01 t,k =| S k 0 | (cid:80) x n ∈ S k 0

Original: { g ( x n ) ̸
Revised: { g ( x n ) (cid:54) =

Original: = k } is the model’s empirical risk over S k 0 .
Revised: k } is the model’s empirical risk over S k 0 .

Original: Reweighting classifier.
Revised: Reweighting classiﬁer.

Original: To directly optimize the 0 / 1 loss, Wu et al.
Revised: To directly optimize the 0 / 1 loss, Wu et al.

Original: [16] further restricted the model’s structure by focusing on the reweighting algorithm.
Revised: [16] further restricted the model’s structure by focusing on the reweighting algorithm.

Original: The basic idea is, the optimal classifier that minimizes R 01 t ( g ) is g ∗ ( x ) =
Revised: The basic idea is, the optimal classiﬁer that minimizes R 01 t ( g ) is g ∗ ( x ) =

Original: arg max k ∈ [ K ] D t ( y = k
Revised: arg max k ∈ [ K ] D t ( y = k

Original: | x ) .
Revised: | x ) .

Original: Under the class shift condition, they rewrote D t ( y = k | x ) as
Revised: Under the class shift condition, they rewrote D t ( y = k | x ) as

Original: [Equation]
Revised: [Equation]

Original: Since the D 0 can be approximate by the offline data S 0 , the leaner can train a probability model g 0 : X (cid:55)→ ∆ K on S 0 to approximate D 

Revised: Since the D 0 can be approximate by the ofﬂine data S 0 , the leaner can train a probability model g 0 : X (cid:55)→ ∆ K on S 0 to approximate D

Original: ( y | x ) .
Revised: ( y | x ) .

Original: Then, by the relationship (13), one can construct a classifier g p : X (cid:55)→ [ K ] parameterized by the reweighting parameter
Revised: Then, by the relationship (13), one can construct a classiﬁer g p : X (cid:55)→ [ K ] parameterized by the reweighting parameter

Original: p ∈
Revised: p ∈

Original: ∆ K to approximate the optimal classifier g ∗ by
Revised: ∆ K to approximate the optimal classiﬁer g ∗ by

Original: [Equation]
Revised: [Equation]

Original: where Z ( x ) = (cid:80)
Revised: where Z ( x ) = (cid:80)

Original: Kk =
Revised: Kk =

Original: [ p ] k D 

Revised: [ p ] k D

Original: ( y = k ) ·
Revised: ( y = k ) ·

Original: [ g 0 ( x )]
Revised: [ g 0 ( x )]

Original: k is the normalization factor and the initial label distribution
Revised: k is the normalization factor and the initial label distribution

Original: D 

Revised: D

Original: ( y = k ) can be estimated with the offline data S 0 .
Revised: ( y = k ) can be estimated with the ofﬂine data S 0 .

Original: When g 0 provides a sufficiently accurate approximation of D t ( y | x ) , one can show that g ∗ = arg min p ∈ ∆ K R 01 t ( g p ) .
Revised: When g 0 provides a sufﬁciently accurate approximation of D t ( y | x ) , one can show that g ∗ = arg min p ∈ ∆ K R 01 t ( g p ) .

Original: Comparisons.
Revised: Comparisons.

Original: In summary, the differences between the risk estimator of Wu et al.
Revised: In summary, the differences between the risk estimator of Wu et al.

Original: [16] and ours are mainly in the following two aspects:
Revised: [16] and ours are mainly in the following two aspects:

Original: (i) Loss function.
Revised: (i) Loss function.

Original: As shown in ( , Wu et al.
Revised: As shown in (12), Wu et al.

Original: ] used the non-convex 0 / 1 loss 1 { g ( x ) ̸ = y } for evaluation.
Revised: [16] used the non-convex 0 / 1 loss 1 { g ( x ) (cid:54) = y } for evaluation.

Original: Theoretically, the non-convexity of 0 / 1 loss makes it hard to confirm the convexity of the 0 / 1 risk, which is the critical condition for the regret bound.
Revised: Theoretically, the non-convexity of 0 / 1 loss makes it hard to conﬁrm the convexity of the 0 / 1 risk, which is the critical condition for the regret bound.

Original: Practically, the gradient of 0 / 1 loss is always zero, which brings troubles to the optimization.
Revised: Practically, the gradient of 0 / 1 loss is always zero, which brings troubles to the optimization.

Original: By contrast, we use a convex surrogate loss in our risk estimator to address the above problems.
Revised: By contrast, we use a convex surrogate loss in our risk estimator to address the above problems.

Original: (ii) Classifier.
Revised: (ii) Classiﬁer.

Original: To match the 0/1 loss function, Wu et al.
Revised: To match the 0/1 loss function, Wu et al.

Original: [16] designed a classifier based on the reweighting algorithm, which directly updates the reweighting parameter p in a simplex.
Revised: [16] designed a classiﬁer based on the reweighting algorithm, which directly updates the reweighting parameter p in a simplex.

Original: Theoretically, as shown in (14), there is a troublesome argmax operation in the formulation of the classifier, which makes it non-convex and then hurts the convexity of the entire risk.
Revised: Theoretically, as shown in (14), there is a troublesome argmax operation in the formulation of the classiﬁer, which makes it non-convex and then hurts the convexity of the entire risk.

Original: Practically, only adjusting the outputs of the initial classifier f p 0 by reweighting might limit the learning capability of the model, especially when the initial classifier does not perform well enough.
Revised: Practically, only adjusting the outputs of the initial classiﬁer f p 0 by reweighting might limit the learning capability of the model, especially when the initial classiﬁer does not perform well enough.

Original: By comparison, we employ a strictly convex classifier and update the whole parameters in a convex parameter space, which confirms the convexity of the risk and the flexibility to better adapt to the environmental changes.
Revised: By comparison, we employ a strictly convex classiﬁer and update the whole parameters in a convex parameter space, which conﬁrms the convexity of the risk and the ﬂexibility to better adapt to the environmental changes.

Original: The classifier in (14) is hard to be convex even if we replace the 0/1 loss with convex surrogate loss (details elaborated below).
Revised: The classiﬁer in (14) is hard to be convex even if we replace the 0/1 loss with convex surrogate loss (details elaborated below).

Original: We finally remark that even if we modify the risk estimator of Wu et al.
Revised: We ﬁnally remark that even if we modify the risk estimator of Wu et al.

Original: [16] by optimizing the convex surrogate loss rather than the 0 / 1 loss, it remains unclear how to apply the OCO framework to this modified estimator.
Revised: [16] by optimizing the convex surrogate loss rather than the 0 / 1 loss, it remains unclear how to apply the OCO framework to this modiﬁed estimator.

Original: The critical challenge is that even if the argmax operation can be avoided, another renormalization process is required due to the reweighting mechanism, making it hard to confirm the convexity of online functions.
Revised: The critical challenge is that even if the argmax operation can be avoided, another renormalization process is required due to the reweighting mechanism, making it hard to conﬁrm the convexity of online functions.

Original: By contrast, our risk estimator well fits the OCO framework and thus can achieve favorable dynamic regret guarantees with suitable online update rules.
Revised: By contrast, our risk estimator well ﬁts the OCO framework and thus can achieve favorable dynamic regret guarantees with suitable online update rules.

Original: D Omitted Details for Section 3. 
Revised: D Omitted Details for Section 3.

Original: This section presents the omitted details for Section 3.1.
Revised: This section presents the omitted details for Section 3.1.

Original: D.1 Proof of Lemma
Revised: D.1 Proof of Lemma

Original: Proof of Lemma 1.
Revised: Proof of Lemma 1.

Original: Recall the definition of the expected risk (2) as
Revised: Recall the deﬁnition of the expected risk (2) as

Original: [Equation]
Revised: [Equation]

Original: where µ y t is the true class prior at iteration t with [ µ y t ] k
Revised: where µ y t is the true class prior at iteration t with [ µ y t ] k

Original: = D ( y = k ) .
Revised: = D ( y = k ) .

Original: Besides, the risk estimator (3) is defined by
Revised: Besides, the risk estimator (3) is deﬁned by

Original: [Equation]
Revised: [Equation]

Original: where (cid:98) µ (cid:98) y t ∈ ∆ K with [ (cid:98) µ (cid:98) y t
Revised: where (cid:98) µ (cid:98) y t ∈ ∆ K with [ (cid:98) µ (cid:98) y t

Original: k
Revised: k

Original: =
Revised: =

Original: (1 / | S t | ) (cid:80) x ∈ S t 1 { f 0 ( x ) = k } is the estimated class prior of the prediction f 0 ( x ) .
Revised: (1 / | S t | ) (cid:80) x ∈ S t 1 { f 0 ( x ) = k } is the estimated class prior of the prediction f 0 ( x ) .

Original: Given that the initial offline data has sufficient samples such that (cid:98) C f [-
-] = C f 0 and (cid:98) R k 0 ( w ) =
Revised: Given that the initial ofﬂine data has sufﬁcient samples such that (cid:98) C f [-
-]0 = C f 0 and (cid:98) R k 0 ( w ) =

Original: R k 0 ( w ) and the model w ∈ W is independent of S t , we have
Revised: R k 0 ( w ) and the model w ∈ W is independent of S t , we have

Original: [Equation]
Revised: [Equation]

Original: where the first equality is due to the definition of (cid:98) R t ( w ) .
Revised: where the ﬁrst equality is due to the deﬁnition of (cid:98) R t ( w ) .

Original: The second inequality comes from the fact that (cid:98) C f 0 and (cid:98) R k 0 are independent of S t .
Revised: The second inequality comes from the fact that (cid:98) C f 0 and (cid:98) R k 0 are independent of S t .

Original: The last equality is a consequence of the unbiasedness of (cid:98) µ (cid:98) y t , i.e., E S t ∼D t
Revised: The last equality is a consequence of the unbiasedness of (cid:98) µ (cid:98) y t , i.e., E S t ∼D t

Original: [ (cid:98) µ (cid:98) y t ] = µ (cid:98) y t , where [ µ (cid:98) y t ]
Revised: [ (cid:98) µ (cid:98) y t ] = µ (cid:98) y t , where [ µ (cid:98) y t ]

Original: k = E x
Revised: k = E x

Original: ∼D t ( x ) [ 1 { f 0 ( x ) = k } ] is the expected class prior of the prediction f 0 ( x ) .
Revised: ∼D t ( x ) [ 1 { f 0 ( x ) = k } ] is the expected class prior of the prediction f 0 ( x ) .

Original: Then, under the condition that (cid:98) C f 

Revised: Then, under the condition that (cid:98) C f

Original: =
Revised: =

Original: C f 0 and (cid:98) R k 0 ( w ) =
Revised: C f 0 and (cid:98) R k 0 ( w ) =

Original: R k 0 ( w ) , we can further obtain
Revised: R k 0 ( w ) , we can further obtain

Original: [Equation]
Revised: [Equation]

Original: where the second equality is due to the relationship µ y t
Revised: where the second equality is due to the relationship µ y t

Original: = C − 1 f 0 µ (cid:98) y t as introduced by (8).
Revised: = C − 1 f 0 µ [Equation] y t as introduced by (8).

Original: The last inequality comes from the definition of R t ( w ) , which completes the proof.
Revised: The last inequality comes from the deﬁnition of R t ( w ) , which completes the proof.

Original: D.2 Concentration of Risk Estimator
Revised: D.2 Concentration of Risk Estimator

Original: In this section, we show that the risk estimator converges to the expected risk at the rate of O ( (cid:112) 1 / | S 0 | ) with high probability.
Revised: In this section, we show that the risk estimator converges to the expected risk at the rate of O ( (cid:112) 1 / | S 0 | ) with high probability.

Original: Lemma 2.
Revised: Lemma 2.

Original: Let δ ∈ (0 , 1 / 4] .
Revised: Let δ ∈ (0 , 1 / 4] .

Original: For any w ∈ W independent of the dataset S t , with probability at least 1 − ( K + 3) δ , the risk estimator [Equation] R t ( w ) in Eq.
Revised: For any w ∈ W independent of the dataset S t , with probability at least 1 − ( K + 3) δ , the risk estimator (cid:98) R t ( w ) in Eq.

Original: (3) satisfies
Revised: (3) satisﬁes

Original: [Equation]
Revised: [Equation]

Original: given that | S 0 | ≥ (25 log(2 K/δ )) / ( κσ ) 2 , where κ = min k ∈
Revised: given that | S 0 | ≥ (25 log(2 K/δ )) / ( κσ ) 2 , where κ = min k ∈

Original: [ K ] D 

Revised: [ K ] D

Original: ( y = k ) and σ is the minimum singular value of C f 0 .
Revised: ( y = k ) and σ is the minimum singular value of C f 0 .

Original: Proof of Lemma 2.
Revised: Proof of Lemma 2.

Original: For notation convenience, we denote by R 0 : W (cid:55)→ R K the vector risk function with its k -th entry
Revised: For notation convenience, we denote by R 0 : W (cid:55)→ R K the vector risk function with its k -th entry

Original: [ R 0 ( w )]
Revised: [ R 0 ( w )]

Original: k = R k 0 ( w ) and its empirical version (cid:98) R 0 : W (cid:55)→ R K with [ (cid:98) R 0 ( w )]
Revised: k = R k 0 ( w ) and its empirical version (cid:98) R 0 : W (cid:55)→ R K with [ (cid:98) R 0 ( w )]

Original: k = (cid:98) R k 0 ( w ) .
Revised: k = (cid:98) R k 0 ( w ) .

Original: Under such a case, we can rewrite the risk estimator as (cid:98) R t ( w ) = (cid:98) µ ⊤ (cid:98) y t (cid:98) C − [Equation] 0 (cid:98) R 0 ( w ) and the expected risk as R t ( w ) = µ ⊤ (cid:98)
Revised: Under such a case, we can rewrite the risk estimator as (cid:98) R t ( w ) = (cid:98) µ (cid:62) (cid:98) y t (cid:98) C − 1 f 0 (cid:98) R 0 ( w ) and the expected risk as R t ( w ) = µ (cid:62) (cid:98)

Original: y t C − 1 f 0 R 0 ( w ) .
Revised: y t C − 1 f 0 R 0 ( w ) .

Original: Then, since S t is independent of S 0 , we have
Revised: Then, since S t is independent of S 0 , we have

Original: [Equation]
Revised: [Equation]

Original: Let W f∈ R K be the diagonal matrix with [ W f 0 ] ii = D 

Revised: Let W f∈ R K be the diagonal matrix with [ W f 0 ] ii = D

Original: ( y = i ) and [ J f 0 ] ij = E ( x ,y )
Revised: ( y = i ) and [ J f 0 ] ij = E ( x ,y )

Original: ∼D 

Revised: ∼D

Original: [ 1 { f 0 ( x ) =
Revised: [ 1 { f 0 ( x ) =

Original: i and y =
Revised: i and y =

Original: j } ] be the confusion matrix defined over the joint distribution of f 0 ( x ) and y .
Revised: j } ] be the confusion matrix deﬁned over the joint distribution of f 0 ( x ) and y .

Original: Besides, denote by [ (cid:99) W f 0 ] ii = (1 / | S 0 | ) (cid:80) ( x ,y ) ∈ S 0 1 { y = i } and (cid:98) J f 

Revised: Besides, denote by [ (cid:99) W f 0 ] ii = (1 / | S 0 | ) (cid:80) ( x ,y ) ∈ S 0 1 { y = i } and (cid:98) J f

Original: =
Revised: =

Original: (1 / | S 0 | ) (cid:80) ( x ,y ) ∈ S 0 1 { f 0 ( x ) =
Revised: (1 / | S 0 | ) (cid:80) ( x ,y ) ∈ S 0 1 { f 0 ( x ) =

Original: i and y =
Revised: i and y =

Original: j } the empirical versions.
Revised: j } the empirical versions.

Original: Definition (10) indicates that
Revised: Deﬁnition (10) indicates that

Original: [Equation]
Revised: [Equation]

Original: Under such a case, the empirical and expected risk can be further rewritten as
Revised: Under such a case, the empirical and expected risk can be further rewritten as

Original: [Equation]
Revised: [Equation]

Original: So, we have
Revised: So, we have

Original: [Equation]
Revised: [Equation]

Original: For term (a), with probability at least 1 − δ , we have
Revised: For term (a), with probability at least 1 − δ , we have

Original: [Equation]
Revised: [Equation]

Original: where the second inequality is due to the Cauchy-Schwarz inequality and the property of the operator norm.
Revised: where the second inequality is due to the Cauchy-Schwarz inequality and the property of the operator norm.

Original: The third inequality holds since ∥ µ (cid:98) y t ∥ 2 ≤ 1 and ∥ (cid:99) W f 0 (cid:98) R 0 ( w ) ∥ 2 ≤ ∥ (cid:99) W f 0 ∥ 2 · ∥ (cid:98) R 0 ( w ) ∥ 2 ≤ √
Revised: The third inequality holds since (cid:107) µ (cid:98) y t (cid:107) 2 ≤ 1 and (cid:107) (cid:99) W f 0 (cid:98) R [Equation] w [Equation]

Revised: √

Original: KB .
Revised: KB .

Original: The last inequality is due to Lemma 4.
Revised: The last inequality is due to Lemma 4.

Original: Next, we analyze the term (b).
Revised: Next, we analyze the term (b).

Original: [Equation]
Revised: [Equation]

Original: where the third inequality is due to the fact that ∥ µ (cid:98) y t ∥ 2 ≤ 1 and ∥ J −1 f 0 ∥ 2 ≤ ∥ W −
Revised: where the third inequality is due to the fact that (cid:107) µ (cid:98) y t (cid:107) 2 ≤ 1 and (cid:107) J −

Original: [Equation]
Revised: [Equation]

Original: C − [Equation] f 
Revised: f 0 (cid:107) 2 ≤ (cid:107) W

Original: [Equation]
Revised: [Equation]

Original: ∥ 2 ≤ 1 / ( κσ ) .
Revised: ≤ 1 / ( κσ ) .

Original: The term ∥ (cid:99) W f 0 (cid:98) R 0 ( w ) − W f 0 R 0 ( w ) ∥ 2 can be further decomposed as
Revised: The term (cid:107) (cid:99) W f 0 (cid:98) R 0 ( w ) − W f 0 R 0 ( w ) (cid:107) 2 can be further decomposed as

Original: [Equation]
Revised: [Equation]

Original: where the first inequality is due to the property of the operator norm.
Revised: where the ﬁrst inequality is due to the property of the operator norm.

Original: The second inequality is a consequence of the fact that ∥ (cid:98) R ( w ) ∥ 2 ≤ B for any w ∈ W and ∥ W f 0 ∥ 2 ≤ 1 .
Revised: The second inequality is a consequence of the fact that (cid:107) (cid:98) R ( w ) (cid:107) 2 ≤ B for any w ∈ W and (cid:107) W f 0 (cid:107) 2 ≤ 1 .

Original: Then, by the matrix Bernstein inequality in [71, Theorem 1.4], we have
Revised: Then, by the matrix Bernstein inequality in [71, Theorem 1.4], we have

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: with probability at least 1 − δ .
Revised: with probability at least 1 − δ .

Original: Now, we turn to bound the term ∥ (cid:98) R 0 ( w ) − R 0 ( w ) ∥ 2 .
Revised: Now, we turn to bound the term (cid:107) (cid:98) R 0 ( w ) − R 0 ( w ) (cid:107) 2 .

Original: Let N min =
Revised: Let N min =

Original: min {| S 10 | , . .
Revised: min {| S 10 | , . .

Original: | S K 0 |} .
Revised: | S K 0 |} .

Original: According to Lemma 5, we can first show that N min ≥ ( κ | S 0 | ) / 2 with probability at least 1 − δ for any δ ∈ (0 , 1 / 4] .
Revised: According to Lemma 5, we can ﬁrst show that N min ≥ ( κ | S 0 | ) / 2 with probability at least 1 − δ for any δ ∈ (0 , 1 / 4] .

Original: Then, we can bound the last term by
Revised: Then, we can bound the last term by

Original: [Equation]
Revised: [Equation]

Original: According to the Hoeffding’s lemma, for each k ∈ [ K ] [Equation] we can bound
Revised: According to the Hoeffding’s lemma, for each k ∈ [ K ] , we can bound

Original: [Equation]
Revised: [Equation]

Original: with probability at least 1 − δ .
Revised: with probability at least 1 − δ .

Original: Combining the K events, which holds with probability 1 − δ , and the event that N min ≥ ( κ | S 0 | ) / 2 , we can obtain that
Revised: Combining the K events, which holds with probability 1 − δ , and the event that N min ≥ ( κ | S 0 | ) / 2 , we can obtain that

Original: [Equation]
Revised: [Equation]

Original: with probability at least 1 − ( K + 1) δ .
Revised: with probability at least 1 − ( K + 1) δ .

Original: Putting (16) and (17) together, we have
Revised: Putting (16) and (17) together, we have

Original: [Equation]
Revised: [Equation]

Original: with probability at least 1 − ( K + 2) δ .
Revised: with probability at least 1 − ( K + 2) δ .

Original: Combining the upper bounds for term (a) and term (b), we can complete the proof by
Revised: Combining the upper bounds for term (a) and term (b), we can complete the proof by

Original: [Equation]
Revised: [Equation]

Original: with probability at least 1 − ( K + 3) δ .
Revised: with probability at least 1 − ( K + 3) δ .

Original: D.3 Static Regret Bound of UOGD Algorithm
Revised: D.3 Static Regret Bound of UOGD Algorithm

Original: The following lemma presents the static regret guarantees of the UOGD algorithm when the step size is set as η = Θ( T − 1 / 2 ) .
Revised: The following lemma presents the static regret guarantees of the UOGD algorithm when the step size is set as η = Θ( T − 1 / 2 ) .

Original: Lemma 3.
Revised: Lemma 3.

Original: Under the same assumptions of Lemma 1, UOGD in Eq.
Revised: Under the same assumptions of Lemma 1, UOGD in Eq.

Original: (4) with a step size η = Θ( T − 1 / 2 ) satisfies
Revised: (4) with a step size η = Θ( T − 1 / 2 ) satisﬁes

Original: [Equation]
Revised: [Equation]

Original: where E 1: T [ · ] denotes the expectation taken over the random draw of dataset { S t } Tt =1 .
Revised: where E 1: T [ · ] denotes the expectation taken over the random draw of dataset { S t } Tt =1 .

Original: The constant σ > 0 denotes the minimum singular value of confusion matrix C f 0 .
Revised: The constant σ > 0 denotes the minimum singular value of confusion matrix C f 0 .

Original: D.4 Useful Lemmas
Revised: D.4 Useful Lemmas

Original: This section presents several useful lemmas for the proofs in Section D.
Revised: This section presents several useful lemmas for the proofs in Appendix D.

Original: Lemma 4.
Revised: Lemma 4.

Original: Let J f 0 and (cid:98) J f 0 be the matrix defined as (15) .
Revised: Let J f 0 and (cid:98) J f 0 be the matrix deﬁned as (15) .

Original: Then, for any δ ∈ [0 . 1) , with probability at least 1 − δ we have
Revised: Then, for any δ ∈ [0 . 1) , with probability at least 1 − δ we have

Original: [Equation]
Revised: [Equation]

Original: given that | S 0 | ≥ (25 log(2 K/δ )) / ( κσ ) 2 , where κ = min k ∈
Revised: given that | S 0 | ≥ (25 log(2 K/δ )) / ( κσ ) 2 , where κ = min k ∈

Original: [ K ] D 

Revised: [ K ] D

Original: ( y = k ) and σ is the minimum singular value of C f 0 .
Revised: ( y = k ) and σ is the minimum singular value of C f 0 .

Original: Proof of Lemma 4.
Revised: Proof of Lemma 4.

Original: Denoting by ∆ J = (cid:98)
Revised: Denoting by ∆ J = (cid:98)

Original: J f 0 − J f 0 , according to [13, Lemma 2], we have
Revised: J f 0 − J f 0 , according to [13, Lemma 2], we have

Original: [Equation]
Revised: [Equation]

Original: with probability at least 1 − δ , where the last inequality is due to the assumption that | S 0 | ≥ (log(2 K/δ )) / ( σκ ) 2 .
Revised: with probability at least 1 − δ , where the last inequality is due to the assumption that | S 0 | ≥ (log(2 K/δ )) / ( σκ ) 2 .

Original: We can check the condition for Lemma
Revised: We can check the condition for Lemma

Original: [Equation]
Revised: [Equation]

Original: where the last inequality holds since σ is the minimum singular value of C f 0 and κ is that of W f 0 .
Revised: where the last inequality holds since σ is the minimum singular value of C f 0 and κ is that of W f 0 .

Original: Then, according to Lemma 10, we can complete the proof by
Revised: Then, according to Lemma 10, we can complete the proof by

Original: [Equation]
Revised: [Equation]

Original: Lemma 5.
Revised: Lemma 5.

Original: Let | S 0 | ≥ (16 log(1 /δ ))
Revised: Let | S 0 | ≥ (16 log(1 /δ ))

Original: /κ 2 .
Revised: /κ 2 .

Original: Then, for any δ ∈ (0 , 1 / 4] , with probability at least 1 − δ the following holds:
Revised: Then, for any δ ∈ (0 , 1 / 4] , with probability at least 1 − δ the following holds:

Original: [Equation]
Revised: [Equation]

Original: where N min = min {| S 10 | , . .
Revised: where N min = min {| S 10 | , . .

Original: | S K 0 |} and κ = min k ∈
Revised: | S K 0 |} and κ = min k ∈

Original: [ K ] D 

Revised: [ K ] D

Original: ( y = k ) .
Revised: ( y = k ) .

Original: Proof of Lemma 5.
Revised: Proof of Lemma 5.

Original: For notation simplicity, we denote by (cid:101) µ 0 the empirical class prior at t = 0 , i.e.
Revised: For notation simplicity, we denote by (cid:101) µ 0 the empirical class prior at t = 0 , i.e.

Original: [ (cid:101) µ 0 ]
Revised: [ (cid:101) µ 0 ]

Original: k = | S k 0 | / | S 0 | and µ 0 the expected class prior with [ µ 0 ] k
Revised: k = | S k 0 | / | S 0 | and µ 0 the expected class prior with [ µ 0 ] k

Original: = D 

Revised: = D

Original: ( y = k ) .
Revised: ( y = k ) .

Original: Then, according to [72,
Revised: Then, according to [72,

Original: Proposition 19], we have
Revised: Proposition 19], we have

Original: [Equation]
Revised: [Equation]

Original: with probability at least 1 − δ , which implies that
Revised: with probability at least 1 − δ , which implies that

Original: [Equation]
Revised: [Equation]

Original: holds with probability at least 1 − δ .
Revised: holds with probability at least 1 − δ .

Original: We complete the proof by noticing that [ µ 0 ] k ≥ κ holds for any k ∈
Revised: We complete the proof by noticing that [ µ 0 ] k ≥ κ holds for any k ∈

Original: [ K ] and further noting the assumption that | S 0 | ≥ 16log(1 /δ ) κ.
Revised: [ K ] and further noting the assumption that | S 0 | ≥ 16log(1 /δ ) κ.

Original: E Omitted Proofs for Section 3. 
Revised: E Omitted Proofs for Section 3.

Original: This section provides the proof omitted in section 3.2.
Revised: This section provides the proof omitted in section 3.2.

Original: We will first present the regret bound for the
Revised: We will ﬁrst present the regret bound for the

Original: UOGD algorithm without step size tuning (Theorem 1), and then show the overall dynamic regret bound of the A TLAS algorithm (Theorem 2).
Revised: UOGD algorithm without step size tuning (Theorem 1), and then show the overall dynamic regret bound of the A TLAS algorithm (Theorem 2).

Original: At the end of this section, we list several useful lemmas.
Revised: At the end of this section, we list several useful lemmas.

Original: Before presenting the proofs, we highlight the main challenge of the regret analysis.
Revised: Before presenting the proofs, we highlight the main challenge of the regret analysis.

Original: As noted by
Revised: As noted by

Original: Section 3.2 and Section C.2, our performance measure (1) is different from the conventional notion of (worst-case) dynamic regret [22], where the performance of the learned model is measured by the observed loss function.
Revised: Section 3.2 and Appendix C.2, our performance measure (1) is different from the conventional notion of (worst-case) dynamic regret [22], where the performance of the learned model is measured by the observed loss function.

Original: However, in the OLaS problem, the model is desired to perform well over the underlying distribution, and its quality is evaluated over the expected risk function R t ( · ) .
Revised: However, in the OLaS problem, the model is desired to perform well over the underlying distribution, and its quality is evaluated over the expected risk function R t ( · ) .

Original: The main challenge here is that the expected risk function R t ( · ) is unavailable in the learning process, which makes it hard to apply the analysis of (worst-case) dynamic regret for our case.
Revised: The main challenge here is that the expected risk function R t ( · ) is unavailable in the learning process, which makes it hard to apply the analysis of (worst-case) dynamic regret for our case.

Original: E.1 Proof of Theorem
Revised: E.1 Proof of Theorem

Original: This part presents the proof of Theorem 1.
Revised: This part presents the proof of Theorem 1.

Original: The key idea is to decompose the overall dynamic regret into two parts by introducing a reference sequence, which changes in a piece-wise stationary manner.
Revised: The key idea is to decompose the overall dynamic regret into two parts by introducing a reference sequence, which changes in a piece-wise stationary manner.

Original: In our analysis, the first part measures the regret of our algorithm when compared with the reference sequence, and the second part reflects the quality of the reference sequence.
Revised: In our analysis, the ﬁrst part measures the regret of our algorithm when compared with the reference sequence, and the second part reﬂects the quality of the reference sequence.

Original: Since the reference sequence appears only in the analysis, we can make a tight regret bound by choosing a proper one to balance the first and second parts.
Revised: Since the reference sequence appears only in the analysis, we can make a tight regret bound by choosing a proper one to balance the ﬁrst and second parts.

Original: Proof of Theorem 1.
Revised: Proof of Theorem 1.

Original: We can decompose the regret bound into two parts by introducing a reference sequence { u t } T t =1 that only changes every ∆ iteration.
Revised: We can decompose the regret bound into two parts by introducing a reference sequence { u t } T t =1 that only changes every ∆ iteration.

Original: More specifically, denoting by I m =
Revised: More speciﬁcally, denoting by I m =

Original: [( m − 1)∆ + 1 , m ∆] the m -th time interval, any comparator u t that falls into I m is taken as the single best decision over the interval, i.e. u t = w ∗I m =
Revised: [( m − 1)∆ + 1 , m ∆] the m -th time interval, any comparator u t that falls into I m is taken as the single best decision over the interval, i.e. u t = w ∗I m =

Original: arg min w ∈W (cid:80) t
Revised: arg min w ∈W (cid:80) t

Original: ∈I m R t
Revised: ∈I m R t

Original: ( w ) for any t ∈
Revised: ( w ) for any t ∈

Original: I m .
Revised: I m .

Original: We have
Revised: We have

Original: E 1: T (cid:34) T (cid:88) t =
Revised: E 1: T (cid:34) T (cid:88) t =

Original: R t ( w t ) (cid:35) −
Revised: R t ( w t ) (cid:35) −

Original: T (cid:88) t =
Revised: T (cid:88) t =

Original: R t ( w ∗ t ) =
Revised: R t ( w ∗ t ) =

Original: E 1: T (cid:34) T (cid:88) t =
Revised: E 1: T (cid:34) T (cid:88) t =

Original: R t ( w t ) (cid:35) −
Revised: R t ( w t ) (cid:35) −

Original: T (cid:88) t =
Revised: T (cid:88) t =

Original: R t ( u t )
Revised: R t ( u t )

Original: +
Revised: +

Original: T (cid:88) t =
Revised: T (cid:88) t =

Original: R t ( u t ) −
Revised: R t ( u t ) −

Original: T (cid:88) t =
Revised: T (cid:88) t =

Original: R t ( w ∗ t ) =
Revised: R t ( w ∗ t ) =

Original: E 1: T (cid:34) T (cid:88) t =
Revised: E 1: T (cid:34) T (cid:88) t =

Original: R t ( w t ) (cid:35) −
Revised: R t ( w t ) (cid:35) −

Original: M (cid:88) m
Revised: M (cid:88) m

Original: = (cid:88) t ∈I m
Revised: = (cid:88) t ∈I m

Original: R t ( w ∗I m ) (cid:124) (cid:123)(cid:122) (cid:125) term (a) +
Revised: R t ( w ∗I m ) (cid:124) (cid:123)(cid:122) (cid:125) term (a) +

Original: M (cid:88) m
Revised: M (cid:88) m

Original: = (cid:88) t ∈I m
Revised: = (cid:88) t ∈I m

Original: R t ( w ∗I m ) −
Revised: R t ( w ∗I m ) −

Original: T (cid:88) t =
Revised: T (cid:88) t =

Original: R t ( w ∗ t ) (cid:124) (cid:123)(cid:122) (cid:125) term (b) , where M = (cid:6) T ∆ (cid:7) ≤ T/ ∆ + 1 is the number of the intervals.
Revised: R t ( w ∗ t ) (cid:124) (cid:123)(cid:122) (cid:125) term (b) , where M = (cid:6) T ∆ (cid:7) ≤ T/ ∆ + 1 is the number of the intervals.

Original: Then, we turn to analyze term (a) and term (b), respectively.
Revised: Then, we turn to analyze term (a) and term (b), respectively.

Original: Analysis for term (a).
Revised: Analysis for term (a).

Original: Term (a) measures the regret of our algorithm when compared with a piece-wise stationary sequence.
Revised: Term (a) measures the regret of our algorithm when compared with a piecewise stationary sequence.

Original: We first show that the regret defined over the expected risk R t ( · ) can be related to the empirical risk estimator (cid:98) R t ( · ) due to its unbiased property.
Revised: We ﬁrst show that the regret deﬁned over the expected risk R t ( · ) can be related to the empirical risk estimator (cid:98) R t ( · ) due to its unbiased property.

Original: term (a) (18) =
Revised: term (a) (18) =

Original: E 1: T (cid:34) T (cid:88) t =
Revised: E 1: T (cid:34) T (cid:88) t =

Original: R t ( w t ) (cid:35) −
Revised: R t ( w t ) (cid:35) −

Original: T (cid:88) t =
Revised: T (cid:88) t =

Original: R t ( u t ) ≤ E 1: T (cid:34) T (cid:88) t = ⟨∇ R t ( w t ) , w t − u t ⟩ (cid:35) =
Revised: R t ( u t ) ≤ E 1: T (cid:34) T (cid:88) t = (cid:104)∇ R t ( w t ) , w t − u t (cid:105) (cid:35) =

Original: E 1: T (cid:34) T (cid:88) t = ⟨∇ R t ( w t ) − ∇ (cid:98) Rt ( w t ) , w t − u t ⟩ (cid:35) (cid:124)
Revised: E 1: T (cid:34) T (cid:88) t = (cid:104)∇ R t ( w t ) − ∇ (cid:98) R

Original: [Equation]
Revised: [Equation]

Original: where the first inequality is due to the convexity of the risk function R t ( · ).
Revised: where the ﬁrst inequality is due to the convexity of the risk function R t ( · )

Revised: [Equation]

Original: For term (a-1), we have term( a -
Revised: For term (a-1) [Equation] we have term

Original: [Equation]
Revised: [Equation]

Original: = E 1: T
Revised: a

Original: [Equation]
Revised: [Equation]

Original: where the last equality is due to the unbiasedness of the risk estimator (cid:98) R t such that ∇ R t ( w t )
Revised: where the last equality is due to the unbiasedness of the risk estimator (cid:98) R t such that ∇ R t ( w t )

Original: =
Revised: =

Original: E t (cid:104) ∇ (cid:98) R t ( w t ) (cid:12)(cid:12)(cid:12) 1 : t − 1 (cid:105) .
Revised: E t (cid:104) ∇ (cid:98) R t ( w t ) (cid:12)(cid:12)(cid:12) 1 : t − 1 (cid:105) .

Original: Thus, it is sufficient to analyze term (a-2) to provide an upper bound for term (a).
Revised: Thus, it is sufﬁcient to analyze term (a-2) to provide an upper bound for term (a).

Original: For the model sequence { w t } Tt =1 generated by the UOGD algorithm in Eq.
Revised: For the model sequence { w t } Tt =1 generated by the UOGD algorithm in Eq.

Original: (4), we have the following lemma, whose proof is deferred to Appendix E.3.
Revised: (4), we have the following lemma, whose proof is deferred to Appendix E.3.

Original: Lemma 6.
Revised: Lemma 6.

Original: Under same assumptions of Theorem 1, UOGD in Eq.
Revised: Under same assumptions of Theorem 1, UOGD in Eq.

Original: (4) with a step size η > 0 satisfies
Revised: (4) with a step size η > 0 satisﬁes

Original: [Equation]
Revised: [Equation]

Original: for any comparator sequence { u t } Tt =1 with u t ∈ W .
Revised: for any comparator sequence { u t } Tt =1 with u t ∈ W .

Original: Moreover, P T = (cid:80) Tt [-=
 ∥-]
Revised: Moreover, P T = (cid:80) Tt [-=
 ∥-]=2 (cid:107)

Original: u t − u t − 1 ∥ 2 measures the variation of the comparator sequence.
Revised: u t − u t − 1 (cid:107) 2 measures the variation of the comparator sequence.

Original: Since the comparator sequence in term (a) only changes M − 1 times, its variation is bounded by
Revised: Since the comparator sequence in term (a) only changes M − 1 times, its variation is bounded by

Original: P T ≤ Γ( M − 1) ≤ (Γ T ) / ∆ .
Revised: P T ≤ Γ( M − 1) ≤ (Γ T ) / ∆ .

Original: By Lemma 6 and taking an expectation over both sides, we have
Revised: By Lemma 6 and taking an expectation over both sides, we have

Original: [Equation]
Revised: [Equation]

Original: Combining the upper bounds of term(a-1) and term (a-2) yields
Revised: Combining the upper bounds of term(a-1) and term (a-2) yields

Original: [Equation]
Revised: [Equation]

Original: Analysis for term (b).
Revised: Analysis for term (b).

Original: For term (b), we can use a similar argument in [17] to show that term
Revised: For term (b), we can use a similar argument in [17] to show that term

Original: [Equation]
Revised: [Equation]

Revised: = (cid:88) t ∈I m (cid:0) R t ( w ∗I m ) − R t ( w ∗ t ) (cid:1) ≤

Revised: M (cid:88) m

Revised: = (cid:88) t ∈I m (cid:0) R t ( w ∗ s m ) − R t ( w ∗ t ) (cid:1) =

Revised: M (cid:88) m

Revised: = (cid:88) t ∈I m (cid:0) R t ( w ∗ s m ) − R s m ( w ∗ s m )

Revised: + R s m ( w ∗ s m )

Revised: − R t ( w ∗ t ) (cid:1) ≤

Revised: M (cid:88) m

Revised: = (cid:88) t ∈I m (cid:0) R t ( w ∗ s m ) − R s m ( w ∗ s m )