Revised: + R s m ( w ∗ t ) − R t ( w ∗ t ) (cid:1) ≤ 2∆

Revised: M (cid:88) m

Revised: = (cid:88) t ∈I m sup

Revised: [Equation]

Revised: [Equation]

Revised: [Equation]

Original:  where s m = ( m − 1)∆ + 1 is the first time step at interval I m .
Revised: ( w ) | = 2∆ T (cid:88) t [Equation] where s m = ( m − 1)∆ + 1 is the ﬁrst time step at interval I m .

Original: In above, the first inequality is due to the optimality of w ∗I m over the interval I m .
Revised: In above, the ﬁrst inequality is due to the optimality of w ∗I m over the interval I m .

Original: The second inequality holds since w ∗ s m = arg min w ∈W R s m ( w ) .
Revised: The second inequality holds since w ∗ s m = arg min w ∈W R s m ( w ) .

Original: According to the label shift condition, we can further bound the variation of the loss function by the variation of the class prior V T =
Revised: According to the label shift condition, we can further bound the variation of the loss function by the variation of the class prior V T =

Original: (cid:80) Tt [-=
 ∥-] µ y t − µ y t − 1 ∥ 1 :
Revised: (cid:80) Tt [-=
 ∥-]=2 (cid:107) µ y t − µ y t − 1 (cid:107) 1 :

Original: [Equation]
Revised: [Equation]

Original:  Thus, we have term ( b ) ≤ 2 B ∆ V T .
Revised: K (cid:88) k = [Equation] Thus, we have term ( b ) ≤ 2 B ∆ V T .

Original: Combining term (a) and term (b), we have
Revised: Combining term (a) and term (b), we have

Original: [Equation]
Revised: [Equation]

Original: where the second inequality is to the parameter setting ∆ = ⌈ (cid:112) Γ 2 T/ ( ηBV T ) ⌉ .
Revised: where the second inequality is to the parameter setting ∆ = (cid:100) (cid:112) Γ 2 T/ ( ηBV T ) (cid:101) .

Original: The third inequality is due to the fact that V T ≤ 2 T .
Revised: The third inequality is due to the fact that V T ≤ 2 T .

Original: The last inequality holds by the AM-GM inequality 3 B 3 √ T 2 VT ≤
Revised: The last inequality holds by the AM-GM inequality 3 B 3 √ T 2 V

Original: [Equation]
Revised: [Equation]

Original: BTV
Revised: ≤

Original: [Equation]
Revised: [Equation]

Original: + B 2 ηT .
Revised: + B [Equation] ηT .

Original: Hence, we complete the proof.
Revised: Hence, we complete the proof.

Original: E.2 Proof of Theorem
Revised: E.2 Proof of Theorem

Original: Proof of Theorem 2.
Revised: Proof of Theorem 2.

Original: We can decompose the dynamic regret into two parts with the piece-wise stationary sequence { u t } Tt =1 introduced in the proof of Theorem 1.
Revised: We can decompose the dynamic regret into two parts with the piece-wise stationary sequence { u t } Tt =1 introduced in the proof of Theorem 1.

Original: [Equation]
Revised: [Equation]

Original:  =
Revised: T (cid:88) t =

Original: R t ( u t ) −
Revised: R t ( u t ) −

Original: T (cid:88) t =
Revised: T (cid:88) t =

Original: R t ( w ∗ t ) =
Revised: R t ( w ∗ t ) =

Original: E 1: T (cid:34) T (cid:88) t =
Revised: E 1: T (cid:34) T (cid:88) t =

Original: R t ( w t ) (cid:35) −
Revised: R t ( w t ) (cid:35) −

Original: M (cid:88) m
Revised: M (cid:88) m

Original: = (cid:88) t ∈I m
Revised: = (cid:88) t ∈I m

Original: R t ( w ∗I m ) (cid:124) (cid:123)(cid:122) (cid:125) term (a) +
Revised: R t ( w ∗I m ) (cid:124) (cid:123)(cid:122) (cid:125) term (a) +

Original: M (cid:88) m
Revised: M (cid:88) m

Original: = (cid:88) t ∈I m
Revised: = (cid:88) t ∈I m

Original: R t ( w ∗I m ) −
Revised: R t ( w ∗I m ) −

Original: T (cid:88) t =
Revised: T (cid:88) t =

Original: R t ( w ∗ t ) (cid:124) (cid:123)(cid:122) (cid:125) term (b) , where term (b) can be bounded by term ( b ) ≤ 2 B ∆ V T following the same arguments in deriving (22).
Revised: R t ( w ∗ t ) (cid:124) (cid:123)(cid:122) (cid:125) term (b) , where term (b) can be bounded by term ( b ) ≤ 2 B ∆ V T following the same arguments in deriving (22).

Original: So, we only need to focus on the analysis of term (a), the regret of the model sequence returned by A TLAS algorithm to the piece-wise stationary compactors.
Revised: So, we only need to focus on the analysis of term (a), the regret of the model sequence returned by A TLAS algorithm to the piece-wise stationary compactors.

Original: For any i -th base UOGD algorithm, we can further decompose term (a) into the meta-regret and the base-regret
Revised: For any i -th base UOGD algorithm, we can further decompose term (a) into the meta-regret and the base-regret

Original: E 1: T (cid:34) T (cid:88) t =
Revised: E 1: T (cid:34) T (cid:88) t =

Original: R t ( w t ) (cid:35) −
Revised: R t ( w t ) (cid:35) −

Original: T (cid:88) t =
Revised: T (cid:88) t =

Original: R t ( u t )(23) =
Revised: R t ( u t ) =

Original: E 1: T (cid:34) T (cid:88) t =
Revised: E 1: T (cid:34) T (cid:88) t =

Original: R t ( w t ) −
Revised: R t ( w t ) −

Original: T (cid:88) t =
Revised: T (cid:88) t =

Original: R t ( w t,i ) (cid:35) (cid:124) (cid:123)(cid:122) (cid:125) meta-regret
Revised: R t ( w t,i ) (cid:35) (cid:124) (cid:123)(cid:122) (cid:125) meta-regret

Original: + E 1: T (cid:34) T (cid:88) t =
Revised: + E 1: T (cid:34) T (cid:88) t =

Original: R t ( w t,i ) (cid:35) −
Revised: R t ( w t,i ) (cid:35) −

Original: T (cid:88) t =
Revised: T (cid:88) t =

Original: R t ( u t )(cid:124)
Revised: R t ( u t )

Original: [Equation]
Revised: [Equation]

Original:  where the meta-regret measures the performance gap between the model returned by A TLAS algorithm and that by the i -th base UOGD algorithm, and the base-regret is the regret of the i -th base UOGD algorithm running with step size η i > 0 .
Revised: 23) where the meta-regret measures the performance gap between the model returned by A TLAS algorithm and that by the i -th base UOGD algorithm, and the base-regret is the regret of the i -th base UOGD algorithm running with step size η i > 0 .

Original: Analysis for meta-regret.
Revised: Analysis for meta-regret.

Original: For the meta-regret, we have
Revised: For the meta-regret, we have

Original: [Equation]
Revised: [Equation]

Original:  where the first inequality is due to the Jensen’s inequality and the last equality is due to the unbiasedness of the risk estimator such that E t [ (cid:98) R t ( w t,i )]
Revised: (24) where the ﬁrst inequality is due to the Jensen’s inequality and the last equality is due to the unbiasedness of the risk estimator such that E t [ (cid:98) R t ( w t,i )]

Original: =
Revised: =

Original: R t ( w t,i ) for any i ∈ [ N ] .
Revised: R t ( w t,i ) for any i ∈ [ N ] .

Original: Then, we can upper bound the meta-regret by the following lemma, whose proof is deferred to Appendix E.3.
Revised: Then, we can upper bound the meta-regret by the following lemma, whose proof is deferred to Appendix E.3.

Original: Lemma 7.
Revised: Lemma 7.

Original: By setting the learning rate ε = σB (cid:113) ln N +2 KT , the meta-algorithm of A TLAS (Algorithm 2) satisfies
Revised: By setting the learning rate ε = σB (cid:113) ln N +2 KT , the meta-algorithm of A TLAS (Algorithm 2) satisﬁes

Original: [Equation]
Revised: [Equation]

Original: for any i ∈ [ N ] , where B ≜ sup ( x ,y ) ∈X×Y , w ∈W | ℓ ( f ( w , x ) , y ) | is defined as the upper bound of the loss function.
Revised: for any i ∈ [ N ] , where B (cid:44) sup ( x ,y ) ∈X×Y , w ∈W | (cid:96) ( f ( w , x ) , y ) | is deﬁned as the upper bound of the loss function.

Original: As a consequence of Lemma 7, we can upper bound the meta-regret as
Revised: As a consequence of Lemma 7, we can upper bound the meta-regret as

Original: [Equation]
Revised: [Equation]

Original: Analysis for base-regret.
Revised: Analysis for base-regret.

Original: Since the base-algorithm of A TLAS algorithm (Algorithm 1) is taken as the UOGD algorithm, following a similar argument for obtaining Lemma 6, we have
Revised: Since the base-algorithm of A TLAS algorithm (Algorithm 1) is taken as the UOGD algorithm, following a similar argument for obtaining Lemma 6, we have

Original: [Equation]
Revised: [Equation]

Original: for any base-algorithm with the index i ∈ [ N ] , where P T is the variation of the comparator sequence
Revised: for any base-algorithm with the index i ∈ [ N ] , where P T is the variation of the comparator sequence

Original: P T =
Revised: P T =

Original: [Equation]
Revised: [Equation]

Original: Tt =2 ∥ u t − u t − 1 ∥ 2 .
Revised: Tt =2 (cid:107) u t − u t − 1 (cid:107) 2 .

Original: Then, by combining the meta-regret and the base-regret, we can obtain that
Revised: Then, by combining the meta-regret and the base-regret, we can obtain that

Original: [Equation]
Revised: [Equation]

Original: for any base-algorithm i ∈
Revised: for any base-algorithm i ∈

Original: [ N ] .
Revised: [ N ] .

Original: It remains to identify the optimal step size η i ∗ to make the bound tight.
Revised: It remains to identify the optimal step size η i ∗ to make the bound tight.

Original: Due to the construction of the step size pool
Revised: Due to the construction of the step size pool

Original: [Equation]
Revised: [Equation]

Original: with N = 1 + ⌈ 12 log 2 (1 + 2 T ) ⌉ , we can make sure that the optimal step size η ∗ =
Revised: with N = 1 + (cid:100) 12 log 2 (1 + 2 T ) (cid:101) , we can make sure that the optimal step size η ∗ =

Original: σ 2 G (cid:113) Γ 2 +2Γ P T KT is covered by the step size pool.
Revised: σ 2 G (cid:113) Γ 2 +2Γ P T KT is covered by the step size pool.

Original: Even better, due to the logarithmic construction of the pool, we claim that there must exist an i ∗ ∈
Revised: Even better, due to the logarithmic construction of the pool, we claim that there must exist an i ∗ ∈

Original: [ N ] satisfies that η i ∗ / 2 ≤ η ∗ ≤ η i ∗ .
Revised: [ N ] satisﬁes that η i ∗ / 2 ≤ η ∗ ≤ η i ∗ .

Original: So, we further bound term (a) asterm
Revised: So, we further bound term (a) as

Original: [Equation]
Revised: [Equation]

Original: by comparing with the i ∗ -th base-algorithm in the decomposition (24).
Revised: by comparing with the i ∗ -th base-algorithm in the decomposition (23).

Original: Overall dynamic regret bound.
Revised: Overall dynamic regret bound.

Original: Combining term (a) and term (b) and noticing that { u t } Tt =1 is the piecewise stationary comparator sequences with P T =
Revised: Combining term (a) and term (b) and noticing that { u t } Tt =1 is the piecewise stationary comparator sequences with P T =

Original: (cid:80) Tt [-=
 ∥-]
Revised: (cid:80) Tt [-=
 ∥-]=2 (cid:107)

Original: u t − u t − 1 ∥
Revised: u t − u t − 1 (cid:107)

Original: Tt =1 ≤ Γ T/ ∆ , we can bound the overall dynamic regret as
Revised: Tt =1 ≤ Γ T/ ∆ , we can bound the overall dynamic regret as

Original: [Equation]
Revised: [Equation]

Original: where the last inequality is due to the fact that √ a + b ≤ √ a + √ b for any a, b ≥ 0 .
Revised: where the last inequality is due to the fact that √ a + b ≤ √ a + √ b for any a, b ≥ 0 .

Original: We can complete the proof by choosing ∆ = ⌈ ( 3 GT Γ √ 2 K 4 σBV T ) 23 ⌉ and obtain the bound asReg
Revised: We can complete the proof by choosing ∆ = (cid:100) ( 3 GT Γ √ 2 K 4 σBV T ) 23 (cid:101) and obtain the bound as

Original: [Equation]
Revised: [Equation]

Original: where the last inequality is due to the fact that V T ≤ 2 T .
Revised: where the last inequality is due to the fact that V T ≤ 2 T .

Original: E.3 Useful Lemmas in the Analysis of Theorem 1 and Theorem
Revised: E.3 Useful Lemmas in the Analysis of Theorem 1 and Theorem

Original: This section presents several useful lemmas that are omitted in the proof of Theorem 1 and Theorem 2.
Revised: This section presents several useful lemmas that are omitted in the proof of Theorem 1 and Theorem 2.

Original: Lemma 6.
Revised: Lemma 6.

Original: Under same assumptions of Theorem 1, UOGD in Eq.
Revised: Under same assumptions of Theorem 1, UOGD in Eq.

Original: (4) with a step size η > 0 satisfies
Revised: (4) with a step size η > 0 satisﬁes

Original: [Equation]
Revised: [Equation]

Original: for any comparator sequence { u t } Tt =1 with u t ∈ W .
Revised: for any comparator sequence { u t } Tt =1 with u t ∈ W .

Original: Moreover, P T = (cid:80) Tt [-=
 ∥-]
Revised: Moreover, P T = (cid:80) Tt [-=
 ∥-]=2 (cid:107)

Original: u t − u t − 1 ∥ 2 measures the variation of the comparator sequence.
Revised: u t − u t − 1 (cid:107) 2 measures the variation of the comparator sequence.

Original: Proof of Lemma 6.
Revised: Proof of Lemma 6.

Original: We can decompose the L.H.S. of the inequality in Lemma 6 as
Revised: We can decompose the L.H.S. of the inequality in Lemma 6 as

Original: [Equation]
Revised: [Equation]

Original: For term(a)
Revised: term

Original: [Equation]
Revised: [Equation]

Original:  we have
Revised: For term (a), we have

Original: [Equation]
Revised: [Equation]

Original: where the first inequality is due to the Cauchy-Schwarz inequality and the second inequality comes from the AM-GM inequality.
Revised: where the ﬁrst inequality is due to the Cauchy-Schwarz inequality and the second inequality comes from the AM-GM inequality.

Original: Then, we bound term (b) by [19, Lemma 7],
Revised: Then, we bound term (b) by [19, Lemma 7],

Original: [Equation]
Revised: [Equation]

Original: Combining the two terms and taking the summation over T iterations, we have
Revised: Combining the two terms and taking the summation over T iterations, we have

Original: [Equation]
Revised: [Equation]

Original:  where we denote by P t = (cid:80)
Revised: ) where we denote by P t = (cid:80)

Original: Tt =2 ∥
Revised: Tt =2 (cid:107)

Original: u t − u t − 1 ∥ 2 the variation of the comparator sequence { u t } Tt =1 .
Revised: u t − u t − 1 (cid:107) 2 the variation of the comparator sequence { u t } Tt =1 .

Original: The third inequality is due to the fact ∥ a ∥ 22 − ∥ b ∥ 22 ≤ ( ∥ a ∥ 2{+(cid:107) 
+} + ∥ b ∥ 2 ) · ∥ a − b ∥ 2 for any a, b ∈ R d .
Revised: The third inequality is due to the fact (cid:107) a (cid:107) 22 − (cid:107) b (cid:107) 22 ≤ ( (cid:107) a {+(cid:107) 
+} + (cid:107) b (cid:107) 2 ) · (cid:107) a − b (cid:107) 2 for any a, b ∈ R d .

Original: We can further bound the first term of (26) by
Revised: We can further bound the ﬁrst term of (25) by

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where σ is the minimum singular value of the matrix C f 0 .
Revised: where σ is the minimum singular value of the matrix C f 0 .

Original: In above, the second inequality is due to the definition R k 0 ( w ) =
Revised: In above, the second inequality is due to the deﬁnition R k 0 ( w ) =

Original: E x ∼D 0 ( x | y
Revised: E x ∼D 0 ( x | y

Original: = k ) [ ℓ ( f ( w , x ) , k )] and the norm of gradient of the loss function ∥∇ w ℓ ( f ( w , x ) , y ) ∥ 2 is bounded by G for any x ∈ X and y ∈
Revised: = k ) [ (cid:96) ( f ( w , x ) , k )] and the norm of gradient of the loss function (cid:107)∇ w (cid:96) ( f ( w , x ) , y ) (cid:107) 2 is bounded by G for any x ∈ X and y ∈

Original: [ K ] .
Revised: [ K ] .

Original: The third inequality is a consequence of the fact that ∥ x ∥ 1 ≤ √ x 2 for any x ∈ R K .
Revised: The third inequality is a consequence of the fact that (cid:107) x (cid:107) 1 ≤ √ x 2 for any x ∈ R K .

Original: The last second inequality hold since (cid:98) µ (cid:98)
Revised: The last second inequality hold since (cid:98) µ (cid:98)

Original: y t ∈ ∆ K − 1 and then we have ∥ (cid:98) µ (cid:98) 
Revised: y t ∈ ∆ K − 1 and then we have (cid:107) (cid:98) µ (cid:98) y t

Original: y t ∥ 22 ≤ 1 .
Revised: (cid:107) 22 ≤ 1 .

Original: We complete the proof by plugging (27) into (26).
Revised: We complete the proof by plugging (26) into (25).

Original: Then, we provide the proof of Lemma 7, which provide an upper bound for the meta-algorithm of the
Revised: Then, we provide the proof of Lemma 7, which provide an upper bound for the meta-algorithm of the

Original: A TLAS algorithm.
Revised: A TLAS algorithm.

Original: Lemma 7.
Revised: Lemma 7.

Original: By setting the learning rate ε = σB (cid:113) ln N +2 KT , the meta-algorithm of A TLAS (Algorithm 2) satisfies
Revised: By setting the learning rate ε = σB (cid:113) ln N +2 KT , the meta-algorithm of A TLAS (Algorithm 2) satisﬁes

Original: [Equation]
Revised: [Equation]

Revised: for any i ∈ [ N ] , where B (cid:44) sup ( x ,y )

Original: for any i ∈ [ N ] , where B ≜ sup ( x ,y ) ∈X×Y , w ∈W | ℓ ( f ( w , x ) , y ) | is defined as the upper bound of the loss function.
Revised: [Equation] , w ∈W | [Equation] ( f ( w , x ) , y ) | is deﬁned as the upper bound of the loss function.

Original: Proof.
Revised: Proof.

Original: Since A TLAS takes the Hedge algorithm as the meta-algorithm, we can exploit the standard analysis to upper bound the meta-regret.
Revised: Since A TLAS takes the Hedge algorithm as the meta-algorithm, we can exploit the standard analysis to upper bound the meta-regret.

Original: According to the regret guarantee of the Hedge algorithm (Lemma 11 in Appendix G) and setting m t = 0 , we can bound the meta-regret bound by 
Revised: According to the regret guarantee of the Hedge algorithm (Lemma 11 in Appendix G) and setting m t = 0 , we can bound the meta-regret bound by T

Original: [Equation]
Revised: [Equation]

Original: where the last term can be further bounded by
Revised: where the last term can be further bounded by

Original: [Equation]
Revised: [Equation]

Original: where the third inequality is due to the Cauchy–Schwarz inequality.
Revised: where the third inequality is due to the Cauchy–Schwarz inequality.

Original: The last inequality comes from the fact that ∥ C − 1 f 0 ∥ 2 ≤ σ − 1 and ∥ (cid:98) µ (cid:98) y t ∥ 2 ≤ 1 .
Revised: The last inequality comes from the fact that (cid:107) C − 1 f 0 (cid:107) 2 ≤ σ − 1 and (cid:107) (cid:98) µ (cid:98) y t (cid:107) 2 ≤ 1 .

Original: Plugging(29) into (28)  yields
Revised: Plugging (28) into (27) yields

Original: T (cid:88) t = N(cid:88)
Revised: T (cid:88) [Equation] N

Original: j = p t,j (cid:98) R t ( w t,j ) − T (cid:88) t = (cid:98) R t ( w t,i ) ≤ ln N + 2 ε + ε T (cid:88) t =
Revised: [Equation]

Original: B 2 KT σ. We complete the proof by setting ε = σB (cid:113) (ln N +2) KT .
Revised: We complete the proof by setting ε = σB (cid:113) (ln N +2) KT .

Original: F Omitted Details for Section 3. 
Revised: F Omitted Details for Section 3.

Original: In this section, we first introduce the omitted algorithm details for the A TLAS -A DA algorithm in
Revised: In this section, we ﬁrst introduce the omitted algorithm details for the A TLAS -A DA algorithm in

Original: Appendix F.1.
Revised: Appendix F.1.

Original: Then, we present the proof of the Theorem 3 (Appendix F.2), followed by the useful lemmas in the regret analysis.
Revised: Then, we present the proof of the Theorem 3 (Appendix F.2), followed by the useful lemmas in the regret analysis.

Original: F.1 Algorithm Details for A TLAS -A DA
Revised: F.1 Algorithm Details for A TLAS -A DA

Original: This part provides algorithm details of the A TLAS -A DA .
Revised: This part provides algorithm details of the A TLAS -A DA .

Original: The main procedures are presented in
Revised: The main procedures are presented in

Original: Algorithm 4 (base-algorithm) and Algorithm 5 (meta-algorithm).
Revised: Algorithm 4 (base-algorithm) and Algorithm 5 (meta-algorithm).

Original: Hint Functions.
Revised: Hint Functions.

Original: As shown in Section 3.3, a guidance for designing the hint function is to approximate the true priors µ y t with hint priors h y t .
Revised: As shown in Section 3.3, a guidance for designing the hint function is to approximate the true priors µ y t with hint priors h y t .

Original: A direct way is to use current data S t to estimate the hint priors, which we call Forward Hint .
Revised: A direct way is to use current data S t to estimate the hint priors, which we call Forward Hint .

Original: However, when the sample size | S t | is small at each round, the estimated priors can be far from the true priors, which motivates us to reuse the historical information.
Revised: However, when the sample size | S t | is small at each round, the estimated priors can be far from the true priors, which motivates us to reuse the historical information.

Original: The intuition is that we can benefit from the historical data if there are some regular shift patterns.
Revised: The intuition is that we can beneﬁt from the historical data if there are some regular shift patterns.

Original: To see this, we design three additional hint functions.
Revised: To see this, we design three additional hint functions.

Original: Details are listed in the following.
Revised: Details are listed in the following.

Original: • Forward Hint .
Revised: • Forward Hint .

Original: We perform transductive learning with the current data S t by h y t = (cid:98) µ y t .
Revised: We perform transductive learning with the current data S t by h y t = (cid:98) µ y t .

Original: • Window Hint .
Revised: • Window Hint .

Original: With the belief that recent data are useful, we set a sliding window size of L m and use the average of the estimated priors in the window as the hint priors:
Revised: With the belief that recent data are useful, we set a sliding window size of L m and use the average of the estimated priors in the window as the hint priors:

Original: [Equation]
Revised: [Equation]

Original: • Periodic Hint .
Revised: • Periodic Hint .

Original: For a periodic situation where a similar pattern recurs at a specific time interval, we consider reusing the historical data in a periodic manner by maintaining a buffer { ¯ µ i | i ∈
Revised: For a periodic situation where a similar pattern recurs at a speciﬁc time interval, we consider reusing the historical data in a periodic manner by maintaining a buffer { ¯ µ i | i ∈

Original: [ L p ] } with the size of L p .
Revised: [ L p ] } with the size of L p .

Original: At round t , we utilize the estimated prior (cid:98) µ y t to update the priors in the buffer:
Revised: At round t , we utilize the estimated prior (cid:98) µ y t to update the priors in the buffer:

Original: [Equation]
Revised: [Equation]

Original: otherwise , Input: step size η i ∈ H 1: Let w 1 ,i be any point in W 2: for t = 2 to T do 3: construct the risk estimator (cid:98) R t − 1 as (3) 4: update the model of base-learner by 
Revised: Input: step size η i ∈ H 1: Let w 1 ,i be any point in W 2: for t = 2 to T do 3: construct the risk estimator (cid:98) R t − 1 as (3) 4: update the model of base-learner by (cid:98)

Original: [Equation]
Revised: [Equation]

Original: 5: send w t,i to the meta-algorithm 6: end for
Revised: 5: send w t,i to the meta-algorithm 6: end for

Original: Input: step size pool H ; learning rate ε 1: initialization: ∀ i ∈ [ N ] , p 1 ,i = 1 /N 2: for t = 2 to T do 3: receive { w t,i } Ni =1 from base-learners 4: update weight p t ∈ ∆ N according to p t,i ∝ exp (cid:16) − ε (cid:0) t − 1 (cid:80) s
Revised: Input: step size pool H ; learning rate ε 1: initialization: ∀ i ∈ [ N ] , p 1 ,i = 1 /N 2: for t = 2 to T do 3: receive { w t,i } Ni =1 from base-learners 4: update weight p t ∈ ∆ N according to p t,i ∝ exp (cid:16) − ε (cid:0) t − 1 (cid:80) s

Original: =1 (cid:98) R s ( w s,i )
Revised: =1 (cid:98) R s ( w s,i )

Original: + H t ( w t,i ) (cid:1)(cid:17) 5: predict final output w t
Revised: + H t ( w t,i ) (cid:1)(cid:17) 5: predict ﬁnal output w t

Original: =
Revised: =

Original: (cid:80) Ni =1 p t,i w t,i 6: end for
Revised: (cid:80) Ni =1 p t,i w t,i 6: end for

Original: where i ∈ [ L p ] and ¯ µ ( t ) i denotes the value of ¯ µ i at round t .
Revised: where i ∈ [ L p ] and ¯ µ ( t ) i denotes the value of ¯ µ i at round t .

Original: λ t ∈ [0 , 1] is a coefficient to balance the historical and present information, which can be set to be 1 /t or a constant value.
Revised: λ t ∈ [0 , 1] is a coefﬁcient to balance the historical and present information which can be set to be 1 /t or a constant value.

Original: Then we set h y t = ¯ µ ( t ) i t , where i t =
Revised: Then we set h y t = ¯ µ ( t ) i t , where i t =

Original: t mod L p .
Revised: t mod L p .

Original: • Online KMeans Hint .
Revised: • Online KMeans Hint .

Original: To further use the feature similarity of the historical data, we design a hint function with the online k-means clustering algorithms [73].
Revised: To further use the feature similarity of the historical data we design a hint function with the online k-means clustering algorithms [73].

Original: We maintain L k prototypes { (¯ x i , ¯ µ i ) | i ∈
Revised: We maintain L k prototypes { (¯ x i , ¯ µ i ) | i ∈

Original: [ L k ] } .
Revised: [ L k ] } .

Original: At round t , we find the prototype i t , which is the most similar to the current unlabeled data S t :
Revised: At round t , we ﬁnd the prototype i t , which is the most similar to the current unlabeled data S t :

Original: [Equation]
Revised: [Equation]

Original: and then we update the corresponding prototype by
Revised: and then we update the corresponding prototype by

Original: [Equation]
Revised: [Equation]

Original: , where (cid:16) ¯ x ( t ) i t , ¯ µ ( t ) i t (cid:17) represents the value of i t -th prototype at round t .
Revised: where (cid:16) ¯ x ( t ) i t , ¯ µ ( t ) i t (cid:17) represents the value of i t -th prototype at round t .

Original: κ t ∈ [0 , 1] is a coefficient to balance the historical and present information, which can be set to be 1 /t or a constant value.
Revised: κ t ∈ [0 , 1] is a coefﬁcient to balance the historical and present information, which can be set to be 1 /t or a constant value.

Original: Then we set h y t = ¯ µ ( t ) i t .
Revised: Then we set h y t = ¯ µ ( t ) i t .

Original: F.2 Proof of Theorem
Revised: F.2 Proof of Theorem

Original: The proof of Theorem 3 shares a similar spirit with that of Theorem 2, where we decompose the overall dynamic regret into meta-regret and base-regret.
Revised: The proof of Theorem 3 shares a similar spirit with that of Theorem 2, where we decompose the overall dynamic regret into meta-regret and base-regret.

Original: The new challenge is that the A TLAS -A DA algorithm involves the hint function H t ( · ) to exploit the structure of distribution change, which requires a more delicate analysis for both the meta- and base-regret.
Revised: The new challenge is that the A TLAS -A DA algorithm involves the hint function H t ( · ) to exploit the structure of distribution change, which requires a more delicate analysis for both the meta- and base-regret.

Original: Proof of Theorem 3.
Revised: Proof of Theorem 3.

Original: Similar to the proof of Theorem 1 and Theorem 2, we can decompose the dynamic regret into two parts with the piece-wise stationary sequence { u t } Tt =1 ,
Revised: Similar to the proof of Theorem 1 and Theorem 2, we can decompose the dynamic regret into two parts with the piece-wise stationary sequence { u t } Tt =1 ,

Original: E 1: T (cid:34) T (cid:88) t =
Revised: E 1: T (cid:34) T (cid:88) t =

Original: R t ( w t ) (cid:35) −
Revised: R t ( w t ) (cid:35) −

Original: T (cid:88) t =
Revised: T (cid:88) t =

Original: R t ( w ∗ t ) =
Revised: R t ( w ∗ t ) =

Original: E 1: T (cid:34) T (cid:88) t =
Revised: E 1: T (cid:34) T (cid:88) t =

Original: R t ( w t ) (cid:35) −
Revised: R t ( w t ) (cid:35) −

Original: M (cid:88) m
Revised: M (cid:88) m

Original: = (cid:88) t ∈I m
Revised: = (cid:88) t ∈I m

Original: R t ( w ∗I m ) (cid:124) (cid:123)(cid:122) (cid:125) term (a) +
Revised: R t ( w ∗I m ) (cid:124) (cid:123)(cid:122) (cid:125) term (a) +

Original: M (cid:88) m
Revised: M (cid:88) m

Original: = (cid:88) t ∈I m
Revised: = (cid:88) t ∈I m

Original: R t ( w ∗I m ) −
Revised: R t ( w ∗I m ) −

Original: T (cid:88) t =
Revised: T (cid:88) t =

Original: R t ( w ∗ t ) (cid:124) (cid:123)(cid:122) (cid:125) term (b) .
Revised: R t ( w ∗ t ) (cid:124) (cid:123)(cid:122) (cid:125) term (b) .

Original: According to (22), we have term (b) ≤ 2 B ∆ V T .
Revised: According to (22), we have term (b) ≤ 2 B ∆ V T .

Original: Besides, when the parameter is set as ∆ = 1 , we have term (b) = 0 .
Revised: Besides, when the parameter is set as ∆ = 1 , we have term (b) = 0 .

Original: Thus, the term (b) can be bounded as term (b) ≤ 1 { ∆ > 1 } · 2 B ∆ V T .
Revised: Thus, the term (b) can be bounded as term (b) ≤ 1 { ∆ > 1 } · 2 B ∆ V T .

Original: As a consequence, we can focus on the analysis of term (a), which can be further decomposed into the meta-regret and the based-regret.
Revised: As a consequence, we can focus on the analysis of term (a), which can be further decomposed into the meta-regret and the based-regret.

Original: [Equation]
Revised: [Equation]

Original: base
Revised: base

Original: [Equation]
Revised: [Equation]

Original: regret
Revised: regret

Original: [Equation]
Revised: [Equation]

Original: where meta-regret measures the performance gap between the model returned by A TLAS -A DA algorithm and that by the i -th base algorithm.
Revised: where meta-regret measures the performance gap between the model returned by A TLAS -A DA algorithm and that by the i -th base algorithm.

Original: The base-regret is the regret of the i -th base algorithm running with step size η i > 0 .
Revised: The base-regret is the regret of the i -th base algorithm running with step size η i > 0 .

Original: Analysis for the meta-regret.
Revised: Analysis for the meta-regret.

Original: Since w t is independent of the sample observed at iteration t , following the same arguments in obtaining (25), we can bound the meta-regret as
Revised: Since w t is independent of the sample observed at iteration t , following the same arguments in obtaining (24), we can bound the meta-regret as

Original: [Equation]
Revised: [Equation]

Original: Remark 5.
Revised: Remark 5.

Original: The main difference between the analysis of the meta-algorithm for A TLAS -A DA and that of A TLAS is that the former method involves an update step with the hint function.
Revised: The main difference between the analysis of the meta-algorithm for A TLAS -A DA and that of A TLAS is that the former method involves an update step with the hint function.

Original: To obtain a meta-regret bound that adapts the quality of the hint function, the previous work [19] crucially relies on the smoothness of the loss function, which is generally hard to be satisfied in the OLaS problem (e.g., the commonly used hinge loss is not smooth).
Revised: To obtain a meta-regret bound that adapts the quality of the hint function, the previous work [19] crucially relies on the smoothness of the loss function, which is generally hard to be satisﬁed in the OLaS problem (e.g., the commonly used hinge loss is not smooth).

Original: A more delicate analysis shows that the smoothness assumption is no longer necessary to achieve an adaptive meta-regret bound.
Revised: A more delicate analysis shows that the smoothness assumption is no longer necessary to achieve an adaptive meta-regret bound.

Original: The result is stated as the following lemma, whose proof is deferred to Appendix F.3.
Revised: The result is stated as the following lemma, whose proof is deferred to Appendix F.3.

Original: Lemma 8.
Revised: Lemma 8.

Original: By setting the learning rate ε =(cid:113)
Revised: By setting the learning rate ε =

Original: [Equation]
Revised: [Equation]

Original: the metaalgorithm of A TLAS -A DA (Algorithm 5) satisfies
Revised: the metaalgorithm of A TLAS -A DA (Algorithm 5) satisﬁes

Original: [Equation]
Revised: [Equation]

Original: for any i ∈ [ N ] , where (cid:101) R t ( w t,i ) = (cid:98) R t ( w t,i ) − (cid:98) R t ( w ref ) and (cid:101) H t ( w t,i ) =
Revised: for any i ∈ [ N ] , where (cid:101) R t ( w t,i ) = (cid:98) R t ( w t,i ) − (cid:98) R t ( w ref ) and (cid:101) H t ( w t,i ) =

Original: H t ( w t,i ) − H t ( w ref ) are the reference loss functions built upon an arbitrary reference point w ref ∈ W .
Revised: H t ( w t,i ) − H t ( w ref ) are the reference loss functions built upon an arbitrary reference point w ref ∈ W .

Original: According to Lemma 8, we can bound the meta-regret by
Revised: According to Lemma 8, we can bound the meta-regret by

Original: [Equation]

Original: =

Original: [Equation]

Original: w t

Original: [Equation]

Original: −

Original: [Equation]

Original: w

Original: [Equation]

Original: i )

Original: [Equation]

Original: [Equation]

Original: (cid:118)(cid:117)(cid:117)(cid:116) (

Original: [Equation]

Original: [Equation]

Original: 2) (cid:32)

Original: [Equation]
Revised: [Equation]

Original: , where the last inequality is due to the Jensen’s inequality and the concavity of the square root function.
Revised: where the last inequality is due to the Jensen’s inequality and the concavity of the square root function.

Original: Analysis for the base-regret.
Revised: Analysis for the base-regret.

Original: Since the model w t is independent of the draw of dataset S t received at iteration t , following a similar argument in obtain (21), we can bound the base-regret asE [Equation]
Revised: Since the model w t is independent of the draw of dataset S t received at iteration t , following a similar argument in obtain (21), we can bound the base-regret as

Original: [Equation]
Revised: [Equation]

Original: Different from the analysis for the base-algorithm of A TLAS , the base-algorithm of A TLAS - ADA exploit a hint function H t ( · ) to better exploit the pattern of the distribution challenge.
Revised: Different from the analysis for the base-algorithm of A TLAS , the base-algorithm of A TLAS - ADA exploit a hint function H t ( · ) to better exploit the pattern of the distribution challenge.

Original: We have the following regret guarantees on the output w t,i of the base-algorithm.
Revised: We have the following regret guarantees on the output w t,i of the base-algorithm.

Original: Lemma 9.
Revised: Lemma 9.

Original: Suppose the hint function H t : W (cid:55)→ R is convex.
Revised: Suppose the hint function H t : W (cid:55)→ R is convex.

Original: The base-algorithm of A TLAS -A DA (Algorithm 4) with a step size η t,i > 0 satisfies T
Revised: The base-algorithm of A TLAS -A DA (Algorithm 4) with a step size η t,i > 0 satisﬁes

Original: [Equation]
Revised: [Equation]

Original: for any comparator sequence { u t } Tt =1 with u t ∈ W .
Revised: for any comparator sequence { u t } Tt =1 with u t ∈ W .

Original: Moreover, P T = (cid:80) Tt [-=
 ∥-]
Revised: Moreover, P T = (cid:80) Tt [-=
 ∥-]=2 (cid:107)

Original: u t − u t − 1 ∥ 2 measures the variation of the comparator sequence.
Revised: u t − u t − 1 (cid:107) 2 measures the variation of the comparator sequence.

Original: By Lemma 9 and taking the expectation over both sides, we can bound the base-regret as
Revised: By Lemma 9 and taking the expectation over both sides, we can bound the base-regret as

Original: E 1: T
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Then, combining the meta- and base-regret, we can obtain that
Revised: Then, combining the meta- and base-regret, we can obtain that

Original: [Equation]
Revised: [Equation]

Original: for any base-algorithm i ∈
Revised: for any base-algorithm i ∈

Original: [ N ] , where G T =
Revised: [ N ] , where G T =

Revised: [Equation]

Revised: Tt =1 E

Revised: [Equation]

Revised: sup w ∈W

Original: [Equation]
Revised: [Equation]

Revised: R

Original: R t ( w ) − ∇ H t ( w ) ∥ 22 (cid:105) measures the reusability of the history information.
Revised: [Equation] ( w ) − ∇ H [Equation] ( w ) (cid:107) 22 (cid:105) measures the reusability of the history information.

Original: It remains to tune the step size to make the bound tight.
Revised: It remains to tune the step size to make the bound tight.

Original: We can bound the term G t by
Revised: We can bound the term G t by

Original: [Equation]
Revised: [Equation]

Original: where the last inequality is due to the fact that sup w ∈W ∥∇ (cid:98) R t ( w t )∥ 2 ≤ ( G √ K )
Revised: where the last inequality is due to the fact that sup w ∈W (cid:107)∇ [Equation] R t ( w t )

Original:  /σ (shown as (27)) and the assumption that sup w ∈W ∥∇ H t ( w ) ∥ 2 ≤ ( G √ K ) /σ . 
Revised: [Equation] 2 ≤ ( G √ K ) /σ (shown as (26)) and the assumption that sup w ∈W (cid:107)∇ H t ( w ) (cid:107) 2 ≤ ( G √ K ) /σ . In such a case, we can show that the optimal step size η [Equation]

Revised: = 12 (cid:113)

Revised: [Equation]

Revised: [Equation]

Original: In such a case, we can show that the optimal step size η ∗ = 12 (cid:113) Γ 2 +2Γ P T 1+ G T lies in the range [ 12 (cid:113) Γ 2 σ 2 σ 2 +4 TG 2 K , 12 √ Γ 2 + 2Γ 2 T ] .
Revised: T 1+ G T lies in the range [ 12 (cid:113) [Equation] [Equation] 2 K , 12 √ Γ 2 + 2Γ 2 T ] .

Original: So, we can construct the following step size pool to cover the optimal step size
Revised: So, we can construct the following step size pool to cover the optimal step size

Original: [Equation]
Revised: [Equation]

Original: with N = 1 + (cid:6) 1 2 log 2 (cid:0) (1 + 2 T )(1 + 4 TG 2 K/σ 2 ) (cid:1)(cid:7) .
Revised: with N = 1 + (cid:6) [Equation] σ 2 ) (cid:1)(cid:7) .

Original: Even better, due to the logarithmic construction of the pool, we claim that there must exist an i ∗ ∈
Revised: Even better, due to the logarithmic construction of the pool, we claim that there must exist an i ∗ ∈

Original: [ N ] satisfies that η i ∗ / 2 ≤ η ∗ ≤ η i ∗ .
Revised: [ N ] satisﬁes that η i ∗ / 2 ≤ η ∗ ≤ η i ∗ .

Original: So, we further bound term (a) as
Revised: So, we further bound term (a) as

Original: [Equation]
Revised: [Equation]

Original: Overall dynamic regret bound.
Revised: Overall dynamic regret bound.

Original: We can then use similar arguments in the proof of Theorem 2 to obtain the final dynamic regret bound.
Revised: We can then use similar arguments in the proof of Theorem 2 to obtain the ﬁnal dynamic regret bound.

Original: Combining term (a) and term (b) and noticing that { u t } Tt =1 is the piecewise stationary comparator sequences with P T =
Revised: Combining term (a) and term (b) and noticing that { u t } Tt =1 is the piecewise stationary comparator sequences with P T =

Original: (cid:80) Tt [-=
 ∥-]
Revised: (cid:80) Tt [-=
 ∥-]=2 (cid:107)

Original: u t − u t − 1 ∥
Revised: u t − u t − 1 (cid:107)

Original: Tt =1 ≤ Γ T/ ∆ , we can bound the overall dynamic regret as
Revised: Tt =1 ≤ Γ T/ ∆ , we can bound the overall dynamic regret as

Original: [Equation]
Revised: [Equation]

Revised: ≤

Revised: [Equation]

Revised: { ∆ > 1 }

Revised: [Equation]

Revised: [Equation]

Revised: G

Revised: [Equation]

Revised: + Γ (cid:112) (ln N

Revised: [Equation]

Revised: [Equation]

Revised: G

Revised: [Equation]

Revised: ≤

Revised: [Equation]

Revised: { ∆ > 1 } · 2 B ∆ V T + 3Γ

Revised: [Equation]

Revised: [Equation]

Revised: G T

Revised: [Equation]

Revised: ∆

Revised: [Equation]

Revised: G

Revised: [Equation]

Revised: (ln N + 2)(1 + G T

Revised: [Equation]

Original: where the last inequality is due to the fact that √ a + b ≤ √ a + √ b for any a, b ≥ 0 .
Revised: where the last inequality is due to the fact that √ a + b ≤ √ a + √ b for any a, b ≥ 0 .

Original: To obtain an upper bound of the above inequality, we consider the two following cases,
Revised: To obtain an upper bound of the above inequality, we consider the two following cases,

Original: Case 1 : 3Γ (cid:112) 2(1 + G T ) T > 4 BV T , in such a case, we can set ∆ =
Revised: Case 1 : 3Γ (cid:112) 2(1 + G T ) T > 4 BV T , in such a case, we can set ∆ =

Original: ⌈ 3Γ √[Equation] [Equation] G
Revised: (cid:100) 3Γ √

Original: [Equation]
Revised: [Equation]

Original: BV

Original: [Equation]
Revised: [Equation]

Revised: (cid:101) .

Original: Then, we can bound the dynamic regret by
Revised: Then, we can bound the dynamic regret by

Original: [Equation]
Revised: [Equation]

Original: Case 2: 3Γ (cid:112) 2(1 + G T ) T ≤ 4 BV T , in such a case, we can set ∆ = 1 .
Revised: Case 2: 3Γ (cid:112) 2(1 + G T ) T ≤ 4 BV T , in such a case, we can set ∆ = 1 .

Original: Then, we have
Revised: Then, we have

Original: E

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where the second inequality is due to the inequality that 3Γ (cid:112) 2(1 + G T ) T ≤ 4 BV T .
Revised: where the second inequality is due to the inequality that 3Γ (cid:112) 2(1 + G T ) T ≤ 4 BV T .

Original: We complete the proof by combining the two cases.
Revised: We complete the proof by combining the two cases.

Original: F.3 Useful Lemmas in Analysis of Theorem
Revised: F.3 Useful Lemmas in Analysis of Theorem

Original: Lemma 8.
Revised: Lemma 8.

Original: By setting the learning rate ε = (cid:113)
Revised: By setting the learning rate ε = (cid:113)

Original: [Equation]
Revised: [Equation]

Revised: [Table]

Original: , the metaalgorithm of A TLAS -A DA (Algorithm 5) satisfies
Revised: , the metaalgorithm of [Table] [Equation] [Table]

Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Original: [Equation]
Revised: [Equation]

Revised: [Table]

Original:  for any i ∈ [ N ] , where (cid:101) R t ( w t,i ) = (cid:98) R t ( w t,i ) − (cid:98) R t ( w ref ) and (cid:101) H t ( w t,i ) =
Revised: [Equation] [Table] [Equation] for any i ∈ [ N ] , where (cid:101) R t ( w t,i ) = (cid:98) R t ( w t,i ) − (cid:98) R t ( w ref ) and (cid:101) H t ( w t,i ) =

Original: H t ( w t,i ) − H t ( w ref ) are the reference loss functions built upon an arbitrary reference point w ref ∈ W .
Revised: H t ( w t,i ) − H t ( w ref ) are the reference loss functions built upon an arbitrary reference point w ref ∈ W .

Original: Proof of Lemma 8.
Revised: Proof of Lemma 8.

Original: The key challenge in the proof is that the meta-algorithm is updated with the loss function (cid:98) R t ( w t,i ) and the hint function H t ( w t,i ) , while we desire a regret that can scale with the gradient to make it compatible with the base-regret bound.
Revised: The key challenge in the proof is that the meta-algorithm is updated with the loss function (cid:98) R t ( w t,i ) and the hint function H t ( w t,i ) , while we desire a regret that can scale with the gradient to make it compatible with the base-regret bound.

Original: To this end, we introduce the reference loss (cid:101) R t ( w t,i ) =
Revised: To this end, we introduce the reference loss (cid:101) R t ( w t,i ) =

Original: (cid:98) R t ( w t,i ) − (cid:98) R t ( w ref ) and (cid:101) H t ( w t,i ) =
Revised: (cid:98) R t ( w t,i ) − (cid:98) R t ( w ref ) and (cid:101) H t ( w t,i ) =

Original: H t ( w t,i ) − H t ( w ref ) .
Revised: H t ( w t,i ) − H t ( w ref ) .

Original: It is easy to verify that the update procedure (4) with the original loss function is identical to that with the reference loss: p t,i = 
Revised: It is easy to verify that the update procedure (4) with the original loss function is identical to that with the reference loss: p t,i = exp (cid:0)

Original: [Equation]
Revised: [Equation]

Original: As a consequence, we can bound the meta-regret by
Revised: As a consequence, we can bound the meta-regret by

Original: T (cid:88) t =
Revised: T (cid:88) t =

Original: N (cid:88)
Revised: N (cid:88)

Original: j = p t,j (cid:98) R t ( w t,j ) −
Revised: j = p t,j (cid:98) R t ( w t,j ) −

Original: T (cid:88) t
Revised: T (cid:88) t

Original: = (cid:98) R t ( w t,i ) =
Revised: = (cid:98) R t ( w t,i ) =

Original: T (cid:88) t =
Revised: T (cid:88) t =

Original: N (cid:88)
Revised: N (cid:88)

Original: j = p t,j (cid:101) R t ( w t,j ) −
Revised: j = p t,j (cid:101) R t ( w t,j ) −

Original: T (cid:88) t = (cid:101) R t ( w t,i ) ≤ ε
Revised: T (cid:88) t = (cid:101) R t ( w t,i ) ≤ ε

Original: T (cid:88) t
Revised: T (cid:88) t

Original: = (cid:0) max i ∈
Revised: = (cid:0) max i ∈

Original: [ N ] {| (cid:101) R t ( w t,i ) − (cid:101)
Revised: [ N ] {| (cid:101) R t ( w t,i ) − (cid:101)

Original: H t ( w t,i ) |} (cid:1) 

Revised: H t ( w t,i ) |} (cid:1)

Original: + ln N + 2 ε ≤(cid:118) (cid:117) (cid:117) (cid:116)
Revised: + ln N + 2 ε ≤(cid:118) (cid:117) (cid:117) (cid:116)

Original: T (cid:88) t
Revised: T (cid:88) t

Original: = (cid:0)max i ∈ [ N ] {| (cid:101) R t [Equation]
Revised: = (cid:0)

Original: [Equation]
Revised: [Equation]

Original: where the first inequality is due to Lemma 11 and the second inequality is by the step size setting of
Revised: where the ﬁrst inequality is due to Lemma 11 and the second inequality is by the step size setting of

Original: [Equation]
Revised: [Equation]

Original: Then, we show that the deviation between the loss function max i ∈ [ N ] {| (cid:101) R t
Revised: Then, we show that the deviation between the loss function max i ∈ [ N ] {| (cid:101) R t

Original: [Equation]
Revised: [Equation]

Original: w t,i ) − (cid:101)
Revised: w t,i ) − (cid:101)

Original: H t ( w t,i ) |} can be converted to the deviation between the gradient.
Revised: H t ( w t,i ) |} can be converted to the deviation between the gradient.

Original: Let G t ( w t,i ) =
Revised: Let G t ( w t,i ) =

Original: (cid:98) R t ( w t,i ) − H t ( w t,i ) , for any i ∈ [ N ] , we have
Revised: (cid:98) R t ( w t,i ) − H t ( w t,i ) , for any i ∈ [ N ] , we have

Original: [Equation]
Revised: [Equation]

Original: where the first equality is due to the mean value theorem and ξ t,i
Revised: where the ﬁrst equality is due to the mean value theorem and ξ t,i

Original: =
Revised: =

Original: c t,i w t,i + (1 − c t,i ) w ref with a certain constant c t,i ∈ [0 , 1] .
Revised: c t,i w t,i + (1 − c t,i ) w ref with a certain constant c t,i ∈ [0 , 1] .

Original: Plugging (32) into (31), we obtain that
Revised: Plugging (31) into (30), we obtain that

Original: T 
Revised: T (cid:88) [Equation] =

Original: [Equation]
Revised: [Equation]

Original: which completes the proof.
Revised: which completes the proof.

Original: Then, we provide the proof of the regret of the base-algorithm for A TLAS -A DA .
Revised: Then, we provide the proof of the regret of the base-algorithm for A TLAS -A DA .

Original: Lemma 9.
Revised: Lemma 9.

Original: Suppose the hint function H t : W (cid:55)→ R is convex.
Revised: Suppose the hint function H t : W (cid:55)→ R is convex.

Original: The base-algorithm of A TLAS -A DA (Algorithm 4) with a step size η t,i > 0 satisfies
Revised: The base-algorithm of A TLAS -A DA (Algorithm 4) with a step size η t,i > 0 satisﬁes

Original: [Equation]
Revised: [Equation]

Original: for any comparator sequence { u t } Tt =1 with u t ∈ W .
Revised: for any comparator sequence { u t } Tt =1 with u t ∈ W .

Original: Moreover, P T = (cid:80) Tt [-=
 ∥-]
Revised: Moreover, P T = (cid:80) Tt [-=
 ∥-]=2 (cid:107)

Original: u t − u t − 1 ∥ 2 measures the variation of the comparator sequence.
Revised: u t − u t − 1 (cid:107) 2 measures the variation of the comparator sequence.

Original: Proof of Lemma 9.
Revised: Proof of Lemma 9.

Original: We decompose the instantaneous loss of the base algorithm as,
Revised: We decompose the instantaneous loss of the base algorithm as,

Original: [Equation]
Revised: [Equation]

Original: For term (a) [Equation] by Cauchy-Schwarz inequality and AM-GM inequality
Revised: For term (a), by Cauchy-Schwarz inequality and AM-GM inequality, we have

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: Then, we bound term (b) by Proposition 4.1 of Campolongo and Orabona [34],
Revised: we bound term (b) by Proposition [Equation] 1 of Campolongo and Orabona [34],

Original: [Equation]
Revised: [Equation]

Original: The term (c) is bounded by Lemma 7 of Zhao et al.
Revised: The term (c) is bounded by Lemma 7 of Zhao et al.

Original: [Equation]
Revised: [Equation]

Original: Combining the three upper bounds yields
Revised: Combining the three upper bounds yields

Original: [Equation]
Revised: [Equation]

Revised: (cid:0) (cid:107) u t − (cid:98) w t

Revised: [Equation]

Original: which completes the proof
Revised: i (cid:107) 22 − (cid:107) u t − (cid:98) w t + [Equation] i (cid:107) 22 (cid:1) [Equation]

Revised: Γ 2 η i

Original: [Equation]
Revised: [Equation]

Revised: which completes the proof.

Original: G Technical Lemmas
Revised: G Technical Lemmas

Original: Lemma 10.
Revised: Lemma 10.

Original: Let A ∈ R K × K be an invertible matrix and ∆ ∈ R K × K be a squared matrix such that ∥ A − 1 ∆ ∥ 2 ≤ 1 / 2 .
Revised: Let A ∈ R K × K be an invertible matrix and ∆ ∈ R K × K be a squared matrix such that (cid:107) A − 1 ∆ (cid:107) 2 ≤ 1 / 2 .

Original: For B =
Revised: For B =

Original: A + ∆ , we have the following two claims:
Revised: A + ∆ , we have the following two claims:

Original: • B is an invertible matrix.
Revised: • B is an invertible matrix.

Original: • ∥ B − 1 −A −
Revised: • (cid:107) B − 1 −

Original: [Equation] [Equation]
Revised: A − 1 (cid:107) 2 ≤ 2 (cid:107) A − 1 (cid:107) 22 · (cid:107) ∆ (cid:107) 2 .

Original: Proof.
Revised: Proof.

Original: The proof follows the same arguments in [74, Page 381].
Revised: The proof follows the same arguments in [74, Page 381].

Original: We first show that B
Revised: We ﬁrst show that B

Original: =
Revised: =

Original: A ( I + A − 1 ∆) is an invertible matrix.
Revised: A ( I + A − 1 ∆) is an invertible matrix.

Original: Since A is an invertible matrix by the assumption, it is sufficient to show that I + A − 1 ∆ is invertible.
Revised: Since A is an invertible matrix by the assumption, it is sufﬁcient to show that I + A − 1 ∆ is invertible.

Original: The statement is true since the spectral radius of A − 1 ∆ satisfies ρ ( A − 1 ∆) ≤ 
Revised: The statement is true since the spectral radius of A − 1 ∆ satisﬁes ρ ( A [Equation] ) ≤ (cid:107)

Original: ∥
Revised: [Equation]

Original: A − 1 ∆ ∥ 2 ≤ 1 / 2 .
Revised: 1 ∆ (cid:107) 2 ≤ 1 / 2 .

Original: The bounded spectral radius implies that − 1 is not an eigenvalue of the matrix A − 1 ∆ , and thus it is invertible.
Revised: The bounded spectral radius implies that − 1 is not an eigenvalue of the matrix A − 1 ∆ , and thus it is invertible.

Original: Then, we can upper bound the norm
Revised: Then, we can upper bound the norm

Original: [Equation]
Revised: [Equation]

Original: 33) where the term ∥ B − 1 ∥ 2 can be further bounded by
Revised: where the term (cid:107) B − 1 (cid:107) 2 can be further bounded by

Original: [Equation]
Revised: [Equation]

Original: Rearranging the above inequality, we have
Revised: Rearranging the above inequality, we have

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: Then, plugging(34) into (33)  and rearranging the terms, we have
Revised: Then, plugging (33) into (32) and rearranging the terms, we have

Original: [Equation]
Revised: [Equation]

Original: which completes the proof.
Revised: which completes the proof.

Original: Lemma 11 (Theorem 19 of Syrgkanis et al.
Revised: Lemma 11 (Theorem 19 of Syrgkanis et al.

Original: Let ℓ t ∈ R N be the loss vector taking ℓ t, i ∈
Revised: Let (cid:96) t ∈ R N be the loss vector taking (cid:96) t,i ∈

Original: R as its i -th entry and m t ∈ R N the hint vector taking m t,i as its i -th entry.
Revised: R as its i -th entry and m t ∈ R N the hint vector taking m t,i as its i -th entry.

Original: The Hedge algorithm updating with p t,i ∝ exp
Revised: The Hedge algorithm updating with p t,i ∝ exp

Original: [Equation]
Revised: [Equation]

Revised: satisﬁes

Revised: [Equation]

Original: for any i ∈ [ N ] , where ε > 0 is the step size.
Revised: for any i ∈ [ N ] , where ε > 0 is the step size.