Title: Online robust non-stationary estimation
Venue: NeurIPS 2023
Accepted: yes
Keywords: Estimation

heavy-tails

distribution shifts

regret
Abstract: The real-time estimation of time-varying parameters from high-dimensional, heavy-tailed and corrupted data-streams is a common sub-routine in systems ranging from those for network monitoring and anomaly detection to those for traffic scheduling in data-centers. For estimation tasks that can be cast as minimizing a strongly convex loss function, we prove that an appropriately tuned version of the {\ttfamily clipped Stochastic Gradient Descent} (SGD) is simultaneously {\em(i)} adaptive to drift, {\em (ii)} robust to heavy-tailed inliers and arbitrary corruptions,  {\em(iii)} requires no distributional knowledge and {\em (iv)} can be implemented in an online streaming fashion. All prior estimation algorithms have only been proven to posses a subset of these practical desiderata. A observation we make is that, neither the $\mathcal{O}\left(\frac{1}{t}\right)$ learning rate for {\ttfamily clipped SGD} known to be optimal for strongly convex loss functions of a \emph{stationary} data-stream, nor the $\mathcal{O}(1)$ learning rate known to be optimal for being adaptive to drift in a \emph{noiseless} environment can be used. Instead, a learning rate of $T^{-\alpha}$ for $ \alpha < 1$ where $T$ is the stream-length is needed to balance adaptivity to potential drift and to combat noise. We develop a new inductive argument and combine it with a martingale concentration result to derive high-probability under \emph{any learning rate} on data-streams exhibiting \emph{arbitrary distribution shift} - a proof strategy that may be of independent interest. Further, using the classical doubling-trick, we relax the knowledge of the stream length $T$. Ours is the first online estimation algorithm that is provably robust to heavy-tails, corruptions and distribution shift simultaneously. We complement our theoretical results empirically on synthetic and real data.

Online robust non-stationary estimation

Abstract

The real-time estimation of time-varying parameters from high-dimensional, heavytailed and corrupted data-streams is a common sub-routine in systems ranging from those for network monitoring and anomaly detection to those for traffic scheduling in data-centers. For estimation tasks that can be cast as minimizing a strongly convex loss function, we prove that an appropriately tuned version of the clipped Stochastic Gradient Descent (SGD) is simultaneously (i) adaptive to drift, (ii) robust to heavy-tailed inliers and arbitrary corruptions, (iii) requires no distributional knowledge and (iv) can be implemented in an online streaming fashion. All prior estimation algorithms have only been proven to posses a subset of these practical desiderata. A observation we make is that, neither the O ( 1 t ) learning rate for clipped SGD known to be optimal for strongly convex loss functions of a stationary data-stream, nor the O (1) learning rate known to be optimal for being adaptive to drift in a noiseless environment can be used. Instead, a learning rate of T -α for α < 1 where T is the stream-length is needed to balance adaptivity to potential drift and to combat noise. We develop a new inductive argument and combine it with a martingale concentration result to derive high-probability under any learning rate on data-streams exhibiting arbitrary distribution shift - a proof strategy that may be of independent interest. Further, using the classical doubling-trick, we relax the knowledge of the stream length T . Ours is the first online estimation algorithm that is provably robust to heavy-tails, corruptions and distribution shift simultaneously. We complement our theoretical results empirically on synthetic and real data.

1 Introduction

Technology improvements have made it easier than ever to collect diverse telemetry at high resolution from any cyber or physical system, for both monitoring and control [31]. This in turn has led to a data deluge, where large amounts of data must be processed at scale [7, 16]. Given the scale and velocity of the data sources, offline processing to make predictions and decisions is computationally cumbersome. For real-time applications such as performance monitoring and anomaly detection, offline/batch processing results in stale predictions [6], [61], [54]. This necessitates computationally cheap, online algorithms that make predictions and decisions on high-dimensional streaming data [41], [21], [58]. Further in many applications, the challenge of being restricted to online algorithms is exacerbated by heavy-tails [2], [3], distribution shift [48], [28] and outliers/anomalies in the data-stream [29], [42], [35, 50]. In practice, although several heuristics to circumvent these issues have been designed [1, 66, 48, 28, 35, 42], there is no systematic study of the impact of these challenges on the achievable statistical accuracy. Motivated by these problems, we study algorithms for high-dimensional online statistical estimation and rigorously establish performance guarantees which exhibit graceful degradation with distribution shift and outliers in the data-stream.

(a)

(b)

(c)

Pareto

Cumulativeregret

15

50

1200

Pareton=1/t

40

1000

Pareton=1/√T

10

Truemean

30

800

Gaussiann=1/t

5

Samples

Corruptions

20

600

Gaussiann=1/√T

400

10

0

200

0

0

0

25

50

75

100

125

150

0

25

50

75

100

125

150

0

25

50

75

100

125

150

1.1 Problem Setup

The online estimation problem we study is modeled as an interaction between an oblivious adversary and the estimator. The adversary before the interaction begins makes two choices (i) a sequence of probability distributions P 1 , · · · , P T on R d and (ii) a sequence of measurable corruption functions ( C t ) T t =1 where for every t ∈ [ T ] 1 , C t : R d × t × R d × t -1 × R p × t -1 → R d , whose roles we explain below.

Sequential interaction protocol : At each time t ∈ [ T ] , the vector X t := Z t + C t , where Z t ∼ P t is sampled independently and C t := C t (( Z s ) s ≤ t , ( C s ) s<t , ( θ s ) s<t ) is the corruption vector computed using the corruption function based on the past random samples and outputs . X t is shown as input to the estimation algorithm. Subsequently after observing X t , the estimation algorithm outputs an estimate θ t ∈ Θ and incurs loss l t := ∥ θ t -θ ∗ t ∥ 2 . Here, L : R d × R p → R is a strongly convex loss function, Θ ⊂ R p a closed convex set denoting the parameter space, and θ ∗ t := arg min θ ∈ Θ E Z ∼ P t [ L ( Z, θ )] is the unique 3 optimizer of the loss function with respect to the distribution P t , chosen by the adversary at time t . For every time t , we denote this minimizer θ ∗ t ∈ Θ as the true parameter at time t . The incurred loss l t is un-observed by the algorithm since the true parameter θ ∗ t is unknown.

Formally, an estimation 4 algorithm A := ( A t ) T t =1 is a sequence of measurable functions such that for all t ∈ [ T ] , the estimate output θ t := A t ( X t , · · · , X 1 ) , where A t : R d × t → R p is a function of all inputs upto and including time t . After outputting an estimate θ t , the algorithm incurs an un-observed loss which is the L2 distance between the estimate and the true parameter.

Causal property of the adversary: Recall that at each time t , the corruption vector is given by C t = C t (( Z s ) s ≤ t , ( C s ) s<t , ( θ s ) s<t ) , which is based only on the past realizations Z 1 , · · · , Z t , C 1 , · · · , C t -1 and the estimator's past outputs θ 1 , · · · , θ t -1 . The adversary does not have access to future randomness Z t +1 , · · · , Z T while choosing the corruption vector C t . This model of corruptions is stronger than those used in online robust estimation (cf. [19], [13]) since in our model, the adversary can choose the time instants of corruption while in previous models, the time instants of corruption are chosen randomly through i.i.d. coin flips. Our model of corruption locations are motivated by empirical observations that corruptions rarely occur randomly and are typically bunced [23, 42]. Thus we assume that the adversary has causal power to choose the location of corruptions rather than corruptions occuring at random instants.

Regret as a Performance Measure: The regret of the algorithm denoted by Reg T is defined as

̸

the total cumulative loss. This definition is known as clean dynamic regret [13], since (i) θ t is compared against the true parameter θ ∗ t , even if the input X t at time t is corrupted , i.e., even if C t = 0 , and (ii) θ t is compared to the time-varying optimal θ ∗ t . Technically the regret of an algorithm A is a random variable that is a function of the probability measures ( P t ) T t =1 and the corruption functions ( C t ) T t =1 and denoted as Reg ( A ) T (( P t ) T t =1 , ( C t ) T t =1 ) . We will denote the regret as Reg T whenever it is clear from context.

We now define the data-stream complexity parameters impacting regret : (i) distribution shift and (ii) corruptions.

Distribution Shift 5 : The cumulative distance between successive optimum points Φ T := ∑ T t =2 ∥ θ ∗ t -θ ∗ t -1 ∥ measures the complexity of distribution-shift on the data-stream. This is a natural measure since if the estimator outputs θ t ∈ Θ at time t , will incur an L 2 loss l t = ∥ θ t -θ ∗ t ∥ .

̸

Corruptions: The corruption measure is the count Λ T := ∑ T t =1 1 ( C t = 0) , of the number of times the random sample Z t is corrupted before inputting to the estimator. The count Λ T is a random-variable since the vector C t is a deterministic function of past random variables.

With this setup, the main desiderata we seek of an estimator is that it (i) is online : O ( d ) run-time per data point on a stream and O ( d ) overall memory, (ii) is free of distributional knowledge , i.e., does not require bounds on moments of ( P t ) t ≥ 1 , distribution shift or corruptions, (iv) attains sub-linear in T regret even when the distributions ( P t ) t ≥ 1 are heavy-tailed , and (v) exhibits graceful degradation of regret with respect to distribution shift and corruptions on the data stream.

1.2 Main Contributions

Problem formulation and lower bounds : We formalize the problem of online estimation under drift and corruptions and define a dimension-free target regret benchmark in Equation (1) that captures all the aforementioned desiderata. We give lower bounds in Propositions 16.1 and 2.6 that shows the necessity of assumptions such as finite diameter for the set Θ ⊂ R d in which the unknown parameters θ ∗ t ∈ Θ lie. Further discussion in Table 1 and in Section 11.

High-probability upper bounds : Theorem 5.1 proves that a tuned version of clipped-SGD achieves all aforementioned desiderata. No previous algorithm achieves all the desiderata simultaneously (Tables 1 and 2). Further in Theorem 4.3, we show that if in addition the data-stream is sub-gaussian, then our bound is optimal in the sense when instantiated with special cases of having no drift and corruptions [24] or in the setting of no corruptions and 0 variance [43], it recovers known optimal results.

Algorithmic insights : Our key finding is that neither the learning rate of O (1) known to be optimal for SGD to adapt to distribution shift in the absence of noise, nor the O (1 /t ) learning rate known to be optimal in a stationary environment can be used. Instead a learning rate of T -α for α < 1 is needed to combat noise and be adaptive to drift simultaneously. See also Figure 1. We show a lower bound in Proposition 6.1 that this insight is fundamental and not an artifact of our analysis. Knowledge of T is relaxed using the doubling trick in Section 14 in the Appendix. Our result improves prior state of art in [8] that show that in the absence of corruptions, SGD without clipping can give sub-linear regret bounds holding in expectation . However, in the presence of heavy-tailed noise, high probability guarantees are more insightful compared to only in expectation [63]. Our algorithmic contribution is that a tuned version of clipped SGD can give high probability sub-linear regret guarantees, that hold even in the presence of non-stationarities and corruptions.

Technical contributions in the proof techniques : From a technical perspective, we employ novel proof techniques to derive high-probability under any learning rate on data-streams exhibiting arbitrary distribution shift . The prior state-of-art analysis in [36, 55] are limited to the specialized setting when the learning rate is η t = 1 / ( t +1) and the data does not exhibit drift. To be adaptive to drift requires newer techniques even in the case of sub-gaussian tails to achieve dimension-free bounds (cf. Section 4.1). We used an induction based technique to apply martingale concentrations in Lemma 19.9. We build upon this technique using contradiction arguments in Lemma 20.10 to extend to the heavy-tailed case. These arguments are of independent interest to prove bounds for other iterative algorithms.

1.3 Motivating application

Online Anomaly Detection (AD) problems consist of a loss function L ( · , · ) and parameter θ t that varies with time t , such that for the input X t received at time t , the algorithm outputs anomaly score S t := L ( X t , θ t ) ([47], [12]). The objective is to output lower scores for samples that are not corrupted, i.e., if at time t the data distribution is Z t ∼ P t , then the optimal anomaly score S ∗ t := L ( X t , θ ∗ t ) is the one produced by the model minimizing the average anomaly score i.e., θ ∗ t ∈ arg min θ ∈ Θ E Z ∼ P t [ L ( Z, θ )] [51]. The distance ∥ θ t -θ ∗ t ∥ between θ t the model used at time t and the optimal θ ∗ t is a measure of AD performance degradation [32, 50], motivating our study of low regret algorithms.

Special cases of such problems include robust online mean estimation [19] and robust online linear regression [20], both of which we consider in simulations in Section 7. The online mean estimation corresponds to L ( X,θ ) := 1 2 ∥ X -θ ∥ 2 2 since arg min θ E Z ∼ P [ ∥ Z -θ ∥ 2 ] = E Z ∼ P [ Z ] . Online linear regression corresponds to splitting the input X ∈ R d 1 + d 2 as X := ( X (1) , X (2) ) , with X (1) ∈ R d 1 and X (2) ∈ R d 2 and using the re-construction loss L ( X,θ ) := 1 2 ∥ X (2) -θ T X (1) ∥ 2 2 as anomaly score. The linear regression setting of detecting anomalies by reconstructing one half of the input X from the other is popularly known as self-supervised AD models [52]. Empirically, in [30], L ( X,θ ) := ∥ ( I d -θ ) X ∥ with θ ∈ R d × d a matrix of rank k < d and I d ∈ R d × d the identity matrix is used to detect anomalies. The linear model of [30] is extended to a non-linear setting in [42], using an auto-encoder [65]. In [17], a tree density model for L ( X,θ ) is studied. These papers' focus is empirical and do not give statistical guarantees.

Organization of the Paper: In Section 2 we state the target benchmark regret along with the assumptions under which we seek it and appropriate lower bounds. In Section 3 we give the clippedSGD algorithm. In Section 4, we give results under the additional assumption of sub-gaussian tails. This helps build intuition and proof techniques for the general result which we give in Section 5. We state implications of our results in Section 6, provide empirical evidence in Section 7, discuss related work in Section 8 and conclude with open problems in Section 9.

2 Formalizing the desiderata

Before formalizing the desiderata in Equation (1), we state some mild assumptions standard in the study of statistical estimation under which we seek guarantees [55, 19].

2.1 Model Assumptions

Assumption 2.1 (Strong convexity) . There exists 0 < m ≤ M < ∞ known to the algorithm, such that for all t , the function θ → E Z ∼ P t [ L ( Z, θ )] is M smooth and m strongly convex.

This is a benign assumption that essentially states that properties of the loss function such as the convexity and smoothness are known to the estimation algorithm.

Assumption 2.2 (Convex domain) . The algorithm knows a closed convex set Θ ⊂ R p such that for all t , θ ∗ t := arg min θ ∈ Θ E Z ∼ P t [ L ( Z, θ )] ∈ Θ is the true parameter to be estimated at time t .

Assumption 2.3 (Finite diameter) . The diameter D := max x,y ∈ Θ ∥ x -y ∥ of Θ is finite, i.e., D < ∞ .

This is a necessary assumption when Λ T > 0 (Proposition 2.6). If Λ T = 0 , we relax this assumption in Corollaries 4.5 and 5.4. Let R t ( θ ) = E Z ∼ P t [ L ( Z, θ )] , denote the population risk at time t .

Assumption 2.4 (Known finite upper bound on the true gradient) . There exists a known finite G < ∞ such that sup θ ∈ Θ ∥∇R t ( θ ) ∥ ≤ G .

This is necessary for many online optimization algorithms [36, 55, 25, 27].

Assumption 2.5 (Existence of second moment) . There exists a matrix Σ ⪰ 0 , unknown to the algorithm such that for all t and θ ∈ Θ , the covariance of the random-vector ∇L ( Z, θ ) is bounded by Σ , i.e., E Z ∼ P t [( ∇L ( Z, θ ) -∇R t ( θ )) · ( ∇L ( Z, θ ) -∇R t ( θ )) T ] ⪯ Σ holds for all t and θ .

The class of distributions admissible by this assumption is large including all sub-gaussian distributions and heavy-tailed distributions that do not have finite higher moments [25], [44]. Observe that no parameteric assumptions on the distribution family is made. Throughout, we denote by σ 2 and ν max (Σ) as the trace and the largest eigen-value of Σ respectively. For a positive semi-definite matrix Σ ∈ R p × p , P (Σ) is the set of all probability measures satisfying assumptions 2.1, 2.4 and 2.5.

2.2 Target benchmark for algorithm design

We say that an estimator is free of distributional knowledge if it does not need as input bounds on either the moments of P t nor on the stream complexity Φ T and Λ T . Our key focus - formalized below in Equation (1) - is: can we design an online algorithm without distributional assumptions that has sub-linear regret and that degrades gracefully with drift and corruption?

Can we design an online algorithm A without distributional knowledge, such that ∃ 1 such that ∀ Σ ⪰ 0 ∈ R p × p , ( P ) T ∈ P (Σ) and ( C ) T , the regret bound

holds with probability at-least 1 -?

Throughout, we use O ( · ) to hide poly-logarithmic factors. Our main result is Theorem 5.1, where we prove that tuned clipped SGD achieves this, and our other desiderata.

2.3 Why is Equation (1) a good benchmark for regret bounds with drift and corruption?

The benchmark demands robustness to heavy-tailed data, since regret is sub-linear for any distribution with finite second moment. The benchmark is dimension free , since dimension terms d, p do not appear in Equation (1). The finite diameter D only affects the bound due to corruptions and not the other terms. In particular, if Λ T = 0 almost-surely, then the benchmark seeks a valid bound even if D = ∞ . In general when Λ T > 0 however, dependence on D cannot be avoided as we show in Proposition 2.6 below (proof in Section 16).

Proposition 2.6 ( Finite diameter is necessary even in the absence of drift ) . There exists a strongly convex loss function L such that for every D > 0 and d ∈ N , domain Θ = { x ∈ R d : ∥ x ∥ ≤ D} such that for every T ∈ N and every algorithm, there exists a probability measure P on R d from which the un-corrupted samples are drawn from (i.e., Φ T = 0 ) and a sequence of corruption functions ( C t ) T t =1 , such that the regret is at-least Ω(Λ T D ) with probability 1 .

Thus, even if there is no drift, finite diameter assumption cannot be avoided. Proof is given in Section 16, relies on the modeling asumption that the corruption locations can be arbitrary. This lower bound is in contrast to prior work [19] which shows that finite diameter is not necessary in the absence of corruptions and when the time instants of corruptions are random. Further in Table 1, we compare the best known lower and upper bounds for the various settings in the presence and absence of distribution shifts and corruptions. As can be seen, in all cases involving either distribution shift or corruptions, our work is the first to give high-probability estimation regret bounds.

Equation (1) gives drift and corruption tolerance as additional performance measures: Since the regret explicitly depends on Φ T and Λ T we can define the drift tolerance and corruption tolerance as performance measures of the algorithm. Drift (Corruption) tolerance is the largest Φ T (largest Λ T ) for which we still guarantee sub-linear in T regret. Thus, if Equation (1) is satisfied by an algorithm, then the drift and corruption tolerance is O ( T 1 -l ) and O ( T 1 -n ) respectively. Thus, higher these tolerances, the better an algorithm is as they indicate that the algorithm's degradation with drifts and corruptions are more graceful. Our method is the only one to have non-zero drift tolerance (Table 2).

3 The clipped SGD Algorithm

In this section, we formally describe the algorithm in Algorithm 1 that achieves the desiderata. Algorithm 1 produces an estimate at time t is given by θ t ←P Θ ( θ t -1 -η t clip ( ∇L ( X t , θ t -1 , λ )) , where P Θ is the projection operator on to Θ and clip ( x, c ) := min(1 , c/ ∥ x ∥ ) x, ∀ x ∈ R d , c ≥ 0 .

Algorithm 1 Clipped-SGD [55]

1: Input : ( η t ) t ≥ 1 , λ > 0 , θ 0 ∈ Θ , T time-horizon 2: for each time t = 1 , 2 , · · · , do 3: Receive sample X t 4: Output θ t ←P Θ ( θ t -1 -η t clip ( ∇L ( X t , θ t -1 ) , λ )) 5: end for

Intuition for why Algorithm 1 can achieve Equation (1) : If there is no distribution shift or corruptions, clipped SGD with appropriate learning rate converges to the true parameter [55]. Now, if there are corruptions, clipping limits the impact on the overall regret. On the other hand, when a distribution shift occurs, the dynamics of Algorithm 1 is equivalent to restarting estimation under the new distribution, which will converge to the true parameter of this shifted distribution [55].

4 Regret bounds when the gradient vectors have sub-gaussian tails

Before giving the general result, we consider the special case of sub-gaussian distributions in this section. We do so because (i) the additional structure of sub-gaussian distributions enable us to prove stronger results in Corollaries 4.6 and 4.7 in the sequel, and (ii) the proof techniques from this special case enables us to build towards the proof of the general setting.

Definition 4.1 (Sub-gaussian random vectors, [57]) . A random vector Y ∈ R d with co-variance matrix Σ is said to be sub-gaussian, if for all λ > 0 and u ∈ R d with ∥ u ∥ = 1 , E [ e λ ⟨ Y,u ⟩ ] ≤ e λ 2 νmax (Σ) 2 , where ν max (Σ) is the highest eigen-value of Σ .

Assumption 4.2 (Sub-gaussian assumption with known upper bound) . For every t , and θ ∈ Θ , the 0 mean random vector ∇L ( Z, θ ) -∇R t ( θ ) , where Z ∼ P t is sub-gaussian with covariance matrix upper-bounded in the positive semi-definite sense by an known positive-semi-definite matrix Σ .

The following is the main result of this section.

Theorem 4.3 (Informal version of Theorem 18.1) . Suppose Assumption 4.2 holds. For every δ ∈ (0 , 1) , if Algorithm 1 is run with constant step-size η = 1 mT α for α ∈ [0 , 1] and clipping value λ ≥ G + σ + O (√ ν max (Σ) ln ( T δ ) ) , then with probability at-least 1 -δ ,

The main achievement in Theorem 4.3 is in explicitly identifying how the choice of α trade-offs the regret contributions from distribution shift, finite sample error and corruptions.

Remark 4.4 ( Corruption and drift tolerance ) . When Theorem 4.3 is instantiated with α ∈ (0 , 1) , Equation (2) yields a drift tolerance of O ( T 1 -α ) and corruption tolerance of O ( T 1 -α 2 ) . In particular, instantiating with α = 2 / 3 yields O ( T 1 / 3 ) and O ( T 2 / 3 ) corruption and drift tolerance respectively.

We now read off several corollaries from this result.

Corollary 4.5 ( Finite diameter assumption can be relaxed in the absence of corruptions ) . Let Λ T = 0 almost-surely, i.e., there are no adversarial corruptions. Then, under the settings of Theorem 4.3 even when the set Θ is unbounded, the regret bound given in Equation (2) holds.

Corollary 4.6 ( Optimal in the stationary case: ) . Under the assumptions of Theorem 4.3, if Φ T = Λ T = 0 , then when Algorithm 1 is run with parameters λ = + ∞ and η = 1 /mT , the regret bound Reg T ≤ O ( √ T m ( √ ν max (Σ) ln( T/δ ) + σ )) holds with probability at-least 1 -δ .

This corollary recovers the well known result [24] of convergence of SGD on strongly convex functions with sub-gaussian gradients.

Corollary 4.7 ( Optimal in the noiseless setting ) . Under the assumptions of Theorem 4.3, if Λ T = 0 , and σ = 0 , i.e., there is no noise, then running Algorithm 1 with λ = + ∞ and η = 1 /m obtains regret bound of Reg T = O (Φ T ) .

This result matches the lower bound in the noise-less setting [67] and recovers the upper-bound by previous analysis of online clipped-SGD specialized to the noiseless setting [43].

Corollary 4.8 ( Special case of α = 2 / 3 ) . When Theorem 4.3 is instantiated with α = 2 / 3 , then w.p. at-least 1 -δ , a regret bound of Reg T ≤ O (( T 1 / 3 Φ T + T 2 / 3 m + T 1 / 3 Λ T ) poly ( σ, G, ln( T/δ ))) .

Remark 4.9 ( The excess risk can be bounded using smoothness ) . Oftentimes, the regret is also measured through the excess risk , i.e., ∑ T t =1 E Z ∼ P t [ L ( Z, θ t ) -L ( Z, θ ∗ t )] . Since we assume that L is M smooth (Assumption 2.1), Theorem 4.3 also bounds the excess risk regret by observing that E Z ∼ P t [ L ( Z, θ t ) -L ( Z, θ ∗ t )] ≤ M 2 ∥ θ t -θ ∗ t ∥ 2 .

4.1 Proof sketch for Theorem 4.3 and technical innovations

The full proof is Appendix 18. We first establish due to the condition on λ that if an input sample is not corrupted, then the gradient will never be clipped. Then in Lemma 19.6, we expand the one-step

recursion of the clipped SGD updates by exploiting the strong-convexity. In order to account for the effect of corruptions, we expand the recursion differently if a time-step is corrupted or not. If a time-step is corrupted, then we accumulate both a bias term of norm at-most λ and an appropriate variance term. If one were only interested in bounds in expectation , then the proof will follow the standard clipped SGD analysis as the expectation of the variance terms are 0 .

To prove a high probability bound, a natural approach is to apply martingale concentrations to bound the variance terms as done in [24] in the absence of drifts and corruptions . A naive way to apply martingale concentrations would be to bound the variance error terms due to drifts and corruption by the worst case error by using the finite diameter D . However, this will not lead to the dimension free bound of Theorem 4.3 -instead will lead to a bound in where the finite diameter D will also multiply the regret term due to finite sample error. In particular, this approach will not result in Corollary 4.5 of being able to relax the finite diameter assumption in the absence of corruptions.

We circumvent this issue by using an inductive argument to bound the variance terms in Lemma 19.9. Concretely, we prove the induction hypothesis that for each time t , error terms until time t is bounded by an appropriate function of the drifts and corruptions. To establish the induction for a time t , we condition on the event the hypothesis holds till time t -1 , and employ martingale concentration to show the error at time t is small. Then we plug the martingale bound back into the one-step recursion and show the induction hypothesis also holds at time t . To get the final un-conditional result, we un-roll all the conditional events by an union bound.

5 Regret bounds in the general heavy-tailed case

The following is the main result of the paper, where we relax Assumption 4.2 and thus making the algorithm free of distributional knowledge and allowing for potentially heavy-tailed data.

Theorem 5.1 (Informal version of Theorem 20.1) . When Algorithm 1 is run with clip value λ = 2 GT α 3 , and step-size η = 1 mT α for α ∈ [0 , 1] , then for every δ ∈ (0 , 1) ,

holds with probability at-least 1 - δ .

The achievement in Theorem 5.1 is in explicitly identifying how the choice of α trade-offs the regret contributions from distribution shift, finite sample error and corruptions, despite heavy-tailed data.

Remark 5.2 ( Price for relaxing assumption 4.2 is sub-optimal bound in the stationary case: ) . The setting in Theorem 5.1 is weaker compared to that of Theorem 4.3 since (i) no sub-gaussian tail assumptions are made, and (ii) no knowledge of the upper-bound matrix Σ is assumed. The price for relaxing these assumptions is a weaker regret, specifically in the term due to finite-sample estimation error. This term scales as O ( T 1 -α 3 ) in Theorem 5.1 while it scales as O ( T 1 -α 2 ) in Theorem 4.3. In particular in the stationary case, when Equation (3) is instantiated with α = 1 , the regret bound reads as Reg T = O ( T 2 / 3 σ 2 ) which is weaker than the optimal O ( √ Tσ ) for the stationary case [55, 39].

Corollary 5.3 ( Setting α = 1 / 2 ) . Under the conditions of Theorem 5.1, if Algorithm 1 is run with α = 1 / 2 , then with probability at-least 1 -δ , it satisfies a regret bound of Reg T ≤ O (( T 2 / 3 Φ T + T 11 / 12 √ Φ T + T 5 / 6 + T 3 / 4 Λ T ) poly ( σ, G, 1 /m, ln( T/δ ))) .

Corollary 5.4 ( Finite diameter assumption can be relaxed in the absence of corruptions ) . Let Λ T = 0 almost-surely, i.e., there are no adversarial corruptions. Then, under the settings of Theorem 5.1 even when the set Θ is unbounded, the regret bound given in Equation (3) holds.

Remark 5.5 ( Corruption and drift tolerance ) . Theorem 5.1 when instantiated with α ∈ (0 , 1) yields a drift tolerance of O ( T 1 -4 α 3 ) and corruption tolerance of O ( T 1 -3 α 2 ) . In particular, instantiating with α = 1 / 2 yields O ( T 1 / 3 ) and O ( T 1 / 4 ) corruption and drift tolerance respectively.

(a)

(b)

(c)

(d)

=O（T0.5),A=0

=0,A =O(70.5)

0=v'0=Φ

10.0

入= 0.5

入 =1

入 = 0.25

10.0

9.5

入 =1

入 = 0.25

入 = 0.5

10.0

9.5

A =1

入 = 0.25

入 = 0.5

10.0

9.5

入=0.5

入 = 1

= 0.25

log(1+Regret)

9.5

入=2

入=3

log(1+Regret)

入 = 2

入=3

log(1+Regret)

9.0

入= 2

入=3

log(1+Regret)

9.0

A=2

入=3

9.0

入 = inf

9.0

入=inf

8.5

入=inf

8.5

入 = inf

8.5

8.5

8.0

8.0

7.5

7.5

8.0

8.0

7.0

7.0

7.5

7.5

6.5

6.5

0.4

0.6

0.8

1.0

0.4

0.6

0.8

1.0

0.4

0.6

0.8

1.0

0.4

0.6

0.8

1.0

α

α

D

α

5.1 Proof sketch of Theorem 5.1 and technical innovations

The proof for this case builds upon the techniques used to prove Theorem 4.3. Unlike in the subgaussian case, we cannot guarantee that only corrupted samples will be clipped in the general case. This introduces an additional bias and variance terms due to clipping of potential un-corrupted samples or inliers . The bias terms of the inliers are handled using techniques from prior works [25, 55]. We bound the variance terms using an inductive argument similar in spirit to that of Theorem 4.3. However, identifying the correct hypothesis involving drifts and outliers so that the induction argument holds is challenging. The challenge is because the induction hypothesis assumed till time t -1 impacts the martingale error term, which in-turn impacts whether the induction hypothesis will hold at time t . We introduce a free parameter to the regret term and deduce the induction hypothesis to hold if a quadratic equation in this parameter does not have any real roots. This extra term contributes to regret degradation compared to the sub-gaussian case (Lemma 20.10).

6 Insights and remarks

Known Time Horizon: This is a benign assumption and can be overcome by the standard doubling trick (Section 2 . 3 of [11]) with additional log 2 ( T ) factor regret (c.f. Appendix 14).

Price for being adaptive to distribution shift: Theorems 4.3 and 5.1 show that, in order to minimize regret due to statistical estimation, we need to set α = 1 , i.e., choose a learning rate of O (1 /T ) . This was shown to be optimal in in the absence of drift and corruptions both under sub-gaussian assumptions [24] and in heavy tail settings [55]. On the other hand, we see from Theorems 4.3 and 5.1, that in the presence of drift Φ T > 0 , a learning rate α < 1 is sufficient to ensure sub-linear regret. The following lower bound (proved in Appendix 15) shows that α < 1 is also necessary.

Proposition 6.1 ( Lower bound showing O (1 /T ) learning rate is not adaptive to drifts ) . There exists a loss function L such that for every T , there exists a sequence of probability measures ( P t ) t ≥ with the diameter D ≤ 2 log( T ) , distribution shift Φ T ≤ 2 log( T ) , Λ T = 0 , such that Algorithm 1 when run with λ ≥ 1 and step size η t := 1 t +1 will incur regret at-least T 3 with probability 1 .

Finite diameter D is necessary in general: We already saw in Proposition 2.6 that Ω(Λ T D ) regret is necessary. We prove the lower-bound by considering mean-estimation in Section 16. Variants of Proposition 2.6 was observed in the literature (for ex. Proposition 1 . 3 [13], Lemma 6 . 1 in [33], Theorem D. 1 in [14], Line 8 of Algorithm 3 in [19]). In theoretical statistics literature, Ω(Λ T D ) is considered un-desirable [13] and thus the models studied restrict corruptions to occur at random times [13, 19] rather than arbitrary as in our setup. However, in practice corruptions are rarely random and typically occur close in time for instance due to machine failures or other external confounding factors [23, 42].

Regret bounds are dimension-free: The problem dimensions d, p do not appear in the regret bound. Moreover, the finite diameter D only appears in the regret term affecting through adversarial corruptions and not in the distribution shift and finite sample error terms. Further, Corollaries 4.5 and 5.4 give regret bounds that hold even when D = ∞ in the absence of corruptions.

7 Experiments

Theorems 4.3 and 5.1 give upper bounds on regret showing that tuning the learning rate depending on the amount of distribution shift and number of corruptions in the stream is beneficial. We empirically observe similar phenomena in Figures 2 and 3 (in the Appendix), thus indicating our theoretical observations are fundamental and not artifact of our bounds. We consider L ( X,θ ) = 1 2 ∥ X -θ ∥ 2 corresponding to mean-estimation in Figure 2 and linear-regression of L ( X,θ ) = 1 2 ∥ X (2) -θ T X (1) ∥ 2 , where X = ( X (1) , X (2) ) with X (1) ∈ R d -1 , X (2) ∈ R , θ ∈ R ( d -1) in Figure 3 in the Appendix in Section 12. We compare clipped-SGD with learning rate 1 /mT α for variety of α with clipping λ = 2 T 0 . 25 and use the doubling trick to remove dependence of T (i.e., use Algorithm 2). For the case of α = 1 , we use the learning rate of η t = 1 / ( m ( t +1)) and λ = 2 √ T as suggested in [55]. All plots are averaged over 30 runs and we plot the median along with the 5 th and 95 th quantile bands. We observe in Figures 2 and 3 that as Φ T or Λ T increases, the optimal α to use decreases, validating the theoretical insight. Further details in the Appendix in Section 12. Evaluations on real-data is conducted in the Appendix in Section 13.

8 Related Work

FITNESS for mean estimation is proposed in [50] that requires variance as input and is adaptive to drifts and corruptions under sub-gaussian distributions with per-sample computational complexity is O ( dT ) . In contrast, our algorithm applies to any strongly convex function, does not require moment bounds, data can be heavy-tailed. The work of [8] studied regret bounds in the absence of corruptions and only give bounds in expectation. A sequence of papers in online estimation including [44, 46, 13, 25, 55, 50] have derived algorithms that are robust in different ways. However, none of them consider the impact of distribution shift . The works of [46, 13] study robust linear regression in the absence of drifts with [46] making Gaussian assumptions on both covariates and response while [13] makes Gaussian assumption only on the responses. The work of [55] show that clipped SGD attains near optimal rates for any strongly convex loss function in the absence of drifts and corruptions . The paper of [19] study online estimation of strongly convex loss functions with corruptions, but do not consider drift. Moreover, their regret bounds are not dimension free (cf. Table 2). Although the paper of [64] gives regret bounds for online learning, a more general setting compared to estimation, do not consider impact of drifts. High probability bounds for optimization with heavy-tailed data have been studied by [44, 25, 15, 60, 36] in recent times. However, none of their analysis considers the impact of drift and corruptions . More related work in Section 10.

9 Conclusions and Open Problems

We studied online estimation in the presence of drifts and corruptions and potentially heavy-tailed inlier data and proved regret bounds for clipped SGD for strongly convex loss functions. Ours is the first proof that an online algorithm can simultaneously be robust to heavy-tails and corruptions and adaptive to drift. A key result was to show how the choice of learning-rate trades off drift, finite sample error and corruptions. Our work leaves several interesting open problems.

The optimal choice of α in Theorems 4.3 and 4.3 is a trade-off between distribution shift, finite sample error and corruptions. Can the optimal α be set without knowing Φ T , Λ T or σ in advance?

In the absence of corruptions, can regret O (( T + T a Φ 1 -a T ) poly ( G,σ, log( T/δ )) , for some a < 1 be achieved? Such an algorithm would simultaneously have both (i) O ( T ) distribution shift tolerance which is the best possible and matching the lower bound established in [50], and (ii) closing the gap of Remark 5.2 for the stationary case.

√

Can m = 1 / 2 in Equation (1) be achieved in the general case with drift and corruptions? Theorem 5.1 shows that only m ≥ 2 / 3 is achievable. For the special case of stationary stream, [55] shows m = 1 / 2 is achievable, matching the lower bound that m ≥ 1 / 2 is necessary.

(Open problem from [55]) Does there exists an online algorithm that can obtain the statistically optimal regret of O ( √ T ( √ Tr (Σ) + √ ν max (Σ) log( T/δ ))) for a stationary stream?

References

Giuseppe Aceto, Alessio Botta, Walter De Donato, and Antonio Pescap` e. Cloud monitoring: A survey. Computer Networks , 57(9):2093-2115, 2013.

Ashkan Aghdai, Fan Zhang, Nadun Dasanayake, Kang Xi, and H Jonathan Chao. Traffic measurement and analysis in an organic enterprise data center. In 2013 IEEE 14th International Conference on High Performance Switching and Routing (HPSR) , pages 49-55. IEEE, 2013.

Mohammad Alizadeh, Shuang Yang, Milad Sharif, Sachin Katti, Nick McKeown, Balaji Prabhakar, and Scott Shenker. pfabric: Minimal near-optimal datacenter transport. ACM SIGCOMM Computer Communication Review , 43(4):435-446, 2013.

Arthur Asuncion and David Newman. Uci machine learning repository, 2007.

Dheeraj Baby and Yu-Xiang Wang. Optimal dynamic regret in exp-concave online learning. In Conference on Learning Theory , pages 359-409. PMLR, 2021.

Jarrod Bakker, Bryan Ng, Winston KG Seah, and Adrian Pekar. Traffic classification with machine learning in a live network. In 2019 IFIP/IEEE Symposium on Integrated Network and Service Management (IM) , pages 488-493. IEEE, 2019.

Richard G Baraniuk. More is less: Signal processing and the data deluge. Science , 331(6018):717-719, 2011.

Omar Besbes, Yonatan Gur, and Assaf Zeevi. Non-stationary stochastic optimization. Operations research , 63(5):1227-1244, 2015.

S´ ebastien Bubeck. Convex optimization: Algorithms and complexity. Foundations and Trends® in Machine Learning , 8(3-4):231-357, 2015.

Olivier Catoni. Challenging the empirical mean and empirical variance: a deviation study. In Annales de l'IHP Probabilit´ es et statistiques , volume 48, pages 1148-1185, 2012.

Nicolo Cesa-Bianchi and G´ abor Lugosi. Prediction, learning, and games . Cambridge university press, 2006.

Varun Chandola, Arindam Banerjee, and Vipin Kumar. Anomaly detection: A survey. ACM computing surveys (CSUR) , 41(3):1-58, 2009.

Sitan Chen, Frederic Koehler, Ankur Moitra, and Morris Yau. Online and distribution-free robustness: Regression and contextual bandits with huber contamination. In 2021 IEEE 62nd Annual Symposium on Foundations of Computer Science (FOCS) , pages 684-695. IEEE, 2022.

Yeshwanth Cherapanamjeri, Efe Aras, Nilesh Tripuraneni, Michael I Jordan, Nicolas Flammarion, and Peter L Bartlett. Optimal robust linear regression in nearly linear time. arXiv preprint arXiv:2007.08137 , 2020.

Ashok Cutkosky and Harsh Mehta. High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems , 34:4883-4895, 2021.

Guido Dartmann, Houbing Song, and Anke Schmeink. Big data analytics for cyber-physical systems: machine learning for the internet of things . Elsevier, 2019.

Ibrahim Delibalta, Kaan Gokcesu, Mustafa Simsek, Lemi Baruh, and Suleyman S Kozat. Online anomaly detection with nested trees. IEEE Signal Processing Letters , 23(12):1867-1871, 2016.

Ilias Diakonikolas and Daniel M Kane. Recent advances in algorithmic high-dimensional robust statistics. arXiv preprint arXiv:1911.05911 , 2019.

Ilias Diakonikolas, Daniel M. Kane, Ankit Pensia, and Thanasis Pittas. Streaming algorithms for high-dimensional robust statistics. In Proceedings of the 39th International Conference on Machine Learning , volume 162 of Proceedings of Machine Learning Research , pages 5061-5117. PMLR, 17-23 Jul 2022.

Jiashi Feng, Huan Xu, and Shie Mannor. Outlier robust online learning. arXiv preprint arXiv:1701.00251 , 2017.

Massimo Gallo, Alessandro Finamore, Gwendal Simon, and Dario Rossi. Real-time deep learning based traffic analytics. In Proceedings of the SIGCOMM'20 Poster and Demo Sessions , pages 76-78. 2020.

Jo˜ ao Gama, Indr˙ e ˇ Zliobait˙ e, Albert Bifet, Mykola Pechenizkiy, and Abdelhamid Bouchachia. A survey on concept drift adaptation. ACM computing surveys (CSUR) , 46(4):1-37, 2014.

Peng Gao, Xusheng Xiao, Zhichun Li, Fengyuan Xu, Sanjeev R Kulkarni, and Prateek Mittal. { AIQL } : Enabling efficient attack investigation from system monitoring data. In 2018 { USENIX } Annual Technical Conference ( { USENIX }{ ATC } 18) , pages 113-126, 2018.

Saeed Ghadimi and Guanghui Lan. Optimal stochastic approximation algorithms for strongly convex stochastic composite optimization, ii: shrinking procedures and optimal algorithms. SIAM Journal on Optimization , 23(4):2061-2089, 2013.

Eduard Gorbunov, Marina Danilova, and Alexander Gasnikov. Stochastic optimization with heavy-tailed noise via accelerated gradient clipping. Advances in Neural Information Processing Systems , 33:15042-15053, 2020.

M Harries. Splice-2 comparative evaluation: electricity pricing (technical report unsw-csetr-9905). Artificial Intelligence Group, School of Computer Science and Engineering, The University of New South Wales, Sydney , 2052, 1999.

Elad Hazan. Introduction to online convex optimization. Foundations and Trends® in Optimization , 2(3-4):157-325, 2016.

Cormac Herley. Automated detection of automated traffic. In 31st USENIX Security Symposium (USENIX Security 22) , pages 1615-1632, 2022.

T Ryan Hoens, Robi Polikar, and Nitesh V Chawla. Learning from streaming data with concept drift and imbalance: an overview. Progress in Artificial Intelligence , 1:89-101, 2012.

Hao Huang and Shiva Prasad Kasiviswanathan. Streaming anomaly detection using randomized matrix sketching. Proceedings of the VLDB Endowment , 9(3):192-203, 2015.

Wazir Zada Khan, MH Rehman, Hussein Mohammed Zangoti, Muhammad Khalil Afzal, Nasrullah Armi, and Khaled Salah. Industrial internet of things: Recent advances, enabling technologies and open challenges. Computers & Electrical Engineering , 81:106522, 2020.

Young-geun Kim, Yongchan Kwon, Hyunwoong Chang, and Myunghee Cho Paik. Lipschitz continuous autoencoders in application to anomaly detection. In International Conference on Artificial Intelligence and Statistics , pages 2507-2517. PMLR, 2020.

Adam Klivans, Pravesh K Kothari, and Raghu Meka. Efficient algorithms for outlier-robust regression. In Conference On Learning Theory , pages 1420-1430. PMLR, 2018.

Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE , 86(11):2278-2324, 1998.

Jia Li, Shimin Di, Yanyan Shen, and Lei Chen. Fluxev: a fast and effective unsupervised framework for time-series anomaly detection. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining , pages 824-832, 2021.

Zijian Liu and Zhengyuan Zhou. Stochastic nonsmooth convex optimization with heavy-tailed noises. arXiv preprint arXiv:2303.12277 , 2023.

G´ abor Lugosi and Shahar Mendelson. Mean estimation and regression under heavy-tailed distributions: A survey. Foundations of Computational Mathematics , 19(5):1145-1190, 2019.

G´ abor Lugosi and Shahar Mendelson. Sub-gaussian estimators of the mean of a random vector. 2019.

Gabor Lugosi and Shahar Mendelson. Robust multivariate mean estimation: the optimality of trimmed mean. The Annals of Statistics , 49(1):393-410, 2021.

Chaitanya Manapragada, Mahsa Salehi, and Geoffrey I Webb. Extremely fast hoeffding adaptive tree. In 2022 IEEE International Conference on Data Mining (ICDM) , pages 319-328. IEEE, 2022.

MHammad Mazhar and Zubair Shafiq. Real-time video quality of experience monitoring for https and quic. In IEEE INFOCOM 2018-IEEE Conference on Computer Communications , pages 1331-1339. IEEE, 2018.

Yisroel Mirsky, Tomer Doitshman, Yuval Elovici, and Asaf Shabtai. Kitsune: an ensemble of autoencoders for online network intrusion detection. arXiv preprint arXiv:1802.09089 , 2018.

Aryan Mokhtari, Shahin Shahrampour, Ali Jadbabaie, and Alejandro Ribeiro. Online optimization in dynamic environments: Improved regret rates for strongly convex problems. In 2016 IEEE 55th Conference on Decision and Control (CDC) , pages 7195-7201. IEEE, 2016.

Alexander V Nazin, Arkadi S Nemirovsky, Alexandre B Tsybakov, and Anatoli B Juditsky. Algorithms of robust stochastic optimization based on mirror descent method. Automation and Remote Control , 80(9):1607-1627, 2019.

Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. Understanding the exploding gradient problem. CoRR, abs/1211.5063 , 2(417):1, 2012.

Scott Pesme and Nicolas Flammarion. Online robust regression via sgd on the l1 loss. Advances in Neural Information Processing Systems , 33:2540-2552, 2020.

Maxim Raginsky, Rebecca M Willett, Corinne Horn, Jorge Silva, and Roummel F Marcia. Sequential anomaly detection in the presence of noise and limited feedback. IEEE Transactions on Information Theory , 58(8):5544-5562, 2012.

Hansheng Ren, Bixiong Xu, Yujing Wang, Chao Yi, Congrui Huang, Xiaoyu Kou, Tony Xing, Mao Yang, Jie Tong, and Qi Zhang. Time-series anomaly detection service at microsoft. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining , pages 3009-3017, 2019.

Byron P Roe, Hai-Jun Yang, Ji Zhu, Yong Liu, Ion Stancu, and Gordon McGregor. Boosted decision trees as an alternative to artificial neural networks for particle identification. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment , 543(2-3):577-584, 2005.

Abishek Sankararaman, Balakrishnan Narayanaswamy, Vikramank Y Singh, and Zhao Song. Fitness:(fine tune on new and similar samples) to detect anomalies in streams with drift and outliers. In International Conference on Machine Learning , pages 19153-19177. PMLR, 2022.

Bernhard Sch¨ olkopf, John C Platt, John Shawe-Taylor, Alex J Smola, and Robert C Williamson. Estimating the support of a high-dimensional distribution. Neural computation , 13(7):14431471, 2001.

Vikash Sehwag, Mung Chiang, and Prateek Mittal. Ssd: A unified framework for self-supervised outlier detection. arXiv preprint arXiv:2103.12051 , 2021.

Ohad Shamir. A variant of azuma's inequality for martingales with subgaussian tails. arXiv preprint arXiv:1110.2392 , 2011.

Tushar Swamy, Alexander Rucker, Muhammad Shahbaz, Ishan Gaur, and Kunle Olukotun. Taurus: a data plane architecture for per-packet ml. In Proceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems , pages 1099-1114, 2022.

Che-Ping Tsai, Adarsh Prasad, Sivaraman Balakrishnan, and Pradeep Ravikumar. Heavy-tailed streaming statistical estimation. In International Conference on Artificial Intelligence and Statistics , pages 1251-1282. PMLR, 2022.

Tim van Erven, Sarah Sachs, Wouter M Koolen, and Wojciech Kotlowski. Robust online convex optimization in the presence of outliers. In Conference on Learning Theory , pages 4174-4194. PMLR, 2021.

Roman Vershynin. High-dimensional probability: An introduction with applications in data science , volume 47. Cambridge university press, 2018.

Jonathan Vestin, Andreas Kassler, Deval Bhamare, Karl-Johan Grinnemo, Jan-Olof Andersson, and Gergely Pongracz. Programmable event detection for in-band network telemetry. In 2019 IEEE 8th international conference on cloud networking (CloudNet) , pages 1-6. IEEE, 2019.

H Victor. A general class of exponential inequalities for martingales and ratios. The Annals of Probability , 27(1):537-564, 1999.

Nuri Mert Vural, Lu Yu, Krishna Balasubramanian, Stanislav Volgushev, and Murat A Erdogdu. Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In Conference on Learning Theory , pages 65-102. PMLR, 2022.

Liangcheng Yu, John Sonchack, and Vincent Liu. Mantis: Reactive programmable switches. In Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication , pages 296-309, 2020.

Jingzhao Zhang, Tianxing He, Suvrit Sra, and Ali Jadbabaie. Why gradient clipping accelerates training: A theoretical justification for adaptivity. arXiv preprint arXiv:1905.11881 , 2019.

Jingzhao Zhang, Sai Praneeth Karimireddy, Andreas Veit, Seungyeon Kim, Sashank Reddi, Sanjiv Kumar, and Suvrit Sra. Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems , 33:15383-15393, 2020.

Jiujia Zhang and Ashok Cutkosky. Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 , 2022.

Lijun Zhang, Shiyin Lu, and Zhi-Hua Zhou. Adaptive online learning in dynamic environments. Advances in neural information processing systems , 31, 2018.

Chong Zhou and Randy C Paffenroth. Anomaly detection with robust deep autoencoders. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining , pages 665-674, 2017.

Martin Zinkevich. Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the 20th international conference on machine learning (icml-03) , pages 928-936, 2003.

10 More related work

Comparison with [8]: The seminal work of [8] started the work on non-stationary estimation in the absence of corruptions. They establish a regret lower bound of Ω(Φ 1 / 3 T 2 / 3 ) for non-stationary estimation in the presence of non-stationarities. They also established an upper bound by showing that SGD without clipping yields matching regret upper bounds in expectation . The upper bound in Theorem 5.1 on the other hand gives high-probability regret bounds that hold even in the presence of corruptions, in addition to non-stationarities. Expressing an upper bound in high-probability is both technically challenging and algorithmically insightful. The insight we make is that for having high-probability bounds, we need to have clipped SGD . On the other hand, since [8] only give bounds holding in expectation, they do not need any clipping in their algorithms. The necessity of clipping in heavy-tailed settings is not an artifact of analysis, but is crucial for good empirical performance, as noted in recent works of [55]. Thus a conceptual contribution we make is that even in the absence of corruptions, a different algorithm compared to [8], namely that of clipping gradients is required to get regret bounds holding in high-probability. From a technical perspective, the proofs for high-probability bounds need different techniques as compared to [8]. For instance, we need several martingale and induction arguments to arrive at high-probability bounds in heavy tails while [8] have a much more simpler proofs just based on convexity, since they only give bounds in expectation.

Robust Stochastic Optimization The paper of [44] proves high-probability bounds for optimizing convex functions under heavy tailed gradients. However, they do not consider corruptions or drifts and assume finite diameter. In contrast our bounds explicitly depend on drifts and corruptions and can handle infinite diameter in the absence of corruptions. The work of [25] give high-probability bounds for offline clipped gradient descent based algorithms for optimizing both convex and strongly convex functions. However, they do not consider drift or corruptions. The work of [60] studies convergence of gradient descent methods in the absence of second moments, but only give bounds in expectation and do not consider drift or corruptions. The paper of [15] give high-probability bounds for stochastic optimization, but their analysis does not consider drift or corruptions. None of these papers simultaneously consider drifts, corruptions and heavy-tailed noise . The paper of [63] extend this to heavy-tailed noise under only the p ∈ (1 , 2] th moment and non-convex functions. However, they only give results in expectation and in a setting without drift and corruptions .

Robust Statistics There is a growing literature [18, 37, 39, 10] that give near-optimal offline algorithms in the presence of corruptions but no drifts. Since our focus is to devise online algorithms that are adaptive to distribution shift, these algorithms are not directly applicable to our setting.

Dynamic Regret Minimization A parallel line of work concerns algorithms being adaptive to drifts for online convex optimization problems [27] where the data are arbitrary rather than sampled from a distribution [67, 65, 56, 5]. However, these algorithms assume that there are no noise in the gradients and no corruptions on the data stream.

Gradient clipping The work of [45] showed clipping gradients as a practical work-around to the 'exploding gradients' problem. Theoretically, [62] study the effect of clipping in offline training under restrictive noise assumptions and do not consider impact of drifts and corruptions.

11 Key algorithmic challenge

All distribution changes initially look like corruptions

̸

In this section, we give a simple example to see whay developing algorithms that are robust to drift and corruptions are challenging. Consider a simple example of estimating the mean in the absence of noise, i.e., when variance is 0 . We will show two possible underlying scenarios that yield the same observation to the forecaster. In both scenarios, the observed samples are all 0 in the first T -Λ T rounds and is R = 0 in all of the last Λ T rounds. The first scenario is one where the true mean in the first T -Λ T samples is 0 and the true mean in the last Λ T samples is R , i.e., this scenario has no adversarial corruptions and a cumulative distribution shift of R . In the second scenario, the true mean for all rounds is 0 , except the last Λ T samples are adversarially corrupted, i.e., this scenario has no distribution shift but has Λ T adversarially corrupted samples. If the true system is scenario 1 , then the regret of a forecaster is the distance from the ground-truth R , while in the second scenario is the distance from the un-corrupted mean 0 . Thus, any algorithm will incur in at-least one of the scenarios, a regret of at-least R Λ T 2 . This example highlights the main tension in the problem : a

=0,=O(T0.5)

(a)

(b)

(c)

(d)

(s1)=v(so⊥)=Φ

0=1v(so1)0=Φ

=0,A=0

11.5

11.0

A = 1

A=8

入 = 64

入 = 100.0

9.80

9.75

入 = 0.25

A = 0.5

入=1

入 = 2

11

A = 1

入 =8

入 = 64

入=100.0

11.5

A = 1

入 = 8

入 = 64

入=100.0

log(1+Regret)

10.5

Regret)

9.70

入=3

log(1+Regret)

log(1+Regret)

11.0

10

10.0

log(1+Re

9.65

9.5

9.60

10.5

6

9.0

9.55

10.0

8.5

9.50

8

8.0

9.5

0.4

0.6

8'0

1.0

0.4

0.6

0.8

1.0

0.4

0.6

0.8

1.0

0.4

0.6

0.8

1.0

α

α

α

α

distribution shift initially presents as outliers and the online algorithm needs samples and thus incurs regret in order to learn this to be the 'new normal'.

12 Additional details on simulations

We give more details on experiments reported in Figures 2 and 3 for synthetic data.

Mean-Estimation Setup At each time t , we sample a 0 mean univariate random variable Y t from the Pareto distribution with mean 0 , variance σ 2 := 5 and tail parameter 3 , so that it has a finite second moment. The un-corrupted d -dimensional vector at time t is denoted by Z t := ( Y t -µ t ) √ d v t , where v t is a unit vector sampled uniformly on the sphere of d = 256 dimensions. The true means µ t are chosen by running a random walk, i.e., µ 0 = 0 and µ t := µ t -1 + w t , where w t ∼ N ( 0 , 1 T 1 -κ I d ) . This ensures that Φ T ≈ T κ . In Figures 2a, 2b, 2c, 2d , we set κ ∈ { 0 . 5 , 0 . 25 , 0 . 15 , 0 } respectively. The instants of corruption are chosen at integer multiples of ⌊ √ T ⌋ and the observed sample X t = 0 , if t = k ⌊ √ T ⌋ , for some non-negative integer k . Otherwise, X t = Z t . For a given underlying mean vector, we run the streaming setup 30 times, and plot the average along with 95% confidence interval on the regret estimates.

Linear Regression Setup: We generate the covariate X t ∈ R 256 from a generalized Pareto distribution with mean 0 , variance σ 2 := 1 , i.e. m = 1 . The true parameter at time 0 was set to θ 0 := [ d -0 . 5 , · · · , d -0 . 5 ] . At each time t , the true parameter θ ∗ t := µ t -1 + w t where w t ∼ N ( 0 , 1 T 1 -κ I d ) . This ensures that Φ T ≈ T κ . In Figures 3a, 3b, 3c, 3d , we set κ ∈ { 0 . 5 , 0 . 25 , 0 . 15 , 0 } respectively.At each time t , the response y t = ⟨ X t , θ ∗ t ⟩ + n t , where n t is a Pareto distribution with 0 mean, variance σ 2 = 3 and shape parameter 2 . 5 . The instants of corruption are chosen at integer multiples of ⌊ √ T ⌋ . At instants of corruption, the covariates are sampled from a normal distribution with identity covariance and mean 100 . The response variable for a corrupted time point is set to 0 . For a given set of underlying mean vector, we run the streaming setup 30 times, and plot the average along with 95% confidence interval on the regret estimates.

Methods: We compare clipped-SGD with learning rates η t := 1 /mT α and clipping values λ = 2 T β for a variety of α and β . Further we implement the doubling trick given in Algorithm 2 so that clipped-SGD does not require knowledge of the time-horizon T . In addition, we also compare against the clipped-SGD version of [55] which uses the time-varying learning rate of 1 / ( t ) learning and clipping value of 2 √ T . Note that the algorithm of [55] has strictly more information than the one we implement as it has knowledge of time horizon T for setting the clipping value, while the one we propose does not and relies on the doubling trick. Further, we do not assume that the set Θ is known and consider the un-projected clipped-SGD for our algorithm, while for that of [55], we consider the smallest L 2 ball around the horizon that contains all the ( θ ∗ t ) T t =1 . Thus, the algorithm we implement is truly parameter-free as it does not have any problem specific parameters in its implementation.

Observations: All plots in Figure 2 show both mean-estimation and linear regression for varying amounts of distribution shift Φ T , while keeping the number of corruptions Λ T = √ T and clipping value λ = 2 T 0 . 25 . For both mean-estimation in Figures 2a, 2b, 2c, 2d and linear regression in Figures

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

=O(T0.5)

=O(T0.25)

Φ =O(70.15)

Φ=0

8

8

8

8

7

7

7

7

6

6

6

6

log(1+Regret)

5

log(1+Regret)

5

log(1+Regret)

log(1+Regret)

5

4

寸

4

4

m

nt=

m

nt=

3

nt=

3

+

nt=

2

+

nt=

0.75

1

2

十

nt=

0.75

1

2

nt=

T0.75

1

2

十

1

1

门t=

70.5

1

1

nt=

7a.5

1

1

nt =

T0.5

1

1

=U

0

nt =

T025

1

0

=

7025

1

0

+

nt

=

70.25

1

0

nt

=

7025

1

0

2000

4000

Time

6000

8000

10000

0

2000

4000

Time

6000

8000

10000

0

2000

4000

Time

6000

8000

10000

0

2000

4000

Time

6000

8000

10000

=0.5

Φ=0.25

Φ=O（70.15)

Φ=0

14

12

12

14

12

10

10

12

log(1+Regret)

8

log(1+Regret)

8

Regret

10

log(1+Regret)

10

Cumulative

8

6

6

4

n=

n=

70.75

1

4

nt=

1

6

4

s=

=

1

4

s=

n =

1

2

n=

TES

1

2

n=

T0.5

1

2

2

nt=

1

0

nt=

70.25

1

0

nt =

70.25

1

0

n=

1

0

nt=

7425

0

2000

4000

6000

0008

10000

0

2000

4000

6000

8000

10000

0

2000

4000

6000

8000

10000

0

2000

4000

6000

8000

10000

Time

Time

Time

Time

3a, 3b, 3c, 3d, we show that as the amount of distribution shift Φ T increases, the learning rate needs to be more aggressive to be adaptive to the distribution shift.

Sample paths In Figure 4, we plot a particular sample path over time for a variety of settings of α .

13 Real-data experiments

The key result of this section is that even on real-data with real performance metrics, adapting the learning-rate to the amount of distribution shift on a data-stream is beneficial. We consider two tasks to demonstrate this : online binary classification and online anomaly detection.

13.1 Classification setting

Datasets: We consider two binary classification datasets available from the UCI repository [4] (i) NSW Electricity data [26] and (ii) MiniBoone data [49]. The NSW Electricity dataset consists of 45312 samples with each data-point having 6 numeric features and one categorical feature representing the day of week. After one-hot-encoding the categorical feature, we have 13 numeric features and a binary target. The MiniBoone dataset has 130065 samples with each data point having 50 numeric features and a binary target. We choose these two as they have been shown to be good binary classification benchmark for data-streams with drift [40]. In particular, the MiniBoone dataset is extreme where the first 36499 has target of 1 while all the last 93565 data points have target of 0 . For both the datasets, the order of streaming is the same order in which they are collected. This is consistent with the evaluation protocol of streaming algorithms in the presence of drifts [40].

(a)

(b)

(c)

ElectricityDataset

MiniBooneDataset

MNIST

10.2

入=

11.4

入=∞0

0.22

入=5

10.1

11.2

0.20

log(1+Regret)

10.0

log(1+Regret)

R-AUC

11.0

9.9

R

P

0.18

9.8

10.8

0.16

9.7

10.6

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

0.14

0.4

0.6

0.8

1.0

α

α

α

Model and Methods: For both data-sets, we consider a simple one layer Logistic regression as the model to train. We set the gradient clipping value λ = 1 for both datasets. For both these datasets, we deviate from our setup and employ the classical online learning evaluation [40]. At each time t , the covariate is first shown to the algorithm which then predicts the target. The prediction of the logistic regression is then threshold at 0 . 5 . The instantaneous loss at time t is the indicator loss whether if predictor after applying the threshold matches the target. The covariate and label pair is then used to take one step of clipped SGD in order to get the model used at the next time step. We experiment with different learning rates and plot their impact in Figure 5.

Observations: We observe in Figure 5 that for the Electricity dataset which has more shifts and is noisy σ > 0 , the optimal choice of α ≈ 0 . 5 , while for the MiniBoone dataset has just one change point (i.e., Φ T ≈ 0 ) and is noiseless σ ≈ 0 (cf [40]). Thus the optimal α ≈ 0 .

13.2 Online Anomaly Detection

Dataset : We considered the standard MNIST dataset [34]. We modified it to a stream containing drift and corruptions/anomalies as follows. We introduce abrupt change points by first streaming in all 0 's followed by 1 's and so on with the last of the stream being 9 's. This way, the stream has 9 abrupt change-points. In addition, within a change-point, the images are slowly rotating clockwise at a fixed angle of 1 degree rotation to model slowly varying change in between two abrupt changes. Anomalies are introduced at random with probability 0 . 05 by sampling a digit other than the current segment of time. For instance 5c, the third point on the stream is an anomaly since the digit is different from 0 which is the inlier digit till the first break-point. Similarly, the last but one the data point on the stream is an anomaly since it is different from 9 , the inlier digit in that segment.

Methods : We flatten the image into a vector of 784 and consider a simple 2 layer auto-encoder with hidden dimension 512 and activation function of ReLU [66]. We initialize the network to be random at the beginning of the stream and consider clipped SGD with clip-value set to 5 for various learning rates as shown in Figure 5c. At each time t , the sample X t is revealed to the algorithm, a single clipped gradient step is taken on this sample to update the model and then the anomaly score is produced from the resulting model.

Observation : In Figure 5c, we plot the anomaly detection accuracy measured through the average precision recall score, where higher the score indicates better AD performance. We see that neither too small nor too high a value of α obtains the highest performance.

<latexi sh

1_b

64="

ZKp fXcYkgd2H

FOI

E

>A

B

3

V

9

wN

J

L

z

+R

o

W

G

Q

M

C

u

T

y

v

/

m

8

5

q

7

r

D

0

n

U

P

<latexi sh

1_b

64="fZkgI5B

m

H

O

+9

zDpU

QYE

>A

7X

c

V

NS8

J3

r

q/

L

0

F

G

o

T

u

2

v

P

w

K

d

j

y

n

M

R

W

C

<latexi sh

1_b

64="8HJ

B

vqr ufZ+G

oK

n

/I

>A

7X

c

V

NS

E

3U

9L

0mk

M

F

Q

z

T

2

O

pg

P

D

w

d

j

y

R

Y

W

C

5

X

1

X

T

Time

X

t

Abrupt change-points

14 Removing the knowledge of time horizon T from Algorithm 1

In this section, we re-state the doubling trick from [11] to remove the dependence of the stream length T on the learning rate tuning in Theorems 4.3 and 5.1.

Proposition 14.1 ([11]) . If for a given α, β, δ, T , clipped-SGD satisfies R T ≤ R ( α, β, T ) ln ( T δ ) with probability at-least 1 -δ , then Algorithm 2 will satisfy regret R T ≤ R ( α, β, T )(log 2 ( T )) 2 ln ( 2 δ ) with probability at-least 1 -δ .

Proof. Over a time interval of T , there are log 2 ( T ) times when Line 5 of Algorithm 2 is executed. Denote by the time-interval between two successive executions of Line 5 as a 'phase' of the algorithm. In the i th phase, the hypothesis of the proposition gives that with probability at-most δ , the regret in phase i denoted by R ( i ) ≥ R ( α, β, T ) ln ( 2 i δ ) . Thus, taking an union bound over all the log 2 ( T )

phases, we get that with probability at-least 1 -log 2 ( T ) δ , the total regret R T is bounded by

15 Proof of Proposition 6.1

The proof is by construction. For each time t = 1 , 2 , · · · , denote by the function f t ( x ) := 1 2 ( x -1 -H t ) 2 , where H t := ∑ t -1 s =1 1 s . Let the initial point θ 0 = 0 and let us set the clipping value of clipped gradient descent to be larger than or equal to 1 .

We will argue by induction that θ t = H t + 1 t for all t = 1 , 2 , · · · . Thus, the loss suffered at time t is l t := 1 -1 t , since the optimal point at all times t is 1 + H t .

For the base-case of time 1 , we see that the gradient of f 1 ( · ) at x = θ 0 = 0 is -1 . Thus, θ 1 = 1 = H 2 and the loss at time 1 is 0 = 1 -1 1 . Assume the induction hypothesis is true for all s = 1 , · · · , t for some t . Consider the time t +1 . The induction hypothesis tells that at time t , θ t = H t +1 . At time t +1 , the function revealed is ( x -1 -H t +1 ) 2 and thus the gradient at any x = H t +1 is -1 . The step size at time t +1 is η t +1 = 1 t +1 and thus θ t +1 = H t +1 + 1 t +1 = H t +2 . Thus, the induction hypothesis holds.

This gives that the total cumulative regret is

16 Proofs of Lower Bounds on Estimation Regret

Before giving the proof of Proposition 2.6, we state and prove a simpler result that finite diameter is necessary in the presence of both non-stationarities and corruptions.

Proposition 16.1 ( Finite diameter is necessary in general ) . There exists a strongly convex loss function L such that for every D > 0 , domain Θ = [ -D / 2 , D / 2] ⊂ R and every T ∈ N , such that for every algorithm, there exists a sequence of probability measures ( P t ) T t =1 and corruptions ( C t ) T t =1 where regret is at-least Ω(Λ T D ) with probability 1 .

We prove the following Proposition on mean-estimation, which will prove Proposition 16.1.

Proposition 16.2 ( Mean estimation needs finite diameter ) . Let Θ = [ -D / 2 , D / 2] ⊂ R , timehorizon T and the number of corruptions be Λ T ≤ T and L ( Z, θ ) = 1 2 ( Z -θ ) 2 . For every algorithm, there exists a sequence of probability measures ( P t ) T t =1 and corruption functions ( C t ) T t =1 , such that the regret incurred is at-least Ω(Λ T D ) with probability 1 .

Proof. The crux of this lower-bound is that the time instants of corruptions are adversarially chosen. The proof follows by construction where the underlying ground-truth is one of two possibilities and there is no noise, i.e., the variance of the probability distributions chosen by the adversary are all 0 . We will show that even against an oblivious adversary, Ω(Λ T D ) regret is un-avoidable.

Assume that the adversary has only two choices - either an environment in which the true mean at all times is D / 2 or one in which the true mean in the first T -Λ T samples is D / 2 and the true mean in the last Λ T samples is -D / 2 . If the adversary picks the first scenario then it does not corrupt any sample and since there is no noise, all the T observed samples in the first scenario will be equal to D / 2 . If

the adversary picks the second scenario as the environment, then it will corrupt the last Λ T samples to show as D / 2 instead of the true -D / 2 . Thus, in either choice of the adversary the observed set of T samples by the forecaster is deterministic D / 2 for all T samples. Thus, the forecaster cannot distinguish whether the underlying environment is from scenario 1 or 2 from observations, even if the forecaster has knowledge of the two possible situations from which the adversary is choosing.

Thus, for any deterministic choice of outputs by the algorithm, the adversary can choose one of these two situations such that the regret on the last Λ T samples of the stream incurs regret at-least Λ T D 4 .

16.1 Proof of Proposition 2.6

Similar to Proposition 2.6, consider two scenarios for mean-estimation. In one scenario, the uncorrupted samples are all drawn from a Dirac mass at 0 , but the first Λ T samples are corrupted with all d coordinates set to D / √ d . In the other scenario, there are no corruptions and the un-corrupted samples are all from Dirac mass at location with all coordinates D / √ d . In both situations, the first Λ T samples are identical. Thus, no estimator in the first Λ T samples can distinguish between these two scenarios and will incur regret at-least (Λ T D ) / 4 .

17 Useful convexity based inequalities

In this section, we collect some inequalities that we will repeatedly use throughout the proofs. Throughout the rest of the paper, for every t ∈ [ T ] , we denote by R t ( θ ) := E Z ∼ P t [ L ( Z, θ )] and by θ ∗ t := arg min θ ∈ Θ R t ( θ ) . Convexity of R t ( · ) and the domain Θ being convex implies that θ ∗ t is well-defined, exists and is unique. Recall that for all time t , R t ( θ ) is strongly convex with parameters M,m respectively. Thus, we know that

Further since θ ∗ t := arg min θ R t ( θ ) , we have that

Putting these two together, we see that

Also, We further use the following lemma.

Lemma 17.1 (Lemma 3 . 11 from [9]) . Let f : R d → R be a M smooth and m strongly convex function. Then for all x, y ∈ R d ,

By substituting x = θ t -1 , y = θ ∗ t and f ( · ) = R t ( · ) and by leveraging the fact that ∇R t ( θ ∗ t ) = 0 , we get the inequality that

Re-arranging, we see that

Further, using the Cauchy-schwartz inequality that ⟨∇R t ( θ t -1 ) , θ t -1 -θ ∗ t ⟩ ≤ ∥∇R t ( θ t -1 ) ∥∥ θ t -1 -θ ∗ t ∥ ≤ D∥∇R t ( θ t -1 ) ∥ , we get from the preceeding inequality that

18 Theorem statement when the gradients are sub-gaussian with known upper-bound

Theorem 18.1 (Formal version of Theorem 4.3) . Suppose Assumption 4.2 holds. For every δ ∈ (0 , 1) , if Algorithm 1 is run with clipping value λ ≥ G + √ Tr (Σ) + C (√ ν max (Σ) ln ( 2 T δ ) ) , where C is given in Proposition 19.1 and step-size η > 0 , then with probability at-least 1 -δ ,

Corollary 18.2. Suppose Assumption 4.2 holds. For every δ ∈ (0 , 1) , if Algorithm 1 is run with clipping value λ ≥ G + √ Tr (Σ) + C (√ ν max (Σ) ln ( 2 T δ ) ) , where C is given in Proposition 19.1 and step-size η = 1 mT α for α ∈ [0 , 1) , then with probability at-least 1 -δ ,

The above theorem immediately yields the following corollary by substituting the learning rate as η = T -2 / 3 .

Corollary 18.3. Suppose Assumption 4.2 holds. For every δ ∈ (0 , 1) , if Algorithm 1 is run with clipping value λ ≥ G + √ Tr (Σ) + C (√ ν max (Σ) ln ( 2 T δ ) ) , where C is given in Proposition 19.1 and step-size η = T -2 / 3 m , then with probability at-least 1 -δ ,

19 Proof of Theorem 18.1

The proof in this section is based on the following result on high-dimensional sub-gaussian random vectors.

Proposition 19.1 (Exercise 6.3.5 [57]) . There exists an absolute constant C > 80 , such that for every sub-gaussian random-vector Z with 0 mean random-vector and covariance matrix Σ and every δ ∈ (0 , 1) , the following

holds, where ν max (Σ) is the largest eigen-value of the upper-bound covariance matrix Σ .

19.0.1 Notations

We follow the same proof architecture as that of [55]. We define three sequences of random variables ( ψ t ) t ≥ 1 and ( ¯ ψ ) t ≥ 1 as follows.

These are random vectors since Z t ∼ P t and both C t and θ t are measurable with respect to the sigma algebra generated by σ ( Z 1 , · · · , Z t , C 1 , · · · , C t -1 ) . Clearly, for all times t ≥ 1 , on the event that C t = 0 , ψ t = ¯ ψ t holds almost-surely. Furthermore, from triangle inequality almost-surely for all time t ≥ 1 , we have

̸

Recall from Assumption 2.5 that for every t and θ ∈ Θ , the covariance matrix of the random vector ∇L ( Z, θ ) -∇R t ( θ ) is bounded from above in the positive semi-definite sense by Σ . Denote by Trace (Σ) and ν max (Σ) as the trace and the highest eigen value respectively of the matrix Σ . Assumption 4.2 implies the following two lemmas.

Denote by the event E no-clip as

19.0.2 Supporting Lemmas based on sub-gaussian concentrations

The main result of this section is stated at the end in Corollary 19.5. In order to state them, we build a series of useful bounds along the way. The following lemma bounds the estimation error of the true gradient using the sample Z t at each time t .

Lemma 19.2.

In words, the above lemma states that for any sequence of ˜ θ 1 , · · · , ˜ θ T , the probability that the norm of the difference between ∇L ( Z t , ˜ θ t ) and its expectation ∇R t ( ˜ θ t ) where Z t ∼ P t exceeds C √ ν max (Σ) ln ( 2 T δ ) is bounded above by δ .

Proof. Fix any deterministic ˜ θ 1 , · · · , ˜ θ T ∈ Θ . Assumption 4.2 yields that for all time t , ∇L ( Z t , ˜ θ t ) -∇R t ( ˜ θ t ) is a 0 mean sub-gaussian random vector with covariance matrix upper-bounded in the positive definite sense by Σ . Observe from Jensen's inequality that for all t and θ ∈ θ

Now, Hoeffding's inequality in Proposition 19.1 gives that

holds with probability at-least 1 -δ T . Now, taking an union bound over t = { 1 , · · · , T } gives that

Now, since ˜ θ 1 , · · · , ˜ θ T ∈ Θ was arbitrary, we can take a supremum on both sides and that concludes the proof.

Corollary 19.3. With probability at-least 1 -δ , for all times t ∈ { 1 , · · · , T } ,

Proof. The proof follows from applying the Lemma 19.2 to the set of random locations θ 1 , · · · , θ T .

Lemma 19.4. If Algorithm 1 is run with clipping value λ ≥ G + √ Tr (Σ)+ C (√ ν max (Σ) ln ( 2 T δ ) ) , where C is given in Proposition 19.1, then with probability at-least 1 -δ gradient is never clipped for any un-corrupted sample, i.e., with probability at-least 1 -δ , event E no-clip holds.

Proof of Lemma 19.4. From the definition, we know that the gradients are never clipped for the un-corrupted samples if ∥∇L ( θ t -1 , Z t ) ∥ ≤ λ holds for all t .

From Corollary 19.3, we know that with probability at-least 1 -δ , for all t ∈ { 1 , · · · , T } , we have

From the triangle inequality we know that

Substituting the bound from Equation (14) into (15), we get that for every t ∈ [ T ] ,

holds with probability at-least 1 -δ .

This immediately yields the following corollary.

Corollary 19.5. If Algorithm 1 is run with clipping value λ ≥ G + √ Tr (Σ)+ C (√ ν max (Σ) ln ( 2 T δ ) ) , where C is given in Proposition 19.1, then with probability at-least 1 -2 δ , for all times t ∈ [ T ] ,

Proof. On the event that both Corollary 19.3 and Lemma 19.4 holds, we have the result. Both those events fail to hold with probability at-most 2 δ .

Denote by the event

Corollary 19.5 states that P [ E ψ ] ≥ 1 -2 δ .

19.1 Lemmas based on expanding the gradient descent recursion

Lemma 19.6. Under event E ψ , for every time t ∈ [ T ] ,

̸

where B = C √ ν max (Σ) ln ( 2 T δ ) + √ Tr (Σ) holds.

19.1.1 Proof of Lemma 19.6

Proof. Consider any time t . We have

Step ( a ) follows since Θ is a convex set, ∥P Θ ( θ t ) -θ ∗ t ∥ ≤ ∥ θ t -θ ∗ t ∥ , since θ ∗ t ∈ Θ . In step ( b ) , we use the fact that ∥ a + b ∥ 2 2 ≤ 2 ∥ a ∥ 2 2 +2 ∥ b ∥ 2 2 , for all a, b ∈ R d . Substituting Equation (6) into (21), we get that

Re-arranging the equation above yields

Further substituting Equation (5) into the display above yields that

where the inequality comes from the fact that if ηm < 1 = ⇒ 2 ηm -2 η 2 m 2 > ηm . We simplify the display above using the inequality in Equation (45) as,

̸

̸

̸

Using the Cauchy-Schartz inequality that ⟨ θ t -1 -θ ∗ t , ψ t -¯ ψ t ⟩ ≤ ∥ θ t -1 -θ ∗ t ∥∥ ¯ ψ t -ψ t ∥ ≤ 2 λ ∥ θ t -1 -θ ∗ t ∥ 1 C t =0 , where the last inequality comes from the fact that for all time t , ∥ ¯ ψ t -ψ t ∥ ≤ 2 λ 1 C t =0 almost-surely. Plugging this into Equation (22) yields

̸

̸

Using the fact that the diameter of the set Θ is D Θ now yields that

̸

Unrolling the recursion yields,

̸

Using the fact that ∑ t -1 s =1 (1 -ηm ) s -1 ≤ 1 ηm , we get that

̸

̸

Changing the variables in the summation, the above inequality can be written as

̸

̸

Simplifying term I from Corollary 19.5 since event E ψ holds by assumption of the Lemma, we get that with probability at-least 1 -2 δ ,

̸

̸

̸

Denote by the event

Lemma 19.8.

holds for all t ∈ [ T ] .

̸

̸

where B = C √ ν max (Σ) ln ( 2 T δ ) + √ Tr (Σ) . Further, simplifying using Cauchy Schwartz inequality, we get the stated result of Lemma 19.6.

̸

̸

19.2 Martingale Analysis of Term II

Denote by the constant

̸

The following proposition holds by simple algebraic manipulation.

Proposition 19.7. For every t ∈ [2 , T ] and s ≤ t ,

Denote by the sequence ( ξ s ) T s =1 of random vectors as

̸

Recall the definition of event E ψ in Equation (17).

Corollary 19.5 give that P [ E ψ ] ≥ 1 -2 δ .

Lemma 19.9. Under the events E ξ and E ψ ,

19.3 Proof of Theorem 18.1

Proof. Corollary 19.5 gives that P [ E C ψ ] ≤ 2 δ . Lemma 19.8 gives that P [ E c ξ ] ≤ δ . An union bound argument thus gives that P [ E C ψ ∪ E C ξ ] ≤ 2 δ . Thus, we have that both events E ψ and E ξ hold with probability at-least 1 -3 δ , i.e.,

Under the event E ψ ∩ E ξ , Lemma 19.9 gives that

Now, applying the fact that √∑ i x i ≤ ∑ i √ x i , we can simplify √ U t as

̸

̸

Observe the following deterministic identities that can be proven by switching the order of summations.

̸

The first inequality above follows from the fact that for all x ∈ (0 , 1) , we have 1 1 - √ 1 -x ≤ 2 x and the last inequality follows from Cauchy-Schwartz inequality that ∑ T i =1 √ x i ≤ √ T ∑ T i =1 x i . Recall that the regret Reg T := ∑ T t =1 ∥ θ t -θ ∗ t ∥ . Thus we get from Equations (32) and (33) that with probability at-least 1 -3 δ ,

holds with probability at-least 1 -3 δ . By a change of variables by re-parametrizing δ → δ 3 yields the stated result.

19.4 Proofs of Lemma 19.9

Proof. We prove this lemma by induction on t . For the base case of t = 1 , Equation (32) holds with probability 1 trivially by definition. Now for the induction hypothesis, assume Equation (32) holds for all s ∈ { 1 , · · · , t -1 } for some t > 1 . Since we are on the event E ξ , we Equation (27) holds. Thus,

̸

̸

From the induction hypothesis, we have that ∥ θ ∗ s -θ s ∥ 2 2 ≤ 2 U s holds for all s ∈ { 1 , · · · , t -1 } . Thus,

Further since we are on the event E ξ , we have that

Thus, plugging this back into Equation (39), we get that

From Equation (28), we see that 2 ηB 2 m ≤ U t . Substituting this in the above equation, we see that

The second inequality follows since B = C √ ν max (Σ) ln ( 2 T δ ) + √ Tr (Σ) , where C ≥ 40 as given in Proposition 19.1. Thus, √ 40 ν max (Σ) ln ( 2 T δ ) B ≤ 1 .

19.5 Proof of Lemma 19.8

We first reproduce an useful result.

Lemma 19.10 (Sub-gaussian Martingale Azuma-Hoeffding inequality [53]) . Suppose Y 1 , · · · , Y T is a martingale sequence with respect to a filtration ( F t ) T t =1 , i.e., E [ Y t |F t -1 ] = 0 for all t . Further, suppose there exists deterministic non-negative numbers ( σ t ) T t =1 such that for every λ ∈ R and t ∈ [ T ] , we have E [exp( λY t ) |F t -1 ] ≤ exp ( λ 2 σ 2 t 2 ) almost-surely. Then for every a > 0 ,

In order to prove Lemma 19.8, we will show that for each t ∈ [ T ] , the event

holds with probability at-least 1 -δ T . Then an union bound will conclude the proof of Lemma 19.8.

To do so, first observe that the sum ∑ t -1 s =1 (1 -ηm ) t -s ⟨ ξ s , ψ s +1 ⟩ is a martingale difference with respect to the filtration ( F t ) T t =1 . This follows from two facts.

Fact I ξ s ∈ F s is measurable with respect to the filtration generated by all the random vectors Z 1 , · · · , Z s and the corruption vectors C 1 , · · · , C s . This follows since the adversary is causal and does not have access to future randomness to decide on corruption levels.

Fact III , The sequence of random variables ((1 -ηm ) t -s ⟨ ξ s , ψ s +1 ⟩ ) t s =1 satisfies the premise of Lemma 19.10 with σ s ≤ (1 -ηm ) 2( t -s ) ∥ ξ s ∥ 2 ν max (Σ) ≤ 2(1 -ηm ) t -s U t ν max (Σ) , for all 1 ≤ s ≤ t .

The third fact follows from the definition that conditional on F s , the covariance matrix of ψ s +1 is upper-bounded by Σ and ∥ ξ s ∥ 2 ≤ 2 U s for all s , and (1 -ηm ) t -s U s ≤ U t holds for all s ≤ t .

In order to apply Lemma 19.10, observe that the sum of variances ∑ t s =1 σ 2 s = 2 U t ν max (Σ) ∑ t s =1 (1 -ηm ) t -s ≤ 2 U t ν max (Σ) ηm . Applying Lemma 19.10, we get that

19.6 Proof of Corollary 4.5

The finite diameter that D < ∞ fact is only used in Lemma 19.6. From observing the proof of Lemma 19.6, it holds even if D = ∞ , albeit the bound is vacuous if Λ T > 0 . Thus, the entire proof holds verbatim even if D = ∞ and gives non-trivial regret guarantees when Λ T = 0 .

20 Theorem statement and Proofs in the general case from Section 5

Theorem 20.1 (Formal version of Theorem 5.1) . For every δ ∈ (0 , 1) , if Algorithm 1 is run with clipping value λ ≥ 2 G , and step-size η > 0 , then with probability at-least 1 -δ ,

Corollary 20.2. For every δ ∈ (0 , 1) , if Algorithm 1 is run with clipping value λ = (2 GT β ) , and step-size η = 1 mT α for α ∈ [0 , 1) , then with probability at-least 1 -δ ,

and by

Corollary 20.3. For every δ ∈ (0 , 1) , if Algorithm 1 is run with clipping value λ = 2 GT α 3 , and step-size η = 1 mT α for α ∈ [0 , 1] , then with probability at-least 1 -δ ,

Corollary 20.4. For every δ ∈ (0 , 1) , if Algorithm 1 is run with clipping value λ ≥ max(2 G,T 1 / 5 ) , and step-size η = T -1 / 2 m , then with probability at-least 1 -δ ,

20.1 Proof of Theorem 20.1

We follow a similar proof architecture as that of Theorem 18.1. We define two sequences of random variables ( ψ t ) t ≥ 1 and ( ¯ ψ ) t ≥ 1 as follows.

These are random vectors since Z t ∼ P t and both C t and θ t are measurable with respect to the sigma algebra generated by σ ( Z 1 , · · · , Z t , C 1 , · · · , C t -1 ) . Clearly, for all times t ≥ 1 , on the event that C t = 0 , ψ t = ¯ ψ t holds almost-surely. Furthermore, from triangle inequality almost-surely for all time t ≥ 1 , we have

̸

Expanding the one-step recursion

Similar to the analysis carried out in the proof of Theorem 18.1, we consider any time t and write the recursion as

Now, this can be expanded verbatim as done in Section 19.1 to yield an identical replica of Equation 24.

̸

̸

Denote by ψ t := ψ ( b ) t + ψ ( v ) t , where ψ ( b ) t := E Z t [ ψ t |F t -1 ] and ψ ( v ) t := ψ t -ψ ( b ) t . Using this in the display above and using that fact that ∥ a + b ∥ 2 2 ≤ 2 ∥ a ∥ 2 2 +2 ∥ b ∥ 2 2 , we get

̸

̸

Further simplifying by adding and subtracting E Z t [ ∥ ψ ( v ) t ∥ 2 2 |F t -1 ] to be above display, we get

̸

Lemma 20.5. If λ ≥ 2 sup θ ∈ Θ ∥∇R t ( θ ) ∥ = 2 G , the following inequalities hold.

Simplifying Equation (48) using bounds in Lemma 20.5, we get

̸

̸

̸

̸

We further simplify the underscored term above as

̸

Using the inequality above into Equation (53) along with the bound that 4 η 2 λ 2 ∑ t -1 s =1 (1 -ηm ) s -1 1 C t -s +1 =0 ≤ 4 ηλ 2 Λ T m , where Λ T := ∑ T t =1 1 C t =0 , we get

̸

̸

̸

For every t ∈ { 1 , · · · , T } , denote by the constant

Proposition 20.7. For every 1 ≤ ≤

̸

For each t ∈ { 1 , · · · , T } , denote by the event E (2) t as

Observe from definition, P [ E (2) 1 ] = 1 . In order to bound the other events, the following lemma holds. Lemma 20.8. For each t ∈ { 2 , · · · , T }

̸

Denote by the event E (1) as

Lemma 20.6.

Denote by the event

̸

Corollary 20.9.

We prove this lemma by induction. The base case of t = 1 holds trivially by definition.

By the induction hypothesis, assume that for some t > 1 , l 2 s ≤ QU s holds for all s < t . Then, from Equation (54) we have that

Step ( a ) follows from the induction hypothesis. Step ( b ) follows since 2 ηm > 1 .

√

Weprove this claim by contradiction. Assume that Q ≥ ( 206 σ 2 λ 2 mη + 4 λ 2 σ 2 ) +2 and that U t + 64 σ 2 mλ QU t + 16 λ √ ηQU t m > QU t . Re-arranging, we see that

The last inequality follows from Equation (55), where U t ≥ 40 ησ 2 m . Re-aranging the last inequality, we see that

It is easy to verify that the above inequality and the fact that Q = ( 206 σ 2 λ 2 mη + 4 λ 2 σ 2 ) +2 cannot hold simultaneously. This can be checked by squaring both sides and solving for the root of the quadratic equation in Q .

where event E (2) is given in Equation (57).

Lemma 20.10. Under events E (1) and E (2) , for all times t ∈ { 1 , · · · , T } ,

20.2 Proof of Theorem 20.1

We know from Lemma 20.6 and Corollary 20.9, that with probability at-least 1 -2 δ , events E (1) ∩E (2) holds. Lemma 20.10 gives that for all t ∈ { 1 , · · · , T } , ∥ θ t -θ ∗ t ∥ 2 ≤ QU t , where Q is given in Lemma 20.10 and U t is given in Equation (55). Now applying the inequality that √ x + y ≤ √ x + √ y , we get that

̸

̸

Recalling that Reg T := ∑ T t =1 ∥ θ t -θ ∗ t ∥ , we have by simplifying using the series expansions in Equations (34, 35, 36, 37), we get that

20.3 Proof of Lemma 20.8

Proof. Fix a time t ∈ { 1 , · · · , T } . We wish to bound the sum ∑ t -1 s =1 2 η (1 -ηm ) s -1 ⟨ θ ∗ t -s -θ t -s , ψ ( v ) t -s +1 ⟩ . In order to do so, we define the following sequence of random vectors

Denote by the sequence ( ξ s ) T s =1 of random vectors as

where Q is defined in Lemma 20.10. From the induction hypothesis, on the event E (2) t -1 , · · · , E (2) 1 , we have

holding almost-surely. We will now bound the sum ∑ t -1 s =1 2 η (1 -ηm ) s -1 ⟨ ξ t -s , ψ ( v ) t -s +1 ⟩ using Freedman's martinglae inequality. We know that,

Further, the sum of conditional variances can be bounded as

20.4 Useful Martingale concentration inequality

Lemma 20.11 (Freedman's inequality[59]) . Suppose Y 1 , · · · , Y T is a bounded martingale with respect to a filtration ( F t ) T t =0 with E [ Y t |F t -1 ] = 0 and P [ | Y t | ≤ B ] = 1 for all t ∈ { 1 , · · · , T } . Denote by V s := ∑ s n =1 Var ( Y n |F n -1 ) be the sum of conditional variances. Then, for every a, v > 0 ,

Re-arranging the above inequality, we see that if

then the RHS of Equation (59) is bounded above by δ 2 .

20.5 Proof of Lemma 20.6

Proof of Lemma 20.6. Fix a t ∈ { 1 , · · · , T } . For s ∈ { 1 , · · · , t -1 } , denote by the random variable Y ( t ) s := 4 η 2 (1 -ηm ) s -1 ( ∥ ψ ( v ) t -s +1 ∥ 2 2 -E Z t -s +1 [ ∥ ψ ( v ) t -s +1 ∥ 2 2 |F t -s ]) . Observe that the sequence ( Y ( t ) s ) t -1 s =1 is a martingale difference sequence with respect to the filtration ( G s ) t -1 s =1 , where G s := F t -s . Furthermore, Lemma 20.5 gives that with probability 1 , | Y ( t ) s | ≤ 4 η 2 (4 λ 2 +4 λ 2 ) ≤ 32 η 2 λ 2 . We can bound the conditional variance as

Now, putting B := 32 η 2 λ 2 and v = 2560 η 3 λ 2 σ 2 m , we get from Equation (60) that with probability at-least 1 -δ/ 2 ,

20.6 Proof of Lemma 20.5 reproduced from [25]

This follows from the analysis of [25] and reproduced here for completeness.

Proof of Equation (49)

Equality ( a ) follows from the fact that θ t -1 ∈ F t -1 .

Proof of Equation (50)

Denote by the event ξ t := 1 ∥∇L ( Z t ,θ t -1 ) ≥ λ and by χ t := 1 ∥∇L ( Z t ,θ t -1 ) -∇R t ( θ t -1 ) ∥ > λ 2 . Since we have λ > 2 G , we have the inequality that

Then, we have that

Now, expanding out ψ ( b ) t , we get

Now computing the probability

The last in-equality is from Chebychev's inequality. Substituting Equation (63) into Equation (62) yields the result.

Proof of Equation (51)

Inequality ( a ) follows from the fact that θ t -1 ∈ F t -1 and step ( b ) follows from the fact that for any random vector Z and filtration F and Y ∈ F , E Z [ ∥ Z -E [ Z |F ] ∥ 2 2 |F ] ≤ E [ ∥ Z -Y ∥ 2 2 |F ] holds almost-surely. To simplify notation, we denote by E t := E [ ·|F t -1 ] . We now bound Equation (64) to conclude.

Inequality ( a ) follows from the fact that for any two vectors a, b , ∥ a + b ∥ 2 2 ≤ 2 ∥ a ∥ 2 2 +2 ∥ b ∥ 2 2 , and the assumption that sup θ ∈ Θ E t [ ∥L ( Z, θ ) -R t ( θ ) ∥ 2 2 ] ≤ σ 2 .

20.7 Proof of Corollary 5.4

The finite diameter that D < ∞ fact is only used in Equation (46) which holds even if D = ∞ , albeit the bound is vacuous if Λ T > 0 . Thus, the entire proof holds verbatim even if D = ∞ and gives non-trivial regret guarantees when Λ T = 0 .

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers an online estimation setting where the learner observes a sequence of samples, which are drawn from a (previously determined but unknown) sequence of probability distributions, and with some fraction of samples arbitrarily corrupted. In each round, the learner makes a decision, and regret is measured based on how far this decision is from that which would minimize their expected loss if no corruption occurred. The loss function is strongly convex, and the goal is to obtain total regret bounds which are sublinear in the time horizon $T$ (but may scale with some standard parameters like diameter of the decision space, (unknown) amount of distribution shift, (unknown) amount of corruptions, (known) gradient norm upper bound, (unknown) gradient covariance upper bound). The authors show that a tuned version of clipped SGD achieves the desired regret bounds, with some partial lower bounds. The proofs use combine a novel inductive argument with martingale concentration techniques to provide high probability regret bounds under arbitrary distribution shift, and the results are verified with some simple experiments pertaining to mean estimation and linear regression.\n\nSTRENGTHS:\n- To the best of my knowledge, and as claimed by the authors, no work has addressed this issue of online, (outlier/heavy-tail) robust, and non-stationary convex optimization, and this setting seems important for many applications.\n- There are several settings where their regret bounds are tight, and many improvements over the state-of-the-art\n- The proposed algorithm is quite simple, and they empirically validate the choice of tuning suggested by their theory\n- The paper is generally well-written, with thorough explanations of the problem setup, prior work, and the different components of their regret bounds (though I struggled a bit with understanding the analysis of their general regret bound)\n\nWEAKNESSES:\n- It would seem more natural to measure regret via the excess risk\n$\\sum_{t=1}^T \\mathbb{E}\\_{Z \\sim P_t}[\\mathcal{L}(Z,\\theta_t)] - \\inf_{\\theta \\in \\Theta}\\mathbb{E}_{Z \\sim P_t}[\\mathcal{L}(Z,\\theta)].$\nDo your results transfer to this metric? If not, why? Cor. 4.6-style results for the stationary setting without corruptions are usually stated with respect to this benchmark, as far as I am aware.\n- In the stationary setting with corruptions, it is not clear to me that the diameter-dependence is necessary (and this is considered quite undesirable in the robust mean estimation literature). The authors' lower bound in Section 16 looks to require non-stationarity. Can a lower bound be provided without this requirement?\n- There is a claim of dimension independence (modulo the diameter dependence), but I view dependence on the trace of $\\Sigma$ as implicitly depending on the dimension in many cases of interest, so this seems a bit misleading\n- It is rather unclear to me whether these results are fundamentally interesting, or whether the solution is somewhat standard and only new because this combination of problem settings hasn't been explicitly studied before. In particular, clipped SGD is a standard solution in this space, though their tuning analysis appears novel.\n\t- If the analysis for the general case is of fundamental interest, I suggest that more space is spent in the appendix (or added page of a final version) describing the induction details - I found the appendix a bit hard to follow and did not verify correctness\n\nMinor Nits:\n- On line 48, the superscript for the footnote looks like a power. Right after that, the way the loss function is introduced read to me as if it was already used.\n- In the abstract and elsewhere, \"high-probability\" is used as an adjective without an accompanying noun (presumable \"regret bounds\").\n- Footnote 4 appears before its reference in Table 1\n- \"an\" -> \"a\" on line 202\n- \"at-least\" -> \"at least\" in Theorem 4.3\n- Notation \"m\" for strong convexity constant conflicts with power of $T$ in Eq. 1\n\nQUESTIONS:\nIn addition to the questions raised in the \"weaknesses\" section, I was curious if this approach could be adapted to incorporate differential privacy (since gradient clipping + noise is a common approach to private learning).\n\nLIMITATIONS:\nAssumptions are made clear, and I don't anticipate any negative impacts.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis works studies robust sequential estimation under a non-stationary environment. A loss function is fixed in advance. The data generating process is non-stationary over time, and hence the optimal parameter \\theta^*_t, which minimizes the expected loss over the distribution at time t, is changing over time. A policy returns in each round an estimated parameter \\theta_t. The goal is to minimize the regret, defined as sum of differences between \\theta^*_t and \\theta. The central question is, is there a policy that is free of distributional knowledge (i.e., moments of the data-generating distributions or stream complexity). They answered this question by presenting a gradient-based algo with sublinear regret. These upper bounds match the known lower bounds in the no-noise and no-drift setting.\n\nSTRENGTHS:\nPresented sublinear regret bounds which matches known lower bounds for the no-drift or no-corruption setting.\n\nWEAKNESSES:\n1. I am not able to find significant novelty in either the problem formulation or the results. Maybe there is a good practical reason to consider this particular formulation but at least in this submission, the authors did not sell it well.\n\n2. Writing: In general this paper is not written well.\n-\tThere are plenty of typos and gramatic mistakes, even in the abstract and formulation section. E.g. “A observation … “ in the abstract. Line 133 “formalize”. E.g. “upto” -> up to. Line 99, “to derive high-probability under any rate..” \n-\tConsistency: do not use a concept before defining it. E.g. in the abstract, “neither the O(1/t)....” What is \"t\"? It seems lower case t is not the same as \"T\" \n-\tVague language. Just to name a few, in the Abstract: “A observation … can be used” – I don’t understand this line. In the def of regret, what norm are we using? Line 96, “the data stream is subgaussian”, do you mean the distribution in each round is subgaussian, or the the entire stream?  \n-\tDo not start a sentence with a mathematical notation; see e.g. “X_t is shown as …”\n-\tThe tone switches between being very informal to very formal abruptly (and why use the word “diserderata” so frequently?)\n\n**The above issues combined suggest that the submission has been written in a rush.** I suggest the authors carefully polish the paper.\n\n3. Lacking of discussion to previous work. It seems that this problem can be reduced to the problem studied in “non-stationary stochastic optimization” (Oper. Res. ‘14) by Besbes et al. Both papers proposed gradient-based algo and used the total-variance distance (“\\Phi_T”) to measure the non-stationarity.  I am wondering what results do previous results in OCO imply, and how the results\n\nQUESTIONS:\nIt seems that this problem can be reduced to the problem studied in “non-stationary stochastic optimization” (Oper. Res. ‘14) by Besbes et al. Does this work already imply sublinear regret for some special case of this work? I understand in this submission the corruption is assumed to be adversarial, but this does not seem to be an essential consideration.\n\nLIMITATIONS:\nn/a", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies the problem of online estimation in a setup that generalizes the stochastic i.i.d. input assumption. The authors consider a setting where the input distribution is allowed to change over time (a certain number of times), and furthermore the input is allowed to be adversarially corrupted (a certain number of times). The authors analyze the standard clipped SGD algorithm to tackle both of these issues simultaneously (Being a simple and implementable algorithm, clipped SGD has many favorable properties in practice). In essence, the paper establishes that the clipped SGD algorithm is \"Lipschitz\" with respect to distribution drift and contaminations.\n\nSTRENGTHS:\nThe paper studies an important problem setting in online convex optimization that gracefully generalizes the standard stochastic i.i.d. input assumption.\n\nWEAKNESSES:\n1. At a high level, my reservations with the paper are that the paper proposes a goal in terms of drift and corruption tolerance (on page 4) that seems a bit arbitrary. I was unable to see if even one of these in isolation is understood and what the correct rates are in those settings. In particular, in all of the explicit examples that I could find in the paper, all the lower bounds in terms of $\\Delta_T$ (or $\\Phi_t$) did not have any multiplicative term with $T$; see, for example, Proposition 2.6 and Section 11.\n\n1. (How the distribution shift is measured) The paper defines the quantity $\\Phi_t$ to be the number of drifts in the input sequence. However, all of the results in the paper have regret scaling with $\\Phi_T$ times a polynomial in $T$. Is this necessary for algorithms that achieve vanishing regret? What are the best upper bounds and lower bounds for the regret in terms of $\\Phi_T$ (without any outliers)? (See the first point for more context)\n\n2. (How the clean error is measured) For outliers, the paper defines the quantity $\\Delta_t$ to count the number of outliers in the input sequence. However, the results then depend on $\\Delta_T$ multiplied by the diameter of the set and a polynomial in $T$. I am not sure if multiplicative dependence on $T$ is necessary for stochastic inputs  (See the points above for more context). Are there lower bounds?\n\n3. (What counts as dimension free? and Comparison with existing work) The paper lists their results as achieving dimension-independent errors, but they have suboptimal dependence on these quantities in the regret bounds. For example, heavy-tailed mean estimation (where the claimed results are somewhat immediate since they multiplicatively depend on the trace of the covariance matrix). The paper [18] is said to have dimension-dependent errors but I was unable to find the entry corresponding to [18] in Table 1 in the paper [18]. Their result on isotropic covariance matrices naturally uses $d$ samples.\n\n\n\n*All references are based on the version uploaded to the supplementary material.*\n\n---\n## Recommendation\n\nThis is perhaps because I am not from this subfield but I am unable to appreciate the technical results of the paper (more comments below). Thus, I recommend weak reject, and I would be happy to change my score if the authors/other reviews convince me otherwise.\n\nQUESTIONS:\nSee above\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-19"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studys online estimation problems on data stream exhibits challenging properties, including distribution drift, heavy tails, and outlier/anonmalies corruptions. Formally, at each time step, (given all the data that has arrived) the algorithm needs to output an estimation on certain unknown parameter so as to minimize the cumulative regret.\n\nConsider the task of mean estimation as an illustrative example: At each time step, the algorithm receives a data point drawn from an unknown distribution and must estimate the mean of that distribution. The core challenges here are threefold:\n1. Distribution drifting: The mean of the distribution from which the data point is drawn can change over time.\n2. Heavy tail: The data's distribution might possess unbounded 3rd or higher-order moments.\n3. Outliers: Observed data points could be significantly distorted or corrupted.\n\nInterestingly, the paper reveals that a modified version of the clipped Stochastic Gradient Descent (SGD) can attain sublinear regret, even when all three of the aforementioned challenges are present. A particularly notable insight from the research is the necessity of an intermediate learning rate for optimal performance. While an $O(1)$ rate is ideal for addressing distribution drift and an $O(1/t)$ rate is best suited for noise, managing both simultaneously requires a rate of $O(1/T^\\alpha)$, where $\\alpha$ lies between 0 and 1.\n\nWhile these findings are derived under certain assumptions regarding the parameter domain and loss functions, the authors further fortify their claims by proving several lower bounds. This demonstrates the indispensability of the stated assumptions.\n\nSTRENGTHS:\nThe paper effectively articulates the problem setting and its significance. Even with my limited familiarity with the subject, it's evident that the problem is complex and the results presented are notably substantive.\n\nWEAKNESSES:\nN/A\n\nQUESTIONS:\n(The paper under review is outside my area of expertise, and I had not initially opted to review it. Unfortunately, I wasn't presented with alternative submissions for evaluation)\n\nWhile I struggled to grasp the entirety of the analysis due to unfamiliarity with the methodologies used, I found the distinction between drift and corruption intriguing. It's a bit surprising to me that, it is possible for an algorithm to even distinguish between *drift* and *corruption*, and it required very different rate ($O(1)$ vs $O(1/t)$) to handle them optimally. The two definitions appear to me to be essentially analogous. The paper's main result seem to confirm this intuition, but the analysis still treats the two issues separately. A more detailed discussion from the authors on this particular point would be beneficial.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 1, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-23"}

comments: SUMMARY:
This paper studies the online robust estimation problem in possibly non-stationary environments. Under the assumption that the loss function is strongly convex, the authors propose an online clipped stochastic gradient descent algorithm with tunable clipping parameter that is able to achieve both adaptiveness in distribution shift and robustness to heavy-tailed inliers and arbitrary corruptions. Moreover, the algorithm does not require distributional knowledge. Theoretical results and experiments are provided.

STRENGTHS:
This paper presents an online estimation algorithm that for the first time provably robust to heavy-tails, corruptions and distribution shift simultaneously. The soundness of the paper is supported by their theoretical results and experiments.

WEAKNESSES:
The presentation of this paper lacks some organization and the problem setup part is not clear at the beginning. For example, $X_t$ is said to be the corrupted input, which is the summation of the sample $Z_t$ And the corruption $C_t$, then the authors want to estimate $\theta_t$, which is not reflected in $X_t = Z_t+C_t$ at all. Moreover, some of the contents in the appendix part (simulations on real data) could be moved to the main body.

QUESTIONS:
N/A

LIMITATIONS:
Yes
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-26