Title: A Batch-to-Online Transformation under Random-Order Model
Venue: NeurIPS 2023
Accepted: yes
Keywords: online learning

random model setting
Abstract: We introduce a transformation framework that can be utilized to develop online algorithms with low $\epsilon$-approximate regret in the random-order model from offline approximation algorithms. We first give a general reduction theorem that transforms an offline approximation algorithm with low average sensitivity to an online algorithm with low $\epsilon$-approximate regret. We then demonstrate that offline approximation algorithms can be transformed into a low-sensitivity version using a coreset construction method. To showcase the versatility of our approach, we apply it to various problems, including online $(k,z)$-clustering, online matrix approximation, and online regression, and successfully achieve polylogarithmic $\epsilon$-approximate regret for each problem. Moreover, we show that in all three cases, our algorithm also enjoys low inconsistency, which may be desired in some online applications.

A Batch-to-Online Transformation under Random-Order Model

Abstract

We introduce a transformation framework that can be utilized to develop online algorithms with low ϵ -approximate regret in the random-order model from offline approximation algorithms. We first give a general reduction theorem that transforms an offline approximation algorithm with low average sensitivity to an online algorithm with low ϵ -approximate regret. We then demonstrate that offline approximation algorithms can be transformed into a low-sensitivity version using a coreset construction method. To showcase the versatility of our approach, we apply it to various problems, including online ( k, z ) -clustering, online matrix approximation, and online regression, and successfully achieve polylogarithmic ϵ -approximate regret for each problem. Moreover, we show that in all three cases, our algorithm also enjoys low inconsistency, which may be desired in some online applications.

1 Introduction

In online learning literature, stochastic and adversarial settings are two of the most well-studied cases. Although the stochastic setting is not often satisfied in real applications, the performance and guarantees of online algorithms in the adversarial case are considerably compromised. This is particularly true for important online tasks such as k -means clustering, which gives a significantly worse guarantee than their offline or stochastic counterparts [Cohen-Addad et al., 2021]. As a result, their practical applicability is greatly limited.

Recently, the random-order model has been introduced as a means of modeling learning scenarios that fall between the stochastic and adversarial settings [Garber et al., 2020, Sherman et al., 2021]. In the random-order model, the adversary is permitted to choose the set of losses, with full knowledge of the learning algorithm, but has no influence over the order in which the losses are presented to the learner. Instead, the loss sequence is uniformly and randomly permuted. This effectively bridges the gap between the stochastic setting, where only the distribution of losses can be chosen by the setting, and the adversarial setting, where the adversary has complete control over the order of the losses presented to the learner.

In this work, we introduce a batch-to-online transformation framework designed specifically for the random-order model. Our framework facilitates the conversion of an offline approximation algorithm into an online learning algorithm with ϵ -approximate regret guarantees. Our primary technical tool is average sensitivity, which was initially proposed by Varma and Yoshida [2021] to describe the algorithm's average-case sensitivity against input perturbations. We demonstrate that any offline approximation algorithm with low average sensitivity will result in a transformed online counterpart that has low ϵ -approximate regret. To achieve small average sensitivity for offline algorithms, we leverage the idea of a coreset [Agarwal et al., 2005, Har-Peled and Mazumdar, 2004], which is a small but representative subset of a larger dataset that preserves important properties of the original data. We present a coreset construction method that attains low average sensitivity, and when combined with the approximation algorithm, yields an overall algorithm with low average sensitivity.

Yuichi Yoshida

To showcase the practicality and versatility of our framework, we apply it to three popular online learning problems: online ( k, z ) -clustering, online matrix approximation, and online regression. In all three cases, our approach yields a polylogarithmic ϵ -approximate regret. Furthermore, due to the low average sensitivity of our algorithms, they also enjoy low inconsistency, which is the cumulative number of times the solution changes. This additional property may prove useful in certain online settings. We note that this inconsistency has also been investigated in the classic online learning and multi-armed bandits literature [Agrawal et al., 1988, Cesa-Bianchi et al., 2013].

2 Related Works

Average sensitivity Varma and Yoshida [2021] first introduced the notion of average sensitivity and proposed algorithms with low average sensitivity on graph problems such as minimum spanning tree, minimum cut, and minimum vertex cover problems. Various other problems have then been analyzed for the average sensitivity, including dynamic programming problems [Kumabe and Yoshida, 2022], spectral clustering [Peng and Yoshida, 2020], Euclidean k -clustering [Yoshida and Ito, 2022], maximum matching problems [Yoshida and Zhou, 2021], and decision tree learning problem [Hara and Yoshida, 2023].

Online (consistent) ( k, z ) -clustering While ( k, z ) -clustering, which includes k -means ( z = 2 ) and k -median ( z = 1 ) as its special cases, has been studied extensively from various perspectives such as combinatorial optimization and probabilistic modeling, it can be NP-hard to obtain the exact solution [Impagliazzo et al., 2001]. Thus most theoretical works have been focused on designing approximation algorithms. In the online setting, Li et al. [2018] proposed a Bayesian adaptive online clustering algorithm that enjoys a minimal sublinear regret. However, the algorithm is allowed to output more than k clusters. Without such assumption, Cohen-Addad et al. [2021] proposed the first algorithm that attains ϵ -approximate regret of O ( k √ d 3 n log( ϵ -1 dkn )) for k -means clustering under adversarial setting.

On a separate vein, Lattanzi and Vassilvitskii [2017] proposed an online consistent ( k, z ) -clustering algorithm that produces a 2 O ( z ) -approximate solution for the data points obtained so far at each step while maintaining an inconsistency bound of O ( k 2 log 4 n ) . This implies that their algorithm only updates the output O ( k 2 log 4 n ) many times. Then, Yoshida and Ito [2022] gave an online algorithm with approximation ratio (1 + ϵ ) and inconsistency poly( d, k, 2 z , ϵ -1 ) · log n in the random-order model. We remark that the way how the losses are computed in Lattanzi and Vassilvitskii [2017], Yoshida and Ito [2022] is different from that of the online setting, which Cohen-Addad et al. [2021] and our paper considered.

Online convex optimization and online principle component analysis (PCA) under the randomorder model The online random-order optimization was proposed in Garber et al. [2020], which established a bound of O (log n ) for smooth and strongly convex losses. This result is then improved by Sherman et al. [2021] while still requiring smooth and convex losses.

The techniques and results are then extended to online PCA with the random-order setting, for which a regret of O ( ζ -1 √ kn ) was established, where ζ is an instance-dependent constant. This recovers the regret for online PCA in the stochastic setting [Warmuth and Kuzmin, 2008, Nie et al., 2016]. We remark that PCA can be viewed as a special case of matrix approximation, in which the matrix being approximated is the covariance matrix of the data, and we discuss the more general problem of matrix approximation in this paper.

3 Preliminaries

For a positive integer n , let [ n ] denote the set { 1 , 2 , . . . , n } . For real values a, b ∈ R , a ∈ (1 ± ϵ ) b is a shorthand for (1 -ϵ ) b ≤ a ≤ (1 + ϵ ) b .

3.1 Offline Learning

We consider a general class of learning problems. Let X be the input space, Θ be the parameter space, and ℓ : Θ ×X → R + be a loss function. For simplicity, we assume the loss is bounded, i.e.,

ℓ ( θ, x ) ≤ 1 . Given a set of n data points X ∈ X n , we are asked to learn a parameter θ ∈ Θ that minimizes the objective value ℓ ( θ, X ) := ∑ x ∈ X ℓ ( θ, x ) . We call this problem the offline learning problem .

When the exact minimization of the loss function ℓ is NP-hard or computationally demanding, one may only hope to obtain an approximate solution efficiently. Specifically, for α > 0 , we say a solution θ ∈ Θ is α -approximate for X ∈ X n if ℓ ( θ, X ) ≤ α · min ˜ θ ∈ Θ ℓ ( ˜ θ, X ) . The value α is called the approximation ratio of the solution. We say a (possibly randomized) algorithm A is α -approximate if the expected approximation ratio of the output solution is at most α .

3.2 Online Learning with Random-Order Model

In the online learning problem , instead of receiving all points at once, the data arrives sequentially throughout a time horizon n . Specifically, the data point comes one by one, where x t comes at time t ∈ [ n ] . At the time t , using the collected data points X t -1 := ( x 1 , . . . , x t -1 ) , we are asked to output a parameter θ t ∈ Θ . Then we receive the data point x t and incur a loss of ℓ ( θ t , x t ) . In this work, we consider the random-order model , in which the data points x 1 , . . . , x n may be chosen adversarially, but their ordering is randomly permuted before the algorithm runs.

To evaluate our performance, we use the notion of regret, which is the cumulative difference between our solution and the best solution in hindsight. In cases where obtaining the exact solution is hard, and one may only hope to obtain an approximate solution efficiently, we use the ϵ -approximate regret .

Definition 3.1 ( ϵ -approximate regret for the random-order model) . Given a (randomized) algorithm A that outputs a sequence of parameters θ 1 , . . . , θ n when given input x 1 , . . . , x n . The ϵ -approximate regret of A for the random-order model is defined as

where the randomness is over the internal randomness of A and the ordering of data points. When ϵ = 0 , we simply call it the regret .

̸

In certain cases, online algorithms are required to maintain a good solution while minimizing inconsistency , which is quantified as the number of times the solution changes. This can be expressed formally as Inconsistency( n ) = E A , { x t } [ ∑ n -1 t =1 I { θ t = θ t +1 } ] , where I is the indicator function.

3.3 Average sensitivity

On a high level, the notion of average sensitivity describes the differences in the performance of a randomized algorithm with respect to input changes. This difference is captured by the total variation distance, which is defined below.

Definition 3.2. For a measurable space (Ω , F ) and probability measures P, Q defined on (Ω , F ) . The total variation distance between P and Q is defined as TV( P, Q ) := sup A ∈F | P ( A ) -Q ( A ) | .

Equipped with this, the average sensitivity of a randomized algorithm is formally defined as the average total variation distance between the algorithm's output on two training data sets that differ by deleting one point randomly. For a dataset X = ( x 1 , . . . , x n ) ∈ X n and i ∈ [ n ] , let X ( i ) denote the set ( x 1 , . . . , x i -1 , x i +1 , . . . , x n ) obtained by deleting the i -th data point. Then, the following definition gives a detailed description of the notion:

Definition 3.3 (Average Sensitivity [Varma and Yoshida, 2021, Yoshida and Ito, 2022]) . Let A be a (randomized) algorithm that takes an input X ∈ X n and outputs A ( X ) . For β : Z + → R + , we say that the average sensitivity of A is at most β if

for any X ∈ X n , where we identify A ( X ) and A ( X ( i ) ) with their distributions.

4 Batch-to-Online Transformation in the Random-Order Model

In this section, we describe a general framework that can transform any offline (1 + ϵ ) -approximate algorithm into an online algorithm with low ϵ -approximate regret. Our goal is to show the following.

Theorem 4.1. Let A be a (randomized) (1 + ϵ ) -approximate algorithm for the offline learning algorithm with average sensitivity β : Z + → R + . Then, there exists an online learning algorithm in the random-order model such that Regret ϵ ( n ) = O ( ∑ n t =1 β ( t ) + 1) .

Our method is described in Algorithm 1. Let A be an approximation algorithm for the offline learning problem. Then, at each time step, based on the collected data X t -1 , we simply output θ t = A ( X t -1 ) .

Algorithm 1: General batch-to-online conversion

To show that Algorithm 1 achieves a low approximate regret when A has a low average sensitivity, the following lemma is useful.

Lemma 4.2. Let A be a (randomized) algorithm for the offline learning problem with average sensitivity β : Z + → R + . Then for any input X ∈ X n , we have

where x = a ± b means a -b ≤ x ≤ a + b .

Proof of Theorem 4.1. Consider Algorithm 1. For any t ∈ [ n ] , we have

where the last equality follows by replacing x i with x t +1 in ℓ ( A ( X ( i ) t ) , x i ) because they have the same distribution, and the last inequality is by the average sensitivity of the algorithm.

Rearranging the terms, we have

where OPT t := min θ ℓ ( θ, X t ) is the optimal value with respect to X t , and the second inequality holds because the approximation ratio of θ t +1 is 1 + ϵ in expectation.

Taking summation over both sides, we have

Fix the ordering x 1 , . . . , x n , and let c i ( i ∈ [ t ]) be the loss incurred by x i in OPT n . In particular, we have OPT n = ∑ n i =1 c i . Note that c i 's are random variables depending on the ordering of data points, but their sum, OPT n , is deterministic. Then, we have OPT t ≤ ∑ t i =1 c i because OPT t minimizes the loss up to time t , Hence, we have

Therefore, we have

5 Approximation Algorithm with Low Average Sensitivity via Coreset

To design approximation algorithms for the offline learning problem with low average sensitivity, we consider the following approach: We first construct a small subset of the input that well preserves objective functions, called a coreset, with small average sensitivity, and then apply any known approximation algorithm on the coreset. Coreset is formally defined as follows:

Definition 5.1 (Har-Peled and Mazumdar [2004], Agarwal et al. [2005]) . Let ℓ : Θ ×X → R + be a loss function and let X ∈ X n . For ϵ > 0 , we say that a weighted set ( Y, w ) with Y ⊆ X and w : Y → R + is an ϵ -coreset of X with respect to ℓ if for any θ ∈ Θ , we have ∑ y ∈ Y w ( y ) ℓ ( θ, y ) ∈ (1 ± ϵ ) ∑ x ∈ X ℓ ( θ, x ) .

Now, we consider a popular method for constructing coresets based on importance sampling and show that it enjoys a low average sensitivity. For a data x ∈ X , its sensitivity σ X ( x ) 1 is its maximum contribution to the loss of the whole dataset, or more formally

It is known that we can construct a coreset as follows: A data point x ∈ X is sampled with probability p ( x ) := σ X ( x ) / ∑ x ′ ∈ X σ X ( x ′ ) , and then its weight in the output coreset is increased by 1 / ˜ p , where

˜ p is a slight perturbation of p ( x ) . This process is to be repeated for a fixed number of times, where the exact number depends on the approximation ratio of the coreset. See Algorithm 2 for details. We can bound its average sensitivity as follows:

Lemma 5.2. The average sensitivity of Algorithm 2 is O ( ϵ -1 m/n ) .

A general bound on the number of times we need to repeat the process, i.e., m in Algorithm 2, to obtain an ϵ -coreset is known (see, e.g., Theorem 5.5 of Braverman et al. [2016]). However, we do not discuss it here because better bounds are known for specific problems and we do not use the general bound in the subsequent sections.

6 Online ( k, z ) -Clustering

In online applications, unlabelled data are abundant and their structure can be essential, and clustering serves as an important tool for analyzing them. In this section, as an application of our general batch-to-online transformation, we describe an online ( k, z ) -clustering method that enjoys low regret.

6.1 Problem setup

The online ( k, z )-clustering problem [Cohen-Addad et al., 2021] is an instance of the general online learning problem described in Section 3. We describe the problem as follows: Let k ≥ 1 be an integer and z ≥ 1 be a real value. Over a time horizon n , at each time step t , a data point x t ∈ R d is given. Using the set of data points X t -1 = { x 1 , . . . , x t -1 } , we are asked to compute a set Z t = { z 1 , . . . , z k } of k points in R d that minimize ℓ ( Z t , x t ) := min j =1 ,...,k ∥ x t -z j ∥ z 2 , which is the z -th power of the Euclidean distance between x t and the closest point in Z t . Note that Z t plays the role of θ t in the general online learning problem. The regret and ϵ -approximate regret are defined accordingly.

6.2 Method and results

One important ingredient to our method is the coreset construction method proposed by Huang and Vishnoi [2020]. The method provides a unified two-stage importance sampling framework, which allows for a coreset with a size that is dimension independent. Specifically, the method constructs an ϵ -coreset of size ˜ O ( min { ε -2 z -2 k, 2 2 z ϵ -4 k 2 }) in ˜ O ( ndk ) time, where the ˜ O hides polylogarithmic factors in n and k . We remark that the importance of sampling steps in the framework is similar to the ones described in Section 5, which thus allows us to analyze its average sensitivity.

Algorithm 3 gives a brief description of our algorithm, while a detailed description is presented in the appendix. The algorithm adheres to the standard transformation approach, whereby an offline approximation algorithm is run on the coreset derived from the aggregated data.

Moreover, there exists an algorithm that enjoys the same regret bound and an inconsistency bound of Inconsistency( n ) = O (( (168 z ) 10 z ϵ -5 z -15 k 5 log( ϵ -1 kn ) + ϵ -2 z -2 k log k log( ϵ -1 kn ) ) log n ) for ( k, z ) -clustering.

Remark 6.1. When z = 2 , previous results for the adversarial setting show an ϵ -approximate regret bound of O ( k √ d 3 n log( ϵ -1 dkn )) [Cohen-Addad et al., 2021]. In comparison, although our regret is for the random-order model, our method and results accommodate a range of values for z , and the regret bound is only polylogarithmically dependent on n and is independent of the dimension d .

7 Online Low-Rank Matrix Approximation

Low-rank matrix approximation serves as a fundamental tool in statistics and machine learning. The problem is to find a rankk matrix that approximates an input matrix A ∈ R d × n as much as possible. In this section, we apply the transformation framework to the offline approximation algorithm to obtain a low regret online algorithm.

7.1 Problem setup

Low-rank matrix approximation By the singular value decomposition (SVD), a rankr matrix A ∈ R d × n can be decomposed as A = UΣV ⊤ , where U ∈ R d × r and V ∈ R n × r are orthonormal matrices, Σ ∈ R r × r is a diagonal matrix with A 's singular values on the diagonal. The best rankk approximation of A is given by

where ∥ · ∥ F denotes the Frobenius norm, Σ k ∈ R k × k is a diagonal matrix with A k 's top k singular values on the diagonal, and U k ∈ R d × k and V k ∈ R n × k are orthonormal matrices obtained from U and V , respectively, by gathering corresponding columns. The best rankk approximation can also be found by projecting A onto the span of its top k singular vectors, that is, A k = U k U ⊤ k A . Then, we can say an orthonormal matrix Z is an ϵ -approximate solution if

The matrix approximation problem serves as an important tool in data analytics and is closely related to numerous machine learning methods such as principal component analysis and least squares analysis. When dealing with streaming data, the online version of the matrix approximation problem becomes a vital tool for designing online versions of the machine learning algorithms mentioned above.

Online matrix approximation Through a time horizon of n , we receive a column of A , a t ∈ R d at each time step t . We are then asked to compute Z t ∈ R d × k that minimizes

Without loss of generality, we will assume that the losses are bounded between [0 , 1] . We remark that similar assumptions are also made in Nie et al. [2016].

The online matrix approximation problem serves as a core component of online machine learning algorithms such as principle component analysis. These algorithms are important to a range of applications, such as online recommendation systems and online experimental design [Warmuth and Kuzmin, 2008, Nie et al., 2016].

7.2 Method and results

In the context of low-rank matrix approximation, the coreset of a matrix is called the projection-cost preserving samples, defined as follows:

Definition 7.1 (Rankk Projection-Cost Preserving Sample Cohen et al. [2017]) . For n ′ < n , a subset of rescaled columns C ∈ R d × n ′ of A ∈ R d × n is a (1 + ϵ ) projection-cost preserving sample if, for all rankk orthogonal projection matrices X ∈ R d × d , (1 -ϵ ) ∥ A -XA ∥ 2 F ≤ ∥ C -XC ∥ 2 F ≤ (1 + ϵ ) ∥ A -XA ∥ 2 F .

Such sketches that satisfy Definition 7.1 can be constructed via importance sampling-based routines, which are modifications of the 'leverage scores'. Specifically, for the i -th column a i of matrix A ,

the ridge leverage score is defined as τ i ( A ) = a ⊤ i ( AA ⊤ + ∥ A -A k ∥ 2 F k I ) † a i , where † denotes the Moore-Penrose pseudoinverse of a matrix [Cohen et al., 2017].

Now, we introduce our online matrix approximation algorithm in Algorithm 4, which builds upon our transformation framework. It computes the approximation of the matrix from the sketch derived from the aggregated matrix using ridge leverage scores.

Theorem 7.2. For any ϵ ∈ (0 , 1) , Algorithm 4 has regret Regret ϵ ( n ) = O ( ϵ -2 k log n log( ϵ -1 kn ) ) . Moreover, there exists an algorithm for online low-rank matrix approximation that enjoys the same regret bound and an inconsistency bound of Inconsistency( n ) = O ( ϵ -2 k log n log( ϵ -1 kn ) ) .

Remark 7.1. The online matrix approximation with the random-order setting has previously been investigated in the context of principle component analysis by Garber et al. [2020]. They established a regret of O ( ζ -1 √ kn ) , where ζ is the smallest difference between two eigenvalues of A t A ⊤ t . In contrast, our result gives a polylogarithmic result on ϵ -regret, which translate to an exact regret of O ( ϵ OPT+ O ( ϵ -2 k log n log( ϵ -1 kn ) )) , with OPT being the minimum possible cumulative loss attained by the hindsight best approximate.

8 Online Regression

In the online regression problem, at each time step t ∈ [ n ] , we are asked to output a vector x t ∈ R d , and then we receive vectors a t ∈ R d and b t ∈ R that incurs the loss of ℓ ( x t , a t , b t ) = ∥ a ⊤ t x t -b ∥ 2 . Without loss of generality, we assume that the losses are bounded between [0 , 1] . We note that similar assumptions are also made in [Cesa-Bianchi et al., 1996, Ouhamma et al., 2021].

With our general reduction framework, we show an ϵ -regret upper bound as follows.

Theorem 8.1. For any ϵ ∈ (0 , 1) , Algorithm 5 has regret Regret ϵ ( n ) = O ( ϵ -2 d log n log( ϵ -1 dn ) ) . Moreover, there exists an algorithm for online regression that enjoys the same regret bound and an inconsistency bound of Inconsistency( n ) = O ( ϵ -2 d log n log( ϵ -1 dn ) ) .

Remark 8.1. In the stochastic setting, the online regression problem has been extensively investigated [Foster, 1991, Littlestone et al., 1995, Cesa-Bianchi et al., 1996, Ouhamma et al., 2021]. Using online ridge regression or forward algorithms, the regret is shown to be O ( d log n ) . In the random-order model setting, Garber et al. [2020], Sherman et al. [2021] give O ( √ n ) -type regret when the matrix A has a small condition number. In comparison, our result attains polylogarithmic ϵ -approximate regret, while maintaining no requirement on the loss function or the condition number. Our result can be translated to an exact regret of O ( ϵ OPT+ O ( ϵ -2 d log n log( ϵ -1 dn ) )) , with OPT being the minimum possible cumulative loss attained by the hindsight best parameter.

8.1 Method and results

Similar to the low-rank matrix approximation problem, we utilize the leverage score method to learn a subspace that preserves information regarding the regression. Specifically, we use the leverage score to learn a ϵ -subspace embedding, which is defined as follows.

Definition 8.2 ( ϵ -Subspace Embedding) . A matrix S ∈ R m × n is said to be an ϵ -subspace embedding of A ∈ R n × d if for any vector x ∈ R d , we have (1 -ϵ ) ∥ A x ∥ ≤ ∥ SA x ∥ ≤ (1 + ϵ ) ∥ A x ∥ .

The subspace embedding serves the same functionality as coreset in the problem of online regression, it preserves the loss of information while enjoying a much lower dimension. In the online regression problem context, we define the leverage score as follows.

Definition 8.3 (Leverage Score) . Let A = UΣV ⊤ be the singular value decomposition of A ∈ R n × d . For i ∈ [ n ] , the i -th leverage score of A , is defined as τ i = ∥ U i, : ∥ 2 2 .

With the leverage score, we propose Algorithm 5. The algorithm follows the general transformation framework, where the regression problem is solved at every step with the sketch derived from the aggregated matrix using leverage score. For notational convenience, we construct the sketch by appending rows instead of columns as we did in Section 7.

Algorithm 5: Online consistent regression

The subspace embedding result of Woodruff [2014] immediately shows the following:

Theorem 8.4. For any ϵ, δ ∈ (0 , 1) , if m = O ( ϵ -2 d log( δ -1 d ) ) , then with probability ≥ 1 -δ , S t is an ϵ -subspace embedding for A t -1 with O ( ϵ -2 d log( δ -1 d ) ) columns.

To obtain Theorem 8.1, we first analyze the average sensitivity of the leverage score sampling. Then, with Theorem 8.4 and the general reduction Theorem 4.1, we obtain the regret bound.

9 Experiments

We here provide a preliminary empirical evaluation of our framework in the context of online k -means clustering, and online linear regression, with the result shown in Figure 1. Our experiments are conducted with various approximation ratios and experimental setups ( ϵ = 0 . 1 , 0 . 01 , 0 . 001 , with k = 3 or k = 5 clusters). We then compare the performance of the proposed algorithm to the hindsight optimal solution. For k -means clustering, we obtain the hindsight optimal solution by applying k -means++ to all the data. In the context of regression, we utilize the least square formula to compute the hindsight optimal solution. Our experimental results demonstrate that the proposed algorithm is highly effective, and its performance aligns with our theoretical findings.

(a)

k

-means,

3

clusters

(b)

k

-means,

5

clusters

(c)

Regression

K-means clustering

K-means clustering

Online Regression

2.5

10

Ours, eps = 0.1

2.0

Ours, eps = 0.01

2.0

8

Ours, eps = 0.001

Hindsight optimal

Averaged cost

Averaged cost

1.5

Averaged cost

1.5

6

1.0

1.0

Ours, eps = 0.1

Ours, eps = 0.1

0.5

Ours, eps = 0.01

0.5

Ours, eps = 0.01

2

Ours, eps = 0.001

Ours, eps = 0.001

Hindsight optimal

Hindsight optimal

0

200

400

600

800

1000

0.0

0

200

400

600

800

1000

0

0

200

400

600

800

1000

Timestep

Timestep

Timestep

10 Conclusion

In this paper, we proposed a batch-to-online transformation framework that designs consistent online approximation algorithms from offline approximation algorithms. Our framework transforms an offline approximation algorithm with low average sensitivity to an online algorithm with low approximate regret. We then show a general method that can transform any offline approximation algorithm into one with low sensitivity by using a stable coreset. To demonstrate the generality of our framework, we applied it to online ( k, z ) -clustering, online matrix approximation, and online regression. Through the transformation result, we obtain polylogarithmic approximate regret for all of the problems mentioned.

References

Pankaj K Agarwal, Sariel Har-Peled, Kasturi R Varadarajan, et al. Geometric approximation via coresets. Combinatorial and computational geometry , 52(1):1-30, 2005.

Rajeev Agrawal, MV Hedge, and Demosthenis Teneketzis. Asymptotically efficient adaptive allocation rules for the multiarmed bandit problem with switching cost. IEEE Transactions on Automatic Control , 33(10):899-906, 1988.

Vladimir Braverman, Dan Feldman, Harry Lang, Adiel Statman, and Samson Zhou. New frameworks for offline and streaming coreset constructions. arXiv preprint arXiv:1612.00889 , 2016.

Nicolo Cesa-Bianchi, Philip M Long, and Manfred K Warmuth. Worst-case quadratic loss bounds for prediction using linear functions and gradient descent. IEEE Transactions on Neural Networks , 7(3):604-619, 1996.

Nicolo Cesa-Bianchi, Ofer Dekel, and Ohad Shamir. Online learning with switching costs and other adaptive adversaries. Advances in Neural Information Processing Systems , 2013.

Michael B Cohen, Cameron Musco, and Christopher Musco. Input sparsity time low-rank approximation via ridge leverage score sampling. In Proceedings of the 2017 Annual ACM-SIAM Symposium on Discrete Algorithms , 2017.

Vincent Cohen-Addad, Benjamin Guedj, Varun Kanade, and Guy Rom. Online k -means clustering. In International Conference on Artificial Intelligence and Statistics , 2021.

Dan Feldman and Michael Langberg. A unified framework for approximating and clustering data. In Proceedings of the 43th Annual ACM SIGACT Symposium on Theory of Computing , 2011.

Dean P Foster. Prediction in the worst case. The Annals of Statistics , pages 1084-1090, 1991.

Dan Garber, Gal Korcia, and Kfir Levy. Online convex optimization in the random order model. In International Conference on Machine Learning , 2020.

Sariel Har-Peled and Soham Mazumdar. On coresets for k -means and k -median clustering. In Proceedings of the 36th Annual ACM SIGACT Symposium on Theory of Computing , 2004.

Satoshi Hara and Yuichi Yoshida. Average sensitivity of decision tree learning. In International Conference on Learning Representations , 2023.

Lingxiao Huang and Nisheeth K Vishnoi. Coresets for clustering in Euclidean spaces: importance sampling is nearly optimal. In Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing , 2020.

Russell Impagliazzo, Ramamohan Paturi, and Francis Zane. Which problems have strongly exponential complexity? Journal of Computer and System Sciences , 63(4):512-530, 2001.

Soh Kumabe and Yuichi Yoshida. Average sensitivity of dynamic programming. In Proceedings of the 2022 Annual ACM-SIAM Symposium on Discrete Algorithms , 2022.

Silvio Lattanzi and Sergei Vassilvitskii. Consistent k -clustering. In International Conference on Machine Learning , 2017.

L Li, B Guedj, and S Loustau. A quasi-bayesian perspective to online clustering. Electronic Journal of Statistics , 12(2):3071-3113, 2018.

Nicholas Littlestone, Manfred K Warmuth, and Philip M Long. On-line learning of linear functions. Computational Complexity , 5:1-23, 1995.

Jiazhong Nie, Wojciech Kotłowski, and Manfred K Warmuth. Online PCA with optimal regret. The Journal of Machine Learning Research , 17(1):6022-6070, 2016.

Reda Ouhamma, Odalric-Ambrym Maillard, and Vianney Perchet. Stochastic online linear regression: the forward algorithm to replace ridge. Advances in Neural Information Processing Systems , 2021.

Pan Peng and Yuichi Yoshida. Average sensitivity of spectral clustering. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , 2020.

Uri Sherman, Tomer Koren, and Yishay Mansour. Optimal rates for random order online optimization. Advances in Neural Information Processing Systems , 2021.

Nithin Varma and Yuichi Yoshida. Average sensitivity of graph algorithms. In Proceedings of the 2021 ACM-SIAM Symposium on Discrete Algorithms , 2021.

Manfred K Warmuth and Dima Kuzmin. Randomized online PCA algorithms with regret bounds that are logarithmic in the dimension. Journal of Machine Learning Research , 9(Oct):2287-2320, 2008.

David P Woodruff. Sketching as a tool for numerical linear algebra. Foundations and Trends® in Theoretical Computer Science , 10(1-2):1-157, 2014.

Yuichi Yoshida and Shinji Ito. Average sensitivity of Euclidean k -clustering. In Advances in Neural Information Processing Systems , 2022.

Yuichi Yoshida and Samson Zhou. Sensitivity analysis of the maximum matching problem. In Innovations in Theoretical Computer Science Conference , pages 58:1-58:20, 2021.

A Proofs for Section 4

Lemma 4.2. Let A be a (randomized) algorithm for the offline learning problem with average sensitivity β : Z + → R + . Then for any input X ∈ X n , we have

where x = a ± b means a -b ≤ x ≤ a + b .

Proof. We have

The other direction can be shown analogously.

B Proofs for Section 5

In this section, we prove Lemma 5.2.

Lemma B.1. For any i ∈ [ n ] and x ∈ X ( i ) , let θ ( i ) = argmax θ ℓ ( θ,x ) ∑ x ′ ∈ X ( i ) ℓ ( θ,x ′ ) . Then, we have

Proof. Denote θ = argmax θ ℓ ( θ,x ) ∑ x ′ ∈ X ℓ ( θ,x ′ ) , then for the left-hand side of the inequality, we have

For the second inequality, we have

Lemma B.2. For any i ∈ [ n ] , we have

and we have

Then, we have

Proof. First, we have

We can bound this quantity from below and above by Lemma B.1,

Then, we have

It then follows,

where the second to last inequality is from Lemma B.2, and the last inequality is by ∑ x ∈ X σ X ( x ) ≥ 1 .

Lemma B.4. For ϵ > 0 , let X and X ′ be sampled from the uniform distribution over [ B, (1 + ϵ ) B ] and [ B ′ , (1 + ϵ ) B ′ ] , respectively. Then, we have

Proof. The proof is implicit in Lemma 2.3 of Kumabe and Yoshida [2022].

Lemma 5.2. The average sensitivity of Algorithm 2 is O ( ϵ -1 m/n ) .

Proof. The average sensitivity of importance sampling can be bounded by the sum of the average total variation distance between selected elements and that between assigned weights, conditioned on that the selected elements are the same. The former is bounded by

where the first equality is Lemma B.3. With Lemma B.4, we have the latter as

Combining the two terms, we have the average sensitivity of importance sampling be bounded as O ( | C | n ) + O ( | C | ϵn ) = O ( | C | ϵn ) .

C Proofs for Section 6

C.1 Algorithm

We now introduced the detailed version of Algorithm 3. In Algorithm 7, we provide a detailed description of the coreset construction method for clustering, which is based on Huang and Vishnoi [2020]. To make the coreset construction enjoys small average sensitivity, we perturbe the weights assigned (Line 6 and Line 11). We show that this preserves the approximation ratio while makes the overall algorithm insensitive. With this, we obtain the online consistent clustering Algorithm 6, which clusters data from a coreset at each step.

Algorithm 6: Online consistent ( k, z ) -clustering

Input: PTAS algorithm D for ( k, z ) -clustering, approximation ratio ϵ, δ ∈ (0 , 1) . 1 for t = 1 , . . . , n do 2 Construct an ϵ -coreset C t -1 = ( S t -1 , ω t -1 ) by running Algorithm 7 on X t -1 . 3 Obtain cluster set Z t by running a PTAS D with approximation ratio of (1 + ϵ ) on C t -1 .

4 Receive x t and ℓ ( Z t , x t ) .

Algorithm 7: Coreset construction for clustering Huang and Vishnoi [2020]

Input: A set of point X , approximation parameter ϵ, δ ∈ (0 , 1) , integer k, z .

1 Set ϵ = ϵ/c for some large constant c > 0 .

2 Compute a k -center set C ∗ t ⊆ R d as an ϵ -approximation of the ( k, z ) -clustering problem over X with the D z -sampling algorithm with an approximation ratio of O (2 z log k ) .

3 For each x ∈ X , compute the closest point to x in C ∗ , c ∗ ( x ) , with ties broken arbitrarily. For each c ∈ C ∗ , denote X c to be the set of points x ∈ X with c ∗ ( x ) = c .

4 For each x ∈ X , let σ 1 ,X ( x ) = 2 2 z +2 ϵ 2 ( ∥ x -c ∗ ( x ) ∥ z 2 ∑ x ∈ X ℓ ( C ∗ ,x ) + 1 | X c ∗ ( x ) | ) .

5 Pick a non-uniform random sample D 1 of N 1 = O ( (168 z ) 10 z ϵ -5 z -15 k 5 log k δ ) points from X , where each x ∈ X is selected with probability σ 1 ,X ( x ) ∑ σ 1 ,X ( y ) .

y ∈ X 6 For each x ∈ D 1 , sample ˜ u X ( x ) from [ ∑ y ∈ X σ 1 ,X ( y ) | D 1 t |· σ 1 ,X ( x ) , (1 + ϵ ) ∑ y ∈ X σ 1 ,X ( y ) | D 1 t |· σ 1 ,X ( x ) ] .

7 Set u X ( x ) = ˜ u X ( x )

8 For each c ∈ C ∗ , compute D c to be the set of points in D 1 whose closest point in C ∗ is c with ties broken arbitrarily.

9 For each x ∈ D 1 , let σ 2 ,X ( x ) = u X ( x ) · ℓ ( C ∗ ,x ) ∑ y ∈ D 1 u X ( y ) · ℓ ( C ∗ ,y )

10 Pick a non-uniform random sample D 2 of N 2 = O ( ϵ -2 z -2 k log k log k ϵδ ) points from X t , where each x ∈ X t is selected with probability σ 2 ,X ( x ) ∑ 1 σ 2 ,X ( y )

y ∈ D 11 For each x ∈ D 2 , sample ˜ w X ( x ) from [ ∑ y ∈ D 1 σ 2 ,X ( y ) | D 2 |· σ 2 ,X ( x ) , (1 + ϵ ) ∑ y ∈ D 1 σ 2 ,X ( y ) | D 2 |· σ 2 ,X ( x ) ] .

12 Set w X ( x ) = ˜ w X ( x )

14 S = D 2 ∪ C ∗

15 w ( x ) = w X ( x )

16 Output ( S, w )

C.2 Analysis

Theorem C.1. Algorithm 7 outputs an ϵ -coreset with probability at least 1 -δ and has an average sensitivity of

Proof. We first show that our coreset construction method gives an ϵ -coreset.

We remark that the way we assigned the weights is a perturbed version of the method presented in Huang and Vishnoi [2020]. Then, to show that the perturbed version of the assignment of the weight still preserves the coreset approximation ratio, we show that the perturbed version still satisfies Corollary 15.3 of Feldman and Langberg [2011] and Theorem 14.5. Then using the same argument as Theorem 15.4 of Feldman and Langberg [2011], our coreset construction method gives an ϵ -coreset.

In the first importance sampling stage, we only perturb ˜ u X ( x ) by a ratio of 1 + ϵ , for each x ∈ D 1 . The same is applied in the second stage with ˜ w X ( x ) , for all x ∈ D 2 . For c ∈ C ∗ , we have

In all cases, the result from [Feldman and Langberg, 2011] still holds, as the weights are scaled by (1 + ϵ ) at most. By the same argument that all perturbed weights are scaled by (1 + ϵ ) at most, we have Theorem 15.4 of Feldman and Langberg [2011] holds with a ratio of ϵ (1 + ϵ ) . This results in an approximation of 1 + ϵ (1 + ϵ ) by applying the argument of Theorem 15.4 of Feldman and Langberg [2011]. Rescale ϵ to ϵ/c for some constant c > 0 gives an approximation ratio of 1 + ϵ and completes our argument that Algorithm 7 produces an ϵ -coreset with probability at least 1 -δ .

To show that our method enjoys low average sensitivity, we upper bound the average sensitivity of the two importance sampling stage of Algorithm 7 separately.

The average sensitivity of importance sampling in the first stage can be bounded by the average total variation distance between selected elements and that between assigned weights, conditioned on the selected elements are the same. By Lemma 5.2, we can upper bound the average sensitivity of the first importance sampling stage to be O ( N 1 /n ) , where N 1 = O ( (168 z ) 10 z ϵ -5 z -15 k 5 log( δ -1 k ) ) .

In the second stage, as points are selected with probability σ 2 ,X ( x ) ∑ y ∈ X σ 2 ,X ( y ) , we bound the total variation distance between selected points in a similar way as to that for the first stage. This gives a bound of O ( N 2 /n ) , where N 2 = O ( ϵ -2 z -2 k log k log k ϵδ ) . To bound the average total variation distance between assigned weights, we again apply the same argument as above and obtain a bound of O ( N 2 /n ) .

Combining these and that the average sensitivity of Line 2 of Algorithm 7 is O ( k/n ) (by Lemma 2.2 of Yoshida and Ito [2022]), we have the average sensitivity as

Theorem 6.1. For any ϵ ∈ (0 , 1) , Algorithm 3 gives a regret bound of

Moreover, there exists an algorithm that enjoys the same regret bound and an inconsistency bound of Inconsistency( n ) = O (( (168 z ) 10 z ϵ -5 z -15 k 5 log( ϵ -1 kn ) + ϵ -2 z -2 k log k log( ϵ -1 kn ) ) log n ) for ( k, z ) -clustering.

Proof. First, we note that the approximation ratio of the algorithm with respect to the aggregated loss is at most

from the choice of δ , i.e., δ = O ( ϵ/n ) .

Also, we note that the overall average sensitivity is

Then substituting this into Theorem 4.1, for z ≥ 1 , we have a regret bound of

For the second claim about inconsistency, we first convert the average sensitivity bound to an inconsistency bound of the same order. This can be done by Lemma 4.2 of [Yoshida and Ito, 2022] and by arguing in a similar way as Lemma 4.5 of [Yoshida and Ito, 2022] by showing that there exists a computable probability transportation for the output of Algorithm 3. This thus yields an inconsistency of O (( (168 z ) 10 z ϵ -5 z -15 k 5 log( ϵ -1 kn ) ϵ -2 z -2 k log k log( ϵ -1 kn ) ) log n ) .

D Proofs for Section 7

Theorem 7.2. For any ϵ ∈ (0 , 1) , Algorithm 4 has regret Regret ϵ ( n ) = O ( ϵ -2 k log n log( ϵ -1 kn ) ) . Moreover, there exists an algorithm for online low-rank matrix approximation that enjoys the same regret bound and an inconsistency bound of Inconsistency( n ) = O ( ϵ -2 k log n log( ϵ -1 kn ) ) .

Proof. Let δ > 0 be determined later. By Theorem 6 from Cohen et al. [2017], with probability 1 -δ , for any rankk orthogonal projection X , by sampling m = O ( ϵ -2 k log( δ -1 k ) ) columns, in Line 4 of Algorithm 5, we have,

With this, we note that the algorithm has an approximation of 1 + ϵ as 1 + ϵ/ 2 + δn ≤ 1 + ϵ by choosing the hidden constant in δ small enough.

Note that by applying Lemma 5.2, this routine has an average sensitivity of O ( m/t ) = O ( ϵ -2 k log( δ -1 k ) /t ) at any step t . Then set Z t to the top k left singular vectors of C t , and we have

To obtain the regret, we calculate the overall average sensitivity as ∑ n t =1 β ( t ) = ∑ n t =1 O ( ϵ -2 k log( δ -1 k ) /t ) = O ( ϵ -2 k log n log( δ -1 k ) ) . Applying Theorem 4.1, and from the choice δ = O ( ϵ/n ) , we have Regret ϵ ( n ) = O ( ϵ -2 k log n log( ϵ -1 kn ) ) .

For the second claim about inconsistency, we prove this convert the average sensitivity bound to an inconsistency bound of the same order. This can be done by Lemma 4.2 of [Yoshida and Ito, 2022] and by arguing in a similar way as Lemma 4.5 of [Yoshida and Ito, 2022] by showing that there exists a computable probability transportation for the output of Algorithm 3.

E Proof for Section 8

Theorem 8.1. For any ϵ ∈ (0 , 1) , Algorithm 5 has regret Regret ϵ ( n ) = O ( ϵ -2 d log n log( ϵ -1 dn ) ) . Moreover, there exists an algorithm for online regression that enjoys the same regret bound and an inconsistency bound of Inconsistency( n ) = O ( ϵ -2 d log n log( ϵ -1 dn ) ) .

Proof. By Theorem 8.4, the algorithm has an approximation of 1 + ϵ as 1 + ϵ/ 2 + δn ≤ 1 + ϵ by choosing the hidden constant in δ small enough.

Similar to the low-rank case, the average sensitivity of leverage score sampling at step t is O ( d log( d/δ ) ϵ 2 t ) . Summing over the average sensitivity

Take this into Theorem 1, and with δ = O ( ϵ/n ) , we have a ϵ -regret of O ( ϵ -2 d log n log( dn/ϵ ) ) .

Similar to that of Theorem 6.1 and Theorem 7.2, we prove the second claim by converting the average sensitivity bound to an inconsistency bound of the same order. This can be done by Lemma 4.2 of [Yoshida and Ito, 2022] and by arguing in a similar way as Lemma 4.5 of [Yoshida and Ito, 2022] by showing that there exists a computable probability transportation for the output of Algorithm 3.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis is a paper about online learning in the random arrival/order model, i.e., the adversary chooses samples but they are randomly permuted before being presented to an online algorithm. The authors propose a general tool that establishes a reduction to the offline/batch setting. The bounds that one can obtain with that tool are in terms of approximate regret, i.e., on top of the standard regret one also loses an epsilon fraction of the optimum.  The key idea behind the reduction is to use a coreset to reduce the sensitivity of the offline algorithm; the rest is fairly straightforward. After presenting the general tools, the authors show how to apply it to three specific problems: clustering, low-rank matrix approximation, regression.\n\nSTRENGTHS:\nThe main result is a general tool that potentially can be applied to different problems.\n\nObtaining this result required a certain level of technical mastery, and was not completely trivial.\n\nThe paper is well written and easy to read.\n\nWEAKNESSES:\nTechnical novelty is very limited. The paper combines known results in a very natural (though technically difficult) way. One could argue that the hard (and novel) part of the job was choosing the right pieces to put them together – but e.g. the idea of using coresets in a very similar context appears already e.g. in [Cohen-Addad et al., 2021] (the authors do not mention that).\n\nThere are no experiments – even though one of the applications is a clustering problem studied by Cohen-Addad et al. [2021], who provided some preliminary experimental results, so it would be fairly easy to compare to them. It would be particularly interesting because the theoretical bounds in the two papers are not directly comparable.\n\nEven though the paper provides new algorithms for three problems previously studied in the literature, because of the specific setup, the theoretical bounds are, as far as I understand, incomparable to the previous ones.\n\nThe results allow only for approximate regret bounds (and only under random arrivals). It seems to me to be a big limitation, and the authors did not manage to convince me that this is not the case.\n\nQUESTIONS:\nI’d be curious to see an experimental comparison with [Cohen-Addad et al., 2021].\n\nDefinition 3.1 (approximate regret) seems to be a standard one, so you could provide a reference and a bit more of a context.\n\nMinor remarks:\n\nLine 116: Is F supposed to be A?\n\nLines 123, 235: \\citet -> \\citep\n\nLine 166: data -> data point\n\nLine 191: logarithmic -> polylogarithmic\n\nAlgorithm 3, Input: “approximation ratio eps \\in (0, 1)” -> “approximation ratio 1+eps, eps \\in (0, 1)”\n\nLine 221: square -> squares\n\nLines 227, 259: assumptions are also assumed -> assumptions are also made \n\nLine 253: outermost brackets should be bigger, there should not be one O inside of another\n\nLine 270: missing closing bracket\n\nLine 276: a -> an\n\nLine 288: \\citep -> \\citet\n\nLine 295: “an offline approximation algorithm” -> “offline approximation algorithms”\n\nLine 364: the second to last inequality requires some justification\n\nLine 395: insensitivity -> insensitive, algorithm 6 -> Algorithm 6\n\nLIMITATIONS:\nThe results yield only approximate regret bounds, which seems to me somewhat limiting.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-03"}

comments: SUMMARY:
This paper provides a general framework for how to use good offline methods to construct online methods under the random-order setting.  It points out the importance of the average sensitivity in the transformation from good offline methods to good online methods.

STRENGTHS:
1. The framework is very general for the algorithm design of online algorithms based on offline algorithms.

2. The connection between regret and average sensitivity is beautifully characterized.

WEAKNESSES:
1. It looks the method cannot cover the setting with $\epsilon=0$. If you can, please clarify it and I will further raise my rating.

2. it is also better to have some numerical experiments of the performance on some specific problems.

QUESTIONS:
1. can this method also solve the case with $\epsilon=0$? At least for the online linear programming problem, offline LPs can be accurately solved, and the regret (epsilon=0) can also be small under the random-order setting. Can this setting also be converted by your paper?

LIMITATIONS:
N/A
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-04

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a general method for converting an offline approximation to a learning problem into an online algorithm for the random-order problem which enjoys low (epsilon-approximate) regret. Specifically, the regret of this method depends on the sensitivity of the offline approximation; intuitively, this is the amount by which the distribution of outputs for the algorithm changes upon a small change to the input. \n\nFirst, the paper presents the construction for converting offline approximation algorithms to online algorithms for the random-order model. This construction is quite simple: at any step, we run the offline approximation on the part of the input that has already arrived, and use the output as the choice of the online algorithm for the next step. This is shown to yield low epsilon-approximate regret when the offline algorithm has low sensitivity. \n\nNext, the paper presents/analyzes low-sensitivity offline algorithms for a few online problems (online clustering, online matrix approximation and online regression); these algorithms can then be plugged into the aforementioned framework in order to yield regret bounds for these online problems. \n\nTo my understanding, the main algorithmic concept used in designing low-sensitivity offline approximation is using coresets, which are subsets of the input that accurately represent the overall input. The paper analyzes a method for constructing such coresets, and shows that it has low-sensitivity. This coreset construction is then used to choose stable subsets of the input, yielding online algorithms with low regret and low inconsistency (which is a measure of changes in the online algorithm's output over time).\n\nSTRENGTHS:\nThe algorithmic framework introduced is general, and could be applied to additional online learning problems. \nThe paper introduces the goal of having low-sensitivity offline approximations for the sake of this offline-to-online conversion; this could be used to guide future research efforts. \nThe algorithmic technique used for designing low-sensitivity algorithms in the paper is nice.\n\nWEAKNESSES:\nThe offline-to-online conversion itself is quite simple, and does not introduce any algorithmic techniques (arguably, this is the first conversion that comes to mind).\n\nQUESTIONS:\nnone\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors present a framework for reducing online to offline learning. More presicely, they consider random order online learning where at each round a random point of some domain $X$ is presented to the learner (as opposed to some adversary choosing the example). They first show that an offline algorithm with low average sensitivity, i.e., the output of the algorithm does not depend too much on any single point, then it can be effectively used to solve the corresponding online problem via a standard \"follow-the-leader\" type reduction. In particular, at round $t$ having observed the points $X_{t-1} = x_1,\\ldots, x_{t-1}$ in the\nonline learning setting, the learner runs the offline algorithm on $X_{t-1}$ and uses its answer to play at the current round. They next show how to obtain offline algorithms with low average sensitivity by constructing a coreset via sensitivity sampling.  They apply their framework to various problems such as online clustering and online linear regression.\n\nSTRENGTHS:\nThe online-to-batch problem considered in this work is well-motivated and interesting. The authors provide a unified approach to obtain online algorithms for random-order online learning tasks and show some non-trivial results for popular online problems such as online regression.  I believe that those results are going to be of interest for the NeurIPS community.\n\nThe applications of the framework proposed in this work yield improved regret bounds compared to the prior work.  In the case of online matrix approximation application, this work provides a regret bound that is only logarithmic in the horizon $n$ while the prior work the regret was roughly $\\sqrt{n}$ and also depended on a ``condition number'' of the observed matrices.\n\nThe paper is very carefully written and the main results and proofs were easy to follow.\n\nWEAKNESSES:\nThis work considers the weaker online learning model of random order (instead of the more standard adversarial model).  In Remark 6.1 (online clustering)\nthe authors compare with online algorithms that dealt with the adversarial setting.  I think a table with precise comparisons with prior work would help this paper.\n\nThe online to offline reduction in the random ordering assuming low average sensitivity is not very surprising and the proof (see proof of Lemma 4.2) is rather straightforward. Also, it seems to crucially rely on the fact that the online learning is random order. Would a this or a similar approach based on low average sensitivity yield any guarantee for online learning in the adversarial setting?\n\nQUESTIONS:\nSee weaknesses.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-14"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper considers the random order model originally proposed by Garber et al. '20, and presents a generic framework to take any (approximate) offline algorithm and via usage of coresets turn it into an online algorithm with low average sensitivity, which can be then used to obtain an (approximate) regret bound. Some specific cases are considered where this framework is applied, including online clustering, online low-rank matrix approximation, and online regression.\n\nSTRENGTHS:\n* The high level idea is elegant, the paper is coherent and well written.\n* The proposed method offers novel algorithmic approach for batch-to-online reductions.\n\nWEAKNESSES:\n* I was missing some context that would allow better understanding the approach presented in the paper; specifically comparison to the approaches considered in [Garber et al., 2020, Sherman et al., 2021]. The role of uniform convergence in coresets construction vs running the offline algorithm directly on the cumulative loss, and average sensitivity vs algorithmic stability. See some of my questions below.\n* Perhaps this is not a weakness per se, but (please correct me if I'm wrong - this was not entirely clear to me) the rates obtained do not strictly improve any prior art excluding Section 6 which improves Cohen-Addad et al., '21 but applies less generally. The rest of the results pertain to $\\epsilon$-approximate regret while the previous works mentioned obtain standard regret bounds.\n\nQUESTIONS:\n* **Coresets vs direct uniform convergence of the cumulative loss.** At a high level, is it correct to say that the coreset construction is based on uniform concentration of the loss, and that in some specific cases this uniform concentration comes cheap? If so, could we alternatively take the offline algorithm and run it on the cumulative loss as is, then claim a regret bound owed to uniform concentration of the cumulative loss directly? Some more questions related to this point:\n\t* Does big-$O$ hide a dimension factor in Lemma 5.2?\n\t* Section 5 presents a coreset construction method, but then in the section that follows (6.2) a different method (that of y Huang and Vishnoi '20) is used. Why?\n* **Average sensitivity vs algorithmic stability.** TV is the infimum (over all couplings of the two RVs) of the expected 0-1 distance. This is an upper bound on the Wasserstein distance which is the infimum (over all couplings of the two RVs) of the expected euclidean distance. This Wasserstein distance is basically average algorithmic stability. Is it correct that average sensitivity is a strictly stronger notion?\n* **Applicability in the full adversarial setup.** It seems to me the method you propose should also work in a fully adversarial setup. Average sensitivity is akin to uniform stability; the output of your offline algorithm \"stabilized\" via coresets does not care about stochasticity in the examples. If the output of an approximate ERM does not change much between rounds this should lead to a regret bound, perhaps? Roughly; $\\sum_{t} \\ell(A(X_t), x_{t+1}) - \\ell(A(X_{t+1}), x_{t+1}) \\leq \\sum_t \\beta(t)$ and by \"Be-the-Leader\" Lemma (Kalai & Vempala '03), $\\sum_{t} \\ell(A(X_{t+1}), x_{t+1}) \\leq \\sum_{t} \\ell(\\theta^\\star, x_{t+1})$.\n* **$\\epsilon$-regret vs competetive-ratio**. How does $\\epsilon$-regret compare to competitive ratio from competitive analysis?\n* Lines 36-37 \"... and when combined with the approximation algorithm ...\" - to my understanding, whether the the offline algorithm that operates on the coreset is approximate or exact is irrelevant to the point you are trying to make, here and in the paper more generally, no? \n\n\n\n## Minor comments\n* \"Although the stochastic setting is not often satisfied in real applications, the performance 15 and guarantees of online algorithms in the adversarial case are considerably compromised\" - the meaning of this sentence is not clear to me.\n* Lemma 4.2: I didn't see the notation \"$x = r \\pm \\beta$\" defined anywhere (I gather it signifies $r-\\beta \\leq x\\leq r+\\beta$).\n* What does it mean to \"run $\\mathcal A$ with approximation ratio of $(1-\\epsilon')$...\" (line 4 of Algorithm 3)? Theorem 6.1 does not mention explicitly any requirements from $\\mathcal A$. Looking into the detailed version of the algorithm (Algorithm 6), it seems $\\mathcal A$ should be a PTAS. Perhaps putting this in the main text could make this a bit clearer. \n* ״We remark that the importance of sampling steps in the framework is similar to 193 the ones described in Section 5, which thus allows us to analyze its average sensitivity.״ --> \"the importance sampling steps\"?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-20"}