SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Jing Dong
The Chinese University of Hong Kong, Shenzhen jingdong@link.cuhk.edu.cn
Yuichi Yoshida
National Institute of Informatics yyoshida@nii.ac.jp
We introduce a transformation framework that can be utilized to develop online algorithms with low  $\epsilon$ -approximate regret in the random-order model from offline approximation algorithms. We first give a general reduction theorem that transforms an offline approximation algorithm with low average sensitivity to an online algorithm with low  $\epsilon$ -approximate regret. We then demonstrate that offline approximation algorithms can be transformed into a low-sensitivity version using a coreset construction method. To showcase the versatility of our approach, we apply it to various problems, including online  $(k,z)$ -clustering, online matrix approximation, and online regression, and successfully achieve polylogarithmic  $\epsilon$ -approximate regret for each problem. Moreover, we show that in all three cases, our algorithm also enjoys low inconsistency, which may be desired in some online applications.
In online learning literature, stochastic and adversarial settings are two of the most well-studied cases. Although the stochastic setting is not often satisfied in real applications, the performance and guarantees of online algorithms in the adversarial case are considerably compromised. This is particularly true for important online tasks such as  $k$ -means clustering, which gives a significantly worse guarantee than their offline or stochastic counterparts [Cohen-Addad et al., 2021]. As a result, their practical applicability is greatly limited.
Recently, the random-order model has been introduced as a means of modeling learning scenarios that fall between the stochastic and adversarial settings [Garber et al., 2020, Sherman et al., 2021]. In the random-order model, the adversary is permitted to choose the set of losses, with full knowledge of the learning algorithm, but has no influence over the order in which the losses are presented to the learner. Instead, the loss sequence is uniformly and randomly permuted. This effectively bridges the gap between the stochastic setting, where only the distribution of losses can be chosen by the setting, and the adversarial setting, where the adversary has complete control over the order of the losses presented to the learner.
In this work, we introduce a batch-to-online transformation framework designed specifically for the random-order model. Our framework facilitates the conversion of an offline approximation algorithm into an online learning algorithm with  $\epsilon$ -approximate regret guarantees. Our primary technical tool is average sensitivity, which was initially proposed by Varma and Yoshida [2021] to describe the algorithm's average-case sensitivity against input perturbations. We demonstrate that any offline approximation algorithm with low average sensitivity will result in a transformed online counterpart that has low  $\epsilon$ -approximate regret. To achieve small average sensitivity for offline algorithms, we leverage the idea of a coreset [Agarwal et al., 2005, Har-Peled and Mazumdar, 2004], which is a small but representative subset of a larger dataset that preserves important properties of the original data. We present a coreset construction method that attains low average sensitivity, and when combined with the approximation algorithm, yields an overall algorithm with low average sensitivity.
To showcase the practicality and versatility of our framework, we apply it to three popular online learning problems: online  $(k,z)$ -clustering, online matrix approximation, and online regression. In all three cases, our approach yields a polylogarithmic  $\epsilon$ -approximate regret. Furthermore, due to the low average sensitivity of our algorithms, they also enjoy low inconsistency, which is the cumulative number of times the solution changes. This additional property may prove useful in certain online settings. We note that this inconsistency has also been investigated in the classic online learning and multi-armed bandits literature [Agrawal et al., 1988, Cesa-Bianchi et al., 2013].
Average sensitivity Varma and Yoshida [2021] first introduced the notion of average sensitivity and proposed algorithms with low average sensitivity on graph problems such as minimum spanning tree, minimum cut, and minimum vertex cover problems. Various other problems have then been analyzed for the average sensitivity, including dynamic programming problems [Kumabe and Yoshida, 2022], spectral clustering [Peng and Yoshida, 2020], Euclidean  $k$ -clustering [Yoshida and Ito, 2022], maximum matching problems [Yoshida and Zhou, 2021], and decision tree learning problem [Hara and Yoshida, 2023].
Online (consistent)  $(k,z)$ -clustering While  $(k,z)$ -clustering, which includes  $k$ -means  $(z = 2)$  and  $k$ -median  $(z = 1)$  as its special cases, has been studied extensively from various perspectives such as combinatorial optimization and probabilistic modeling, it can be NP-hard to obtain the exact solution [Impagliazzo et al., 2001]. Thus most theoretical works have been focused on designing approximation algorithms. In the online setting, Li et al. [2018] proposed a Bayesian adaptive online clustering algorithm that enjoys a minimal sublinear regret. However, the algorithm is allowed to output more than  $k$  clusters. Without such assumption, Cohen-Addad et al. [2021] proposed the first algorithm that attains  $\epsilon$ -approximate regret of  $O(k\sqrt{d^3n}\log (\epsilon^{-1}dkn))$  for  $k$ -means clustering under adversarial setting.
On a separate vein, Lattanzi and Vassilvitskii [2017] proposed an online consistent  $(k,z)$ -clustering algorithm that produces a  $2^{O(z)}$ -approximate solution for the data points obtained so far at each step while maintaining an inconsistency bound of  $O(k^2\log^4 n)$ . This implies that their algorithm only updates the output  $O(k^2\log^4 n)$  many times. Then, Yoshida and Ito [2022] gave an online algorithm with approximation ratio  $(1 + \epsilon)$  and inconsistency  $\mathrm{poly}(d,k,2^z,\epsilon^{-1})\cdot \log n$  in the random-order model. We remark that the way how the losses are computed in Lattanzi and Vassilvitskii [2017], Yoshida and Ito [2022] is different from that of the online setting, which Cohen-Addad et al. [2021] and our paper considered.
Online convex optimization and online principle component analysis (PCA) under the random-order model The online random-order optimization was proposed in Garber et al. [2020], which established a bound of  $O(\log n)$  for smooth and strongly convex losses. This result is then improved by Sherman et al. [2021] while still requiring smooth and convex losses.
The techniques and results are then extended to online PCA with the random-order setting, for which a regret of  $O\left(\zeta^{-1}\sqrt{kn}\right)$  was established, where  $\zeta$  is an instance-dependent constant. This recovers the regret for online PCA in the stochastic setting [Warmuth and Kuzmin, 2008, Nie et al., 2016]. We remark that PCA can be viewed as a special case of matrix approximation, in which the matrix being approximated is the covariance matrix of the data, and we discuss the more general problem of matrix approximation in this paper.
3 Preliminaries
For a positive integer  $n$ , let  $[n]$  denote the set  $\{1, 2, \dots, n\}$ . For real values  $a, b \in \mathbb{R}$ ,  $a \in (1 \pm \epsilon)b$  is a shorthand for  $(1 - \epsilon)b \leq a \leq (1 + \epsilon)b$ .
3.1 Offline Learning
We consider a general class of learning problems. Let  $\mathcal{X}$  be the input space,  $\Theta$  be the parameter space, and  $\ell : \Theta \times \mathcal{X} \to \mathbb{R}_+$  be a loss function. For simplicity, we assume the loss is bounded, i.e.,
$\ell(\theta, x) \leq 1$ . Given a set of  $n$  data points  $X \in \mathcal{X}^n$ , we are asked to learn a parameter  $\theta \in \Theta$  that minimizes the objective value  $\ell(\theta, X) := \sum_{x \in X} \ell(\theta, x)$ . We call this problem the offline learning problem.
When the exact minimization of the loss function  $\ell$  is NP-hard or computationally demanding, one may only hope to obtain an approximate solution efficiently. Specifically, for  $\alpha > 0$ , we say a solution  $\theta \in \Theta$  is  $\alpha$ -approximate for  $X \in \mathcal{X}^n$  if  $\ell(\theta, X) \leq \alpha \cdot \min_{\tilde{\theta} \in \Theta} \ell(\tilde{\theta}, X)$ . The value  $\alpha$  is called the approximation ratio of the solution. We say a (possibly randomized) algorithm  $\mathcal{A}$  is  $\alpha$ -approximate if the expected approximation ratio of the output solution is at most  $\alpha$ .
3.2 Online Learning with Random-Order Model
In the online learning problem, instead of receiving all points at once, the data arrives sequentially throughout a time horizon  $n$ . Specifically, the data point comes one by one, where  $x_{t}$  comes at time  $t \in [n]$ . At the time  $t$ , using the collected data points  $X_{t-1} := (x_{1}, \ldots, x_{t-1})$ , we are asked to output a parameter  $\theta_{t} \in \Theta$ . Then we receive the data point  $x_{t}$  and incur a loss of  $\ell(\theta_{t}, x_{t})$ . In this work, we consider the random-order model, in which the data points  $x_{1}, \ldots, x_{n}$  may be chosen adversarially, but their ordering is randomly permuted before the algorithm runs.
To evaluate our performance, we use the notion of regret, which is the cumulative difference between our solution and the best solution in hindsight. In cases where obtaining the exact solution is hard, and one may only hope to obtain an approximate solution efficiently, we use the  $\epsilon$ -approximate regret.
Definition 3.1 ( $\epsilon$ -approximate regret for the random-order model). Given a (randomized) algorithm  $\mathcal{A}$  that outputs a sequence of parameters  $\theta_1, \ldots, \theta_n$  when given input  $x_1, \ldots, x_n$ . The  $\epsilon$ -approximate regret of  $\mathcal{A}$  for the random-order model is defined as
$$
\operatorname {R e g r e t} _ {\epsilon} (n) := \underset {\mathcal {A}, \{x _ {t} \}} {\mathbb {E}} \left[ \sum_ {t = 1} ^ {n} \ell (\theta_ {t}, x _ {t}) - (1 + \epsilon) \cdot \min  _ {\tilde {\theta} \in \Theta} \sum_ {t = 1} ^ {n} \ell (\tilde {\theta}, x _ {t}) \right].
$$
where the randomness is over the internal randomness of  $\mathcal{A}$  and the ordering of data points. When  $\epsilon = 0$ , we simply call it the regret.
In certain cases, online algorithms are required to maintain a good solution while minimizing inconsistency, which is quantified as the number of times the solution changes. This can be expressed formally as Inconsistency  $(n) = \mathbb{E}_{\mathcal{A},\{x_t\}}[\sum_{t=1}^{n-1}\mathbb{I}\{\theta_t \neq \theta_{t+1}\}]$ , where  $\mathbb{I}$  is the indicator function.
3.3 Average sensitivity
On a high level, the notion of average sensitivity describes the differences in the performance of a randomized algorithm with respect to input changes. This difference is captured by the total variation distance, which is defined below.
Definition 3.2. For a measurable space  $(\Omega, \mathcal{F})$  and probability measures  $P, Q$  defined on  $(\Omega, \mathcal{F})$ , The total variation distance between  $P$  and  $Q$  is defined as  $\mathrm{TV}(P, Q) \coloneqq \sup_{A \in \mathcal{F}} |P(A) - Q(A)|$ .
Equipped with this, the average sensitivity of a randomized algorithm is formally defined as the average total variation distance between the algorithm's output on two training data sets that differ by deleting one point randomly. For a dataset  $X = (x_{1},\ldots ,x_{n})\in \mathcal{X}^{n}$  and  $i\in [n]$ , let  $X^{(i)}$  denote the set  $(x_{1},\dots,x_{i - 1},x_{i + 1},\dots,x_{n})$  obtained by deleting the  $i$ -th data point. Then, the following definition gives a detailed description of the notion:
Definition 3.3 (Average Sensitivity [Varma and Yoshida, 2021, Yoshida and Ito, 2022]). Let  $\mathcal{A}$  be a (randomized) algorithm that takes an input  $X\in \mathcal{X}^n$  and outputs  $\mathcal{A}(X)$ . For  $\beta :\mathbb{Z}_{+}\rightarrow \mathbb{R}_{+}$ , we say that the average sensitivity of  $\mathcal{A}$  is at most  $\beta$  if
$$
\frac {1}{n} \sum_ {i = 1} ^ {n} \operatorname {T V} (\mathcal {A} (X), \mathcal {A} (X ^ {(i)})) \leq \beta (n),
$$
for any  $X \in \mathcal{X}^n$ , where we identify  $\mathcal{A}(X)$  and  $\mathcal{A}(X^{(i)})$  with their distributions.
4 Batch-to-Online Transformation in the Random-Order Model
In this section, we describe a general framework that can transform any offline  $(1 + \epsilon)$ -approximate algorithm into an online algorithm with low  $\epsilon$ -approximate regret. Our goal is to show the following.
Theorem 4.1. Let  $\mathcal{A}$  be a (randomized)  $(1 + \epsilon)$ -approximate algorithm for the offline learning algorithm with average sensitivity  $\beta : \mathbb{Z}_{+} \to \mathbb{R}_{+}$ . Then, there exists an online learning algorithm in the random-order model such that  $\mathrm{Regret}_{\epsilon}(n) = O\left(\sum_{t=1}^{n} \beta(t) + 1\right)$ .
Our method is described in Algorithm 1. Let  $\mathcal{A}$  be an approximation algorithm for the offline learning problem. Then, at each time step, based on the collected data  $X_{t - 1}$ , we simply output  $\theta_t = \mathcal{A}(X_{t - 1})$ .
Algorithm 1: General batch-to-online conversion
Input: Offline approximation algorithm 1 for  $t = 1,\dots ,n$  do 2  $\begin{array}{r}\big{\lfloor}\mathrm{Obtain}\theta_t\mathrm{by~runing}\mathcal{A}\mathrm{on}X_{t - 1}.\\ \big{\lfloor}\mathrm{Receive}x_t\mathrm{and}\ell (\theta_t,x_t). \end{array}$  3
To show that Algorithm 1 achieves a low approximate regret when  $\mathcal{A}$  has a low average sensitivity, the following lemma is useful.
Lemma 4.2. Let  $\mathcal{A}$  be a (randomized) algorithm for the offline learning problem with average sensitivity  $\beta : \mathbb{Z}_{+} \to \mathbb{R}_{+}$ . Then for any input  $X \in \mathcal{X}^{n}$ , we have
$$
\frac {1}{n} \sum_ {i = 1} ^ {n} \underset {\mathcal {A}} {\mathbb {E}} [ \ell (\mathcal {A} (X ^ {(i)}), x _ {i}) ] = \frac {1}{n} \sum_ {i = 1} ^ {n} \underset {\mathcal {A}} {\mathbb {E}} [ \ell (\mathcal {A} (X), x _ {i}) ] \pm \beta (n),
$$
where  $x = a\pm b$  means  $a - b\leq x\leq a + b$
Proof of Theorem 4.1. Consider Algorithm 1. For any  $t \in [n]$ , we have
$$
\begin{array}{l} \underset {\mathcal {A}, \{x _ {i} \}} {\mathbb {E}} \left[ \ell \left(\theta_ {t + 1}, x _ {t + 1}\right) - \frac {1}{t} \ell \left(\theta_ {t + 1}, X _ {t}\right) \right] = \underset {\mathcal {A}, \{x _ {i} \}} {\mathbb {E}} \left[ \frac {1}{t} \sum_ {i = 1} ^ {t} \left(\ell \left(\theta_ {t + 1}, x _ {t + 1}\right) - \ell \left(\theta_ {t + 1}, x _ {i}\right)\right) \right] \\ = \underset {\mathcal {A}, \{x _ {i} \}} {\mathbb {E}} \left[ \frac {1}{t} \sum_ {i = 1} ^ {t} \left(\ell \left(\mathcal {A} \left(X _ {t}\right), x _ {t + 1}\right) - \ell \left(\mathcal {A} \left(X _ {t}\right), x _ {i}\right)\right) \right] \\ \leq \underset {\mathcal {A}, \left\{x _ {i} \right\}} {\mathbb {E}} \left[ \frac {1}{t} \sum_ {i = 1} ^ {t} \left(\ell \left(\mathcal {A} \left(X _ {t}\right), x _ {t + 1}\right) - \ell \left(\mathcal {A} \left(X _ {t} ^ {(i)}\right), x _ {i}\right)\right) \right] + \beta (t) \tag {By Lemma 4.2} \\ = \underset {\mathcal {A}, \{x _ {i} \}} {\mathbb {E}} \left[ \frac {1}{t} \sum_ {i = 1} ^ {t} \left(\ell (\mathcal {A} (X _ {t}), x _ {t + 1}) - \ell (\mathcal {A} (X _ {t} ^ {(i)}), x _ {t + 1})\right) \right] + \beta (t) \\ \leq \underset {\mathcal {A}, \{x _ {i} \}} {\mathbb {E}} \left[ \frac {1}{t} \sum_ {i = 1} ^ {t} \mathrm {T V} (\mathcal {A} (X _ {t}), \mathcal {A} (X _ {t} ^ {(i)})) \right] + \beta (t) \leq 2 \beta (t), \\ \end{array}
$$
where the last equality follows by replacing  $x_{i}$  with  $x_{t + 1}$  in  $\ell(\mathcal{A}(X_t^{(i)}), x_i)$  because they have the same distribution, and the last inequality is by the average sensitivity of the algorithm.
Rearranging the terms, we have
$$
\underset {\mathcal {A}, \{x _ {i} \}} {\mathbb {E}} \left[ \ell \left(\theta_ {t + 1}, x _ {t + 1}\right) \right] \leq \underset {\mathcal {A}, \{x _ {i} \}} {\mathbb {E}} \left[ \frac {\ell \left(\theta_ {t + 1} , X _ {t}\right)}{t} \right] + 2 \beta (t) \leq \underset {\{x _ {i} \}} {\mathbb {E}} \left[ \frac {(1 + \epsilon) \mathrm {O P T} _ {t}}{t} \right] + 2 \beta (t),
$$
where  $\mathrm{OPT}_t\coloneqq \min_\theta \ell (\theta ,X_t)$  is the optimal value with respect to  $X_{t}$ , and the second inequality holds because the approximation ratio of  $\theta_{t + 1}$  is  $1 + \epsilon$  in expectation.
Taking summation over both sides, we have
$$
\underset {\mathcal {A}, \{x _ {i} \}} {\mathbb {E}} \left[ \sum_ {t = 1} ^ {n} \ell (\theta_ {t}, x _ {t}) \right] = \underset {\mathcal {A}, \{x _ {i} \}} {\mathbb {E}} \left[ \ell (\theta_ {1}, x _ {1}) \right] + \underset {\mathcal {A}, \{x _ {i} \}} {\mathbb {E}} \left[ \sum_ {t = 1} ^ {n - 1} \ell (\theta_ {t + 1}, x _ {t + 1}) \right]
$$
$$
\leq 1 + \underset {\{x _ {i} \}} {\mathbb {E}} \left[ \sum_ {t = 1} ^ {n - 1} \frac {(1 + \epsilon) \mathrm {O P T} _ {t}}{t} \right] + 2 \sum_ {t = 1} ^ {n - 1} \beta (t).
$$
Fix the ordering  $x_{1}, \ldots, x_{n}$ , and let  $c_{i}$  ( $i \in [t]$ ) be the loss incurred by  $x_{i}$  in  $\mathrm{OPT}_n$ . In particular, we have  $\mathrm{OPT}_n = \sum_{i=1}^{n} c_{i}$ . Note that  $c_{i}$ 's are random variables depending on the ordering of data points, but their sum,  $\mathrm{OPT}_n$ , is deterministic. Then, we have  $\mathrm{OPT}_t \leq \sum_{i=1}^{t} c_{i}$  because  $\mathrm{OPT}_t$  minimizes the loss up to time  $t$ . Hence, we have
$$
\begin{array}{l} \underset {\{x _ {i} \}} {\mathbb {E}} \left[ \sum_ {t = 1} ^ {n} \frac {\mathrm {O P T} _ {t}}{t} \right] \leq \underset {\{x _ {i} \}} {\mathbb {E}} \left[ \sum_ {t = 1} ^ {n} \frac {\sum_ {i = 1} ^ {t} c _ {i}}{t} \right] = \underset {\{x _ {i} \}} {\mathbb {E}} \left[ \sum_ {i = 1} ^ {n} c _ {i} \sum_ {t = i} ^ {n} \frac {1}{t} \right] = \sum_ {i = 1} ^ {n} \underset {\{x _ {i} \}} {\mathbb {E}} [ c _ {i} ] \sum_ {t = i} ^ {n} \frac {1}{t} \\ = \frac {\mathrm {O P T} _ {n}}{n} \cdot \sum_ {i = 1} ^ {n} \sum_ {t = i} ^ {n} \frac {1}{t} = \frac {\mathrm {O P T} _ {n}}{n} \cdot n = \mathrm {O P T} _ {n}. \\ \end{array}
$$
Therefore, we have
$$
\mathbb {E} _ {\mathcal {A}, \{x _ {i} \}} \left[ \sum_ {t = 1} ^ {n} \ell (\theta_ {t}, x _ {t}) \right] - (1 + \epsilon) \mathrm {O P T} _ {n} = O \left(\sum_ {t = 1} ^ {n} \beta (t) + 1\right).
$$
5 Approximation Algorithm with Low Average Sensitivity via Coreset
To design approximation algorithms for the offline learning problem with low average sensitivity, we consider the following approach: We first construct a small subset of the input that well preserves objective functions, called a coreset, with small average sensitivity, and then apply any known approximation algorithm on the coreset. Coreset is formally defined as follows:
Definition 5.1 (Har-Peled and Mazumdar [2004], Agarwal et al. [2005]). Let  $\ell : \Theta \times \mathcal{X} \to \mathbb{R}_+$  be a loss function and let  $X \in \mathcal{X}^n$ . For  $\epsilon > 0$ , we say that a weighted set  $(Y, w)$  with  $Y \subseteq X$  and  $w: Y \to \mathbb{R}_+$  is an  $\epsilon$ -coreset of  $X$  with respect to  $\ell$  if for any  $\theta \in \Theta$ , we have  $\sum_{y \in Y} w(y) \ell(\theta, y) \in (1 \pm \epsilon) \sum_{x \in X} \ell(\theta, x)$ .
Now, we consider a popular method for constructing coresets based on importance sampling and show that it enjoys a low average sensitivity. For a data  $x \in X$ , its sensitivity  $\sigma_X(x)^1$  is its maximum contribution to the loss of the whole dataset, or more formally
$$
\sigma_ {X} (x) = \sup  _ {\theta \in \Theta} \frac {\ell (\theta , x)}{\ell (\theta , X)}. \tag {1}
$$
Algorithm 2: Coreset Construction Based on Sensitivity Sampling
Input: Loss function  $\ell :\Theta \times \mathcal{X}\to \mathbb{R}_{+}$  dataset  $X\in \mathcal{X}^n$ $m\in \mathbb{N}$  , and  $\epsilon >0$  1 For each  $x\in X$  , compute  $\sigma_X(x)$  and set  $p(x) = \sigma_X(x) / \sum_{x'\in X}\sigma_X(x')$    
2 Let  $S$  be an empty set.   
3 for  $i = 1,\ldots ,m$  do   
4 Sample  $x$  with probability  $p(x)$    
5 Sample  $\tilde{p}$  from  $[p(x),(1 + \epsilon /2)p(x)]$  uniformly at random.   
6 if  $w(x)$  is undefined then   
7 S S U {x}.   
8 w(x) 1/..   
9 else   
10 w(x) 1 w(x) + 1/..   
11 return (S,w).
It is known that we can construct a coreset as follows: A data point  $x \in X$  is sampled with probability  $p(x) \coloneqq \sigma_X(x) / \sum_{x' \in X} \sigma_X(x')$ , and then its weight in the output coreset is increased by  $1 / \tilde{p}$ , where
1The reader should not confuse sensitivity, which is a measure for data points, with average sensitivity, which is a measure for algorithms.
$\tilde{p}$  is a slight perturbation of  $p(x)$ . This process is to be repeated for a fixed number of times, where the exact number depends on the approximation ratio of the coreset. See Algorithm 2 for details. We can bound its average sensitivity as follows:
Lemma 5.2. The average sensitivity of Algorithm 2 is  $O\left(\epsilon^{-1}m / n\right)$ .
A general bound on the number of times we need to repeat the process, i.e.,  $m$  in Algorithm 2, to obtain an  $\epsilon$ -coreset is known (see, e.g., Theorem 5.5 of Braverman et al. [2016]). However, we do not discuss it here because better bounds are known for specific problems and we do not use the general bound in the subsequent sections.
6 Online  $(k,z)$ -Clustering
In online applications, unlabelled data are abundant and their structure can be essential, and clustering serves as an important tool for analyzing them. In this section, as an application of our general batch-to-online transformation, we describe an online  $(k,z)$ -clustering method that enjoys low regret.
6.1 Problem setup
The online  $(k,z)$ -clustering problem [Cohen-Addad et al., 2021] is an instance of the general online learning problem described in Section 3. We describe the problem as follows: Let  $k\geq 1$  be an integer and  $z\geq 1$  be a real value. Over a time horizon  $n$ , at each time step  $t$ , a data point  $x_{t}\in \mathbb{R}^{d}$  is given. Using the set of data points  $X_{t - 1} = \{x_1,\ldots ,x_{t - 1}\}$ , we are asked to compute a set  $Z_{t} = \{z_{1},\dots ,z_{k}\}$  of  $k$  points in  $\mathbb{R}^d$  that minimize  $\ell (Z_t,x_t)\coloneqq \min_{j = 1,\dots,k}\| x_t - z_j\| _2^z$ , which is the  $z$ -th power of the Euclidean distance between  $x_{t}$  and the closest point in  $Z_{t}$ . Note that  $Z_{t}$  plays the role of  $\theta_{t}$  in the general online learning problem. The regret and  $\epsilon$ -approximate regret are defined accordingly.
6.2 Method and results
One important ingredient to our method is the coreset construction method proposed by Huang and Vishnoi [2020]. The method provides a unified two-stage importance sampling framework, which allows for a coreset with a size that is dimension independent. Specifically, the method constructs an  $\epsilon$ -coreset of size  $\tilde{O}\left(\min \left\{\varepsilon^{-2z - 2}k,2^{2z}\epsilon^{-4}k^2\right\}\right)$  in  $\tilde{O}(ndk)$  time, where the  $\tilde{O}$  hides polylogarithmic factors in  $n$  and  $k$ . We remark that the importance of sampling steps in the framework is similar to the ones described in Section 5, which thus allows us to analyze its average sensitivity.
Algorithm 3 gives a brief description of our algorithm, while a detailed description is presented in the appendix. The algorithm adheres to the standard transformation approach, whereby an offline approximation algorithm is run on the coreset derived from the aggregated data.