SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Theorem 3.3 (Informal). Let  $c \in (0,1)$ . Given black-box access to a  $(\hat{\beta},\hat{B})$ -approximation oracle for  $k$ -centers (cf. Problem 2.4) and under Assumption F.2, there is a  $\rho$ -replicable algorithm for statistical  $k$ -centers (cf. Problem 2.3), that outputs a  $(O(\beta +\hat{\beta}),O(B + \widehat{B} +(\beta +\hat{\beta} +1)c)\Delta)$ -approximate solution with probability at least 0.99. Moreover, it has sample complexity
$$
\tilde {O} \left(\frac {n ^ {2} k \left(1 / c\right) ^ {3 d}}{\rho^ {2} q ^ {2}}\right).
$$
Recall that there is a simple greedy 2-approximation for the sample  $k$ -center problem whose approximation ratio cannot be improved unless  $\mathrm{P} = \mathrm{NP}$  [Hochbaum and Shmoys, 1985].
We defer the discussion around  $k$ -centers to Appendix F. In particular, see Theorem F.8 in Appendix F for the formal statement of Theorem 3.3.
4 Overview of  $(k,p)$ -Clustering
In this section, we present our approach to the  $(k,p)$ -clustering problem. First, we replicably approximate the distribution with a finite set of points by extending the approach of Frahling and Sohler [2005] to the distributional setting. Then, we solve the combinatorial  $(k,p)$ -clustering problem on this coreset using an approximation oracle in a black-box manner. In the following subsections, we give a more detailed overview for each step of our approach. For the full proofs and technical details, we kindly refer the reader to Appendix D.4 - D.6. In summary:
1) Replicably build a variant of a quad tree [Finkel and Bentley, 1974] (cf. Section 4.2).
2) Replicably produce a weighted coreset using the quad tree (cf. Section 4.1).
3) Apply the optimization oracle for the combinatorial problem on the coreset.
For general norms, this approach leads to an exponential dependence on  $d$ . However, we are able to handle the case of Euclidean distances by extending existing dimensionality reduction techniques for sample Euclidean  $(k,p)$ -clustering [Makarychev et al., 2019] to the distributional case (cf. Section 5). Thus, for the widely used Euclidean norm, our algorithm has poly  $(d)$  sample complexity.
4.1 Coresets
Definition 4.1 ((Strong) Coresetsets). For a distribution  $\mathbb{P}$  with support  $\mathcal{X} \subseteq \mathcal{B}_d$  and  $\varepsilon \in (0,1)$ , a (strong)  $\varepsilon$ -coreset for  $\mathcal{X}$  is a distribution  $\mathbb{P}'$  on  $\mathcal{X}' \subseteq B_d$  which satisfies
$$
(1 - \varepsilon) \cdot \mathbb {E} _ {x \sim \mathbb {P}} \kappa (x, F) ^ {p} \leq \mathbb {E} _ {x ^ {\prime} \sim \mathbb {P} ^ {\prime}} [ \kappa (x ^ {\prime}, F) ^ {p} ] \leq (1 + \varepsilon) \cdot \mathbb {E} _ {x \sim \mathbb {P}} \kappa (x, F) ^ {p}
$$
for every set of centers  $F \subseteq \mathcal{B}_d$ ,  $|F| = k$ .
Essentially, coresets help us approximate the true cost on the distribution  $\mathbb{P}$  by considering another distribution  $\mathbb{P}'$  whose support  $\mathcal{X}'$  can be arbitrarily smaller than the support of  $\mathbb{P}$ .
Inspired by Frahling and Sohler [2005], the idea is to replicably consolidate our distribution  $\mathbb{P}$  through some mapping  $R: \mathcal{X} \to \mathcal{X}$  whose image has small cardinality  $|R(\mathcal{X})| < \infty$  so that for any set of centers  $F$ ,
$$
(1 - \varepsilon) \mathbb {E} _ {x} \kappa (x, F) ^ {p} \leq \mathbb {E} _ {x} \kappa (R (x), F) ^ {p} \leq (1 + \varepsilon) \mathbb {E} _ {x} \kappa (x, F) ^ {p},
$$
where  $\varepsilon \in (0,1)$  is some error parameter. In other words,  $(\mathbb{P}_R,R(\mathcal{X}))$  is an  $\varepsilon$ -coreset. Note that given the function  $R$ , we can replicably estimate the probability mass at each point in  $R(\mathcal{X})$  and then apply a weighted  $(k,p)$ -clustering algorithm.
4.2 Replicable Quad Tree
We now explain how to replicably obtain the mapping  $R: \mathcal{X} \to \mathcal{X}$  by building upon the work of Frahling and Sohler [2005]. The pseudocode of the approach is provided in Algorithm 4.1. While Frahling and Sohler [2005] present their algorithm using hierarchical grids, we take an alternative presentation using the quad tree [Finkel and Bentley, 1974], which could be of independent interest.
First, we recall the construction of a standard quad tree in dimension  $d$ . Suppose we have a set of  $n$  points in  $[-1/2, 1/2]^d$ . The quad tree is a tree whose nodes represent hypercubes containing points and can be built recursively as follows: The root represents the cell  $[-1/2, 1/2]^d$  and contains all points. If a node contains more than one point, we split the cell into disjoint subsets, equally sized cells. For each non-empty cell, we add it as a child of the current node and recurse into it. The recursion stops when a node contains only 1 point. In the distributional setting, the stopping criterion is either when the diameter of a node is less than some length or when the node contains less than some mass, where both quantities are a function of the depth of the node. See Algorithm 4.1.
A quad tree implicitly defines a function  $R: \mathcal{X} \to \mathcal{X}$  as follows. Given a point  $x \in \mathcal{X}$  and the root node of our tree, while the current node has a child, go to the child containing  $x$  if such a child exists, otherwise, go to any child. At a leaf node, output the center of the cell the leaf represents. Intuitively, the quad tree consolidates regions of the sample space into single points. The construction can be made replicable since the decision to continue the recursion or not is the only statistical operation and is essentially a heavy-hitters operations which can be performed in a replicable fasion.
Let  $\mathcal{G}_i$  denote the union of all  $2^{id}$  possible cells at the  $i$ -th level. We write  $\mathbb{P}_i$  to denote the discretized distribution to  $\mathcal{G}_i$ . In other words,  $\mathbb{P}_i = \mathbb{P}|_{\sigma(\mathcal{G}_i)}$  is the restriction of  $\mathbb{P}$  to the smallest  $\sigma$ -algebra containing  $\mathcal{G}_i$ . Moreover, we write  $\Lambda$  to denote a replicable estimate of OPT with relative error  $\varepsilon$ , say  $1/\beta(1+\varepsilon)$  OPT  $\leq \Lambda \leq (1+\varepsilon)$  OPT for some absolute constant  $\beta \geq 1$ . We demonstrate how to obtain such a replicable estimate in Appendix D.5.
Algorithm 4.1 Replicable Quad Tree
1: rQuadTree(distribution  $\mathbb{P}$ , accuracy  $\varepsilon$ , exponent  $p$ , replicability  $\rho$ , confidence  $\delta$ ):  
2: Init the node on the first level  $\mathcal{Z}[0] \gets \{[-1/2, 1/2]^d\}$ .  
3: for depth  $i \gets 1$ ;  $\mathcal{Z}[i - 1] \neq \emptyset$  AND  $(2^{-i + 1}\Delta)^p > \varepsilon \Lambda/5$ ;  $i \gets i + 1$  do  
4:  $\{\mathbb{P}_i$  is the discretized distribution over  $2^{id}$  cells on the  $i$ -th layer\}  
5:  $\{\gamma$  is a parameter to be determined later.\}  
6:  $\{t$  is an upper bound on the number of layers.\}  
7: Compute  $\mathcal{H} = \mathrm{rHeavyHitters}\left(\mathbb{P}_i, v = \frac{\gamma \cdot \Lambda}{2^{-pi}}, \frac{v}{2}, \frac{\rho}{t}, \frac{\delta}{t}\right)$   
8: for node  $Z \in \mathcal{Z}[i - 1]$  do  
9: for heavy hitter cells  $H \in \mathcal{H}$  such that  $H \subseteq Z$  do  
10: children $(Z) \gets \mathrm{children}(Z) \cup \{H\}$   
11:  $\mathcal{Z}[i] \gets \mathcal{Z}[i] \cup \{H\}$ .  
12: end for  
13: end for  
14: end for  
15: Output root node.
Our technique differs from that of Frahling and Sohler [2005] in at least three ways. Firstly, they performed their analysis for finite, uniform distributions with access to the entire distribution, while
our results hold assuming only sample access to a general bounded distribution<sup>5</sup>. Secondly, Frahling and Sohler [2005] bound the number of layers as a function of the cardinality of the support. For us, this necessitates the extra termination condition when the side lengths of our grids fall below a fraction of  $\Lambda$  as our distribution may have infinite support. Finally, Frahling and Sohler [2005] estimate OPT by enumerating powers of 2. This suffices for their setting since their distributions are discrete and bounded. However, we require a more nuanced approach (cf. Appendix D.5) as we do not have a lower bound for OPT. We tackle this by showing uniform convergence of the clustering solution costs, which we establish via metric entropy and Rademacher complexity (cf. Appendix C).
4.3 Putting it Together
Once we have produced the function  $R$  implicitly through a quad tree, there is still the matter of extracting a replicable solution from a finite distribution. We can accomplish this by replicably estimating the probability mass at each point of  $R(\mathcal{X})$  and solving an instance of the weighted sample  $k$ -medians ( $k$ -means). This leads to the following results. For details, see Appendix D.4 - D.6.
Theorem 4.2 (Theorem 3.1; Formal). Let  $\varepsilon, \rho \in (0,1)$  and  $\delta \in (0,\rho /3)$ . Given black-box access to a  $\beta$ -approximation oracle for weighted  $k$ -medians (cf. Problem 2.2), there is a  $\rho$ -replicable algorithm for statistical  $k$ -medians (cf. Problem 2.1) such that, with probability at least  $1 - \delta$ , it outputs a  $(1 + \varepsilon)\beta$ -approximation. Moreover, it has sample complexity
$$
\tilde {O} \left(\left(\frac {k ^ {2} d ^ {2}}{\varepsilon^ {1 2} \rho^ {6} \cdot \mathrm {O P T} ^ {1 2}} + \frac {k ^ {3} 2 ^ {1 8 d} \Delta^ {3 d + 3}}{\rho^ {2} \varepsilon^ {3 d + 5} \cdot \mathrm {O P T} ^ {3}}\right) \log \frac {1}{\delta}\right).
$$
Theorem 4.3 (Theorem 3.1; Formal). Given black-box access to a  $\beta$ -approximation oracle for weighted  $k$ -means (cf. Problem 2.2), there is a  $\rho$ -replicable algorithm for statistical  $k$ -means (cf. Problem 2.1) such that, with probability at least  $1 - \delta$ , it replicably outputs a  $(1 + \varepsilon)\beta$ -approximation. Moreover, it has sample complexity
$$
\tilde {O} \left(\left(\frac {k ^ {2} d ^ {2}}{\varepsilon^ {1 2} \rho^ {6} \cdot \mathrm {O P T} ^ {1 2}} + \frac {k ^ {3} 2 ^ {3 9 d} \Delta^ {3 d + 6}}{\rho^ {2} \varepsilon^ {6 d + 8} \cdot \mathrm {O P T} ^ {3}}\right) \log \frac {1}{\delta}\right).
$$
5 The Euclidean Metric, Dimensionality Reduction, and  $(k,p)$ -Clustering
In this section, we focus on the Euclidean metric  $\kappa(y,z) \coloneqq \|y - z\|_2$  and show how the sample complexity for Euclidean  $(k,p)$ -clustering can be made polynomial in the ambient dimension  $d$ . Note that results for dimensionality reduction exist for the combinatorial Euclidean  $(k,p)$ -clustering problem [Charikar and Waingarten, 2022, Makarychev et al., 2019]. However, these do not extend trivially to the distributional case. We remark that we require our dimensionality reduction maps to be data-oblivious since we are constrained by replicability requirements.
The cornerstone result in dimensionality reduction for Euclidean distance is the Johnson-Lindenstrauss lemma (cf. Theorem E.1), which states that there is a distribution over linear maps  $\pi_{d,m}$  from  $\mathbb{R}^d\to \mathbb{R}^m$  that approximately preserves the norm of any  $x\in \mathbb{R}^d$  with constant probability for some target dimension  $m$ . In Makarychev et al. [2019], the authors prove Theorem E.4, which roughly states that it suffices to take  $m = \tilde{O} (p^{4} / \varepsilon^{2}\log (1 / \delta))$  in order to preserve  $(k,p)$ -clustering costs when the domain is finite. Firstly, we extend Theorem E.4 to the distributional setting by implicitly approximating the distribution with a weighted  $\varepsilon$ -net. Then, we implicitly map the  $\varepsilon$ -net onto the low-dimensional space and solve the clustering problem there. An important complication we need to overcome is that this mapping preserves the costs that correspond to partitions of the data and not arbitrary solutions. Because of that, it is not clear how we can "lift" the solution from the low-dimensional space to the original space. Thus, instead of outputting  $k$  points that correspond to the centers of the clusters, our algorithm outputs a clustering function  $f:\mathcal{X}\rightarrow [k]$ , which takes as input a point  $x\in \mathcal{X}$  and returns the label of the cluster it belongs to. The replicability guarantees of the algorithm state that, with probability  $1 - \rho$ , it will output the same function across two executions. In Appendix E.4, we describe this function. We emphasize that for each  $x\in \mathcal{X}$ , the running time of  $f$  is polynomial in  $k,d,p,1 / \varepsilon ,\log (1 / \delta)$ . Essentially, this function maps  $x$  onto the low-dimensional
Our results also generalize to unbounded distributions with sufficiently small tails
Roughly speaking, a solution corresponds to a partition when the center of each cluster is its center of mass.
space using the same projection map  $\pi$  as our algorithm and then finds the nearest center of  $\pi(x)$  in the low-dimensional space. For full details, we refer the reader to Appendix E.3. We are now ready to state the result formally.
Theorem 5.1 (Theorem 3.2; Formal). Let  $\varepsilon, \rho \in (0,1)$  and  $\delta \in (0,\rho /3)$ . Given a  $\beta$ -approximation oracle for weighted Euclidean  $k$ -medians ( $k$ -means), there is a  $\rho$ -replicable algorithm that outputs a clustering function such that with probability at least  $1 - \delta$ , the cost of the partition is at most  $(1 + \varepsilon)\beta$  OPT. Moreover, the algorithm has sample complexity
$$
\tilde {O} \left(\mathrm {p o l y} \left(\frac {k d}{\rho \mathrm {O P T}}\right) \left(\frac {2 \sqrt {m}}{\varepsilon}\right) ^ {O (m)} \log \frac {1}{\delta}\right),
$$
where  $m = O\left(\frac{1}{\varepsilon^2}\log \frac{k}{\delta\varepsilon}\right)$
6 Running Time for  $(k,p)$ -Clustering
All of our algorithms terminate in  $O(\mathrm{poly}(n))$  time where  $n$  denotes the sample complexity. See Table A.1 for more detailed time complexity of each stage of our approach. Moreover, we make  $O\bigl (\log \bigl (\beta /\bigl (\varepsilon \mathrm{OPT}\bigr)\bigr)\bigr)$  calls to the  $\beta$ -approximation oracle for the combinatorial  $(k,p)$ -clustering problem within our OPT estimation subroutine and then one more call to the oracle in order to output a solution on the coreset.
7 Replicable  $k$ -Centers
Due to space limitations, we briefly sketch our approach for the  $k$ -centers problem and kindly refer the reader to Appendix F. As explained before, the assumptions we make in this setting state that there exists some "good" solution, so that when we draw  $n$  i.i.d. samples from  $\mathbb{P}$  we observe at least one sample from each cluster, with constant probability. We first take a fixed grid of side  $c$  in order to cover the unit-diameter ball. Then, we sample sufficiently many points from  $\mathbb{P}$ . Subsequently, we "round" all the points of the sample to the centers of the cells of the grid that they fall into and estimate the probability mass of every cell. In order to ensure replicability, we take a random threshold from a predefined interval and discard the points from all the cells whose mass falls below the threshold. Finally, we call the approximation oracle using the points that remain. Unlike the  $(k,p)$ -clustering problem (cf. Problem 2.1), to the best of our knowledge, there does not exist any dimensionality reduction techniques that apply to the  $k$ -centers problem. The main result is formally stated in Theorem F.8.
We now provide a practical demonstration<sup>7</sup> of the replicability of our approach on synthetic data in 2D. In Figure 8.1, we leverage the sklearnn [Pedregosa et al., 2011] implementation of the popular  $k$ -means++ algorithm for  $k = 3$  and compare the output across two executions on the two moons distribution. In the first experiment, we do not perform any preprocessing and run  $k$ -means++ as is, resulting in different centers across two executions. In the second experiment, we compute a replicable coreset for the two moons distribution before running  $k$ -means++ on the coreset. This leads to the same centers being outputted across the executions. Note that the computation for the coreset is performed independently for each execution, albeit with a shared random seed for the internal randomness. See also Figure 8.2 for the results of a similar experiment on a mixture of truncated Gaussian distributions.
<sup>7</sup>https://anonymous.4open.science/r/replicable_clustering_experiments-E380
Figure 8.1: The results of running vanilla vs replicable  $k$ -Means++ on the two moons distribution for  $k = 3$ .
Figure 8.2: The results of running vanilla vs replicable  $k$ -Means++ on a mixture of truncated Gaussians distributions for  $k = 3$ .
9 Conclusion & Future Work
In this work, we designed replicable algorithms with strong performance guarantees using black-box access to approximation oracles for their combinatorial counterparts. There are many follow-up research directions that this work can lead to. For instance, our coreset algorithm adapts the coreset algorithm of Frahling and Sohler [2005] by viewing their algorithm as a series of heavy hitter estimations that can be made replicable. it may be possible to interpret more recent approaches for coreset estimation as a series of statistical operations to be made replicable in order to get replicable algorithms in the statistical  $(k,p)$ -clustering setting [Hu et al., 2018]. It would also be interesting to examine the sensitivity of (replicable) clustering algorithms to the choice of parameters such as the choice of exponent in the cost function or the measure of similarity of the data. Another relevant direction is to explore sample complexity lower ounces for statistical clustering, where little is known even in the non-replicable setting.
Amin Karbasi acknowledges funding in direct support of this work from NSF (IIS-1845032), ONR (N00014-19-1-2406), and the AI Institute for Learning-Enabled Optimization at Scale (TILOS). Grigoris Velegkas is supported by TILOS, the Onassis Foundation, and the Bodossaki Foundation. Felix Zhou is supported by TILOS.
Konstantin Makarychev, Yury Makarychev, and Ilya Razenshteyn. Performance of johnson-lindenstrauss transform for k-means and k-medians clustering. In Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing, pages 1027-1038, 2019.
Vitali D Milman and Gideon Schechtman. Asymptotic Theory of Finite Dimensional Normed Spaced, volume 1200. Springer Berlin, 1986.
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011.
Joelle Pineau, Koustuv Sinha, Genevieve Fried, Rosemary Nan Ke, and Hugo Larochelle. Icrl reproducibility challenge 2019. ReScience C, 5(2):5, 2019.
Joelle Pineau, Philippe Vincent-Lamarre, Koustuv Sinha, Vincent Larivière, Alina Beygelzimer, Florence d'Alché Buc, Emily Fox, and Hugo Larochelle. Improving reproducibility in machine learning research: a report from the neurips 2019 reproducibility program. Journal of Machine Learning Research, 22, 2021.
Gilles Pisier. The volume of convex bodies and Banach space geometry, volume 94. Cambridge University Press, 1999.
Alexander Rakhlin and Andrea Caponnetto. Stability of  $k$ -means clustering. Advances in neural information processing systems, 19, 2006.
Christian Sohler and David P Woodruff. Strong coresets for k-median and subspace approximation: Goodbye dimension. In 2018 IEEE 59th Annual Symposium on Foundations of Computer Science (FOCS), pages 802-813. IEEE, 2018.
AW van der Vaart and Jon A Wellner. Weak convergence and empirical processes with applications to statistics. Journal of the Royal Statistical Society-Series A Statistics in Society, 160(3):596-608, 1997.
Ulrike Von Luxburg and Shai Ben-David. Towards a statistical theory of clustering. In *Pascal workshop on statistics and optimization of clustering*, pages 20-26. London, UK, 2005.
Ulrike Von Luxburg et al. Clustering stability: an overview. Foundations and Trends® in Machine Learning, 2(3):235-274, 2010.
Martin J Wainwright. High-dimensional statistics: A non-asymptotic viewpoint, volume 48. Cambridge University Press, 2019.
Yuichi Yoshida and Shinji Ito. Average sensitivity of euclidean k-clustering. In Advances in Neural Information Processing Systems, 2022.
A Tables
We present the running times required for our sub-routines in order to ensure each stage is  $\rho$ -replicable and succeeds with probability at least 0.99. The overall output is a constant ratio approximation for Euclidean  $k$ -medians ( $k$ -means).
Table A.1: Running Time Overview for Euclidean  $k$ -Medians ( $k$ -Means)
ALGORITHM	RUNNING TIME
OPT ESTIMATION	O(poly(kd/OPTρ))
DIMENSIONALITY REDUCTION	O(poly(d/OPT)(k/ρ)O(log log(k/ρ))
CORESET	O(poly(d/OPT)(k/ρ)O(log log(k/ρ))
PROBABILITY MASS ESTIMATION	O(poly(d/OPT)(k/ρ)O(log log(k/ρ))
B Useful Facts
Proposition B.1 (Bretagnolle-Huber-Carol Inequality; [Vaart and Wellner, 1997]). Suppose the random vector  $(Z^{(1)},\ldots ,Z^{(N)})$  is multinomially distributed with parameters  $(p^{(1)},\ldots ,p^{(N)})$  and  $n$ . Let  $\widehat{p}^{(j)}\coloneqq \frac{1}{n} Z^{(j)}$ . Then
$$
\mathbb {P} \left\{\sum_ {j = 1} ^ {N} \left| \widehat {p} ^ {(j)} - p ^ {(j)} \right| \geq 2 \varepsilon \right\} \leq 2 ^ {N} \exp \left(- 2 \varepsilon^ {2} n\right).
$$
In particular, for any  $\varepsilon, \rho \in (0,1)$ , sampling
$$
n \geq \frac {\ln \frac {1}{\rho} + N \ln 2}{2 \varepsilon^ {2}}
$$
points from a finite distribution implies that  $\sum_{j=1}^{N}|\hat{p}^{(j)} - p^{(j)}| < 2\varepsilon$  with probability at least  $1 - \rho$ . Remark B.2 (Weighted  $k$ -Means/Medians). We remark that if we have access to a  $\beta$ -approximation oracle for unweighted  $k$ -means/medians, we can implement a weighted one by considering multiple copies of the points. In particular, in our applications, we get weights that are polynomials in the parameters of concern so this will not affect the stated runtime guarantees.
C Uniform Convergence of  $(k,p)$ -Clustering Costs
We would like to estimate OPT for statistical  $k$ -medians and statistical  $k$ -means by solving the combinatorial problem on a sufficiently large sample size. However, while the convergence of cost for a particular set of centers is guaranteed by standard arguments, e.g. Chernoff bounds, we require the stronger statement that convergence holds simultaneously for all possible choices of centers.
Similar results can be found [Ben-David, 2007], with the limitation that centers are either chosen from the sample points, or chosen as centers of mass of the clusters they induce. While this suffices to achieve constant ratio approximations, we would like to choose our centers anywhere in  $\mathcal{B}_d$ . One reason for doing so is for replicability as we have no control over the location of samples from two independent executions so we need to output centers that are not overly reliant on its specific input. Another is for dimensionality reduction, which we will see later.
C.1 History of Uniform Laws of Large Numbers
Uniform laws of large numbers generalize convergence results for a finite number of random variables to possibly uncountable classes of random variables. The Rademacher complexity and related Gaussian complexity are canonical techniques in developing these laws and also have a lengthy history in the study of Banach spaces using probabilistic methods [Pisier, 1999, Milman and Schechtman,
1986, Ledoux and Talagrand, 1991]. Metric entropy, along with related notions of expressivity of various function classes, can be used to control the Rademacher complexity and are also central objects of study in the field of approximation theory [DeVore and Lorentz, 1993, Carl and Stephani, 1990].
C.2 Rademacher Complexity
Let  $\mathcal{F}$  denote a class of functions from  $\mathbb{R}^d\to \mathbb{R}$ . For any fixed collection of  $n$  points  $x_{1}^{n}:= (x_{1},\ldots ,x_{n})$ , we write
$$
f \left(x _ {1} ^ {n}\right) := \left(f \left(x _ {1}\right), \dots , f \left(x _ {n}\right)\right)
$$