Title: Coresets for Clustering Under Stochastic Noise
Venue: NeurIPS 2025
Accepted: yes
Keywords: coreset

clustering

stochastic noise

error metrics
Abstract: We study the problem of constructing coresets for $(k, z)$-clustering when the input dataset is corrupted by stochastic noise drawn from a known distribution. In this setting, evaluating the quality of a coreset is inherently challenging, as the true underlying dataset is unobserved. To address this, we investigate coreset construction using surrogate error metrics that are tractable and provably related to the true clustering cost. We analyze a traditional metric from prior work and introduce a new error metric that more closely aligns with the true cost. Although our metric is defined independent of the noise distribution, it enables approximation guarantees that scale with the noise level. We design a coreset construction algorithm based on this metric and show that, under mild assumptions on the data and noise, enforcing an $\varepsilon$-bound under our metric yields smaller coresets and tighter guarantees on the true clustering cost than those obtained via classical metrics. In particular, we prove that the coreset size can improve by a factor of up to $\mathrm{poly}(k)$, where $n$ is the dataset size. Experiments on real-world datasets support our theoretical findings and demonstrate the practical advantages of our approach.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies coresets for k-means clustering when the input dataset is corrupted by random noise. Specifically, the paper models noise by random perturbation of each data point. It then presents a a surrogate error metric, which is basically an upper bound on the error of the coreset but relative to the unperturbed dataset. The main result is then 2 algorithms for constructing a coreset from the perturbed dataset. The crux is that the algorithm's accuracy/quality is measured wrt the unperturbed points, even though the algorithm cannot see them. The algorithms are analyzed theoretically, showing smaller coreset size, and they are validated via a few experiments.\n\nSTRENGTHS AND WEAKNESSES:\nI like the paper's research direction of studying noisy data instead of worst-case data. As a side comment, such models where the input is a combination of adversarial + random choices, are called semi-random inputs or smoothed analysis in related literature:\nDavid Arthur, Bodo Manthey, Heiko Röglin: Smoothed Analysis of the k-Means Method. J. ACM 58(5): 19:1-19:31 (2011)\nJonathan A. Kelner, Jerry Li, Allen Liu, Aaron Sidford, Kevin Tian: Semi-Random Matrix Completion via Flow-Based Adaptive Reweighting. NeurIPS 2024\n\nHowever, I find the paper's methodology is unconvincing. First, the main error measure $Err(S,P)$ is only an upper bound on the error (the paper calls it a surrogate), and I doubt it is the correct measure that we would like to minimize. Second, the two algorithms do not bring new insights or methods to the area, in fact the first one (theorem 3.1) simply says to apply any known coreset algorithm, and thus has no new component (the contribution is analysis of the error wrt the unperturbed dataset). The second algorithm (for theorem 3.3) is follows a rather standard recipe, basically computing an O(1)-approximate clustering and sampling from each cluster a carefully chosen number of points. I don't see how or why these algorithms are tailored for a perturbation model. Overall, I don't see a significant technical novelty in the algorithms, and the analysis might be involved but seems (from its overview) to use standard tools.\n\nSimilarly, the experiments do not convince me, primary because I am not fond of the error measure $Err(S,P)$, but also because they do  not evaluate the quality of the coreset directly, i.e., solving the clustering problem directly for the input $P$ vs solving for the coreset $S$ and using this center set. \n\nI have also many comments about specific issues in the paper, but they are minor. For example, line 217 demonstrates the applicability but works for a rather small $\\epsilon=1/poly(k)$ and large stability parameter $\\gamma$.  In addition, theorems 3.1 and 3.3 should probably include WHP somewhere in the statement. In addition, the $r_P$ measure makes a lot of sense for worst-case inputs, so why use them here also in the noisy setting? Finally, the bounds in the two theorems involve many terms and the big picture is lost. Finally, it seems that you use k-means++ experimentally both for the initial O(1)-approximate solution and as a heuristic for finding the optimal solution (because it's NP-hard), which might cause inconsistency in the experimental results (the algorithm \"starts\" at the optimal solution).\n\nQUESTIONS:\nnone\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-20"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nClustering is a fundamental machine learning task. For large datasets, coresets -- small, weighted subsets -- are used to approximate the clustering cost while reducing computational requirements. However, most coreset constructions assume noise-free data, which is rarely the case in practice. The paper addresses the construction of coresets for (k, z)-clustering (which includes k-means and k-median) when the observed dataset is corrupted by stochastic (random, additive) noise with a known distribution -- a common scenario in privacy, sensor data, and robustness applications.\n\nThe paper analyzes the traditional surrogate error metric (Err), used to measure coreset quality in noise-free settings, and demonstrates that it may significantly overestimate error when data is noisy due to uniform cost inflation. Next, the authors introduce a new approximation-ratio-based metric, which more closely aligns with the true (unobservable) clustering quality under noise. Then, they show under certain assumptions on the dataset, under certain noisy models, their proposed sampling procedure produces a smaller -- by poly(k) factor -- coreset. The authors complement their theoretical results with empirical evaluations.\n\nSTRENGTHS AND WEAKNESSES:\nCoreset construction in the presence of noise is highly relevant for practical machine learning yet underexplored. This paper fills an important gap. The theoretical guarantees under the two natural noise models provided in this paper are quite interesting. However, relying on certain assumptions about the dataset makes the results slightly weak, especially since such an assumption is necessary. Further, I am not sure whether the additive terms in Err and r_P (in Theorem 3.1) are unavoidable or not. Overall, I think this paper shows some nice, interesting results on coreset constructions for noisy datasets.\n\nQUESTIONS:\nCould you please comment on whether the additve factors on Err and r_P in Theorem 3.1 are unavoidable or not? Are those additive terms optimal? Are there any natural interpretations of those terms or do they appear solely because of technical reasons?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe core challenge addressed in this paper is the construction of coresets for $(k,z)$-clustering in the $d$-dimensional Euclidean space, where input points are perturbed by noise. The authors consider a main noise model where each point remains untouched with some probability, and with the complementary probability each of its coordinates is perturbed by noise drawn from a coordinate-specific centered and unit-variance distribution. They then extend this model to a setting where the distribution is allowed to have larger variance $\\sigma^2$, and to noise models where coordinate noises are no longer independent. This perturbation means the algorithm operates without ever seeing the true, original point set. As a result, the traditional error metric, which gauges how much a coreset solution's cost differs from that of the complete data (for every solution), might not be the most appropriate measure when noise is present. \n\nThis paper’s main results are the following:\n\n- First, they show that, in a noisy setting, the canonical error metric used to evaluate coreset quality falls short of being the right one, as it may significantly overestimate coreset error. They, thus, define a new distribution-independent quality measure that does not suffer from the brittleness of measuring absolute cost deviation.\n\n- The second result of this paper consists of showing that, with respect to the canonical error metric, a coreset of state-of-the-art size suffices to guarantee an error of $\\varepsilon$ plus an additive error polynomial in $\\frac{\\theta nd}{OPT_P}$.\n\n- The third and main result is an analog of the second with respect to the new error metric: It shows that, under a cost-stability assumption, the new metric allows for a (up to $poly(k)$) smaller coreset size, and tighter control on the approximation ratio. \n\nFor the second result, its analysis relies on composition bounds to estimate the clustering cost. This involves controlling the deviation of the clustering cost on the true point set from its estimate, using concentration and Bernstein's moment conditions.\n\nThe third result is substantially more challenging to obtain. For instance, the natural composition that used to hold with respect to the canonical metric, no longer holds here, and a composition to an inflated $(1+\\varepsilon)$ version has to be considered. The analysis of the third result first demonstrates that the designed coreset algorithm has a small error on the perturbed point set. Second, it establishes the error gap between the true point set and its noisy counterpart. The algorithm initially constructs a coreset in a noise-free setting by partitioning points into well-separated clusters with uniform sampling. This method is then extended to noisy environments using geometric properties ensured by cost-stability. Noise-induced challenges, such as increased cluster diameter and weakened closeness guarantees, are addressed through a \"cleaned up\" phase for noisy points. This process results in a negligible effect from removed points and achieves the desired error bound.\n\nSTRENGTHS AND WEAKNESSES:\nThis paper presents a well-structured extension of coreset construction for clustering problems, specifically when handling noisy input. The exposition is generally clear. I found the simple example comparing the noise-free and noise-aware metrics particularly effective; it plainly shows why the proposed surrogate measure is appropriate. The additional discussion in the appendix is also insightful, demonstrating that the standard error measure can be brittle even without noise.\n\nWhile the analysis for the second result is more standard, it plays a nice role by providing a benchmark against which the third result—which achieves sharper bounds—can be compared. For this third and main result, a key strength is the development of new techniques for the newly defined error metric, better suited for noisy environments. The \"Key Ideas\" paragraphs in the main body are effective in helping readers build intuition and grasp the problem. Despite its technical depth, the treatment maintains conceptual clarity.\n\nOverall, the contributions and presentation are solid. The paper introduces the noisy coreset construction in $d$-dimensional Euclidean space, illustrating the challenges it poses, and providing nice results on the coreset size for this setting.\n\nQUESTIONS:\n- Reading the paper, I was wondering whether the mergeability property of coresets would be preserved under the new error metric, and I am glad you present this as a challenge in the last section. Do you have some intuition on which kind of instances mergeability might cease to hold?\n\n- Your noise models are currently defined only for Euclidean space. Have you considered extending this framework to general metric spaces, for example, by allowing point perturbations within a ball of a certain radius, where the radius itself is drawn from a distance distribution?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper investigates how to adapt a coreset produced from a noise-corrupted dataset for the underlying clean data in $(k,z)$-clustering. It shows that the classical strong-coreset error $Err$ can become vacuous once noise is added. To address this issue, this paper then proposes a relaxed metric $Err_{\\alpha}$ that considers only center sets whose cost on the coreset is within an $\\alpha$-factor of the optimum. A separation construction illustrates that $Err_{\\alpha}$ can be exponentially smaller than $Err$. Under mild assumptions on cost-stability, balanced clusters and bounded radius, the authors prove that any standard sensitivity-sampling coreset built on the noisy data remains a good coreset for the clean data when evaluated with $\\mathrm{Err}_{\\alpha}$, maintaining an $\\tilde O(k\\log k/\\varepsilon)$ coreset size. Experiments on several public datasets with Gaussian noise confirm that the new metric allows 20–35 % smaller coresets while preserving clustering quality.\n\nSTRENGTHS AND WEAKNESSES:\nOne of the key strengths is the conceptual clarity and practical relevance of the new error notion, which focuses evaluation on solutions practitioners actually search for rather than on all possible centre sets. The theoretical analysis is careful, technically sound and matches empirical observations; the resulting algorithm is simple to implement and inherits the best-known size guarantees. Presentation is clear, with motivating examples, lucid proofs and reproducible experiments that cover multiple noise models and real data sets.\n\nThe main weaknesses stem from the assumptions behind the theory. Cost-stability, well separation and balance may be violated in highly skewed or overlapping clusters, and the noise model is i.i.d. additive with sub-exponential tails, leaving correlated or heavy-tailed settings unaddressed. In addition, the need for a constant-factor estimate of the clean optimal cost is glossed over.\n\nQUESTIONS:\nI would appreciate clarification on the following issues: \n\n1. How does the guarantee degrade when the cost-stability constant $\\gamma$ becomes small and clusters are less well separated?\n\n2. What is the practical impact if the estimate of the clean optimal cost is off by a factor of two or more?\n\nLIMITATIONS:\nSince this is mainly a theoretical result, there is no limitations and potential negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}