Title: Improved Algorithms for Clustering with Distance Oracles
Venue: NeurIPS 2025
Accepted: no
Keywords: k-means clustering

distance oracles

query complexity

k-means++

noisy distances
Abstract: Bateni et al. has recently introduced the {\em weak-strong distance oracle model} to study clustering problems in settings with limited distance information. Given query access to the strong-oracle and weak-oracle in the weak-strong oracle model, the authors design approximation algorithms for $k$-means and $k$-center clustering problems. In this work, we design algorithms with improved guarantees for $k$-means and $k$-center clustering problems in the weak-strong oracle model. The $k$-means++ algorithm is commonly used to solve $k$-means where complete distance information is available. One of the main contributions of this work is to show that $k$-means++ algorithm can be adapted to work in the weak-strong oracle model using only a small number of strong-oracle queries, which is the critical resource in this model. In particular, our $k$-means++ based algorithm gives a constant approximation for $k$-means and uses $O(k^2 \log^2{n})$ strong-oracle queries. This improves on the algorithm of Bateni et al. that uses $O(k^2 \log^4n \log^2 \log n)$ strong-oracle queries for a constant factor approximation of $k$-means. For the $k$-center problem, we give a simple {\em ball-carving} based $6(1 + \epsilon)$-approximation algorithm that uses $O(k^3 \log^2{n} \log{\frac{\log{n}}{\epsilon}})$ strong-oracle queries. This is an improvement over the $14(1 + \epsilon)$-approximation algorithm of Bateni et al. that uses $O(k^2 \log^4{n} \log^2{\frac{\log{n}}{\epsilon}})$ strong-oracle queries.  To show the effectiveness of our algorithms, we perform empirical evaluations on real-world datasets and show that our algorithms significantly outperform the algorithms of Bateni et al.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe submission proposed novel algorithms for k-center and k-means clustering in a model which aims at minimizing the amount of information required about the data points. In particular, the model considers both \"strong\" queries (which provide exact information but should be minimized), and \"weak\" queries (which provide potentially faulty information).\n\nSTRENGTHS AND WEAKNESSES:\nThe submission is well-written overall. Its contribution primarily lies on the foundational side, but supports its findings with experimental evaluations. I especially appreciated the inclusion of the \"comparison with known results\" and \"Technical overview\" subsections/paragraphs - these are often missing in NeurIPS submissions. Nevertheless, I do have some concerns about the submissions' presentation and contributions, and at present these are preventing me from more strongly suggesting acceptance.\n\n[Minor] The messaging regarding the main results is somewhat confusing. The Main Results section begins by discussing results for the Strong-oracle, but as far as I understand these are all direct observations and the non-trivial contributions all apply to the weak-strong oracle model. \n\n[Medium] The model assumes that weak queries have a probability of returning an entirely random value, and that these values are persistent (i.e., that repeating the same query will not provide different results). For the latter, I understand that without persistence one could repeat the query multiple times in order to circumvent the whole idea behind the model - but nevertheless, I wonder whether the assumption can be justified or motivated by real-world applications. For the former, I would have expected that the weak queries simply provide answers which have a probability distribution around the real hidden values... again, I understand that this is different, but there seems to be no justification for why the proposed model is better suited for at least some applications. \n\nNote that I am aware that the preceding work [6] made the same assumptions, but nevertheless the model should be justified here as well (especially since here the authors \"double down\" on these assumptions with their main contributions). Moreover, in my quick parse of [6] I didn't spot a careful justification of these assumptions there either.\n\n[Minor] The current explanation of k-means in subsection 1.1 does not make it clear that the k \"centers\" are points from the metric space and not elements of X. I suggest replacing \"centers\" on the 4th row of 1.1 with \"points in \\mathcal{X} (called \\emph{centers})\".\n\n[Suggestion] At the top of page 3, it would be nice to mention the constant in \"constant factor bi-criteria approximation\".\n\n[Minor] The wall of text on page 5 could be improved, e.g., by introducing a new paragraph at \"Let’s see whether the same argument applies...\"\n\n[Suggestion] The justification for why the proposed research should appear at ICLR could be strengthened by outlining that clustering of unknown/incomplete data has been studied by the machine learning research community in a variety of settings. For instance, there are several algorithmic-complexity papers on clustering incomplete data that could be mentioned:\n\n--Tomohiro Koana, Vincent Froese, Rolf Niedermeier: The complexity of binary matrix completion under diameter constraints. J. Comput. Syst. Sci. 132: 45-67 (2023)\n\n--Robert Ganian, Thekla Hamm, Viktoriia Korchemna, Karolina Okrasa, Kirill Simonov: The Complexity of k-Means Clustering when Little is Known. ICML 2022: 6960-6987\n\n--Eduard Eiben, Fedor V. Fomin, Petr A. Golovach, William Lochet, Fahad Panolan, Kirill Simonov:\nEPTAS for k-means Clustering of Affine Subspaces. SODA 2021: 2649-2659\n\n--Eduard Eiben, Robert Ganian, Iyad Kanj, Sebastian Ordyniak, Stefan Szeider: The Parameterized Complexity of Clustering Incomplete Data. AAAI 2021: 7296-7304\n\n[Minor] The formulation of Theorem 1.1 is too vague: it doesn't clarify what the impact of epsilon is (whether it affects the runtime, the approximation factor or both only becomes clear later). Overall, I'd suggest stating the result in  a way which is explicit about the runtime and factor, similarly as Theorem 1.2.\n\n[Comment] On the technical side, the results are non-trivial and of sufficient technical depth to justify presentation at NeurIPS. However, the presentation of the results and comparison to the direct predecessor [6] fails to mention that the improvements only target a rather specific (and small) subset of the scope of [6]. In this sense, the contributions can be seen as somewhat narrow and comparatively weaker (but perhaps still of sufficient interest to consider acceptance).\n\n[Major] The preceding work [6] introduces and considers strong point queries as well as strong distance queries. In fact, the main contributions of [6] seem to target strong point queries first and foremost - for instance, these are what the experiments in [6] focus on. The submission however does not mention this at all in the discussion and comparison to [6]; everything up to the experimental section focuses on strong distance queries, and then strong point queries are abruptly introduced as a side note when they needed in the experimental section. I am wondering:\n1) Do the results obtained here also yield improvements to the number of strong point queries required (Theorems 1 and 2 in [6]), or are the techniques exclusive to strong distance queries?\n2) How did you translate theoretical algorithms that consider strong distance queries to experimental evaluations that count the number of strong point queries, and is this explained in the submission?\n\nQUESTIONS:\nThe authors are welcome to respond to any of the points raised in the \"Strengths and Weaknesses\".\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-20"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper designs new algorithms for clustering problems in the weak-strong distance oracle model, where the distance between points is available only via a distance oracle that offers two types of queries: a strong query always returns the true distance, and a weak query that with probability $\\delta=1/3$ returns an arbitrarily corrupted value. This model describes scenarios where strong queries are computationally expensive, e.g. hard to apply or implement, hence the main measure of algorithmic efficiency is the number of strong queries. \n\nThis model was introduced in [6], which provided approximation algorithms for a few problems. The submission achieves improved approximation factor for two clustering problems, k-means and k-center, with slightly better efficiency, i.e., number of strong queries but also of weak queries. The new k-means algorithm is based on the well-known k-means++ algorithm, a very different approach from [6] which is based on Meyerson's online facility location. The algorithms' effectiveness is demonstrated also via an empirical evaluation using synthetic and real-life datasets.\n\nSTRENGTHS AND WEAKNESSES:\nThe main strength is the theoretical improvement in approximation factor and number of strong/weak queries. Moreover, the k-means is based on an algorithm that is famous also as practical heuristic, hence the submission tells us how to adapt this common heuristic to use two types of queries. \n\nAnother strength is the model itself, which is abstracts nicely having access to two different distance oracle. The model was proposed in prior work (not here), but it is very timely, as access to expensive models (deep networks) is becoming prevalent, and the strong-weak model offers and examines the tradeoff between cost and accuracy. \n\nThe main weakness I see is that the model is not fully justified e.g. what practical scenarios it models reasonably well, particularly for the true distance being returned with probability $1-\\delta$? It is more reasonably for a noisy physical measurement than for a deep network. Moreover, this is one of two models introduced in [6], and plausibly one that is the less relevant for deep network representation. \n\nA second weakness is that the empirical evaluation is not very convincing, roughly for the same issues. The experiments with synthetic datasets have limited value, because the SBM is random but has some structure, and because the experiments use a specific corruption pattern, which is not necessarily hard or adversarial, and the consequence is that table 2 reports various numbers but there is no lesson to be learned. In addition, I see improvement in comparison with [6], however I could not understand what is measured here and how it's related to the theoretical part (remark 4.1), perhaps it corresponds to the other model introduced in [6]. I am ignoring the experiments with real-life datasets, because they appear only in the appendix. \n\nOverall, I feel the strengths overweigh the weaknesses.\n\nQUESTIONS:\nnone\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-28"}

comments: SUMMARY:
The authors develop approximation algorithms for clustering problem in the strong and weak oracle model, where an oracle provides exact or noisy distances between the input points, respectively. Their theoretical results are complemented with an experimental evaluation on real-world datasets.

STRENGTHS AND WEAKNESSES:
Strenghts:
- the problems studied in the paper are interesting and relevant to the NeurIPS community
- some of the theoretical results are novel and interesting
- an experimental evaluation on real-world datasets is included in the paper

Weaknesses:
- some of the results show only a marginal improvement over previous ones (in particular they remove some factors which are polynomial in k and logn while introducing a dependency on 1/epsilon^6 )
- some of their claims appear not to be accurate (see below), as a result it is difficult to appreciate their novel technical contributions overall

Some other comments:
- the fact that k-means++ can be adapted to the weak-oracle model is interesting, however, to be fair there are well-known algorithms based on coreset for this problem
- the lower bound on the weak-oracle model is novel to best of my knowledge and perhaps it should be stressed more in the paper

QUESTIONS:
- the authors claim that one of the main contributions of their work is to remove the assumption that the aspect-ratio be polynomial which is needed in [6]. However, in appendix A of [6] the authors claim that this can be easily removed. Could the authors please clarify?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 2
DATE: 2025-06-30

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper provides new improved algorithms for solving $k$-means and $k$-center problems in the *weak-strong oracle model* recently introduced in a COLT 2024 paper by Bateni et al. In this weak-strong oracle model, the point embeddings in $\\mathbb{R}^d$ are not available directly but instead for any $2$ points $x,y$, there is an option to either do a *strong-oracle* query $SO(x,y)$ which returns the exact distance between the points $d(x,y)$, or do a *weak-oracle* query $WO(x,y)$ which returns the exact distance $d(x,y)$ with probability $1-\\delta$ or returns some arbitrary value with probability $\\delta$ for some fixed $\\delta \\in (0,1/2)$. This is an interesting model, especially for modern times in the era of LLMs, where it might be much cheaper to get noisy information from very cheap (or free) queries from less expensive models (like GPT-4o) while it might be extremely expensive for any single query to more high-quality models (like o1 pro). The submission provides algorithms with both, better theoretical approximation factors, and also better empirical performance than the corresponding algorithms in Bateni et al.\n\nSTRENGTHS AND WEAKNESSES:\nQuality: Overall the submission is technically sound. The theoretical claims are well-supported with proofs provided in the Appendix. I have gone over the algorithm descriptions along with the intuitions for the proofs given in the main paper and they seem to be sound. The experimental results are also well-documented and seem to be okay (but I have a qualm which I will mention in the questions section of the review).\n\nClarity: Overall clearly written paper. But I urge the authors to make an effort to chop up the huge wall-of-text paragraphs that are flowing in the paper in several sections like subsection 1.3 and section 3. I understand you have the 9-page constraint to which you have to stick, but in the current form note that potential readers (especially new grad students who might be unfamiliar with a lot of the arguments you are making) might find these huge walls-of-text very daunting to read.\n\nSignificance: Yes, the results of the paper are significant to the community in my opinion. This weak-strong oracle model which this paper is studying seems to be interesting and potentially useful so I believe there will be interest from other researchers.\n\nOriginality: The weak-strong oracle model is a very new model. Therefore it is unsurprising that not much work has been done on this model yet. The results in this paper seem to definitely be decent improvements over the original paper that proposed this model. Now whether these results are very original is a different question altogether. Majority of the arguments used in the paper seem to follow from the standard literature on clustering. One of the key contributions of this paper is in analyzing k-means++ in this model, which doesn't seem to be the approach used by the earlier paper...\n\nQUESTIONS:\nRegarding the experiments, one qualm I have is that since the entire current paper never discussed point queries in the theoretical portion, it seems unfair to impose them on the readers in the experimental section. I did not read Bateni et al. earlier so it was unnecessary to have the exact comparison using their reported values. You could have run your own experiments using the algorithms of Bateni et al. on the datasets to compute the number of strong-oracle queries which you have been discussing about through your entire paper.\n\nAlso, regarding the choice of constant probability bi-criteria bounds in the footsteps of Aggarwal et al 2009: There have been follow up works on getting expectation bounds for bi-criteria approximation by Wei in NeurIPS 2016 [1] and Makarychev et al in NeurIPS 2020 [2]. In general, I am sure you are aware of the several reasons why proving bounds on the expectation are the defacto choice in the approximation algorithms literature over proving constant probability bounds. Please go over these papers and at least make a comment at the end of your paper that your results could be improved to expectation bounds following in the footsteps of these papers.\n\n1) \"A Constant-Factor Bi-Criteria Approximation Guarantee for k-means++\" by Wei, NeurIPS 2016\n2) \"Improved Guarantees for k-means++ and k-means++ Parallel\" by Makarychev et al, NeurIPS 2020.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}