Title: Accelerating data-driven algorithm selection for combinatorial partitioning problems
Venue: NeurIPS 2025
Accepted: yes
Keywords: data-driven algorithm selection

sub-sampling

clustering

max-cut

Goemans-williamson
Abstract: Data-driven algorithm selection is a powerful approach for choosing effective heuristics for computational problems. It operates by evaluating a set of candidate algorithms on a collection of representative training instances and selecting the one with the best empirical performance. However, running each algorithm on every training instance is computationally expensive, making scalability a central challenge. In practice, a common workaround is to evaluate algorithms on smaller proxy instances derived from the original inputs. However, this practice has remained largely ad hoc and lacked theoretical grounding. We provide the first theoretical foundations for this practice by formalizing the notion of size generalization: predicting an algorithm's performance on a large instance by evaluating it on a smaller, representative instance, subsampled from the original instance. We provide size generalization guarantees for three widely used clustering algorithms (single-linkage, k-means++, and Gonzalez's k-centers heuristic) and two canonical max-cut algorithms (Goemans-Williamson and Greedy). We characterize the subsample size sufficient to ensure that performance on the subsample reflects performance on the full instance, and our experiments support these findings.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces *size generalization*, the problem of predicting the accuracy on the full dataset given a subset of it. They then analyze this for several different classical tasks, such as $k$-center, $k$-means, single-linkage clustering and max-cut. They prove bounds on the sizes of the samples and the numbers of accesses to ground truth labels required to achieve the size generalization.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- I am honestly quite shocked that this direction does not have more literature on it. I am familiar with coresets for clustering and, separately, with a lot of topics in deep learning research. So when I saw the description in the introduction, I thought there must surely be significant work on this idea of predicting the accuracy on a full dataset using only a subset of it. After all, I know a similar thing is done for coresets (but different in that coresets attempt to find a sample which preserves the cost), and the field of deep learning is actively interested in analyzing which training run will land where (in a more empirical sense). However, after spending time on a literature review, I could not find papers discussing this specific task of size generalization done in a theoretically rigorous way. This is quite surprising to me and implies that the paper is filling a nice gap in the literature. Unless I am mistaken and have missed something, I consider this a significant strength of the paper.\n- The resulting bounds feel consistent with what one might expect given familiarity with sublinear-time algorithms, sensitivity sampling, etc.\n- Some of the proof techniques are quite nice. I particulalry like the continuous extension of k-center and the analysis of the minimax distance with respect to sampling.\n\nWeaknesses:\n- I found this paper quite difficult to read and wish the results were presented differently. As it stands, this paper feels like three papers: size generalization for center-based clustering, size generalization for single-linkage and size generalization for max-cut. These results are presented by giving the proof techniques specific to each problem case. However, I find myself wanting an understanding of the general themes inherent to the task of size generalization. That is, I cannot see much overlap at all between the methods employed by the proof sketches in, for example, Theorems 2.2, 2.7 and Theorem 3.4. Even if these are, in fact, quite separate approaches, it would be good to have an overview somewhere after the introduction which describes the general techniques one might expect and how to approach a size-generalization problem. How does this relate to PAC-learning techniques? Are there techniques which can consistently be applied to size generalization and, if not, why not?\n- While I strongly commend the general topic of size generalization (as discussed in the strengths), I find the chosen motivation for studying these specific algorithms... detached from the state of the field. That is to say, I do not actually see the primary use case for size-generalization to be in combinatorial algorithms or clustering: there is usually sufficient compute to run k-means on any size dataset required.\n  - Instead, it feels much more relevant to me to study these ideas on neural networks, where training runs are *significantly* more expensive and where the empirical utility is much higher. Indeed, at least half of the list of empirical use cases for size-generalization listed on lines 43-46 are about neural network convergence. This is not counting the entire field of scaling laws which has seen huge growth in the last 3-4 years and is oriented *precisely* around the topic of size-generalization.\n  - Complete nitpick: as a person who has dipped toes into both TCS and the more empirical side of machine learning, I find it frustrating when TCS paper introductions motivate algorithms by saying they are used for tasks when, in reality, they are no longer used for that task. For example, clustering is never used for image segmentation anymore and the reference to chip design is from 1988.\n\nIn summary, I consider the task of studying size-generalization a very relevant one and the paper's results feel clean. However, it seems that the utility of the topic would be better suited to other algorithms and the paper did not provide blueprints which would explain which (if any) of the proof techniques can be extended to the topic of size-generalization more broadly. Of course, I am not requesting that the authors prove additional size-generalization bounds for new optimization problems -- I am moreso explaining why the clarity and significance scores are not higher given my praise for this paper's strengths.\n\nQUESTIONS:\nMy primary question is regarding the similarities and differences between the proof techniques for the various optimization problems. Are there common themes that can be extracted and applied more generally? If not, why not?\n\nLIMITATIONS:\nI think a formal limitations discussion would be helpful. The current discussed limitations are oriented around technical details of the proof techniques. However, the paper's primary limitation seems to actually be that size-generalization is most actively studied in the deep learning community right now (with scaling laws being the most extreme example) but the paper does not interface with that literature at all. Which is perfectly fine! But worth mentioning.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 4, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work introduces the concept of *size generalization*, which means predicting an algorithm’s performance on a large instance by evaluating it on a smaller subsample drawn from that instance. This work establishes size generalization bounds when using smaller instances—sampled either uniformly or randomly from the original data—for three widely used clustering algorithms (single-linkage, k-means++, and Gonzalez’s k-centers heuristic) as well as two max-cut algorithms (Goemans-Williamson and Greedy).\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n* This work introduces the notion of *size generalization*, which has practical significance in many scenarios.\n* It also provides size generalization bounds when using smaller instances—sampled either uniformly or randomly from the original data—for three widely used clustering algorithms and two max-cut algorithms.\n\n## Weaknesses\n\n* The experimental section is overly simplistic for real‐world applications. In the clustering experiments, every instance is partitioned into k=2 clusters—an inappropriate choice given the complexity of the datasets. Moreover, even for large datasets, the authors fix the problem size at n=500, which contradicts their stated goal of addressing big-data scalability. Finally, the results do not demonstrate whether the proposed algorithms can reliably approximate the ground-truth cost when the sampled data size is very small (for example, sublinear in\nn).\n\n* The proposed method requires two “oracles” to run in a semi-supervised fashion: (1) a distance oracle that returns the distance between any two points, and (2) a ground-truth oracle that provides the true clustering or cut value for any subset—each assumed to answer in constant time. Such idealized oracles are rarely available in practical, real-world settings, calling into question the method’s feasibility.\n\n* The authors should compare their fast-selection framework against existing sublinear-time solutions for k-means, k-center, and Max-Cut—such as “Sublinear Time Approximate Clustering, Mishra et al, SODA'01” “Sublinear Algorithms for Greedy Selection in High Dimensions, Chen et al, UAI'22”, “Sublinear Algorithms for MAXCUT and Correlation Clustering, Bhaskara et al., ICALP'18”, and many other results in this \"sub-linear time algorithms\" area (two good surveys from Ronitt Rubinfeld & Asaf Shapira,  and  Artur Czumaj & Christian Sohler). Although these works optimize a different cost function, they directly address sublinear-time solution of the original problem; if one can compute an approximate solution in sublinear time, the need for a separate fast-selection step may be less compelling.\n\n* There appears to be an error in Fig. 1(a) that may cause confusion: the legends for the original clustering algorithm and the subsampled version seem to be swapped.\n\nQUESTIONS:\n* *Size generalization* aims to predict an algorithm’s performance on a large instance by evaluating it on a smaller subsample, thereby improving efficiency. I wonder why the timing performance of both the subsampled and the original algorithms was not reported in the experiments.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the central challenge of **scalability in data-driven algorithm selection**. Data-driven algorithm selection is a powerful approach that evaluates candidate algorithms on a representative set of training instances and empirically chooses the best-performing one. However, running each algorithm on all training instances is computationally expensive, making it impractical for large or numerous instances.\n\nTo resolve this computational bottleneck, it is common practice to **evaluate algorithms on smaller surrogate instances** derived from the original input, but this practice has **lacked theoretical justification** until now .\n\nThis research provides the first theoretical foundation for this practice and formalizes a new concept called **\"size generalization\"** . Size generalization refers to the question (Question 1.1) of whether \"the performance of an algorithm on large instances can be quickly estimated by evaluating it on smaller, representative instances subsampled from the original instance\" .\n\nThe **main contributions** of this paper are as follows:\n\n*   **Formalization and provision of a theoretical foundation for size generalization** :\n    *   It introduces the new concept of \"size generalization\" to address the computational bottleneck in data-driven algorithm selection .\n    *   It provides the first **formal basis and theoretical guarantees** for the concept that an algorithm's performance on large instances can be estimated by evaluating it on smaller, representative subsamples .\n    *   This offers a natural framework for efficiently selecting algorithms for a single large problem instance, even when training instances are unavailable .\n*   **Size generalization guarantees for key combinatorial optimization problems** :\n    *   For **clustering problems**, it provides size generalization guarantees for three widely used algorithms: **single linkage, k-means++, and Gonzalez's k-center heuristic** .\n        *   For center-based algorithms (smoothed variants of Gonzalez's k-center heuristic and k-means++), it shows that, under appropriate assumptions, the required sample size does not depend on the size of the original instance .\n        *   For single linkage, it characterizes the conditions under which its performance can be estimated from subsamples, despite its theoretical instability .\n    *   For the **Max-cut problem**, it analyzes size generalization guarantees for two representative algorithms: **Goemans-Williamson (GW) and Greedy** .\n        *   For the GW algorithm, it proves a convergence bound for the SDP objective value without assumptions on the graph structure, strengthening prior work . This demonstrates that GW's performance can be efficiently estimated using subgraphs .\n        *   For the Greedy algorithm, building on prior work, it identifies the rate at which the cut density of a subgraph converges to that of the full graph .\n*   **Characterization of sufficient subsample sizes** :\n    *   It characterizes the sufficient subsample size for the performance on a subsample to reflect the performance on the complete instance .\n*   **Experimental validation** :\n    *   Experiments in both clustering and max-cut problems validate that the best-performing algorithm on subsamples can effectively predict the best algorithm on the full dataset . This demonstrates the proposed theoretical insights .\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\nAddressing computational bottlenecks in data-driven algorithm selection: This paper tackles the primary challenge in data-driven algorithm selection, where running candidate algorithms on all training instances becomes computationally expensive. It introduces a new concept called \"size generalization\", demonstrating that an algorithm's performance on large instances can be efficiently estimated by evaluating it on smaller, representative instances. This significantly enhances the efficiency of algorithm selection.\n\nEstablishing a theoretical foundation for size generalization and applying it to a wide range of algorithms: This work provides the first formal theoretical justification for the common practice of evaluating algorithms on smaller proxy instances. Specifically, it presents size generalization guarantees for three widely used clustering algorithms: single linkage, k-means++, and Gonzalez's k-centers heuristic, as well as for two canonical Max-cut algorithms: Goemans-Williamson (GW) and Greedy. Furthermore, it strengthens prior work by providing convergence guarantees for the Goemans-Williamson SDP for Max-cut without imposing any assumptions on the graph structure.\n\nComprehensive empirical validation and reproducibility of theoretical findings: The theoretical discoveries in the paper are supported by extensive experiments using diverse datasets (MNIST, GM, Omniglot, noisy circles) and random graphs (Erdös-Réyni, Barbell, random geometric, Barabási-Albert). Experimental results show that as sample size increases, the accuracy or cut density of proxy algorithms converges to the performance of the original algorithms, highlighting the alignment between theory and practice. Additionally, code to reproduce the main experimental results is provided, ensuring the transparency and reproducibility of the research.\n\nWeaknesses\nHigh sample complexity or theoretical instability in some algorithms:\n\nGonzalez's k-centers heuristic: Directly applying the framework to this algorithm poses a challenge due to its deterministic nature leading to high sample complexity (ϑk,f(X) = n). While a smoothed variant (SoftmaxCenters) is introduced to address this, it creates a trade-off with the algorithm's strict approach.\n\nSingle linkage: This algorithm is known to be theoretically unstable, and its sample complexity for size generalization (Theorem 2.7) depends on factors such as cluster separability (ϑk,SL(X)) and minimum cluster size, potentially requiring large subsamples when these values are high.\n\nLimitations regarding algorithm-specific assumptions:\n\nMax-cut Greedy algorithm: The analysis of the Greedy algorithm is limited to unweighted graphs.\n\nGoemans-Williamson (GW) algorithm: While the SDP objective value converges, the distribution of GW cut values after random rounding can differ even with different optimal SDP solutions, indicating non-Lipschitz behavior between the SDP solution and the GW cut value. This implies that estimating the cut value itself, in addition to the SDP value, involves both multiplicative and additive errors.\n\nLimited scope of combinatorial optimization problems addressed: This paper focuses on two major combinatorial optimization problems: clustering and Max-cut. While it establishes a powerful framework for size generalization, its extension to other combinatorial optimization problems such as integer programming and dynamic programming is mentioned as \"interesting future directions\", implying that its current applicability is limited.\n\nQUESTIONS:\nThe problem awareness and approach of this paper were highly intriguing. As pointed out in the third weakness, while the paper focuses on two major combinatorial optimization problems—clustering and Max-cut—it does not seem to provide a concrete direction for extending the framework to other combinatorial optimization problems such as integer programming or dynamic programming. Is that a correct understanding?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces and tackles the task of \"size generalization\", which is estimating algorithms' performance on a large dataset by running it on a much smaller subset of instances. This task is meaningful because it provides a principled way to accelerate algorithm selection, which would be extremely costly if every candidate algorithm is run on every instance. Specifically, the authors provide size generalization guarantees for three clustering algorithms and two max-cut algorithms. \n\nTo model the generalization behaviour of k-means++ and Gonzalez’s k-centers heuristic, the paper builds an MCMC-based surrogate and proves a sample bound in terms of a smoothness parameter $\\varsigma_{k,f}$ that quantifies how sharply the seeding distribution concentrates. Notably, the sub-sample size needed to obtain the bound is independent of the total sample size. To analyse linkage-based clustering algorithms, the paper tracks how min–max path distances shift under subsampling and introduces a separation parameter $\\varsigma_{k,\\mathrm{SL}}$; size generalization holds when this remains moderate. For Goemans–Williamson max-cut, the paper compares the SDP objective on an induced subgraph to that on the full graph via a dual-pruning argument, then obtains accuracy guarantees. For the Greedy differencing heuristic, the paper casts the evolving cut weight as a martingale, applies concentration inequalities, and shows that sampling $t = O(\\varepsilon^{-2})$ vertices suffices to approximate the full-graph cut within $O(\\varepsilon)$. \n\nExperiments on four image or synthetic datasets and four random-graph families confirm that the proxy algorithms’ accuracies and cut densities rapidly approach those on the full instances, often allowing the preferred heuristic to be identified from a tiny fraction of the data.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The task of size generalization formalizes a indeed very practical question. In fact, most people in the field of machine learning test their models on small datasets, and implicitly assumes size generalization in most times. It is good to see efforts in providing theoretical supports of size generalization, first on combinatorial algorithmic tasks.\n\n2. The paper develops several set of tools for obtaining size generalization bounds, which can be inspiring to more combinatorial algorithms. However, I am curious whether these techniques are directly applicable to a larger set of problems and algorithms.\n\nWeaknesses\n\n1. All size generalization bounds in the paper rely on some data assumptions, such as the smoothness parameters $\\vartheta_{k,f}$ and $\\vartheta_{k,\\mathrm{SL}}$, yet the authors provide little guidance on how to estimate or upper bound these quantities before running experiments. This limitation prevents people from directly using the bounds to plan computation budgets in real world scenarios. For example, when the data condition is unknown, I wonder if there is a naive workaround with a probe and stop strategy: run the target algorithm on randomly selected subsets of size 1, 2, 4, 8, 16, \\dots, compare the objective values or algorithm rankings between consecutive subset sizes, and terminate when the performance curve stabilises within a predefined tolerance. If we can prove the objective converges before reaching certain size, then this empirical scheme can act as a pragmatic substitute for the theoretical sample complexity.\n\n2. In this paper, basically each algorithm needs a set of new analysis. This approach is intrinsically limited in scope, because extending size generalization results to more / unknown combinatorial algorithms will likely require a more general framework.\n\nQUESTIONS:\n- How applicable are the size generalization bounds and proving techniques on other combinatorial problems and algorithms? Is there a more general framework we can extract out of the separated bounds and proofs in this paper?\n\n- Can the bounds developed in this paper guide us in selecting sub-sample sizes in real-world scenario? Is there a good way to decide or guess the smoothing parameters or other data assumptions?\n\nLIMITATIONS:\nPlease see the weakness section.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}