Title: Localized Data Shapley: Accelerating Valuation for Nearest Neighbor Algorithms
Venue: NeurIPS 2025
Accepted: yes
Keywords: Data valuation

Shapley values

k-Nearest Neighbors

Clustering
Abstract: Data Shapley values provide a principled approach for quantifying the contribution of individual training examples to machine learning models. However, computing these values often requires computational complexity that is exponential in the data size, and this has led researchers to pursue efficient algorithms tailored to specific machine learning models. Building on the prior success of the Shapley valuation for $K$-nearest neighbor (KNN) models, in this paper, we introduce a localized data Shapley framework that significantly accelerates the valuation of data points. Our approach leverages the distance-based local structure in the data space to decompose the global valuation problem into smaller, localized computations. Our primary contribution is an efficient valuation algorithm for a threshold-based KNN variant and shows that it provides provable speedups over the baseline under mild assumptions. Extensive experiments on real-life datasets demonstrate that our methods achieve a substantial speedup compared to previous approaches.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a localized data Shapley framework to accelerate the valuation of data points for K-nearest neighbor (KNN) models. The authors leverage distance-based local structure to decompose the global valuation problem into smaller computations. They propose an efficient algorithm for threshold-based KNN that achieves provable speedups under perturbation resilience assumptions, with experiments showing up to 25x speedup compared to baseline methods.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n- Clear presentation and solid theoretical foundation. The paper is really well-written with a logical flow from preliminaries (Section 2) through the baseline approach (Section 3) to the proposed methods. The mathematical notation is consistent, and the recursive formula in Theorem 2 provides a nice foundation for understanding how Shapley values can be computed efficiently for KNN models.\n- Practical algorithm design with theoretical guarantees. The landmark selection using farthest-first traversal (FFT) is a smart choice that balances theoretical guarantees with practical efficiency. The algorithm gracefully handles the worst case.\n- Novel use of perturbation resilience for algorithm analysis. The connection between perturbation resilience (Definition 2) and the efficiency of landmark-based near neighbor search is creative. The authors cleverly show that 3-perturbation resilient instances allow Algorithm 1 to restrict computations within clusters, leading to the complexity in Theorem 4.\n- Impressive empirical performance gains. The experimental results in Table 1 are quite compelling - the proposed Algorithm 1 achieves speedups up to 25x across different datasets.\n\n### Weaknesses\n- Limited discussion on determining $k^\\*$. While the paper assumes knowledge of the optimal cluster number $k^\\*$ for the theoretical analysis, there is insufficient guidance on how to determine or estimate this value in practice. The authors mention that \"perturbation resilience condition is hard to verify in practice\" (line 353), but don't provide any heuristics or methods to estimate $k^\\*$ for real datasets, which limits the practical applicability of the theoretical guarantees.\n- Missing ablation study on $n_L$ in final experiments. Section 6.1 shows the effect of $n_L$ on the 2dplanes dataset (Figure 1a), but Section 6.2 only reports results with a fixed $n_L = 50$ for all datasets. Given that the theoretical speedup depends heavily on $n_L$ and the paper claims \"significant speedup even when $n_L$ is as large as $\\sqrt{n}$\" (line 293), it would strengthen the paper to show how performance varies with different $n_L$ values across all datasets in Table 1.\n- Restrictive threshold-based KNN variant. The main algorithm focuses on threshold-based KNN (TKNN), which has \"additional desirable privacy-friendly properties\" (line 114), but this variant is less commonly used than standard KNN. While the authors claim TKNN is \"more robust\" (line 112), they don't provide empirical evidence comparing the actual prediction performance of TKNN vs standard KNN, making it unclear whether users would want to adopt this variant just for faster Shapley computation.\n\nQUESTIONS:\n- Could you provide some practical guidance on estimating $k^\\*$ or choosing $n_L$ for a new dataset where the clustering structure is unknown?\n- In line 217, you mention that $\\tau_{D_{\\textrm{mark}}} \\le \\tau$ is \"easily achieved by continuing the iterative process of FFT.\" How many landmarks does this typically require in practice? Does this ever result in $n_L > k^\\*$?\n\nLIMITATIONS:\nSee weaknesses above.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a new technique to calculate the Data Shapley values for KNN models. The proposed method addresses the significant computational cost of existing KNN-Shapley when applied to large test sets. The evaluation shows some promising efficient improvements.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The technique proposed is novel, and appears to be an interesting addition to the line of work on the Shapley value for KNN models.\n2. The evaluation shows promising efficiency gains.\n\nWeaknesses: \n1. It is important to point out that the utility function underlying the Shapley value that the paper is focused on is a different utility function from the original KNN-Shapley value. Specifically, the paper considers the utility for some distance-weighted threshold-based KNN. The original paper of KNN-Shapely just considered the standard unweighted KNN. If one is allowed to change the utility function, then Wang et al. [1] considers the unweighted threshold-based KNN and the computation complexity of this method is only linear in both training and test size, getting rid of the log N factor. The paper should compare the computational efficiency with this method. \n2. On a related note, since the paper changes the utility function, it is crucial to examine whether the resulting Shapely value can still detect the data quality differences as effectively as KNN-Shapley and TKNN-Shapley [1].\n[1] Wang et al. Threshold KNN-Shapley: A Linear-Time and Privacy-Friendly Approach to Data Valuation\n3. The proposed algorithm requires to sort all data points in training and test sets  for each landmark point, which results in a complexity of O((N_train+N_test)log(N_train+N_test) N_landmark). If the test points are coming on the fly, this computation cost would be problematic as we need to re-do the sorting whenever a new test point arrives. Yet KNN-Shapley does not have this landmark building setup and circumvents the costs. The paper lacks some discussion on how this landmark building costs would affect the overall efficiency in different contexts.\n\nQUESTIONS:\n1. Compare with TKNN-Shapley in terms of compute efficiency.\n2. Evaluate the utility of proposed data values on popular downstream tasks.\n3. Discuss the online test query setting.\n\nI'm happy to raise my scores if the weaknesses are addressed.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper tackles the high cost of computing data Shapley values for KNN. \nIt introduces a “localized data Shapley” algorithm that limits valuation complexity to points within a small radius around each test example, found quickly via landmark-based clustering and a thresholded KNN search. The authors prove that, under mild perturbation-resilience assumptions, this strategy returns the exact Shapley values while cutting the runtime from quadratic to (nearly) linear in dataset size, and they show up to 25 times speed-ups on real datasets with up to a million points.\n\nSTRENGTHS AND WEAKNESSES:\nStrength\n- Authors addressed a well-motivated and increasingly relevant problem: scalable data valuation.\n- Introduced a novel algorithm combining threshold-based KNN with landmark-based clustering—both theoretically grounded and practically impactful.\n- Offered a provable runtime guarantees under realistic structural assumptions (perturbation resilience).\n- The empirical results showing up to 25× speedups over prior methods which was nice to have. However, I didn't have time to run their code, I might do that during the response period. \n\nWeaknesses \n- Some limitations in verifying perturbation resilience in practice are acknowledged, but not fully explored.\n- Algorithm may degrade in high-dimensional settings due to curse of dimensionality—this isn’t deeply discussed.\n- Slightly incremental in the sense that it builds heavily on prior KNN Shapley work, though the localization idea is novel. \n– While the authors provide strong theoretical contributions, the experimental section could benefit from deeper analysis and broader exploration of dataset characteristics and more complex dataset to better demonstrate practical impact. Since the paper is submitted under general ML (not theory), a more robust experimental design is expected.\n\nNot a weakness just a friendly suggestion: the citations currently start from [22] at the beginning of the paper, and it might be helpful to renumber them starting from [1]. This would make it easier to follow and track the references as we read through the paper.\n\nQUESTIONS:\n1. Table 1 suggests mixed results on CIFAR-10/EMNIST (512-D). Could you add a controlled sweep over dimensionality (e.g., random projections) to isolate the “curse of dimensionality” effect?\n2. Any practical heuristics for choosing τ in high dimensional spaces?\n3. The paper focuses on thresholded KNN; can the landmark-based trick extend to weighted or learned-metric KNN (cv. Wang & Jia ’24)? A short discussion or a negative result would help.\n4. How often does a real-world dataset satisfy the 3- (or 4-) perturbation-resilient k-center condition?\n5. Could you include an empirical proxy (e.g., ratio of inter- to intra-cluster distances) or a stress-test that shows the algorithm’s degradation when the assumption is violated?\n\nLIMITATIONS:\nAuthors noted some technical limits, for example:\n- Perturbation-resilience is hard to verify, but the authors justify it theoretically and show the method remains correct, falling back to baseline runtime if unmet.\n- They note efficiency may taper in high dimensions => showed through results on CIFAR-10/EMNIST (512-D) datasets.\n- They address τ and nL sensitivity with ablation plots and suggest using the “knee” point for tuning.\n\nIt would be nice to add a short section on how automated Shapley scores might exacerbate data-market inequities or enable privacy inference, and propose concrete safeguards (e.g., DP noise, audit trails).\nAlso discuss whether landmark clustering could leak sensitive structural info and how that risk can be mitigated in practice can show the depth of authors understanding of the potential impact of their works.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers the problem of efficiently computing Shapley values for a special class of functions. In general, Shapley values have $2^n$ terms, where $n$ is the number of Shapley values. However, for restricted classes of functions $v$, we can hope to compute the Shapley values more efficiently. Work by Jia et al. shows how to compute the Shapley value with respect to one test point for $k$ nearest neighbors in time $O(dn + n log n)$ where $d$ is the dimension of the points. However, scaling this calculation to $n$ test points would naively take $O(dnn + n^2 log n)$.\n\nIn this work, the authors build a \"landmark\" data structures that enables faster computation when the dataset is \"stable\" in a technical sense. They find experimentally that this method can perform better than prior works for computing the Shapley values.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n* *Very* nice presentation of the method. The main ideas were all clearly and accessibly discussed.\n\n* The main result that Shapley values can be more efficiently computed under a stability assumption is quite nice.\n\n* The theoretical analysis and guarantees is a nice application of \"perturbation resilience.\"\n\nWeaknesses:\n\n* The speed up seems somewhat marginal in practice? This could be an issue with the presentation, please see below in the question.\n\n* To me, perhaps because I don't work in this exact space, the application of this work is somewhat limited.\n\nQUESTIONS:\nIn table 1, each row corresponds to a different dataset. The baseline only has $\\tau/d=.05$ but Algorithm 1 has $\\tau/d =.05, .1, .2$. If you're going to vary $\\tau/d$, it seems fair to also vary it for the baseline approach. Can you please run these experiments?\n\nRegarding your discussion after Theorem 4, let's imagine $n_test \\approx n$, and let's assume that the partitions are roughly equally sized so that $s \\approx n / n_L$ (you could make this formal under the right distributional assumption with some log factors). Ignoring log factors, your bound is $O(n_L n d + n n/ n_L d)$. Optimizing over $n_L$, we would choose $n_L = \\sqrt{n}$, then your bound is $O(n^{1.5 d})$. Would you present your result in this way? I think it makes the guarantee more digestible (with some mild assumptions), and makes it clear that you're getting a reasonable speed up from $n^2 d$ to $n^{1.5} d$.\n\nCould you run experiments where you plot baseline vs algorithm 1 performance by size of dataset? It's hard to tell from the raw numbers how the *rate* of the difference is changing.\n\nIf a plot looks persuasive and you present your guarantee this way, I could change my opinion on the \"marginal\" impact of your work, and increase my score.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe article addresses the problem of computing Shapley values for data valuation in K-nearest neighbor (KNN) models. By introducing a landmark-based near-neighbor search, the authors reduce the search space and significantly improve the computational efficiency of Data Shapley for threshold-based KNN. They analyze the theoretical complexity of their algorithm under a perturbation resilience assumption and support their findings with empirical validation on both synthetic and real-world datasets. The results demonstrate that the proposed methodology offers substantial improvements over classical KNN-based Shapley value computations.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n- The article is clearly written and well-structured.\n- The addressed problem—efficiently computing data Shapley values for KNN models—is relevant.\n- The work is mathematically solid and theoretically grounded.\n- The authors analyze the theoretical complexity of their method and provide sufficient conditions under which it improves upon existing time bounds in the literature.\n- The proposed method shows strong empirical performance, outperforming the classical KNN Shapley approach.\n\nWeaknesses\n- The theoretical assumption on perturbation resilience is difficult to verify in practice.\n- The empirical evaluation compares the proposed method only to the classical baseline, without including other competitive data valuation methods. Moreover, comparisons are done only with respect to execution time.\n\nQUESTIONS:\n1. Are there known scenarios where perturbation resilience does not hold? If so, does your method still outperform the baseline in those cases?\n2. You report improvements in execution time, but do not present any accuracy comparisons. How accurate is your method in approximating the Shapley values compared to the baseline?\n3. How does your method perform against other state-of-the-art data valuation techniques—both in terms of computation time and accuracy?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}