SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We stress again that the point of this experiment is not claiming that the membership inference attack that we developed here is an optimal MI attack on data value scores; instead, this is a proof-of-concept for the claim that data value scores can leak private information about other data points in the dataset, and we instantiate a possible privacy attack that exploits this privacy risks. We believe it is interesting future work to improve the attack performance as well as explore other possible threat models.
τ = -0.9	τ = -0.8	τ = -0.7	τ = -0.6	τ = -0.5	τ = -0.4	τ = -0.3	τ = -0.2	τ = -0.1
2DPlanes	0.549	0.799	0.738	0.688	0.53	0.665	0.6	0.558	0.628
Phoneme	0.777	0.679	0.736	0.673	0.704	0.692	0.588	0.522	0.5
CPU	0.75	0.672	0.638	0.635	0.512	0.518	0.58	0.545	0.508
Fraud	0.752	0.529	0.577	0.594	0.558	0.715	0.678	0.7	0.645
Creditcard	0.55	0.501	0.664	0.685	0.59	0.56	0.597	0.552	0.52
Apsfail	0.506	0.529	0.608	0.574	0.558	0.507	0.569	0.571	0.526
Click	0.718	0.56	0.545	0.6	0.568	0.735	0.535	0.56	0.52
Wind	0.528	0.65	0.7	0.585	0.585	0.58	0.568	0.562	0.632
Pol	0.772	0.62	0.748	0.715	0.6	0.592	0.59	0.532	0.672
Table 4: Results of the AUROC of our MI attack on TKNN-Shapley.
τ = -0.9	τ = -0.8	τ = -0.7	τ = -0.6	τ = -0.5	τ = -0.4	τ = -0.3	τ = -0.2	τ = -0.1
2DPlanes	0.51	0.474	0.488	0.518	0.494	0.51	0.479	0.466	0.478
Phoneme	0.502	0.505	0.512	0.506	0.5	0.484	0.552	0.538	0.482
CPU	0.468	0.489	0.486	0.51	0.523	0.524	0.468	0.466	0.527
Fraud	0.479	0.474	0.492	0.513	0.48	0.484	0.485	0.484	0.578
Creditcard	0.518	0.495	0.511	0.522	0.501	0.483	0.448	0.521	0.535
Apsfail	0.484	0.476	0.471	0.492	0.486	0.5	0.493	0.434	0.454
Click	0.492	0.491	0.49	0.484	0.488	0.482	0.544	0.452	0.434
Wind	0.52	0.496	0.514	0.492	0.53	0.488	0.504	0.522	0.43
Pol	0.491	0.503	0.51	0.496	0.492	0.488	0.458	0.523	0.487
Table 5: Results of the AUROC of our MIA attack on DP-TKNN-Shapley  $(\varepsilon = 0.5)$
τ = -0.9	τ = -0.8	τ = -0.7	τ = -0.6	τ = -0.5	τ = -0.4	τ = -0.3	τ = -0.2	τ = -0.1
2DPlanes	0.5	0.523	0.513	0.534	0.518	0.492	0.484	0.445	0.556
Phoneme	0.507	0.529	0.524	0.472	0.476	0.488	0.543	0.512	0.47
CPU	0.492	0.487	0.504	0.485	0.473	0.489	0.453	0.49	0.508
Fraud	0.508	0.501	0.502	0.506	0.497	0.501	0.496	0.436	0.53
Creditcard	0.482	0.494	0.522	0.505	0.505	0.495	0.494	0.498	0.429
Apsfail	0.472	0.498	0.499	0.486	0.488	0.48	0.521	0.545	0.441
Click	0.502	0.529	0.501	0.508	0.518	0.496	0.545	0.47	0.456
Wind	0.533	0.539	0.478	0.504	0.493	0.498	0.529	0.499	0.419
Pol	0.498	0.513	0.509	0.51	0.482	0.492	0.489	0.52	0.528
Table 6: Results of the AUROC of our MIA attack on DP-TKNN-Shapley  $(\varepsilon = 1.0)$
B.3 Challenges in making KNN-Shapley being differentially private
In this section, we give more details of the inherent difficulties in making the KNN-Shapley  $\phi_{z_i}^{\mathrm{KNN}}(D_{-z_i})$  (i.e., Theorem 7) to be differentially private.
B.3.1 Large global sensitivity
We find it very challenging to tightly bound the global sensitivity of  $\phi_{z_i}^{\mathrm{KNN}}(D_{-z_i})$ . Moreover, we show that the global sensitivity of  $\phi_{z_i}^{\mathrm{KNN}}(D_{-z_i})$  can significantly exceed the magnitude of  $\phi_{z_i}^{\mathrm{KNN}}$  by showing a lower bound. Specifically, we prove that the global sensitivity bound is at least around  $O(1)$  by constructing a specific pair of neighboring datasets.
Theorem 9 (Lower Bound for the global sensitivity of  $\phi_z^{\mathrm{KNN}}(D_{-z}))$ . For a data point  $z = (x,y)$  and validation data point  $z^{\mathrm{(val)}} = (x^{\mathrm{(val)}},y^{\mathrm{(val)}})$ , denote the global sensitivity of  $\phi_z^{\mathrm{KNN}}(D_{-z};z^{\mathrm{(val)}})$  as
$\Delta (\phi_z^{KNN};z^{\mathrm{(val)}})\coloneqq \sup_{D_{-z}\sim D_{-z}'}\big|\phi_z^{KNN}(D_{-z};z^{\mathrm{(val)}}) - \phi_z^{KNN}(D_{-z}';z^{\mathrm{(val)}})\big|$ . We have
$$
\Delta \left(\phi_ {z} ^ {K N N}\right) \geq \frac {1}{2} \left(\mathbb {1} \left[ y = y ^ {(\text {v a l})} \right] - 1 / C\right)
$$
where  $C$  is the number of classes for the corresponding classification task.
Proof. The proof idea is to construct two neighboring datasets:  $D = \{z_2\}$ ,  $D' = \{z_1, z_2\}$  where  $z_1 = (x_1, y_1)$ ,  $z_2 = (x_2, y_2)$  are two data points and we let  $y_1 = y^{(\mathrm{val})}$ . Moreover, we let  $d(z_1, z^{(\mathrm{val})}) \leq d(z_2, z^{(\mathrm{val})})$ . From KNN-Shapley's formula in Theorem 7, we have
$$
\phi_ {z _ {2}} ^ {\mathrm {K N N}} (D) = \mathbb {1} \left[ y _ {2} = y ^ {(\text {v a l})} \right] - 1 / C
$$
and if  $K\geq 2$  , we have
$$
\phi_ {z _ {2}} ^ {\mathrm {K N N}} \left(D ^ {\prime}\right) = \frac {1}{4} \left(\mathbb {1} \left[ y _ {2} = y ^ {(\mathrm {v a l})} \right] - \mathbb {1} \left[ y _ {1} = y ^ {(\mathrm {v a l})} \right]\right) + \frac {1}{2} \left(\mathbb {1} \left[ y _ {2} = y ^ {(\mathrm {v a l})} \right] - 1 / C\right)
$$
, and if  $K = 1$ , we have
$$
\phi_ {z _ {2}} ^ {\mathrm {K N N}} \left(D ^ {\prime}\right) = \frac {1}{2} \left(\mathbb {1} \left[ y _ {2} = y ^ {(\text {v a l})} \right] - 1 / C\right)
$$
Since  $\mathbb{1}[y_1 = y^{(\mathrm{val})}] = 1$  (our condition), we have
$$
\left| \phi_ {z _ {2}} ^ {\mathrm {K N N}} (D) - \phi_ {z _ {2}} ^ {\mathrm {K N N}} (D ^ {\prime}) \right| \geq \frac {1}{2} \left(\mathbb {1} \left[ y _ {2} = y ^ {(\mathrm {v a l})} \right] - 1 / C\right)
$$
The above theorem tells us that the global sensitivity for KNN-Shapley is at the order of  $O(1)$ . On the other hand, we can see from the formula of KNN-Shapley in Theorem 7 that the magnitude  $\phi_z^{KNN}$  for many of the data points  $z$  is at the order of  $O(1 / N)$ . Hence, if we apply the Gaussian mechanism and add random noise proportional to the global sensitivity bound, the resulting privatized data value score could substantially deviate from its non-private counterpart, thereby compromising the utility of the privatized data value scores.
Tight Global Sensitivity for the older version of KNN-Shapley from [29]. As we said earlier, it is hard to bound the global sensitivity for  $\phi_z^{\mathrm{KNN}}$ . Moreover, even if we are able to bound the global sensitivity, the bound will highly likely be large compared with the magnitude of  $\phi_z^{\mathrm{KNN}}$ , as we can see from Theorem 9.
In order to find a reasonable baseline for comparing with DP-TKNN-Shapley, we consider the older version of the KNN-Shapley developed in [29], where we show that its global sensitivity can be tightly bounded (but still large).
Theorem 10 (Global sensitivity of  $\phi_z^{\mathrm{KNN - OL}}(D_{-z})$  from [29]). For a data point  $z = (x,y)$  and validation data point  $z^{(\mathrm{val})} = (x^{(\mathrm{val})},y^{(\mathrm{val})})$ , denote the global sensitivity of  $\phi_z^{\mathrm{KNN - OL}}(D_{-z};z^{(\mathrm{val})})$  as  $\Delta (\phi_z^{\mathrm{KNN - OL}};z^{(\mathrm{val})})\coloneqq \sup_{D_{-z}\sim D_{-z}'}\big|\phi_z^{\mathrm{KNN - OL}}(D_{-z};z^{(\mathrm{val})}) - \phi_z^{\mathrm{KNN - OL}}(D_{-z}'；z^{(\mathrm{val})})\big|$ . We have
$$
\Delta \left(\phi_ {z} ^ {K N N - O L D}; z ^ {(\mathrm {v a l})}\right) \leq \frac {1}{K (K + 1)}
$$
Proof. For any dataset  $D = D_{-z} \cup \{z\}$ , we sort the data points according to the distance to  $x^{(\mathrm{val})}$  and we denote  $z_{j}$  for the  $j$ th closest data point to  $z^{(\mathrm{val})}$ . WLOG, suppose  $z_{i} \coloneqq z$ . We first write out the non-recursive expression for the older KNN-Shapley  $\phi_{z_i}^{\mathrm{KNN - OLD}}$ :
$$
\phi_ {z _ {i}} ^ {\mathrm {K N N - O L D}} = \frac {\mathbb {1} [ y _ {N} = y _ {\mathrm {v a l}} ]}{\max (K , N)} + \sum_ {j = i} ^ {N - 1} \frac {\mathbb {1} [ y _ {j} = y _ {\mathrm {v a l}} ] - \mathbb {1} [ y _ {j + 1} = y _ {\mathrm {v a l}} ]}{K} \frac {\min (K , j)}{j}
$$
If  $N \leq K$ , then we have  $\phi_{z_i}^{\mathrm{KNN - OLD}} = \frac{\mathbb{1}[y_i = y^{(\mathrm{val})}]}{K}$  for all  $i$  (i.e., no privacy leakage in this case).
If  $N > K$ , there are two cases:
Case 1:  $i\geq K$  , then we have
$$
\phi_ {z _ {i}} ^ {\mathrm {K N N - O L D}} = \frac {\mathbb {1} [ y _ {i} = y ^ {(\mathrm {v a l})} ]}{i} - \sum_ {j = i + 1} ^ {N} \frac {1}{(j - 1) j} \mathbb {1} [ y _ {j} = y ^ {(\mathrm {v a l})} ]
$$
Hence, if we add/remove a data point  $z_{j} = (x_{j},y_{j})$  s.t.  $d(x_{j},x^{(\mathrm{val})})\geq d(x_{i},x^{(\mathrm{val})})$  , we have
$$
\left| \phi_ {z _ {i}} ^ {\mathrm {K N N - O L D}} \left(D _ {- z _ {i}}\right) - \phi_ {z _ {i}} ^ {\mathrm {K N N - O L D}} \left(D _ {- z _ {i}} ^ {\prime}\right) \right| \leq \frac {1}{(j - 1) j} \leq \frac {1}{K (K + 1)}
$$
If we add a data point  $z_{j} = (x_{j},y_{j})$  s.t.  $d(x_{j},x^{(\mathrm{val})}) < d(x_{i},x^{(\mathrm{val})})$ , we have
$$
\begin{array}{l} \left| \phi_ {z _ {i}} ^ {\text {K N N - O L D}} \left(D _ {- z _ {i}}\right) - \phi_ {z _ {i}} ^ {\text {K N N - O L D}} \left(D _ {- z _ {i}} ^ {\prime}\right) \right| \\ = \left| \left(\frac {\mathbb {1} [ y _ {i} = y ^ {(\mathrm {v a l})} ]}{i} - \sum_ {j = i + 1} ^ {N} \frac {1}{(j - 1) j} \mathbb {1} [ y _ {j} = y ^ {(\mathrm {v a l})} ]\right) - \left(\frac {\mathbb {1} [ y _ {i} = y ^ {(\mathrm {v a l})} ]}{i + 1} - \sum_ {j = i + 2} ^ {N} \frac {1}{(j - 1) j} \mathbb {1} [ y _ {j} = y ^ {(\mathrm {v a l})} ]\right) \right| \\ = \left| \left(\frac {1}{i} - \frac {1}{i + 1}\right) \mathbb {1} [ y _ {i} = y ^ {(\mathrm {v a l})} ] - \sum_ {j = i + 1} ^ {N} \left(\frac {1}{(j - 1) j} - \frac {1}{(j + 1) j}\right) \mathbb {1} [ y _ {j} = y ^ {(\mathrm {v a l})} ] \right| \\ \leq \frac {1}{i (i + 1)} \\ \leq \frac {1}{K (K + 1)} \\ \end{array}
$$
When  $i \geq K + 1$ , the sensitivity analysis for remove a data point  $z_{j} = (x_{j},y_{j})$  s.t.  $d(x_{j},x^{(\mathrm{val})}) < d(x_{i},x^{(\mathrm{val})})$  is similar to the analysis above where we also have
$$
\left| \phi_ {z _ {i}} ^ {\mathrm {K N N - O L D}} \left(D _ {- z _ {i}}\right) - \phi_ {z _ {i}} ^ {\mathrm {K N N - O L D}} \left(D _ {- z _ {i}} ^ {\prime}\right) \right| \leq \frac {1}{K (K + 1)}
$$
Case 2:  $i < K$ , then we have
$$
\phi_ {z _ {i}} ^ {\mathrm {K N N - O L D}} = \frac {\mathbb {1} [ y _ {i} = y ^ {(\mathrm {v a l})} ]}{K} - \sum_ {j = K + 1} ^ {N} \frac {1}{(j - 1) j} \mathbb {1} [ y _ {j} = y ^ {(\mathrm {v a l})} ]
$$
By a similar analysis, we can also show that
$$
\left| \phi_ {z _ {i}} ^ {\mathrm {K N N} - 0 \mathrm {L D}} \left(D _ {- z _ {i}}\right) - \phi_ {z _ {i}} ^ {\mathrm {K N N} - 0 \mathrm {L D}} \left(D _ {- z _ {i}} ^ {\prime}\right) \right| \leq \frac {1}{K (K + 1)}
$$
The only remaining case that we haven't discussed yet is when  $i = K$ , and we remove a data point  $z_{j} = (x_{j},y_{j})$  s.t.  $d(x_{j},x^{(\mathrm{val})}) < d(x_{i},x^{(\mathrm{val})})$ . In this case, we have
$$
\begin{array}{l} \left| \phi_ {z _ {i}} ^ {\text {K N N - O L D}} \left(D _ {- z _ {i}}\right) - \phi_ {z _ {i}} ^ {\text {K N N - O L D}} \left(D _ {- z _ {i}} ^ {\prime}\right) \right| \\ = \left| \left(\frac {\mathbb {1} [ y _ {i} = y ^ {(\text {v a l})} ]}{K} - \sum_ {j = K + 1} ^ {N} \frac {1}{(j - 1) j} \mathbb {1} [ y _ {j} = y ^ {(\text {v a l})} ]\right) - \left(\frac {\mathbb {1} [ y _ {i} = y ^ {(\text {v a l})} ]}{K} - \sum_ {j = K + 1} ^ {N} \frac {1}{(j - 1) j} \mathbb {1} [ y _ {j} = y ^ {(\text {v a l})} ]\right) \right| \\ = 0 \leq \frac {1}{K (K + 1)} \\ \end{array}
$$
We stress that the bound  $\frac{1}{K(K + 1)}$  is tight. For example, for the case where  $i = K$ , then if we add another data point  $z^{*} = (x^{*},y^{*})$  s.t.
$$
d \left(x _ {i}, x ^ {(\mathrm {v a l})}\right) = d \left(x _ {K}, x ^ {(\mathrm {v a l})}\right) \leq d \left(x ^ {*}, x ^ {(\mathrm {v a l})}\right) \leq d \left(x _ {K + 1}, x ^ {(\mathrm {v a l})}\right)
$$
the change of the value  $\phi_{z_i}^{\mathrm{KNN - OLD}}$  will be  $\frac{1}{K(K + 1)}$ .
B.3.2 Difficulty (computational challenge) in incorporating subsampling technique
"Privacy amplification by subsampling" [4] is a technique where the subsampling of a dataset amplifies the privacy guarantees due to the reduced probability of an individual's data being included. Being able to incorporate such a technique is often important for achieving a decent privacy-utility tradeoff. However, the recursive nature of KNN-Shapley computation makes it hard to incorporate the subsampling techniques.
Specifically, recall that in practical data valuation scenarios, it is often desirable to compute the data value scores for all of  $z_{i} \in D$ . To apply the subsampling technique, we first need to create a subsampled dataset and compute the KNN-Shapley for the target data point  $z_{i}$ , i.e., we need to compute  $\phi_z^{\mathrm{KNN}}(\mathsf{sample}(D_{-z}))$ . The subsampled dataset is usually constructed by sampling each data point independently with a probability  $q$  (this is usually referred to as Poisson subsampling [4]). If we view  $q$  as a constant, then the computation of  $\phi_z^{\mathrm{KNN}}(\mathsf{sample}(D_{-z}))$  requires a runtime of  $\widetilde{O}(N)$  for the computation of each  $\phi_{z_i}^{\mathrm{KNN}}$ . This results in a final runtime of  $\widetilde{O}(N^2)$  for the computation of all  $(\phi_{z_i}^{\mathrm{KNN}})_{z_i \in D}$ , which is a significant increase in computational demand compared to the non-private KNN-Shapley. The recursive nature of KNN-Shapley computation significantly complicates the attempt of improving the computational efficiency. That is, it is not clear how to reuse the subsampled dataset to compute the KNN-Shapley score for those that are not sampled. Therefore, it seems that an  $\widetilde{O}(N^2)$  runtime is necessary if we want to incorporate the subsampling technique.
B.4 Baseline for experiments in Section 6.2.1
DP-KNN-Shapley. Given such an upper bound for the global sensitivity of  $\phi_z^{\mathrm{KNN - OLD}}$ , we can use Gaussian mechanism (Theorem 4) to privatize  $\phi_z^{\mathrm{KNN - OLD}}$ , i.e., we compute  $\widehat{\phi}_z^{\mathrm{KNN - OLD}}(D_{-z}) \coloneqq \phi_z^{\mathrm{KNN - OLD}}(D_{-z}) + \mathcal{N}\left(0, \frac{1}{K(K + 1)}\right)$ . We note that such a bound is still not satisfactory as the magnitude  $\phi_z^{\mathrm{KNN - OLD}}$  for many of the data points  $z$  is at the order of  $O(1 / N)$ . Nevertheless, this is a reasonable baseline (if it is not the only one) that we can use for comparison in DP-related experiments.
DP-KNN-Shapley with subsampling. Despite the high computational cost associated with the incorporation of the subsampling technique, we still compare our approach with this computationally intensive baseline for completeness. More specifically, we compute  $\widehat{\phi}_{z}^{\mathrm{KNN - OLD}}(D_{-z})\coloneqq$ $\phi_z^{\mathrm{KNN - OLD}}(\mathrm{sample}(D_{-z})) + \mathcal{N}\left(0,\left(\frac{1}{K(K + 1)}\right)^2\right)$
C Details for TKNN-Shapley
C.1 Consistency Result for TKNN
For a data point  $x$  and a threshold  $\tau$ , we denote
$$
S _ {x, \tau} := \left\{x ^ {\prime} \mid d \left(x, x ^ {\prime}\right) \leq \tau \right\}
$$
the ball of radius  $\tau$  centered at  $x$ . Recall that the prediction rule of TKNN when given a training set  $D$  is
$$
m _ {D} (x; \tau) := \left\{ \begin{array}{l l} 0 & | \mathrm {N B} _ {x, \tau} (D) | = 0 \\ \frac {1}{| \mathrm {N B} _ {x , \tau} (D) |} \sum_ {x ^ {\prime} \in \mathrm {N B} _ {x, \tau} (D)} m (x ^ {\prime}) & | \mathrm {N B} _ {x, \tau} (D) | > 0 \end{array} \right.
$$
Theorem 11. Suppose  $m$  is the target function that is Lipschitz on  $\text{supp}(\mu)$  where  $\mu$  is the probability measure of data distribution. As  $n \to \infty$ , if  $\tau_n \to 0$  and  $n\mu(S_{x,\tau}) \to \infty$  for all  $x \in \text{supp}(\mu)$ , then
$$
\lim _ {n \to \infty} \mathbb {E} _ {x \sim \mu , D _ {n} \sim \mu^ {n}} \left[ (m _ {D _ {n}} (x) - m (x)) ^ {2} \right] = 0
$$
$$
\begin{array}{l} \mathbb {E} _ {x, D _ {n}} \left[ \left(m _ {D _ {n}} (x) - m (x)\right) ^ {2} \right] \\ = \mathbb {E} _ {x, D _ {n}} [ (m _ {D _ {n}} (x) - m (x)) ^ {2} | | \mathbb {N B} _ {x, \tau_ {n}} (D _ {n}) | > 0 ] \operatorname * {P r} _ {x, D _ {n}} [ | \mathbb {N B} _ {x, \tau_ {n}} (D _ {n}) | > 0 ] \\ + \mathbb {E} _ {x, D _ {n}} [ (0 - m (x)) ^ {2} | | \mathbb {N B} _ {x, \tau_ {n}} (D _ {n}) | = 0 ] \Pr_ {x, D _ {n}} [ | \mathbb {N B} _ {x, \tau_ {n}} (D _ {n}) | = 0 ] \\ \end{array}
$$
$$
\begin{array}{l} \lim  _ {n \rightarrow \infty} \Pr_ {x, D _ {n}} \left[ | \mathbb {N B} _ {x, \tau_ {n}} (D _ {n}) | = 0 \right] = \lim  _ {n \rightarrow \infty} \mathbb {E} _ {x} \left[ \Pr_ {D _ {n}} \left[ | \mathbb {N B} _ {x, \tau_ {n}} (D _ {n}) | = 0 \right]\right] \\ = \lim  _ {n \rightarrow \infty} \mathbb {E} _ {x} \left[\left(1 - \mu \left(S _ {x, \tau_ {n}}\right)\right) ^ {n} \right] \\ \leq \lim _ {n \to \infty} \mathbb {E} _ {x} \left[ \frac {1}{1 + n \mu (S _ {x , \tau_ {n}})} \right] \\ = 0 \\ \end{array}
$$
Suppose  $m$  has Lipschitz constant  $L$ , i.e., for any pair of data points  $x, x' \in \operatorname{supp}(\mu)$ , we have
$$
\left| m (x) - m \left(x ^ {\prime}\right) \right| \leq L \| x - x ^ {\prime} \|
$$
$$
\begin{array}{l} \mathbb {E} _ {x, D _ {n}} \left[ (m _ {D _ {n}} (x) - m (x)) ^ {2} | | \mathrm {N B} _ {x, \tau_ {n}} (D _ {n}) | > 0 \right] \\ = \mathbb {E} _ {x, D _ {n}} \left[ \left(\frac {1}{| \mathrm {N B} _ {x , \tau_ {n}} (D _ {n}) |} \sum_ {x ^ {\prime} \in \mathrm {N B} _ {x, \tau_ {n}} (D _ {n})} (m (x ^ {\prime}) - m (x))\right) ^ {2} | | \mathrm {N B} _ {x, \tau_ {n}} (D _ {n}) | > 0 \right] \\ \leq \mathbb {E} _ {x, D _ {n}} \left[ \frac {1}{| \mathrm {N B} _ {x , \tau_ {n}} (D _ {n}) |} \sum_ {x ^ {\prime} \in \mathrm {N B} _ {x, \tau_ {n}} (D _ {n})} (m (x ^ {\prime}) - m (x)) ^ {2} | | \mathrm {N B} _ {x, \tau_ {n}} (D _ {n}) | > 0 \right] \\ \leq \mathbb {E} _ {x, D _ {n}} \left[ \frac {1}{| \mathrm {N B} _ {x , \tau_ {n}} (D _ {n}) |} \sum_ {x ^ {\prime} \in \mathrm {N B} _ {x, \tau_ {n}} (D _ {n})} (m (x ^ {\prime}) - m (x)) ^ {2} | | \mathrm {N B} _ {x, \tau_ {n}} (D _ {n}) | > 0 \right] \\ \leq \mathbb {E} _ {x, D _ {n}} \left[ \frac {1}{| \mathbb {N B} _ {x , \tau_ {n}} (D _ {n}) |} \sum_ {x ^ {\prime} \in \mathbb {N B} _ {x, \tau_ {n}} (D _ {n})} L ^ {2} \| x - x ^ {\prime} \| ^ {2} | | \mathbb {N B} _ {x, \tau_ {n}} (D _ {n}) | > 0 \right] \\ \leq \mathbb {E} _ {x, D _ {n}} \left[ \frac {1}{| \mathrm {N B} _ {x , \tau_ {n}} (D _ {n}) |} \sum_ {x ^ {\prime} \in \mathrm {N B} _ {x, \tau_ {n}} (D _ {n})} L ^ {2} \tau_ {n} ^ {2} | | \mathrm {N B} _ {x, \tau_ {n}} (D _ {n}) | > 0 \right] \\ = L ^ {2} \tau_ {n} ^ {2} \\ \end{array}
$$
which  $\to 0$  as  $n\to \infty$
C.2 Proofs for TKNN-Shapley
Given a dataset  $S$  and a test data point  $(x^{\mathrm{(val)}},y^{\mathrm{(val)}})$ , let  $\mathbb{N}\mathbb{B}_{x^{\mathrm{(val)}},\tau}(S) := \{(x,y)|(x,y) \in S, d(x,x^{\mathrm{(val)}}) \leq \tau\}$  the nearest neighbor of  $x^{\mathrm{(val)}}$  that is within a pre-specified threshold  $\tau$ . Recall that in this case, the utility function of soft-label TKNN classifier becomes
$$
v (S; (x ^ {(\mathrm {v a l})}, y ^ {(\mathrm {v a l})})) := \left\{ \begin{array}{l l} 1 / C & | \mathbb {N B} _ {x ^ {(\mathrm {v a l})}, \tau} (S) | = 0 \\ \frac {1}{| \mathbb {N B} _ {x ^ {(\mathrm {v a l})} , \tau} (S) |} \sum_ {(x, y) \in \mathbb {N B} _ {x ^ {(\mathrm {v a l})}, \tau} (S)} \mathbb {1} [ y = y ^ {(\mathrm {v a l})} ] & | \mathbb {N B} _ {x ^ {(\mathrm {v a l})}, \tau} (S) | > 0 \end{array} \right.
$$
where  $C$  is the number of classes for the classification task, and  $1 / C$  is the random guess accuracy.
Semvalue. The class of data values that satisfy all the Shapley axioms except efficiency is called semvalues. It was originally studied in the field of economics and recently proposed to tackle the data valuation problem [39]. Unlike the Shapley value, semvalues are not unique. The following theorem by the seminal work of [13] shows that every semvalue of a player  $z$  (in our case the player is a data point) can be expressed as the weighted average of marginal contributions  $v(S \cup \{z\}) - v(S)$  across different subsets  $S \subseteq N \setminus \{z\}$ .
Theorem 12 (Representation of Semivalence [13]). A value function  $\phi_{\mathrm{semi}}$  is a semivalence, if and only if there exists a weight function  $w: [N] \to \mathbb{R}$  such that  $\sum_{k=1}^{N} \binom{N-1}{k-1} w(k) = N$  and the value function  $\phi$  can be expressed as follows:
$$
\phi_ {z _ {i}} \left(D _ {- z _ {i}}; v, w\right) := \frac {1}{N} \sum_ {k = 1} ^ {N} w (k) \sum_ {\substack {S \subseteq D _ {- z _ {i}}, \\ | S | = k - 1}} [ v (S \cup \{z \}) - v (S) ] \tag{6}
$$
Semivalues subsume both the Shapley value and the Banzhaf value with  $w_{\mathrm{shap}}(k) = \binom{N-1}{k-1}^{-1}$  and  $w_{\mathrm{banz}}(k) = \frac{N}{2^{N-1}}$ , respectively. In the following, we first discuss the general semvalue for soft-label TKNN classifier with a weight function  $w$ . We then plug in the specific weight function for the Shapley value.
Our goal is to derive  $\phi_{z_i}\left(D_{-z_i};v,w\right)$ , the semvalue of  $z_{i} = (x_{i},y_{i})$  with a weight function  $w$  when using soft-label TKNN classifier, i.e.,
$$
\phi_{z_{i}}\left(D_{-z_{i}};v,w\right):= \frac{1}{N}\sum_{k = 1}^{N}w(k)\sum_{\substack{S\subseteq D_{-z_{i}},\\ |S| = k - 1}}[v(S\cup \{z_{i}\}) - v(S)]
$$
Consider a validation data point  $(x^{(\mathrm{val})},y^{(\mathrm{val})})$ , and recall that  $v(S)\coloneqq v(S;(x^{(\mathrm{val})},y^{(\mathrm{val})}))$ . Denote
$$
\mathbf {c} _ {x ^ {(\mathrm {v a l})}, \tau} = \mathbf {c} _ {x ^ {(\mathrm {v a l})}, \tau} (D _ {- z _ {i}}) := 1 + | \mathbb {N B} _ {x ^ {(\mathrm {v a l})}, \tau} (D _ {- z _ {i}}) | \quad (1 + \# \text {n e i g h b o r s} x ^ {(\mathrm {v a l})} \text {i n} D _ {- z _ {i}})
$$
$$
\mathbf {c} _ {z (\text {v a l})}, \tau = \mathbf {c} _ {z (\text {v a l})}, \tau (D _ {- z _ {i}}) := \sum_ {(x, y) \in \mathbb {N B} _ {x (\text {v a l})}, \tau} \mathbb {1} [ y = y ^ {(\text {v a l})} ] \quad (\# \text {s a m e - l a b e l n e i g h b o r s i n} D _ {- z _ {i}})
$$
Case 1: if  $\| x_i - x^{(\mathrm{val})} \| > \tau$ , we have  $v(S \cup z_i) - v(S) = 0$  for any  $S$ , hence  $\phi_{z_i}(D_{-z_i}; v, w) = 0$ .
Case 2: if  $\| x_{i} - x^{(\mathrm{val})}\| \leq \tau$ , we have the following:
$$
v (S \cup z _ {i}) - v (S) =
$$
$$
\left\{ \begin{array}{l l} \frac {\mathbb {1} [ y _ {i} = y ^ {(\mathrm {v a l})} ]}{1 + | \mathbb {N B} _ {x ^ {(\mathrm {v a l}) , \tau}} (S) |} + \left(\frac {1}{1 + | \mathbb {N B} _ {x ^ {(\mathrm {v a l}) , \tau}} (S) |} - \frac {1}{| \mathbb {N B} _ {x ^ {(\mathrm {v a l}) , \tau}} (S) |}\right) \sum_ {(x, y) \in \mathbb {N B} _ {x ^ {(\mathrm {v a l}) , \tau}} (S)} \mathbb {1} [ y = y _ {\mathrm {v a l}} ] & \left| \mathbb {N B} _ {x ^ {(\mathrm {v a l})}, \tau} (S) \right| > 0 \\ \mathbb {1} [ y _ {i} = y ^ {(\mathrm {v a l})} ] - 1 / C & \left| \mathbb {N B} _ {x ^ {(\mathrm {v a l})}, \tau} (S) \right| = 0 \end{array} \right.
$$
This means that the marginal contribution on  $S$  only depends on the subset  $\mathbb{N}\mathbb{B}_{x^{(\mathrm{val})},\tau}(S)\subseteq S$ . Plugging in the above equation for  $v(S\cup z_i) - v(S)$ , we have
$$
\begin{array}{l} \phi_ {z _ {i}} \left(D _ {- z _ {i}}; v, w\right) \\ = \frac{1}{N}\sum_{k = 0}^{N - 1}w(k + 1)\sum_{\substack{S\subseteq D_{-z_{i}},\\ |S| = k}}[v(S\cup \{z_{i}\}) - v(S)] \\ = \frac{1}{N}\sum_{k = 0}^{N - 1}w(k + 1)\sum_{\substack{S_{0}\subseteq \mathbb{NB}_{x(\text{val}),\tau}(D_{-z_{i}}),\\ |S_{0}|\leq k}}\binom {N - \mathbf{c}_{x(\text{val})},\tau}{k - |S_{0}|}\left[v(S_{0}\cup \{z_{i}\}) - v(S_{0})\right] \\ = \frac{1}{N}\sum_{k = 0}^{N - 1}w(k + 1)\sum_{j = 0}^{k}\sum_{\substack{S_{0}\subseteq \mathbb{N}B_{x^{\mathrm{(val)}},\tau}(D_{-z_{i}}),\\ |S_{0}| = j}}\binom {N - \mathbf{c}_{x^{\mathrm{(val)}},\tau}}{k - j}\left[v(S_{0}\cup \{z_{i}\}) - v(S_{0})\right] \\ = \frac {1}{N} \sum_ {k = 0} ^ {N - 1} w (k + 1) \left\{\underbrace {\sum_ {j = 1} ^ {k} \sum_ {S _ {0} \subseteq \mathbb {N B} _ {x ^ {(\mathrm {v a l}) , \tau} (D - z _ {i})} ,} \binom {N - \mathbf {c} _ {x ^ {(\mathrm {v a l})}, \tau}} {k - j} \left[ \frac {\mathbb {1} [ y _ {i} = y ^ {(\mathrm {v a l})} ]}{1 + j} + \left(\frac {1}{1 + j} - \frac {1}{j}\right) \sum_ {(x , y) \in S _ {0}} \mathbb {1} [ y = y ^ {(\mathrm {v a l})} ] \right]} _ {(*)} \right. \\ \end{array}
$$