Mist labeled Data Detection	Noisy Data Detection
Dataset	ε	DP-TKNN-Shapley	DP-KNN-Shapley(no subsampling)	DP-KNN-Shapley(with subsampling)	DP-TKNN-Shapley	DP-KNN-Shapley(no subsampling)	DP-KNN-Shapley(with subsampling)
2dplanes	0.1	0.883 (0.017)	0.49 (0.024)	0.733 (0.011)	0.692 (0.014)	0.494 (0.023)	0.615 (0.01)
0.5	0.912 (0.009)	0.488 (0.022)	0.815 (0.006)	0.706 (0.004)	0.494 (0.012)	0.66 (0.004)
1	0.913 (0.009)	0.504 (0.019)	0.821 (0.005)	0.705 (0.007)	0.495 (0.011)	0.665 (0.004)
phoneme	0.1	0.816 (0.011)	0.5 (0.014)	0.692 (0.011)	0.648 (0.028)	0.475 (0.042)	0.566 (0.01)
0.5	0.826 (0.007)	0.497 (0.011)	0.738 (0.003)	0.683 (0.014)	0.536 (0.033)	0.588 (0.004)
1	0.826 (0.005)	0.486 (0.01)	0.741 (0.002)	0.685 (0.016)	0.494 (0.071)	0.59 (0.005)
CPU	0.1	0.932 (0.007)	0.49 (0.028)	0.881 (0.005)	0.805 (0.037)	0.42 (0.074)	0.709 (0.011)
0.5	0.946 (0.004)	0.507 (0.029)	0.928 (0.002)	0.838 (0.007)	0.472 (0.092)	0.746 (0.003)
1	0.948 (0.002)	0.512 (0.008)	0.931 (0.002)	0.839 (0.003)	0.455 (0.079)	0.748 (0.002)
Creditcard	0.1	0.661 (0.008)	0.511 (0.026)	0.584 (0.011)	0.558 (0.017)	0.479 (0.115)	0.519 (0.014)
0.5	0.663 (0.008)	0.503 (0.021)	0.624 (0.011)	0.571 (0.058)	0.513 (0.089)	0.541 (0.008)
1	0.668 (0.005)	0.491 (0.026)	0.627 (0.009)	0.585 (0.048)	0.525 (0.107)	0.545 (0.006)
Click	0.1	0.566 (0.017)	0.512 (0.013)	0.511 (0.024)	0.55 (0.012)	0.439 (0.094)	0.497 (0.017)
0.5	0.567 (0.013)	0.502 (0.018)	0.529 (0.012)	0.555 (0.01)	0.506 (0.071)	0.503 (0.018)
1	0.567 (0.007)	0.502 (0.021)	0.532 (0.01)	0.559 (0.009)	0.488 (0.076)	0.504 (0.017)
Wind	0.1	0.861 (0.021)	0.489 (0.022)	0.836 (0.01)	0.622 (0.167)	0.464 (0.282)	0.557 (0.017)
0.5	0.883 (0.002)	0.511 (0.021)	0.873 (0.004)	0.668 (0.055)	0.554 (0.282)	0.601 (0.007)
1	0.882 (0.002)	0.486 (0.022)	0.875 (0.004)	0.681 (0.032)	0.595 (0.257)	0.605 (0.008)
Fraud	0.1	0.947 (0.004)	0.497 (0.02)	-	0.813 (0.033)	0.468 (0.127)	-
0.5	0.952 (0.002)	0.502 (0.026)	-	0.85 (0.017)	0.548 (0.192)	-
1	0.953 (0.003)	0.502 (0.008)	-	0.859 (0.007)	0.497 (0.162)	-
Apsfail	0.1	0.903 (0.078)	0.501 (0.018)	-	0.693 (0.2)	0.456 (0.357)	-
0.5	0.952 (0.001)	0.521 (0.023)	-	0.873 (0.014)	0.489 (0.139)	-
1	0.952 (0.0)	0.49 (0.031)	-	0.882 (0.003)	0.494 (0.179)	-
Pol	0.1	0.844 (0.006)	0.508 (0.012)	-	0.604 (0.082)	0.53 (0.212)	-
0.5	0.858 (0.006)	0.5 (0.029)	-	0.673 (0.039)	0.626 (0.153)	-
1	0.857 (0.001)	0.516 (0.02)	-	0.695 (0.02)	0.622 (0.195)	-
MNIST	0.1	0.801 (0.002)	0.492 (0.007)	-	0.841 (0.002)	0.384 (0.129)	-
0.5	0.948 (0.002)	0495 (0.005)	-	0.873 (0.001)	0.389 (0.125)	-
1	0.958 (0.002)	0.495 (0.005)	-	0.881 (0.002)	0.4 (0.002)	-
CIFAR10	0.1	0.924 (0.002)	0.496 (0.014)	-	0.821 (0.002)	0.558 (0.013)	-
0.5	0.936 (0.002)	0.497 (0.012)	-	0.847 (0.004)	0.560 (0.009)	-
1	0.949 (0.003)	0.497 (0.012)	-	0.852 (0.001)	0.560 (0.006)	-
AGNews	0.1	0.725 (0.002)	0.491 (0.002)	-	0.611 (0.003)	0.421 (0.065)	-
0.5	0.733 (0.005)	0.492 (0.002)	-	0.621 (0.001)	0.426 (0.075)	-
1	0.738 (0.002)	0.494 (0.002)	-	0.627 (0.001)	0.426 (0.075)	-
DBPedia	0.1	0.652 (0.005)	0.499 (0.002)	-	0.579 (0.002)	0.360 (0.07)	-
0.5	0.654 (0.002)	0.499 (0.002)	-	0.586 (0.002)	0.360 (0.09)	-
1	0.657 (0.001)	0.5 (0.002)	-	0.588 (0.001)	0.361 (0.07)	-
Table 8: Privacy-utility tradeoff of DP-TKNN-Shapley and DP-KNN-Shapley for mislabeled/noisy data detection task on 13 datasets. We show the AUROC at different privacy budgets  $\varepsilon$  (We set  $\delta = 10^{-5}$  for MNIST, CIFAR10, AGNews, and PBpedia, and we set  $\delta = 10^{-4}$  for the rest of dataset, following the common rule of  $\delta \approx 1 / N$ ). The higher the AUROC is, the better the method is. We show the standard deviation of AUROC across 5 independent runs in(). The results for DP-KNN-Shapley with subsampling are omitted for half of the datasets as it requires a significant amount of runtime.