SlowGuess/NeurIPS_2023_OCR huggingface dataset:
As we can see, Faith-Shap, Low-degree, and Harmonica are consistent and efficient for all cases. Other methods can only fit a few inputs, the 3rd order Shapley interaction index misses all the cases because it is not efficient, and LIME misses all the cases because  $f_{3}$  is not a linear function.
Table 3: Interpretations by LIME, SHAP, Shapley Interaction Index, Shapley Taylor Index, Faith-Shap, Low-degree, and Harmonica on the 2nd order polynomial function  $f_{2}$ .
Algorithms	(-1,-1,-1)	(-1,-1,+1)	(-1,+1,-1)	(-1,+1,+1)	(+1,-1,-1)	(+1,-1,+1)	(+1,+1,-1)	(+1,+1,+1)
Ground Truth	-0.718	-0.015	+0.449	-0.818	+0.599	+1.468	-0.432	+0.365
LIME	-0.417	+0.083	-1.083	-0.583	+0.583	+1.083	-0.083	+0.417
SHAP	-0.365	+0.242	-1.292	-0.685	+0.685	+1.292	-0.242	✓
Shapley Interaction Index (3rd order)	-0.485	+0.224	-0.943	-0.896	+0.724	+1.390	-0.510	+0.496
Shapley Taylor Index (3rd order)	✓	+0.194	-0.606	-0.970	+0.751	+1.625	-0.642	✓
Faith-Shap (3rd order)	✓	✓	✓	✓	✓	✓	✓	✓
Low-degree (3rd order)	✓	✓	✓	✓	✓	✓	✓	✓
Harmonica (3rd order)	✓	✓	✓	✓	✓	✓	✓	✓
Table 4: Interpretations by LIME, SHAP, Shapley Interaction Index, Shapley Taylor Index, Faith-Shap, Low-degree, and Harmonica on the 3rd order polynomial function  $f_{3}$ .
Faith-SHAP has an intricate representation theorem, assigning coefficients to different terms under the Möbius transform. Since the basis induced by the Möbius transform is not orthonormal, it is unclear to us whether Faith-SHAP can theoretically compute the accurate coefficients for higher-order functions. However, the running time of Faith-SHAP has an exponential dependency on  $n$ , so empirically weighted sampling on subsets of features is needed [62]. This might be the main reason why our algorithms outperform Faith-SHAP in experiments on real-world datasets.
G More on Experiment Details
In this section, we will first describe the detailed experimental settings. For the two language tasks, i.e., SST-2 and IMDb, we use the same CNN neural network. The model has a test accuracy of  $85.6\%$ . For the readability of results, we treat sentences as units instead of words - masking several
words in a sentence may render the entire paragraph difficult to understand and even meaningless while masking a critical sentence has meaningful semantic effects. Therefore, the radius is defined as the maximum number of masked sentences for IMDb. The word embedding layer is pre-trained by GloVe [46] and the maximum word number is set to 25,000. Besides the embedding layer, the network consists of several convolutional kernels with different kernel sizes (3, 4, and 5). After that, we use several fully connected layers, non-linear layers, and pooling layers to process the features. A Sigmoid function is attached to the tail of the network to ensure that the output can be seen as a probability distribution. Our networks are trained with an Adam [32] optimizer with a learning rate of 1e-2 for 5 epochs. For the vision task, we choose the official ResNet [28] architecture, which is available on PyTorch and we do not discuss the architecture details here. All the experiments are run on a server with 4 Nvidia 2080 Ti GPUs. A running time comparison can be found in Table 5. More information about the run-time Python environment and implementation details can be found in our code.
Table 5: Running time of different algorithms on SST2 dataset. The main characteristics affecting the running speed are listed in the table.
Algorithm	Main Characteristics Affecting Speed	Running Time (s)
Harmonica-2nd Harmonica-3rd LIME	Parallel perturbation, sample=2000	225
Parallel perturbation, sample=2000	616
Captum.attr.LimeBase, Seq. perturbation, Sample=2000, Exp. cosine sim.	1499
Integrated Gradients SHAP	Gradient step=500	82
Captum.attr.KernelShap, Seq. perturbation, Sample=2000	1235
Integrated Hessians Shapley Taylor Index Faith-Shap	Gradient step=50	3550
Parallel random perturbations	651
Parallel random perturbations	538
The accurate representation of the high non-linearity of deep neural networks (DNNs) can be quite complex, often leading to computationally expensive or practically infeasible interpretability methods. To strike a balance between computational efficiency and faithful representation, we employ higher-order polynomial approximations to capture the non-linear aspects of the target DNNs. Specifically, we leverage Fourier basis (or polynomial basis) to represent Boolean functions over feature subsets, thus providing a compact and computationally efficient way to approximate DNNs. This is particularly useful for removal-based explanations, which inherently deal with Boolean functions representing the presence or absence of features.
It is crucial to point out that our choice of using higher-order polynomials is not arbitrary but is guided by the principle of "truthfulness." We introduce a metric called the "truthful gap," mathematically defined as  $\mathbb{T}_V(f,g)$  (as per Equation 1), to quantify how well our polynomial approximations capture the information in a specific subspace  $V$  of the original function  $f$ . This notion of truthfulness serves as a rigorous measure to validate the efficacy of our higher-order polynomial approximations.
Our empirical results, particularly those outlined in Section 5, indicate that higher-order polynomials, specifically of orders 2 and 3, yield a favorable trade-off. They offer a substantial reduction in interpretation error and a low truthful gap compared to other techniques, substantiating the reasonableness and effectiveness of our approach.
Notice that our algorithm is a post-hoc model-agnostic interpretation algorithm, we only need to use the original neural network  $f$  to infer on given input as an oracle. This means that one can easily change the network architecture without any additional changes.
H More Experimental Results
In this section, we present ablation studies that examine the impact of various factors such as image segmentation format, neural network architecture, and choice of baselines. Our results demonstrate the robustness and efficacy of our approach in different experimental settings.
H.1 Image Segmentation Format
We employ the SLIC algorithm to generate segmentation superpixels on the ImageNet dataset. We initially divide each  $224 \times 224$  image into 16 superpixels for a balance between human readability and computational efficiency. The results, displayed in Table 6 and Figure 4, validate the effectiveness of our approach. Further experiments with 10 superpixels are reported in Table 7, confirming the robustness of our method.
H.2 Neural Network Architectures
We also conduct experiments using diverse neural network architectures such as ResNet-18 and VGG16. Tables 8 and 9 respectively present the interpretation errors on ImageNet for these architectures.
H.3 Choice of Baselines
The choice of baseline is critical for removal-based interpretation methods. We investigate the interpretation errors when using the average pixel value and a blurred image as baselines. The results are shown in Tables 10 and 11.
Table 6: Interpretation error on MS-COCO. Bold indicates the best performance, and underline indicates the second best.
Radius
1	2	4	8	16
Harmonica-2nd	0.0291	0.0332	0.0363	0.0367	0.0367
Harmonica-3rd	0.0183	0.0218	0.0241	0.0244	0.0244
LIME	0.2264	0.2451	0.2478	0.2479	0.2479
SHAP	0.1952	0.2036	0.2041	0.2042	0.2042
IG	0.3473	0.3515	0.3464	0.3462	0.3438
IH	1.3717	1.5047	1.5141	1.5141	1.5141
Shapley Taylor	0.3554	0.3313	0.3269	0.3267	0.3267
Faith-Shap	0.2428	0.2397	0.2349	0.2347	0.2347
Table 7: Interpretation error on ImageNet with the number of superpixels set to 10.
Radius
1	2	4	8	16
Harmonica-2nd	0.0970	0.1054	0.1221	0.1416	0.1420
Harmonica-3rd	0.0705	0.0825	0.0997	0.1179	0.1183
LIME	0.1995	0.2121	0.2570	0.3052	0.3051
SHAP	0.1449	0.2077	0.2743	0.3064	0.3063
IG	0.2615	0.3567	0.4423	0.4628	0.4626
IH	1.6707	2.2977	2.9443	3.1871	3.1877
Shapley Taylor	0.0982	0.1417	0.1761	0.1815	0.1814
Faith-Shap	0.0970	0.1250	0.1587	0.1811	0.1810
I Detailed Results on Interpretation Error
In this section, we provide interpretation error results under other norms (Figure 6, 7, and 8) and the corresponding numerical results (Table 12, 13 and 14).
Table 8: Interpretation error on ImageNet using ResNet18 (ACC@1=69.758%).
Radius
1	2	4	8	16
Harmonica-2nd	0.1468	0.1433	0.1353	0.1357	0.1414
Harmonica-3rd	0.1247	0.1261	0.1231	0.1272	0.1319
LIME	0.2278	0.2209	0.2150	0.2382	0.2578
SHAP	0.1034	0.1436	0.1963	0.2323	0.2375
IG	0.1937	0.2613	0.3466	0.4028	0.4039
IH	0.6422	0.8686	1.1591	1.3939	1.4254
Shapley Taylor	0.1292	0.1530	0.1716	0.1744	0.1775
Faith-Shap	0.1239	0.1426	0.1532	0.1607	0.1693
Table 9: Interpretation error on ImageNet using VGG16 (ACC@1=71.592%).
Radius
1	2	4	8	16
Harmonica-2nd	0.1468	0.1404	0.1298	0.1303	0.1368
Harmonica-3rd	0.1290	0.1267	0.1204	0.1225	0.1289
LIME	0.2376	0.2280	0.2180	0.2448	0.2664
SHAP	0.1042	0.1461	0.2012	0.2407	0.2477
IG	0.1708	0.2331	0.3123	0.3659	0.3694
IH	0.5988	0.8132	1.1010	1.3663	1.4149
Shapley Taylor	0.1155	0.1435	0.1677	0.1720	0.1728
Faith-Shap	0.1085	0.1306	0.1456	0.1548	0.1621
Table 10: Interpretation error on ImageNet using the average pixel value as the baseline.
Method	Radius
1	2	4	8	16
Harmonica-2nd	0.1341	0.1279	0.1201	0.1243	0.1304
Harmonica-3rd	0.1012	0.1025	0.1022	0.1095	0.1156
LIME	0.2205	0.2142	0.2095	0.2327	0.2520
SHAP	0.1022	0.1410	0.1934	0.2307	0.2357
IG	0.1824	0.2468	0.3280	0.3822	0.3835
IH	0.6292	0.8497	1.1356	1.3752	1.4115
Shapley Taylor	0.1293	0.1665	0.2157	0.2550	0.2534
Faith-Shap	0.1219	0.1530	0.1918	0.2288	0.2302
Table 11: Interpretation error on ImageNet using the blurred image as the baseline.
Method	Radius
1	2	4	8	16
Harmonica-2nd	0.0207	0.0209	0.0210	0.0218	0.0225
Harmonica-3rd	0.0200	0.0203	0.0204	0.0213	0.0220
LIME	0.0260	0.0276	0.0310	0.0372	0.0394
SHAP	0.0184	0.0258	0.0360	0.0460	0.0477
IG	0.0215	0.0299	0.0415	0.0524	0.0541
IH	0.0268	0.0373	0.0525	0.0684	0.0718
Shapley Taylor	0.0350	0.0370	0.0385	0.0372	0.0360
Faith-Shap	0.0343	0.0358	0.0368	0.0358	0.0350
Radius	L2norm	L1norm	L0norm	Radius	L2norm	L1norm	L0norm	Radius	L2norm	L1norm	L0norm
1	0.0536	0.0460	0.1154	1	0.0434	0.0363	0.0995	1	0.0981	0.0870	0.3572
2	0.0576	0.0463	0.1219	2	0.0484	0.0376	0.1046	2	0.1010	0.0861	0.3458
4	0.0639	0.0483	0.1350	4	0.0561	0.0405	0.1157	4	0.1043	0.0863	0.3421
8	0.0768	0.0549	0.1696	8	0.0700	0.0474	0.1456	8	0.1135	0.0919	0.3659
16	0.0945	0.0651	0.2173	16	0.0876	0.0575	0.1900	16	0.1285	0.1015	0.4034
32	0.0990	0.0677	0.2275	32	0.0921	0.0600	0.2003	32	0.1323	0.1038	0.4111
∞	0.0991	0.0677	0.2279	∞	0.0922	0.0601	0.2005	∞	0.1322	0.1037	0.4112
Harmonica2	Harmonica3	LIME
Radius	L2norm	L1norm	L0norm	Radius	L2norm	L1norm	L0norm	Radius	L2norm	L1norm	L0norm
1	0.0792	0.0568	0.1819	1	0.1133	0.0681	0.2010	1	0.0865	0.0477	0.1215
2	0.1089	0.0836	0.3167	2	0.1551	0.1055	0.3361	2	0.1232	0.0756	0.2118
4	0.1470	0.1186	0.4847	4	0.2081	0.1563	0.5066	4	0.1753	0.1183	0.3406
8	0.1865	0.1554	0.6252	8	0.2595	0.2075	0.6476	8	0.2343	0.1691	0.4859
16	0.2106	0.1783	0.6891	16	0.2852	0.2336	0.7058	16	0.2678	0.1994	0.5724
32	0.2137	0.1814	0.6958	32	0.2872	0.2357	0.7099	32	0.2709	0.2028	0.5823
∞	0.2137	0.1814	0.6961	∞	0.2874	0.2359	0.7105	∞	0.2711	0.2029	0.5823
SHAP	Integrated Gradients	Integrated Hessians
Radius	L2norm	L1norm	Radius	L2norm	L1norm	L0norm
1	0.0623	0.0472	0.1090	1	0.0602	0.0451	0.1034
2	0.0853	0.0649	0.1971	2	0.0813	0.0614	0.1795
4	0.1144	0.0874	0.3069	4	0.1091	0.0824	0.2794
8	0.1426	0.1081	0.3909	8	0.1387	0.1042	0.3707
16	0.1612	0.1217	0.4404	16	0.1586	0.1190	0.4271
32	0.1642	0.1241	0.4486	32	0.1616	0.1213	0.4356
∞	0.1642	0.1240	0.4489	∞	0.1616	0.1213	0.4357
Shapley Taylor Index2	Faith-Shap2
Table 12: The interpretation error of Harmonica and other baseline algorithms evaluated on the SST-2 dataset for different neighborhoods with a radius ranging from 1 to  $\infty$  under  ${L}^{2},{L}^{1}$  and  ${L}^{0}$  norms.
Radius	L2norm	L1norm	L0norm	Radius	L2norm	L1norm	L0norm	Radius	L2norm	L1norm	L0norm
1	0.0225	0.0193	0.0041	1	0.0175	0.0149	0.0027	1	0.0624	0.0525	0.1175
2	0.0234	0.0193	0.0036	2	0.0193	0.0157	0.0022	2	0.0671	0.0546	0.1358
4	0.0300	0.0228	0.0141	4	0.0264	0.0195	0.0098	4	0.0740	0.0583	0.1606
8	0.0442	0.0298	0.0434	8	0.0407	0.0265	0.0367	8	0.084	0.0640	0.1903
16	0.0506	0.0330	0.0574	16	0.0472	0.0296	0.0506	16	0.0885	0.0664	0.2017
32	0.0515	0.0334	0.0596	32	0.0481	0.0301	0.0528	32	0.0892	0.0667	0.2033
∞	0.0515	0.0334	0.0597	∞	0.0481	0.0301	0.0529	∞	0.0892	0.0667	0.2033
Harmonica2	Harmonica3	LIME
Radius	L2norm	L1norm	L0norm	Radius	L2norm	L1norm	L0norm	Radius	L2norm	L1norm	L0norm
1	0.0849	0.0650	0.2266	1	0.1228	0.0810	0.2702	1	0.1178	0.0635	0.1929
2	0.1123	0.0903	0.3589	2	0.1576	0.1169	0.4102	2	0.1567	0.0997	0.3125
4	0.1406	0.1162	0.4891	4	0.1882	0.1494	0.5333	4	0.1969	0.1415	0.4501
8	0.1598	0.1341	0.5610	8	0.2031	0.1657	0.5885	8	0.2227	0.1702	0.5423
16	0.1662	0.1402	0.5817	16	0.2067	0.1697	0.6004	16	0.2308	0.1795	0.5702
32	0.1671	0.1411	0.5842	32	0.2071	0.1702	0.6014	32	0.2319	0.1807	0.5739
∞	0.1671	0.1411	0.5843	∞	0.2071	0.1702	0.6015	∞	0.2319	0.1807	0.5738
SHAP	Integrated Gradients	Integrated Hessians
Radius	L2norm	L1norm	Radius	L2norm	L1norm	L0norm
1	0.0444	0.0365	1	0.0395	0.0313	0.0337
2	0.0590	0.0470	2	0.0469	0.0378	0.0466
4	0.0684	0.0538	4	0.0547	0.0440	0.0657
8	0.0698	0.0542	8	0.0620	0.0494	0.0808
16	0.0719	0.0558	16	0.0684	0.0543	0.0851
32	0.0741	0.0575	32	0.0740	0.0588	0.0859
∞	0.0743	0.0577	∞	0.0744	0.0590	0.0859
Shapley Taylor Index2	Faith-Shap2
Table 13: The interpretation error of Harmonica and other baseline algorithms evaluated on the IMDb dataset for different neighborhoods with a radius ranging from 1 to  $\infty$  under  $L^2$ ,  $L^1$  and  $L^0$  norms.
(a) Laptop
(b) Flatworm
(a) Insertion
Figure 4: Illustrative examples for applying our method to vision tasks.
(b) Deletion
Figure 5: Insertion and deletion results on SST2.
J Detailed Results on Harmonica-local
J.1 Additional Experiments
We further explore Harmonica's performance when limited to a local space instead of the whole space. It is worth noting that  $L_{r}$  is a regularization loss for Harmonica, ensuring the sparseness of each interpretation model  $g_{i}$ . Additionally,  $L_{c}$  is a regularization loss penalizing the difference between interpretation models. The value of balance coefficients depends on the application scenario and can be determined by the end user.
(a)  $L^2$  norm
Figure 6: Visualization of interpretation error  $\mathbb{I}_{p,\mathcal{N}_x}(f,g)$  evaluated on SST-2 dataset.
(b)  $L^1$  norm
(c)  $L^0$  norm
(a)  $L^2$  norm
(b)  $L^1$  norm
Figure 7: Visualization of interpretation error  $\mathbb{I}_p,\mathcal{N}_x(f,g)$  evaluated on IMDb dataset.
(a)  $L^2$  norm
Figure 8: Visualization of interpretation error  $\mathbb{I}_{p,\mathcal{N}_x}(f,g)$  evaluated on ImageNet dataset.
Radius	L2norm	L1norm	L0norm
1	0.1290	0.1108	0.4248
2	0.1308	0.1073	0.4116
4	0.1373	0.1094	0.4332
8	0.1584	0.1264	0.5156
∞	0.1693	0.1342	0.5405
Harmonica
Radius	L² norm	L¹ norm	L⁰ norm
1	0.1048	0.0880	0.3202
2	0.1108	0.0887	0.3260
4	0.1220	0.0955	0.3723
8	0.1443	0.1139	0.4698
∞	0.1566	0.1230	0.5030
Harmonica
Radius	L2norm	L1norm	L0norm
1	0.2422	0.2208	0.7274
2	0.2347	0.2036	0.6976
4	0.2346	0.1918	0.6540
8	0.2897	0.2304	0.6893
∞	0.3261	0.2579	0.7196
LIME
Radius	L2norm	L1norm	L0norm
1	0.1197	0.0867	0.2892
2	0.1658	0.1261	0.4566
4	0.2306	0.1862	0.6483
8	0.2943	0.2409	0.7318
∞	0.3115	0.2523	0.7362
SHAP
Radius	L² norm	L¹ norm	L⁰ norm
1	0.2322	0.1650	0.4871
2	0.3141	0.2375	0.6406
4	0.4200	0.3346	0.7722
8	0.5010	0.4094	0.8300
∞	0.5113	0.4177	0.8340
Integrated Gradients
Radius	L2norm	L1norm	L0norm
1	1.3681	0.9504	0.8443
2	1.8603	1.4006	0.9025
4	2.5168	2.0427	0.9394
8	3.1095	2.6844	0.9543
∞	3.2139	2.8140	0.9562
Integrated Hessians
Radius	L2norm	L1norm	L0norm	Radius	L2norm	L1norm	L0norm
1	0.1337	0.1039	0.3939	1	0.1238	0.0948	0.3443
2	0.1681	0.1309	0.4906	2	0.1499	0.1161	0.4338
4	0.2028	0.1600	0.5830	4	0.1718	0.1351	0.5126
8	0.2226	0.1771	0.6365	8	0.1960	0.1554	0.5872
∞	0.2274	0.1804	0.6418	∞	0.2099	0.1654	0.6071
Shapley Taylor Index² Faith-Shap²
Table 14: The interpretation error of Harmonica and other baseline algorithms evaluated on the ImageNet dataset for different neighborhoods with a radius ranging from 1 to  $\infty$  under  ${L}^{2},{L}^{1}$  and  ${L}^{0}$  norms.
Figure 9, 10 and 11 compares Harmonica-local and Harmonica algorithms on SST-2, IMDb and ImageNet, respectively. Here we set the radius of  $\mathcal{N}_x$  to be 4. The results reveal that Harmonica-local indeed performs better within the local region but fails to cover a far region.
(a)  $L^2$  norm
(a)  $L^2$  norm
Figure 10: Visualization of interpretation error  $\mathbb{I}_{p,\mathcal{N}_x}(f,g)$  evaluated on IMDb dataset of algorithm Harmonica and Harmonica-local.
Figure 9: Visualization of interpretation error  $\mathbb{I}_{p,\mathcal{N}_x}(f,g)$  evaluated on SST-2 dataset of algorithm Harmonica and Harmonica-local.
(c)  $L^0$  norm
(a)  $L^2$  norm
Figure 11: Visualization of interpretation error  $\mathbb{I}_{p,\mathcal{N}_x}(f,g)$  evaluated on ImageNet of algorithm Harmonica and Harmonica-local.
K Detailed Results on Truthful Gap
Estimating truthful gap For convenience, we define the set of bases  $C^d$  up to degree  $d$  as  $C^d = \chi_S |S \subseteq [n], |S| \leq d$ . We evaluate the truthful gap on the set of bases  $C^3$ ,  $C^2$ , and  $C^1$ . By definition in Eqn. (3), we have  $\mathbb{T}_{V_C}(f,g) = \sum_{\chi_S \in C} \langle f - g, \chi_S \rangle^2 = \left( \mathbb{E}_{x \sim \{-1,1\}^n} \left[ (f(x) - g(x)) \sum_{\chi_S \in C} \chi_S(x) \right] \right)^2$ .
Then we perform a sampling-based estimation of the truthful gap. Worth mentioning that since the size of set  $C^d$  satisfies  $|C^d| = \sum_{i=0}^{d} \binom{n}{i}$  and the max number of words, sentences, or superpixels
$n^* \leq 50, \sum_{\chi_S \in C} \chi_S(x)$ , as the summation function of orthonormal basis, is easy to compute on every sample  $x \in \{-1,1\}^n$  (for  $n^*$  very large, we will perform another sampling step on this function).
Table 1 shows the  $C^3$  truthful gap evaluated on different datasets. We can see that Harmonica outperforms all the other baseline algorithms. Figure 12 shows the truthful gap evaluated on the SST-2 dataset. We can see that Harmonica achieves the best performance for  $C^2$  and  $C^3$ . For the simple linear case  $C^1$ , Harmonica is almost as good as LIME. Figure 13 shows the truthful gap evaluated on the IMDb dataset under the same settings as the SST-2 dataset. Figure 14 shows the truthful gap evaluated on the ImageNet dataset. We can see that Harmonica outperforms all the other baselines consistently.
(a) bases  $C^3$
(b) bases  $C^2$
Figure 12: Visualization of truthful gap  $\mathbb{T}_C(f,g)$  evaluated on SST-2 dataset.
(c) bases  $C^1$
(a) bases  $C^3$
Figure 13: Visualization of truthful gap  $\mathbb{T}_C(f,g)$  evaluated on IMDb dataset.
(b) bases  $C^2$
(a) bases  $C^3$
Figure 14: Visualization of truthful gap  $\mathbb{T}_C(f,g)$  evaluated on ImageNet dataset.
L Detailed Results on Harmonica-anchor
L.1 Harmonica-anchor
Figure 15, 16, and 17 show the interpretation error results evaluated on three datasets. All Harmonica-anchor algorithms further reduce the interpretation error compared to Harmonica. As we increase the number of anchors, the interpretation error slightly reduces, which is consistent with our intuition. Table 15, 16, and 17 show the numerical results.
(b)  $L^1$  norm
Figure 15: Visualization of interpretation error  $\mathbb{I}_{p,\mathcal{N}_x}(f,g)$  evaluated on SST-2 of Harmonica-anchor (2nd order) with different anchor number  $k$ .
(a)  $L^2$  norm
Figure 16: Visualization of interpretation error  $\mathbb{I}_{p,\mathcal{N}_x}(f,g)$  evaluated on IMDb of Harmonica-anchor (2nd order) with different anchor number  $k$ .
(a)  $L^2$  norm
Figure 17: Visualization of interpretation error  $\mathbb{I}_{p,\mathcal{N}_x}(f,g)$  evaluated on ImageNet of Harmonica-anchor (2nd order) with different anchor number  $k$ .
L.2 Harmonica-anchor-constrained
Here we provide the interpretation error of Harmonica-anchor-constrained evaluated on SST-2 dataset in Figure 18. As we increase the constraint coefficient  $\lambda_{2}$ , the interpretation error increases, but still smaller than that of Harmonica.
Radius	L2norm	L1norm	L0norm
1	0.0217	0.0178	0.0137
2	0.0257	0.0196	0.0186
4	0.0360	0.0258	0.0417
8	0.0559	0.0396	0.1007
16	0.0723	0.0518	0.1579
32	0.0753	0.0541	0.1688
∞	0.0753	0.0541	0.1687
Harmonica-anchor (k=3)
Harmonica-anchor (k=5)
Radius	L2norm	L1norm	L0norm
1	0.0195	0.0159	0.0114
2	0.0231	0.0175	0.0144
4	0.0330	0.0232	0.0356
8	0.0535	0.0374	0.0948
16	0.0719	0.0514	0.1598
32	0.0753	0.0540	0.1714
∞	0.0753	0.0540	0.1717
Harmonica-anchor (k=7)
Harmonica-anchor (k=9)
Table 15: The interpretation error of Harmonica-anchor algorithms evaluated on the SST-2 dataset for different  $k$  with a radius ranging from 1 to  $\infty$  under  ${L}^{2},{L}^{1}$  and  ${L}^{0}$  norms.
Radius	L2norm	L1norm	L0norm
1	0.0078	0.0062	0.0013
2	0.0088	0.0067	0.0005
4	0.0124	0.0088	0.0015
8	0.0199	0.0135	0.0074
16	0.0231	0.0154	0.0114
32	0.0235	0.0157	0.0121
∞	0.0236	0.0157	0.0121
Harmonica-anchor (k=3)
Radius	L2norm	L1norm	L0norm
1	0.0057	0.0044	0.0010
2	0.0062	0.0045	0.0003
4	0.0088	0.0058	0.0008
8	0.0151	0.0097	0.0042
16	0.0181	0.0116	0.0078
32	0.0186	0.0118	0.0085
∞	0.0186	0.0119	0.0086
Harmonica-anchor (k=7)
Radius	L2norm	L1norm	L0norm
1	0.0057	0.0044	0.0010
2	0.0062	0.0045	0.0003
4	0.0088	0.0058	0.0008
8
16
32
∞
Table 16: The interpretation error of Harmonica-anchor algorithms evaluated on the IMDb dataset for different  $k$  with a radius ranging from 1 to  $\infty$  under  ${L}^{2},{L}^{1}$  and  ${L}^{0}$  norms.
Radius	L2norm	L1norm	L0norm
1	0.0573	0.0473	0.1281
2	0.0681	0.0518	0.1524
4	0.1025	0.0779	0.2855
8	0.1382	0.1066	0.4240
∞	0.1485	0.1145	0.4562
Harmonica-anchor (k=3)
Harmonica-anchor (k=5)
Radius	L2norm	L1norm	L0norm
1	0.0442	0.0362	0.0719
2	0.0561	0.0418	0.1053
4	0.0954	0.0706	0.2496
8	0.1358	0.1032	0.4036
∞	0.1461	0.1110	0.4352
Harmonica-anchor (k=7)
Harmonica-anchor (k=9)
Table 17: The interpretation error of Harmonica-anchor algorithms evaluated on the ImageNet dataset for different  $k$  with a radius ranging from 1 to  $\infty$  under  ${L}^{2},{L}^{1}$  and  ${L}^{0}$  norms.
(b)  $L^1$  norm
Figure 18: Visualization of interpretation error  $\mathbb{I}_{p,\mathcal{N}_x}(f,g)$  evaluated on SST-2 dataset of Harmonica-anchor-constrained (2nd order) with different coefficient  $\lambda_{2}$ . Here all the Harmonica-anchor-constrained algorithms use  $k = 5$ .
Radius	L2norm	L1norm	L0norm	Radius	L2norm	L1norm	L0norm
1	0.0220	0.0182	0.0115	1	0.0234	0.0176	0.0250
2	0.0236	0.0166	0.0157	2	0.0287	0.0191	0.0329
4	0.0348	0.0210	0.0382	4	0.0403	0.0251	0.0579
8	0.0587	0.0363	0.0977	8	0.0608	0.0382	0.1086
16	0.0777	0.0484	0.1500	16	0.0808	0.0494	0.1538
32	0.0813	0.0504	0.1581	32	0.0853	0.0517	0.1623
∞	0.0812	0.0503	0.1579	∞	0.0851	0.0516	0.1620
Harmonica-anchor-constrained  $(\lambda_{2} = 0)$
Harmonica-anchor-constrained  $(\lambda_{2} = 1.0)$
Radius	L2norm	L1norm	L0norm
1	0.0304	0.0238	0.0317
2	0.0357	0.0258	0.0427
4	0.0461	0.0315	0.0701
8	0.0650	0.0435	0.1214
16	0.0839	0.0539	0.1663
32	0.0883	0.0561	0.1750
∞	0.0881	0.0560	0.1745
Harmonica-anchor-constrained  $(\lambda_{2} = 5.0)$
Table 18: The interpretation error of Harmonica-anchor-constrained algorithms evaluated on the SST-2 dataset for different  $\lambda_{2}$  with a radius ranging from 1 to  $\infty$  under  $L^2$ ,  $L^1$  and  $L^0$  norms.
M Discussion on the Low-degree algorithm
We also investigate the sample complexity of Harmonica and Low-degree algorithms, which demonstrates that Harmonica achieves better performance with the same sample size.
(a)  $L^2$  norm
Figure 19: Visualization of interpretation error  $\mathbb{I}_{p,\mathcal{N}_x}(f,g)$  evaluated on SST-2 dataset, while Harmonica and Low-degree algorithms using different sample size varying from 2000 to 10000.
From Theorem 2 and Theorem D.1, we know that the sample complexity of the Harmonica algorithm  $(\tilde{O}(\frac{1}{\epsilon}))$  is much more efficient than the Low-degree algorithm  $(\tilde{O}(\frac{1}{\epsilon^2}))$ . Figure 19 shows that when evaluating the interpretation error on SST-2 dataset, with the same sample size, the Harmonica