Title: Learning to Understand: Identifying Interactions via the Möbius Transform
Venue: NeurIPS 2024
Accepted: yes
Keywords: Shapley Value

Importance Scores

Transforms

Signal Processing

Interactions

Group Testing
Abstract: One of the key challenges in machine learning is to find interpretable representations of learned functions. The Möbius transform is essential for this purpose, as its coefficients correspond to unique *importance scores* for *sets of input variables*. This transform is closely related to widely used game-theoretic notions of importance like the *Shapley* and *Bhanzaf value*, but it also captures crucial higher-order interactions. Although computing the Möbius Transform of a function with $n$ inputs involves $2^n$ coefficients, it becomes tractable when the function is *sparse* and of *low-degree* as we show is the case for many real-world functions. Under these conditions, the complexity of the transform computation is significantly reduced. When there are $K$ non-zero coefficients, our algorithm recovers the Möbius transform in $O(Kn)$ samples and $O(Kn^2)$ time asymptotically under certain assumptions, the first non-adaptive algorithm to do so. We also uncover a surprising connection between group testing and the Möbius transform. For functions where all interactions involve at most $t$ inputs, we use group testing results to compute the Möbius transform with $O(Kt\log n)$ sample complexity and $O(K\mathrm{poly}(n))$ time.  A robust version of this algorithm withstands noise and maintains this complexity. This marks the first $n$ sub-linear query complexity, noise-tolerant algorithm for the Möbius transform. While our algorithms are conceptualized in an idealized setting, they indicate that the Möbius transform is a potent tool for interpreting deep learning models.

Learning to Understand: Identifying Interactions via the M¨ obius Transform

Abstract

One of the key challenges in machine learning is to find interpretable representations of learned functions. The M¨ obius transform is essential for this purpose, as its coefficients correspond to unique importance scores for sets of input variables . This transform is closely related to widely used game-theoretic notions of importance like the Shapley and Bhanzaf value , but it also captures crucial higher-order interactions. Although computing the M¨ obius Transform of a function with n inputs involves 2 n coefficients, it becomes tractable when the function is sparse and of low degree as we show is the case for many real-world functions. Under these conditions, the complexity of the transform computation is significantly reduced. When there are K non-zero coefficients, our algorithm recovers the M¨ obius transform in O ( Kn ) samples and O ( Kn 2 ) time asymptotically under certain assumptions, the first non-adaptive algorithm to do so. We also uncover a surprising connection between group testing and the M¨ obius transform. For functions where all interactions involve at most t inputs, we use group testing results to compute the M¨ obius transform with O ( Kt log n ) sample complexity and O ( K poly( n )) time. A robust version of this algorithm withstands noise and maintains this complexity. This marks the first n sub-linear query complexity, noise-tolerant algorithm for the M¨ obius transform. In several examples, we observe that representations generated via sparse M¨ obius transform are up to twice as faithful to the original function, as compared to Shapley and Banzhaf values, while using the same number of terms.

1 Introduction

As machine learning models become increasingly complex, our ability to interpret them has not kept pace. A natural question to ask is: What is the most fundamental interpretable representation of the functions we learn? The Shapley value [1], a concept from cooperative game theory, has become a popular way to interpret model predictions [2] by assigning importance scores to individual inputs such as features, data samples or tokens. This value represents the weighted average marginal contribution of an input, quantifying the change in the function's output when that input is included. Recent research has expanded the scope of interpretability to encompass sets of inputs [3, 4], capturing the collective influence of input combinations and their synergies on model predictions. Central to this advancement is the M¨ obius Transform [5], a mathematical transformation that projects functions onto a fundamental interpretable basis known in game theory as the unanimity function basis .

The M¨ obius transform has a more powerful and nuanced explanation capability than the Shapley value. Consider a sentiment analysis model (BERT [6] fine-tuned on the IMDB dataset [7]) explained

f























Her

acting

never

fails

to

impress























=

F























Her

acting

never

fails

to

impress























+

F























Her

acting

never

fails

to

impress























+

F























Her

acting

never

fails

to

impress























+

F























Her

acting

never

fails

to

impress























+

· · ·

+

F























Her

acting

never

fails

to

impress























+

F























Her

acting

never

fails

to

impress























+

F























Her

acting

never

fails

to

impress























+

· · ·

+

F























Her

acting

never

fails

to

impress























+

. . .

+0

.

98

0

.

00

-

0

.

80

-

0

.

96

+0

.

59

+0

.

87

-

0

.

75

-

0

.

58

+2

.

59

0th Order

1st Order

2nd Order

3rd Order and Higher

f























Her

acting

never

fails

to

impress























=

F























Her

acting

never

fails

to

impress























+

F























Her

acting

never

fails

to

impress























+

F























Her

acting

never

fails

to

impress























+

. . .

+

F























Her

acting

never

fails

to

impress























+

. . .

-

0

.

96

0

.

00

-

0

.

96

+0

.

59

-

0

.

58

SV

[

Her

]

SV

[

acting

]

SV

[

never

]

SV

[

fails

]

SV

[

to

]

SV

[

impress

]

+0

.

05

+0

.

01

+0

.

32

+0

.

19

+0

.

10

+0

.

31

using both Shapley values and the M¨ obius transform as depicted in Fig. 1. The model's objective is to classify the sentiment of the review as positive or negative. The M¨ obius transform assigns a score to all word subsets within a sentence. For instance, in the sentence 'Her acting never fails to impress' each subset of words is evaluated-positive interactions receive positive scores, and negative interactions, negative scores. Summing these scores yields the overall sentiment +0 . 98 . This granular analysis reveals the model's understanding of linguistic constructs like double negatives, as seen in the interaction between never and fails , and the inherent positivity of words like impress . When the word never is masked, interactions involving never are excluded, shifting the sentiment negatively to -0 . 96 .

This level of detail is not readily available with the Shapley value, which assigns scores to individual words without considering their interplay. The value of the M¨ obius transform is apparent, but given its complex structure, is it possible to compute efficiently?

In general, to compute a M¨ obius transform over n features requires 2 n inferences (masking over all 2 n subsets of features), as well as n 2 n time using a divide-and-conquer approach similar to that of the Fast Fourier Transform (FFT) algorithm. GPT-4 currently supports in the range of 8000 words-per-prompt, and context length will continue to grow with new architectures [9]. Running inference 2 8000 times is not even close to possible, and even if you could, 2 8000 coefficients are hardly interpretable! In Fig. 1 we see that many coefficients are insignificant . This is typical. The solution to the computational problem is to just focus on computing the largest M¨ obius interactions and ignore the small ones. Is this possible in a systematic way? Yes-assuming that only K M¨ obius coefficients (which K values are significant is unknown) are non-zero, our algorithm enables us to intelligently query the model to significantly reduce the number of samples of that are required to O ( Kn ) with O ( Kn 2 ) time. We also explore the regime where the non-zero interactions occur between at most t inputs, with t ≪ n , showing that only O ( Kt log( n )) samples are required in O ( K poly( n )) time. We also have a robust algorithm that allows for some noise in the sampling process, effectively relaxing the constraint that the insignificant coefficients are exactly zero while maintaining the same complexities.

Defining the M¨ obius Transform We define a value function for a model with n inputs across subsets S ⊆ [ n ] denoted as f ( S ) . The construction of this function varies based on the model: in Fig. 1, words not in S might be masked or omitted. In other cases, we might take a conditional expectation over words not in S . To facilitate later discussion on group testing, we express the function as f : Z n 2 → R , where f ( S ) = f ( m ) with S = { i : m i = 1 } . The relationship between f : Z n 2 → R

and its M¨ obius transform F : Z n 2 → R is characterized by the forward and inverse transforms:

where k ≤ m means that k i ≤ m i ∀ i . This transform acts as a bridge, connecting various importance metrics, which can be expressed as projections onto a subset of the M¨ obius basis. The Shapley value SV( i ) and Banzhaf value BZ( i ) for feature i is elegantly represented within this framework:

These relationships are foundational to the definition of the Shapley and Bhanzaf values themselves. The left equality appears as Eq. 10 in Lloyd Shapley's original report from 1952 [1] where the concept of Shapley value was first introduced and is central to his derivation of a closed-form expression.

1.1 Related Works and Applications

This work is inspired by the literature on sparse Fourier transforms, which began with [10, 11, 12]. The sparse Boolean Fourier (Hadamard) transform [13, 14] is most relevant.

Group Testing This manuscript establishes a strong connection between the interaction identification problem and group testing [15]. Group testing was first described by Dorfman [16], who noted that when testing soldiers for syphilis, pooling blood samples from many soldiers, and testing the pooled blood samples reduced the total number of tests needed. [17] is the first work to exploit group testing in a feature selection/importance problem, using a group testing matrix in their algorithm. [18] also mentions group testing in relation to Shapley values.

M¨ obius Transform M¨ obius transforms [5] have been studied in the pseudo-Boolean (set) function literature, and dates back to at least [19]. [20] develops a framework for computing sparse transforms of pseudo-Boolean functions. They do not directly consider the M¨ obius transform as we define it, but one can apply their algorithm to compute a K sparse transform in O ( nK ) adaptive samples and O ( K 2 n ) time. In the sparse and noiseless setting, our algorithm improves on this by being fully nonadaptive and having lower time complexity in most non-trivial settings. [20] does not consider the important low degree setting and does not consider robustness to noise (approximate sparsity), which are critical aspects of this work. In [21], the authors show that a classifier satisfying certain properties can be well represented by a sparse and low degree M¨ obius transform.

Explainability [2] proposes model explanation via pseudo-Boolean functions approximated by Shapley values, effectively utilizing only first-order M¨ obius coefficients. Constructing these functions, [22, 23, 24, 25] especially for generative models with complex outputs [26, 27, 28], is an ongoing research area. [3] presents the Taylor-Shapley interaction index (STII), scoring interactions up to size t . For sets smaller than t , STII are exactly M¨ obius coefficients. [4] introduces the Faithful Shapley Interaction index (FSI), which computes scores via projection onto up to t th order M¨ obius coefficients. [29] develops methods for computing FSI, STII, and other interaction indices. The relationship between the M¨ obius transform, FSI, STII, Shapley value, and Banzhaf value is detailed in Appendix A.

Data Valuation In data valuation [18] the goal is to assign an importance score to data, either to determine a fair price [30] or to curate a more efficient dataset [31]. A feature of this problem is the high cost of getting a sample since we need to determine the accuracy of our model when trained on different subsets of data, making sample complexity of critical importance. [32, 33] try to approximate this by looking at the accuracy of partially trained models, though this introduces sampling noise.

1.2 Main Contributions

Our algorithm and proofs are deeply interdisciplinary , and the contributions of this paper are theoretical. We use modern ideas spanning across signal processing, algebra, coding and information theory, and group testing to address the important problem of interpretability at the forefront of machine learning. The main contributions of this manuscript are:

For a function with K non-zero M¨ obius coefficients chosen uniformly at random, the Sparse M¨ obius Transform (SMT) algorithm exactly recovers the transform F in O ( Kn ) samples and O ( Kn 2 ) time in the limit as n →∞ with K growing at most as 2 nδ with δ ≤ 1 3 .

10

0

10

1

10

2

10

3

10

4

10

5

10

6

Sparsity

0

.

00

0

.

25

0

.

50

0

.

75

1

.

00

R

2

(Faithfulness)

XGBoost for Breast Cancer Diagnosis

10

0

10

1

10

2

10

3

10

4

10

5

Sparsity

0

.

00

0

.

25

0

.

50

0

.

75

1

.

00

R

2

(Faithfulness)

BERT for Sentiment Analysis

10

0

10

1

10

2

10

3

10

4

Sparsity

0

.

00

0

.

25

0

.

50

0

.

75

1

.

00

R

2

(Faithfulness)

BERT for Multiple Choice

0

3

6

9

12

15

18

21

Degree

0

.

00

0

.

25

0

.

50

0

.

75

1

.

00

R

2

(Faithfulness)

0

3

6

9

12

15

18

Degree

0

.

00

0

.

25

0

.

50

0

.

75

1

.

00

R

2

(Faithfulness)

0

3

6

9

12

15

Degree

0

.

00

0

.

25

0

.

50

0

.

75

1

.

00

R

2

(Faithfulness)

We develop a formal connection with group testing and present a variant of SMT that works when all non-zero interactions are low order. If the maximum order of interaction is t = Θ( n α ) where α < 0 . 409 then we can compute the M¨ obius transform in O ( Kt log( n )) samples in O ( K poly( n )) time with error going to zero as n →∞ with growing K .

Using robust group testing, we develop an algorithm that, under certain assumptions, computes the M¨ obius transform in O ( Kt log( n )) samples, with vanishing error as n →∞ with growing K .

In addition to our asymptotic analysis, we provide synthetic and real-world experiments that verify that our algorithm performs well even in the finite n regime. Furthermore, our results are non-adaptive meaning that all samples can be computed in parallel. Code has been made publicly available 1 .

Notation Lowercase boldface x and uppercase boldface X denote vectors and matrices respectively. x ≥ y means that x i ≥ y i ∀ i . Multiplication is always standard real field multiplication, but addition between two elements in Z 2 should be interpreted as a logical OR ∨ . We define subtraction, of x -y for x ≥ y by standard real field subtraction. ¯ x corresponds to bit-wise negation for Boolean x , and x ⊙ y represents an element-wise multiplication.

2 Understanding Assumptions: Sparsity and Low Degree

Computing the forward transform (1) typically requires sampling all 2 n input combinations, an infeasible task, even for modest n . For an arbitrary f , one cannot do any better. In fact, the same is true of the Shapley value, yet, computational tools like SHAP [2] exist because practical functions of interest are not arbitrary . To help understand this, we define faithfulness for an explanation model ˆ f :

Note that this corresponds to the standard definition of R 2 in statistics when f is zero-mean, and we generally define f such that this is the case. A good explanation model should have a high R 2 ,

a succinct representation, and most importantly, be easily computed. For the M¨ obius transform, we aim to learn coefficients ˆ F ( k ) efficiently and construct ˆ f using the inverse transform (1). With no restrictions on ˆ F ( k ) we can achieve R 2 = 1 , but this fails to meet our simplicity criterion. Fortunately, many real-world functions are sparse -only a few ˆ F ( k ) coefficients need to be nonzero to yield R 2 ≈ 1 . Fig. 2 considers three machine learning models for breast cancer diagnosis, sentiment analysis, and question answering respectively. In all three cases, we find that we only need a small number of M¨ obius coefficients to achieve R 2 ≈ 1 . Furthermore, real-world functions are low degree , such that those small number of non-zero coefficients satisfy | k | ≤ t for some small t . This results in a much more compact representation and as we shall see, also enables efficient computation.

Fig. 2 validates our assumption for the deep-learning models mentioned above. Prior research [36, 21], have presented empirical and theoretical evidence that sparsity and low degree properties are common in well-trained models. Further investigation of the spectral properties of explanation functions could be a promising avenue for future research. Our formal statements of assumptions are given below:

̸

Assumption 2.2. ( K t -Degree Interactions) f : Z n 2 ↦→ R has a M¨ obius transform of the following form: k 1 , . . . , k K are sampled uniformly from { k : | k | ≤ t, k ∈ Z n 2 } , and have F ( k i ) = 0 , ∀ i ∈ [ K ] , but F ( k ) = 0 other k ∈ Z n 2 .

Assumption 2.1. ( K Uniform Interactions) f : Z n 2 ↦→ R has a M¨ obius transform of the following form: k 1 , . . . , k K are sampled uniformly at random from Z n 2 , and have F ( k i ) = 0 , ∀ i ∈ [ K ] , but F ( k ) = 0 for all other k ∈ Z n 2 .

̸

Assumption Limitations By assuming that the non-zero coefficients are uncorrelated and uniformly distributed, we aim to understand the fundamental difficulty in learning a sparse M¨ obius transform. Correlation between non-zero coefficients means identifying one coefficient would tell us information about the locations of the others, which can be further exploited. The existence of a scheme that works well under the uniform setting suggests that it is possible to solve the problem where correlations between interactions exist. We also consider exact sparsity in our assumptions. In practice, these 'zero' coefficients may instead have some small magnitude. We investigate this in Section 4.

3 Algorithm Overview

3.1 Subsampling and Aliasing

First we perform functional subsampling : For some b < n we construct u according to

where we have the freedom to choose m ℓ . Critically, the M¨ obius transform of u , denoted U , is related to F via the well-known signal processing phenomenon of aliasing :

where A ( j ) corresponds to an aliasing set determined by m ℓ . Fig. 3 shows this subsampling procedure on a 'sparsified' version of our sentiment analysis example using different m ℓ . Our goal is to choose m ℓ such that the non-zero values of F ( k ) are uniformly spread across the aliasing sets, since that makes them easier to recover. If only a single k with non-zero F ( k ) ends up in an aliasing set A ( j ) , we call it a singleton . In Fig. 3, our

Algorithm 1 Sparse M¨ obius Transform (SMT)

first subsampling generated two singletons, while our second one generated only one. Maximizing the number of singletons is one of our goals since we can ultimately use those singletons to construct the M¨ obius transform. In this work, we have determined two different subsampling procedures that perform well under our two assumptions:

u

1

(00)

=

f

(110011)

u

1

(01)

=

f

(110111)

u

1

(10)

=

f

(111011)

u

1

(11)

=

f

(111111)

u

2

(00)

=

f

(111100)

u

2

(01)

=

f

(111101)

u

2

(10)

=

f

(111110)

u

2

(11)

=

f

(111111)

U

1

(00)

=

0

U

1

(01)

=

F

(

k

3

)

U

1

(10)

=

F

(

k

1

)

U

1

(11)

=

F

(

k

2

) +

F

(

k

4

)

U

2

(00)

=

F

(

k

1

) +

F

(

k

2

) +

F

(

k

3

)

U

2

(01)

=

F

(

k

4

)

U

2

(10)

=

0

U

2

(11)

=

0





















Her

acting

never

fails

to

impress





















=





















0

0

1

0

0

0





















=

k

1



















Her

acting

never

fails

to

impress



















=



















0

0

0

1

0

0



















=

k

3





















Her

acting

never

fails

to

impress





















=





















0

0

1

1

0

0





















=

k

2



















Her

acting

never

fails

to

impress



















=



















0

0

1

1

0

1



















=

k

4

Aliasing 1

Aliasing 2

Non-zero Interactions

Transform

Singleton

Multiton

















Lemma 3.1. Choose m ℓ = H T ℓ , which results in A ( j ) = { k : Hk = j } . H is chosen as follows:

Under Assumption 2.2 with t = Θ( n α ) for some α ≤ 0 . 409 , there exists a matrix H chosen from b rows of a near constant column weight group testing matrix.

Under Assumption 2.1, we choose H = [ I b × b 0 b,n -b ] , or any column permutation of this matrix.

With the given H , non-zero indices are mapped to the 2 b sampling sets A ( j ) independently and uniformly at random asymptotically, thus maximizing the number of singletons when b = Θ(log( K )) .

In both cases the matrix H can be viewed as a part of a group testing matrix . This is explicit under Assumption 2.2, but under Assumption 2.1, H can be viewed as part of an individual testing matrix, where elements are tested one-by-one. These matrices are optimal in an information-theoretic sense because they achieve an optimal rate asymptotically in their respective setting. We can think of rate as the amount of information about k what we get from the matrix product Hk (see [15] for a formal definition). For example, if the matrix Hk was always the same for all k , then we would get little information about k from Hk , and we would say H has a low rate . Conversely, if Hk is different for different k then the product Hk provides more information about k . Information theoretically, maximizing the rate of H can be thought of as maximizing the entropy [37] of Hk . It is this connection between rate and the randomness of the product Hk that enables us to prove Lemma 3.1.

A detailed discussion of Lemma 3.1 is in Appendix C.2, with an enhanced version for independence across multiple H , as is required for our overall result. The proof of this lemma touches many areas of mathematics, including the theory of monoids, information theory, and optimal group testing.

3.2 Singleton Detection and Identification

Singletons are useful, but we cannot immediately use them to recover F ( k ) . Wefirst need to know that a given U ( j ) is a singleton . Secondly, we need to identify the value of k that singleton corresponds to. Section 4 discusses both tasks. For now, we discuss the rest of the algorithm assuming that we can accomplish both tasks.

3.3 Message Passing to Resolve Collisions

Since we don't know the non-zero indices beforehand, collisions between multiple non-zero indices in the same aliasing set are inevitable. These are called multitons . One approach to deal with these multitons is to repeat the procedure over again:

Recovered Singleton

Subtract

Sampling Group

c

= 1

Sampling Group

c =

2

c = 1 , . . . , C . Each time, we get different aliasing sets A c ( j ) resulting in different singletons, and thus find different k with non-zero F ( k ) . While this approach works, a better approach is to combine this idea with message passing to use known non-zero indices and values ( k , F ( k )) to resolve these multitons and turn them into singletons. The type of message passing algorithm we use is called graph peeling . The aliasing structure can be represented as a bipartite graph. Each U c ( j ) is a check node , and each non-zero coefficient F ( k ) is a variable node . The variable node F ( k ) is connected to the check node U c ( j ) if H c k = j . Fig. 4 constructs this bipartite graph for the aliasing in Fig. 3. Note that U 1 (11) = F ( k 2 ) + F ( k 4 ) is a multiton; however, in the other sub-samping group U 2 (01) = F ( k 4 ) is a singleton. Once we resolve U 2 (01) , we can simply subtract F ( k 4 ) from U 1 (11) , allowing us to create a new singleton, and extract F ( k 2 ) . The remaining values of F both appear as singletons in the first sampling group, so we can resolve all 4 non-zero interactions F with only 8 ( 7 unique) samples. Peeling algorithms were popularized in information and coding theory for decoding fountain codes [38] and have since been applied to variety of applications in communications [39] and signal processing [13, 40]. They can be analyzed using density evolution theory [41], which we also use as part of our proof.

4 Singleton Detection and Identification

We have discussed how to subsample efficiently to maximize singletons and how to use message passing to recover as many coefficients as possible. Now we discuss (1) how to identify singletons and (2) how to determine the k ∗ corresponding to the singleton. The following result is key:

Lemma 4.1. Consider H ∈ Z b × n 2 , and f : Z n 2 ↦→ R , and some d ∈ Z n 2 . If U is the M¨ obius transform of u , and F is the M¨ obius transform of f we have:

The proof can be found in Appendix C.4. The form of (6) allows us to shrink the aliasing set in a controlled way. Define d c, 0 := 0 n , and D c ∈ Z P × n 2 for some P > 0 . The i th row of D c is denoted d c,p , p = 1 . . . , P . Using these vectors, we construct C ( P +1) different subsampled functions u c,p :

Wecompute the M¨ obius transform of each u c,p denoted by U c,p and construct a vector-valued function U c ( j ) := [ U c, 0 ( j ) , . . . , U c,P ( j )] T . The goal of singleton detection is to identify when U c ( j ) reduces to a single term, and for what value k that term corresponds to. To do so, we define the Type( · ) :

Type( U c ( j )) = H Z denotes a zeroton , for which there does not exist F ( k ) = 0 s.t. Hk = j .

̸

3. Type( U c ( j )) = H M denotes a multiton with more than one k with F ( k ) = 0 s.t. Hk = j .

̸

Type( U c ( j )) = H S ( k , F ( k )) denotes a singleton with only one k with F ( k ) = 0 s.t. Hk = j .

̸

To describe our type estimation rule, we define the following ratios between elements of U c ( j ) :

and construct the vector y c := [ y c, 1 , . . . , y c,P ] T . Then, our estimate for the type is given by

By considering the definition of U c it is possible to show that if Type( U c ( j )) = H S ( k ∗ , F ( k ∗ )) , then y c = D c k ∗ . When we have a singleton taking D c = I , and thus P = n suffices to recover k ∗ . So long as the non-zero coefficients, F ( k ) are not chosen in an adversarial way, this choice of D c also ensures that Detect ( U c ( j )) = Type ( U c ( j )) . For the purposes of our formal proof, we will assume that non-zero F ( k ) are drawn from an absolutely continuous joint distribution. We can't do better if we don't have any extra information about k ∗ , but we can if we know | k ∗ | ≤ t as we show below. Going back to our example in Fig. 3, with D c = I we use a total of 8 × 7 = 56 samples as opposed to 2 6 = 64 . While this improvement is modest at this scale, for larger problems the improvement is dramatic.

Singleton Identification in the Low Degree Setting Let's say we want to determine the singleton from U 1 (10) in Fig. 3, and we know | k ∗ | ≤ 1 . By exploiting group testing, it is possible to design D c with fewer rows, and thus fewer overall measurements:

Decode

y

k

1

k

1

=

D

c

=

y

Her

acting

never

fails

to

impress

0

0

0

1

1

1

0

0

1

1

0

0

1

1

1

0

1

0

1

0

1

The matrix product D c k ∗ has a different output for each | k | ≤ 1 . In this case, the result y corresponds to the binary index of the location of the 1 . It requires P = 3 , rather than the P = 6 for D c = I . If all non-zero F ( k ) had satisfied | k | ≤ 1 , we could use this matrix for our example in Fig. 3. However, we only have | k | ≤ 3 for non-zero F ( k ) in this example, so D c as in (5) does not suffice. In the case of general | k | ≤ t , [42] says that for any scaling of t with n , there exists a group testing design D c with P = O ( t log( n )) that can recover k ∗ in the limit as n → ∞ with vanishing error in poly( n ) time. If we also assume that F ( k ) are Detect ( U c ( j )) has vanishing error (see Appendix C.7.2).

Extension to Noisy Setting We now relax the assumption that most of the coefficients are exactly zero. To do this, we assume each subsampled M¨ obius coefficient is corrupted by noise:

where Z c,p ( j ) i.i.d. ∼ N (0 , σ 2 ) . There are two main changes that must be made compared to the noiseless case. First, we must place an assumption on the magnitude of non-zero coefficients | F ( k i ) | , such that the signal-to-noise ratio (SNR) remains fixed. Secondly, the matrix D c must be modified. It now consists of two parts: D c = [ D 1 c ; D 2 c ] . We design D 2 c ∈ Z P 2 × n 2 as a standard noise robust Bernoulli group testing matrix with P 2 = O ( t log( n )) tests, which suffices for singleton identification under any fixed SNR [43]. However, unlike the noiseless case, the samples from the rows of D 2 c are not enough to ensure a vanishing error for singleton detection in the Detect ( · ) procedure. To solve this, we design D 1 c ∈ Z P 1 × n 2 as a Bernoulli group testing matrix with a different parameter. In Appendix C.7.4, we show this modified version of Detect ( · ) has vanishing error if P 1 = O ( t log( n )) .

5 Results

Now that we have discussed all components of the algorithm, we present our theoretical guarantees:

Theorem 5.1. (Recovery with K Uniform Interactions) Let f satisfy Assumption 2.1 for some K = O (2 nδ ) with δ ≤ 1 3 and let the non-zero coefficients of F be drawn from an absolutely continuous distribution. For { H c } C c =1 chosen as in Lemma C.3 with b = O (log( K )) , C = 3 and D c = I , Algorithm 1 exactly computes the transform F in O ( Kn ) samples and O ( Kn 2 ) time complexity with probability at least 1 -O (1 /K ) .

Theorem 5.2. (Noise-Robust Recovery with K t -Degree Interactions) Let f satisfy Assumption 2.2 for K = O (poly( n )) and t = Θ( n α ) with α ≤ 0 . 409 . Assume either:

The non-zero coefficients of F are drawn from an arbitrary continuous distribution, or

U c,p is corrupted by noise as in (10) and let non-zero coefficients satisfy | F ( k ) | = ρ .

Then, for { H c } C c =1 chosen as in Lemma C.4 with b = O (log( K )) , C = 3 , and D c chosen as a suitable group testing matrix, Algorithm 1 exactly computes the transform F in O ( Kt log( n )) samples and O ( K poly( n )) time complexity with probability at least 1 -O (1 /K ) in both the noiseless case (1) and noisy case (2).

2

100

2

200

2

300

2

400

2

500

2

600

2

700

2

800

2

900

2

1000

100

200

300

400

500

600

700

800

900 1000

n

N

10

4

10

5

10

6

Sample Complexity

0%

20%

40%

60%

80%

100%

Perfect Reconstruction %

(a)

0

5

10

15

20

25

SNR (dB)

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

R

2

(Faithfulness)

t

= 6

t

= 8

t

= 10

t

= 12

(b)

2

8

2

10

2

12

2

14

2

16

2

18

2

20

2

22

2

24

2

26

2

28

2

30

8

10

12

14

16

18

20

22

24

26

28

30

n

N

0

1

2

3

4

5

6

7

8

9

10

Runtime Complexity (sec)

SHAP IQ

LASSO

SMT

(c)

0

6

12

18

24

30

36

42

48

54

Sparsity

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

R

2

(Faithfulness)

XGBoost for Breast Cancer Diagnosis

(

n

= 30)

0

5

10

15

20

25

30

35

40

Sparsity

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

R

2

(Faithfulness)

BERT for Sentiment Analysis

(

n

≈

25)

0

3

6

9

12

15

18

21

Sparsity

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

R

2

(Faithfulness)

BERT for Multiple Choice

(

n

= 15)

Shapley Values

Banzhaf Values

Faith-Banzhaf Indices

SMT (Algorithm 1)

The proof of Theorem 5.1 and 5.2 is provided in Appendix C.5. It combines results from all of the parts of the algorithm we have discussed: aliasing, singleton detection and graph peeling. The requirement | F ( k ) | = ρ is only due to limitations of group testing theory. In practice, we observe that a lower bound on the magnitude suffices. In addition to our theoretical results, we also conduct numerical experiments on synthetic and real word models, which are discussed below.

5.1 Synthetic Simulations

We tested SMT's efficacy on functions satisfying Assumption 2.1 and 2.2, setting non-zero F ( k ) uniformly in [ -1 , 1] . SMT is implemented as in Algorithm 1, with group testing decoding via linear programming (see Appendix F.2). Fig. 6a is a phase transition plot that shows the percent of cases where SMT achieves R 2 = 1 with fixed K = 100 at different sample complexities and values of n . We vastly outperform the naive approach: when n = 1000 , we get perfect reconstruction with only 10 -294 percent of total samples! Furthermore, the number of samples necessary to achieve perfect reconstruction scales linearly in n as predicted. Fig. 6b assesses SMT under noise for various values

of t , plotting R 2 against SNR with K = 500 , n = 500 , and P = 1000 . Fig. 6c plots the runtime for SMT and competing methods. Test functions f have K = 10 non-zero M¨ obius coefficients at locations that satisfy | k | = 5 (restricted to equality due to limitations in the SHAP-IQ code at the time of running). We compare against SHAP-IQ [29] configured to compute 5 th order FSI, as well as the method of [4] which computes 5 th order FSI via LASSO. As shown in Appendix A, the t th order FSI are exactly the t th order M¨ obius coefficients for our chosen f . This figure exemplifies that SMT is the sole feasible method for identifying interactions on the scale of n ≥ 100 . Additional simulations and discussion can be found in Appendix E.

5.2 Real-World Models

Our objective is a computable, faithful, and compact representation of real-world machine-learned functions. Fig. 7 addresses this goal head-on, by plotting R 2 against the number of terms used in the representation (sparsity) for SMT and other popular model explanation approaches. We consider three different tasks: The first is an XGboost model for breast cancer diagnosis, and the other two are transformer-based BERT models for the tasks of sentiment analysis and multiple choice question answering respectively. Appendix B discusses the setup in great detail. For Shapley and Banzhaf values, to generate an r -sparse representation, we use the top r magnitude values. For SMT and FaithBanzhaf, we do a slightly more sophisticated refinement procedure using LASSO [44], described in the Appendix. We observe that for the breast cancer and sentiment analysis tasks, SMT can generate representations that, with the same number of terms, achieve a much higher R 2 . This is done by identifying interactions between inputs that are important to the model output. Interestingly, in the case of the multiple choice model, there is less of a difference between the Faith-Banzhaf Indices and the SMT representations. This is likely because in the corresponding dataset, answers to the questions are usually contained in single sentences, making interactions less important.

6 Conclusion

Identifying interactions between inputs is an important open research question in machine learning, with applications to explainability, data valuation, and many other problems. We approached this problem by studying the M¨ obius transform, which is a representation over the fundamental interaction basis. We introduced several new tools to the problem of identifying interactions. The use of ideas from sparse signal processing and group testing has allowed SMT to operate in regimes where all other methods fail due to computational burden. Our theoretical results guarantee asymptotic exact reconstruction and are complemented by numerical simulations that show SMT performs well with finite parameters and also under a noisy model.

Limitations Our assumption of independently sampled interactions was made for informationtheoretic hardness and may not hold in some settings where correlated interactions exist. For instance, in the sentiment problem in Fig. 1, words with strong 2 nd order interactions are likely to appear together in important 3 rd order interactions. In such settings, correlation is exploitable, so a more specific algorithm can likely exploit this correlation and eliminate this assumption. Another limitation is that we have focused on taking a sparse M¨ obius transform in this work. In practice, we may be more interested in taking a sparse projection onto a subset of low-order terms. For unitary transforms, projection can be achieved by truncation, however, with non-orthogonal transforms like the M¨ obius Transform, projection is not straightforward. This is an important distinction, because there can be functions which are well-approximated by a sparse low degree M¨ obius projection, but do not have a sparse transform.

Future Work Applying SMT to real-world tasks like understanding protein language models [45], LLM chatbots [46] or diffusion models [47] would be insightful. Working with large and complicated models will likely require further improvements to robustness-both in terms of dealing with noise from small but non-zero interactions and dealing with potential correlations between interactions. Some interesting ideas in this direction could be using more standard statistical ideas like in [29] or considering concepts from adaptive group testing. Finally, due to the connection with the Shapley or Banzhaf values (2), methods for computing the M¨ obius transform can also be used for computing these values by first computing the M¨ obius transform and then using (2).

References

L. S. Shapley, A Value for N-Person Games . Santa Monica, CA: RAND Corporation, 1952.

S. M. Lundberg and S.-I. Lee, 'A unified approach to interpreting model predictions,' in Proceedings of the 31st International Conference on Neural Information Processing Systems , ser. NIPS'17. Red Hook, NY, USA: Curran Associates Inc., 2017, p. 4768-4777.

M. Sundararajan, K. Dhamdhere, and A. Agarwal, 'The Shapley Taylor interaction index,' in International Conference on Machine Learning , Jul 2020, pp. 9259-9268. [Online]. Available: https://proceedings.mlr.press/v119/sundararajan20a.html

C.-P. Tsai, C.-K. Yeh, and P. Ravikumar, 'Faith-shap: The faithful Shapley interaction index,' Journal of Machine Learning Research , vol. 24, no. 94, pp. 1-42, 2023.

M. Grabisch, J.-L. Marichal, and M. Roubens, 'Equivalent Representations of Set Functions,' Mathematics of Operations Research , vol. 25, no. 2, pp. 157-178, May 2000. [Online]. Available: https://pubsonline.informs.org/doi/10.1287/moor.25.2.157.12225

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, 'BERT: Pre-training of deep bidirectional transformers for language understanding,' in North American Chapter of the Association for Computational Linguistics , 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID: 52967399

J. Lee, 'IMDB Finetuned BERT-base-uncased,' 2023, accessed Jan 2024. [Online]. Available: https://huggingface.co/JiaqiLee/imdb-finetuned-bert-base-uncased

J. M. P´ erez, J. C. Giudici, and F. Luque, 'pysentimiento: A python toolkit for sentiment analysis and SocialNLP tasks,' 2021.

T. Dao, D. Fu, S. Ermon, A. Rudra, and C. R´ e, 'Flashattention: Fast and memory-efficient exact attention with IO-awareness,' Advances in Neural Information Processing Systems , vol. 35, pp. 16 344-16 359, 2022.

H. Hassanieh, P. Indyk, D. Katabi, and E. Price, 'Simple and practical algorithm for sparse Fourier transform,' in SIAM Symposium on Discrete Algorithms (SODA) , 2012, pp. 1183-1194. [Online]. Available: https://epubs.siam.org/doi/abs/10.1137/1.9781611973099.93

P. Stobbe and A. Krause, 'Learning Fourier sparse set functions,' in International Conference on Artificial Intelligence and Statistics (AISTATS) , ser. Proceedings of Machine Learning Research, La Palma, Canary Islands, Apr 2012, pp. 1125-1133. [Online]. Available: https://proceedings.mlr.press/v22/stobbe12.html

S. Pawar and K. Ramchandran, 'Computing a k -sparse n -length Discrete Fourier Transform using at most 4 k samples and O ( klogk ) complexity,' in IEEE International Symposium on Information Theory (ISIT) , 2013, pp. 464-468.

X. Li, J. K. Bradley, S. Pawar, and K. Ramchandran, 'The SPRIGHT algorithm for robust sparse Hadamard Transforms,' in IEEE International Symposium on Information Theory (ISIT) , 2014, pp. 1857-1861.

A. Amrollahi, A. Zandieh, M. Kapralov, and A. Krause, 'Efficiently learning fourier sparse set functions,' Advances in Neural Information Processing Systems , vol. 32, 2019.

M. Aldridge, O. Johnson, and J. Scarlett, 'Group testing: An information theory perspective,' Foundations and Trends® in Communications and Information Theory , vol. 15, no. 3-4, p. 196-392, 2019. [Online]. Available: http://dx.doi.org/10.1561/0100000099

R. Dorfman, 'The detection of defective members of large populations,' The Annals of mathematical statistics , vol. 14, no. 4, pp. 436-440, 1943.

Y. Zhou, U. Porwal, C. Zhang, H. Q. Ngo, X. Nguyen, C. R´ e, and V. Govindaraju, 'Parallel feature selection inspired by group testing,' in Advances in Neural Information Processing Systems , vol. 27, 2014.

R. Jia, D. Dao, B. Wang, F. A. Hubis, N. Hynes, N. M. G¨ urel, B. Li, C. Zhang, D. Song, and C. J. Spanos, 'Towards efficient data valuation based on the shapley value,' in International Conference on Artificial Intelligence and Statistics (AISTATS) , vol. 89, 16-18 Apr 2019, pp. 1167-1176. [Online]. Available: https://proceedings.mlr.press/v89/jia19a.html

J. C. Harsanyi, 'A bargaining model for the cooperative n -person game,' Ph.D. dissertation, Department of Economics, Stanford University, Stanford, CA, USA, 1958.

C. Wendler, A. Amrollahi, B. Seifert, A. Krause, and M. P¨ uschel, 'Learning set functions that are sparse in non-orthogonal Fourier bases,' in AAAI Conference on Artificial Intelligence , vol. 35, 2021, pp. 10 283-10 292.

Q. Ren, J. Gao, W. Shen, and Q. Zhang, 'Where we have arrived in proving the emergence of sparse interaction primitives in DNNs,' in International Conference on Learning Representations , 2024. [Online]. Available: https://openreview.net/forum?id=3pWSL8My6B

K. Aas, M. Jullum, and A. Løland, 'Explaining individual predictions when features are dependent: More accurate approximations to Shapley values,' Artificial Intelligence , vol. 298, p. 103502, 2021.

H. Chen, J. D. Janizek, S. Lundberg, and S.-I. Lee, 'True to the model or true to the data?' 2020. [Online]. Available: https://arxiv.org/pdf/2006.16234

D. Janzing, L. Minorics, and P. Bl¨ obaum, 'Feature relevance quantification in explainable AI: A causal problem,' in International Conference on Artificial Intelligence and Statistics , 2020, pp. 2907-2916.

J. Ren, Z. Zhou, Q. Chen, and Q. Zhang, 'Can we faithfully represent absence states to compute shapley values on a DNN?' in International Conference on Learning Representations , 2023. [Online]. Available: https://openreview.net/forum?id=YV8tP7bW6Kt

J. Enouen, H. Nakhost, S. Ebrahimi, S. O. Arik, Y. Liu, and T. Pfister, 'TextGenSHAP: Scalable post-hoc explanations in text generation with long documents,' 2023. [Online]. Available: https://arxiv.org/pdf/2312.01279

V. Miglani, A. Yang, A. Markosyan, D. Garcia-Olano, and N. Kokhlikyan, 'Using Captum to explain generative language models,' in Workshop for Natural Language Processing Open Source Software (NLP-OSS 2023) , Singapore, Dec. 2023, pp. 165-173. [Online]. Available: https://aclanthology.org/2023.nlposs-1.19

L. M. Paes, D. Wei, H. J. Do, H. Strobelt, R. Luss, A. Dhurandhar, M. Nagireddy, K. N. Ramamurthy, P. Sattigeri, W. Geyer et al. , 'Multi-level explanations for generative language models,' 2024. [Online]. Available: https://arxiv.org/pdf/2403.14459

F. Fumagalli, M. Muschalik, P. Kolpaczki, E. H¨ ullermeier, and B. E. Hammer, 'SHAP-IQ: Unified approximation of any-order shapley interactions,' in Conference on Neural Information Processing Systems , 2023. [Online]. Available: https://openreview.net/forum?id=IEMLNF4gK4

J. S. Kang, R. Pedarsani, and K. Ramchandran, 'The fair value of data under heterogeneous privacy constraints in federated learning,' Transactions on Machine Learning Research , 2024. [Online]. Available: https://openreview.net/forum?id=ynG5Ak7n7Q

J. T. Wang and R. Jia, 'Data Banzhaf: A robust data valuation framework for machine learning,' in International Conference on Artificial Intelligence and Statistics , vol. 206, 25-27 Apr 2023, pp. 6388-6421. [Online]. Available: https://proceedings.mlr.press/v206/wang23e.html

A. Ghorbani and J. Zou, 'Data Shapley: Equitable valuation of data for machine learning,' in International Conference on Machine Learning , vol. 97, 09-15 Jun 2019, pp. 2242-2251. [Online]. Available: https://proceedings.mlr.press/v97/ghorbani19c.html

A. Ghorbani, M. Kim, and J. Zou, 'A distributional framework for data valuation,' in International Conference on Machine Learning , vol. 119, 13-18 Jul 2020, pp. 3535-3544. [Online]. Available: https://proceedings.mlr.press/v119/ghorbani20a.html

T. Chen and C. Guestrin, 'XGBoost: A scalable tree boosting system,' in ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , San Francisco, 2016, pp. 785-794. [Online]. Available: http://doi.acm.org/10.1145/2939672.2939785

A. Barbosa, 'Roberta large fine tuned on race,' 2021. [Online]. Available: https://huggingface. co/LIAMF-USP/roberta-large-finetuned-race/tree/main

Q. Ren, Y. Xu, J. Zhang, Y. Xin, D. Liu, and Q. Zhang, 'Towards the dynamics of a DNN learning symbolic interactions,' 2024. [Online]. Available: https://arxiv.org/pdf/2407.19198

T. M. Cover, Elements of Information Theory . John Wiley & Sons, 1999.

M. Luby, 'LT codes,' in IEEE Symposium on Foundations of Computer Science , 2002, pp. 271280.

K. R. Narayanan and H. D. Pfister, 'Iterative collision resolution for slotted aloha: An optimal uncoordinated transmission policy,' in International Symposium on Turbo Codes and Iterative Information Processing (ISTC) , 2012, pp. 136-139.

Y. E. Erginbas, J. Kang, A. Aghazadeh, and K. Ramchandran, 'Efficiently computing sparse fourier transforms of q-ary functions,' in IEEE International Symposium on Information Theory (ISIT) , 2023, pp. 513-518.

S.-Y. Chung, T. Richardson, and R. Urbanke, 'Analysis of sum-product decoding of low-density parity-check codes using a gaussian approximation,' IEEE Transactions on Information Theory , vol. 47, no. 2, pp. 657-670, 2001.

W. H. Bay, J. Scarlett, and E. Price, 'Optimal non-adaptive probabilistic group testing in general sparsity regimes,' Information and Inference: A Journal of the IMA , vol. 11, no. 3, pp. 1037-1053, 02 2022. [Online]. Available: https://doi.org/10.1093/imaiai/iaab020

J. Scarlett and O. Johnson, 'Noisy non-adaptive group testing: A (near-)definite defectives approach,' IEEE Transactions on Information Theory , vol. 66, no. 6, pp. 3775-3797, 2020.

R. Tibshirani, 'Regression shrinkage and selection via the lasso,' Journal of the Royal Statistical Society Series B: Statistical Methodology , vol. 58, no. 1, pp. 267-288, 1996.

Z. Lin, H. Akin, R. Rao, B. Hie, Z. Zhu, W. Lu, N. Smetanin, R. Verkuil, O. Kabeli, Y. Shmueli, A. dos Santos Costa, M. Fazel-Zarandi, T. Sercu, S. Candido, and A. Rives, 'Evolutionary-scale prediction of atomic-level protein structure with a language model,' Science , vol. 379, no. 6637, pp. 1123-1130, 2023. [Online]. Available: https://www.science.org/doi/abs/10.1126/science. ade2574

OpenAI, 'GPT-4 technical report,' 2023.

D. Kingma, T. Salimans, B. Poole, and J. Ho, 'Variational diffusion models,' in Advances in Neural Information Processing Systems , vol. 34, 2021, pp. 21 696-21 707.

P. L. Hammer and R. Holzman, 'Approximations of pseudo-boolean functions; applications to game theory,' Zeitschrift f¨ ur Operations Research , vol. 36, no. 1, pp. 3-21, 1992. [Online]. Available: https://doi.org/10.1007/BF01541028

W. Wolberg, O. Mangasarian, N. Street, and W. Street, 'Breast Cancer Wisconsin (Diagnostic),' UCI Machine Learning Repository, 1995, DOI: https://doi.org/10.24432/C5DW2B.

D. Q. Nguyen, T. Vu, and A. T. Nguyen, 'BERTweet: A pre-trained language model for English Tweets,' in Conference on Empirical Methods in Natural Language Processing: System Demonstrations , 2020, pp. 9-14.

A. L. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Y. Ng, and C. Potts, 'Learning word vectors for sentiment analysis,' in Annual Meeting of the Association for Computational Linguistics: Human Language Technologies . Portland, Oregon, USA: Association for Computational Linguistics, June 2011, pp. 142-150. [Online]. Available: http://www.aclweb.org/anthology/ P11-1015

G. Lai, Q. Xie, H. Liu, Y. Yang, and E. Hovy, 'RACE: Large-scale ReAding comprehension dataset from examinations,' in Conference on Empirical Methods in Natural Language Processing , Copenhagen, Denmark, Sep. 2017, pp. 785-794. [Online]. Available: https: //aclanthology.org/D17-1082

A. Coja-Oghlan, O. Gebhard, M. Hahn-Klimroth, and P. Loick, 'Information-theoretic and algorithmic thresholds for group testing,' IEEE Transactions on Information Theory , vol. 66, no. 12, pp. 7911-7928, 2020.

C. L. Chan, S. Jaggi, V. Saligrama, and S. Agnihotri, 'Non-adaptive group testing: Explicit bounds and novel algorithms,' IEEE Transactions on Information Theory , vol. 60, no. 5, pp. 3019-3035, 2014.

A Relationship between M¨ obius Transform and Other Importance Metrics

We begin with some notation. We define the M¨ obius basis function (which are all possible products of inputs) as:

Now we define the following sub-spaces of pseudo-Boolean function in terms of the linear span of M¨ obius basis functions:

Now we define the projection operator Proj µ ( f, D ) , as the projection of the function f onto the function space D with respect to the measure µ . If g ( m ) = Proj µ ( f, D ) , we write its decomposition as g ( m ) = ∑ k ∈ Z n 2 c ( f, D , µ, k ) b k ( m ) . Note that linear independence implies the uniqueness of this representation.

Shapley Value The Shapley values SV( i ) [1] of the inputs m i , i = 1 , . . . , n with respect to the function f are [48]:

where σ is the Shapley kernel. SV( i ) = F ( e i ) when f is a linear function.

Banzhaf Index The Banzhaf index BZ( i ) of the inputs m i , i = 1 , . . . , n with respect to the function f are [48]:

where µ is the uniform measure. BZ( i ) = F ( e i ) when f is a linear function.

Faith Shapley Interaction Index The t th order Faith Shapley interaction index SV t ( k ) for | k | ≤ t [4] is where σ is the Shapley kernel. SV t ( k ) = F ( k ) when f is a t th order function, i.e., F ( k ) = 0 when | k | > t .

Faith Banzhaf Interaction Index The t th order Faith Shapley interaction index BZ t ( k ) for | k | ≤ t [4] is

where µ is the uniform measure. BZ t ( i ) = F ( k ) when f is a t th order function, i.e., F ( k ) = 0 when | k | > t .

Shapley-Taylor Interaction Index The t th order Shapley-Taylor Interaction Index [3] STII t ( k ) is:

where σ is the Shapley kernel. Explicitly, it can be shown that:

As a consequence of the above, we have STII t ( k ) = SV t ( k ) = F ( k ) when f is a t th order function, i.e., F ( k ) = 0 for | k | > t .

B Experiment Details

Let f be the real-world function we wish to explain. In subsections B.1, B.2, and B.3, we describe how we formed these functions for the tasks of breast cancer diagnosis, sentiment analysis, and multiple choice answering respectively. For our experiments, we plot the R 2 (faithfulness) for a variety of explanation models ˆ f , measured through:

In Figure 2, we consider settings where n ≈ 20 , such that we can run optimization procedures to find faithful approximations that are sparse and low degree .

∥ where we use the notation ∥ f ∥ 2 2 = ∑ m ∈ Z n 2 f ( m ) 2 .

Achievable Low Degree: To find the best approximation ˆ f of up to degree t , we solve the following quadratic programming problem:

Achievable Sparsity: On the other hand, we cannot efficiently find the optimal faithful K -sparse approximation due to the problem's combinatorial nature. Instead, informed by the strong faithfulness of low degree approximations, we employ the following heuristic to obtain some sparse approximation.

Let S K ⊆ Z n 2 be a set containing the first K coordinates with the lowest degree, where ties are randomly broken. With this set, we solve the following quadratic programming problem:

In Figure 7, we consider the four explanation models described below.

Shapley Values: We approximate Shapley values by iterating through permutations of the inputs [2]. For an efficient implementation of the algorithm, we use the SHAP Python package [2]. To measure the faithfulness captured by Shapley values at some sparsity level r , we consider approximations that only include the topr Shapley values by magnitude.

Banzhaf Values: We approximate Banzhaf values using the Maximum Sample Reuse Monte Carlo procedure described in [31]. To measure the faithfulness captured by Banzhaf values at some sparsity level r , we consider approximations that only include the topr Banzhaf values by magnitude.

Faith-Banzhaf Indices: We calculate Faith-Banzhaf indices using the regression formulation described in [4]. To measure the faithfulness captured by sparse approximations of Faith-Banzhaf indices, we modify the regression problem by adding an ℓ 1 penalty on the values of the Faith-Banzhaf indices. We vary the penalty coefficient to obtain different levels of sparsity.

SMT: We run SMT (Algorithm 1) to obtain a sparse M¨ obius representation ˆ F with support supp( ˆ F ) . Then, we fine-tune the values of the coefficients by solving the following regression problem over a uniformly sampled set of points D ⊆ Z n 2 :

To measure the faithfulness captured by sparse approximations, we modify the regression problem by adding an ℓ 1 penalty on the values of the M¨ obius coefficients. Then, we vary the penalty coefficient λ to obtain different levels of sparsity:

B.1 XGBoost for Breast Cancer Diagnosis

We train an XGBoost model for classification using the Wisconsin Breast Cancer dataset [49]. This dataset contains the mean, standard deviation, and largest value of ten measurements, resulting in thirty features. For Figure 2, we use only the mean and standard deviation, resulting in twenty features. For Figure 2, we use the first ten data points in the training set and for Figure 7, we present the aggregated results over the first twenty.

To explain the XGBoost model h (the probability associated with a positive classification) on each data point x ∈ X , we use an interventional expected value formulation: we freeze some of the features and take an expectation over all data points by infilling the remaining features. Formally,

where we use the notation x m = { x i : m i = 1 } .

B.2 BERT for Sentiment Analysis

We employ the sentiment analysis model from [8], which is built upon BERTweet [50], a RoBERTa model trained on English tweets. We take movie reviews from the IMDb Movie Reviews dataset [51]. For a particular review, we define its function as a mapping from maskings of words (using the [UNK] token) to the model's logit value associated to the correct sentiment classification.

For Figure 2, we use the first ten sentences in the dataset with 17, 18, or 19 words, where words separated through spaces in the review. Below, we include the reviews and their low degree and sparse approximations calculated with equations 20 and 22 respectively.

0

3

6

9

12

15

18

Degree

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

R

2

(Faithfulness)

(a)

0

3

6

9

12

15

18

Degree

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

R

2

(Faithfulness)

(b)

0

3

6

9

12

15

18

Degree

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

R

2

(Faithfulness)

(c)

0

3

6

9

12

15

18

Degree

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

R

2

(Faithfulness)

(d)

B.3 BERT for Multiple Choice

For multiple choice answering, we use a RoBERTa model [35] fine-tuned on RACE [52]: a largescale reading comprehension dataset. This dataset contains over 28,000 passages, each containing

0

3

6

9

12

15

18

Degree

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

R

2

(Faithfulness)

(e)

0

3

6

9

12

15

18

Degree

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

R

2

(Faithfulness)

(f)

0

3

6

9

12

15

18

Degree

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

R

2

(Faithfulness)

(g)

0

3

6

9

12

15

18

Degree

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

R

2

(Faithfulness)

(h)

0

3

6

9

12

15

18

Degree

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

R

2

(Faithfulness)

(i)

0

3

6

9

12

15

18

Degree

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

R

2

(Faithfulness)

(j)

10

0

10

1

10

2

10

3

10

4

10

5

Sparsity

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

R

2

(Faithfulness)

(a)

10

0

10

1

10

2

10

3

10

4

10

5

Sparsity

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

R

2

(Faithfulness)

(b)

10

0

10

1

10

2

10

3

10

4

10

5

Sparsity

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

R

2

(Faithfulness)

(c)

10

0

10

1

10

2

10

3

10

4

10

5

Sparsity

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

R

2

(Faithfulness)

(d)

10

0

10

1

10

2

10

3

10

4

10

5

Sparsity

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

R

2

(Faithfulness)

(e)

10

0

10

1

10

2

10

3

10

4

10

5

Sparsity

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

R

2

(Faithfulness)

(f)

10

0

10

1

10

2

10

3

10

4

10

5

Sparsity

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

R

2

(Faithfulness)

(g)

10

0

10

1

10

2

10

3

10

4

10

5

Sparsity

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

R

2

(Faithfulness)

(h)

10

0

10

1

10

2

10

3

10

4

10

5

Sparsity

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

R

2

(Faithfulness)

(i)

10

0

10

1

10

2

10

3

10

4

10

5

Sparsity

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

R

2

(Faithfulness)

(j)

corresponding multiple-choice questions. For our experiments, we found the first ten passages with 15 sentences, and took their first multiple-choice question.

To construct the function, we consider sentence-level maskings of the passages using the [PAD] token. We pass the masked passage, alongside the multiple choice question into the RoBERTa model, and measure the logit value of the question's correct answer.

C Missing Proofs

C.1 Boolean Arithmetic

Table 1 the addition and multiplication table for arithmetic between x, y ∈ Z 2 . We also note that Z 2 is typically used to refer to the integer ring modulo 2 . The arithmetic we are describing here is actually that of a monoid . Since the audience for this paper is people interested in machine learning, we continue to use Z 2 since it is commonly used to simply refer to the set { 0 , 1 } .

Table 1: Addition, Multiplication and Subtraction table for Boolean arithmetic in this paper. Subtraction is for y -x .

C.2 Discussion of Aliasing of the M¨ obius Transform

When a function has many small or zero Mobius coefficients (interactions), our goal is to subsample (4) in such a way that the aliasing causes the non-zero coefficients to end up in different aliasing sets (5) (as opposed to all of them being aliased together, making them more difficult to reconstruct). Lemma C.1 is a key tool that we will use in this work to design subsampling patterns that result in good aliasing patterns.

Lemma C.1. Consider H ∈ Z b × n 2 , b < n and f : Z n 2 ↦→ R . Let

If U is the Mobius transform of u , and F is the Mobius transform of f we have:

This lemma is a powerful tool, allowing us to control the aliasing sets through the matrix H . The proof can be found in Appendix C.3, and is straightforward, given the relationship between u and

f . Understanding why we choose this relationship, however, is more complicated. Underlying this choice is the algebraic theory of monoids and abstract algebra.

As we have mentioned, our ultimate goal is to design H to sufficiently 'spread out' the non-zero indices among the aliasing sets. Below, we define a simple and useful construction for H .

Definition C.2. Consider { i 1 , . . . , i b } = I ⊂ [ n ] , with | I | = b , and H ∈ Z b × n 2 . Let h i correspond to the i th row of H , given by h i = e i j , the length n unit vector in coordinate i j . Then if we subsample according to (23) we have:

which happens to result in aliasing sets A ( j ) = { k : k i = j i ∀ i ∈ I } all of equal size 2 b . The above choice H actually induces a rather simple sampling procedure when we follow (23). For instance if I = [ b ] , we have:

In other words, in this case, we construct samples by freezing n -b of the inputs to 1 and then varying the remaining b inputs across all the 2 b possible options. In the case where the non-zero Mobius interactions are chosen uniformly at random, this construction does a good job at spacing them out across the various aliasing sets. The following result formalizes this.

̸

Lemma C.3. (Uniform interactions) Let k 1 , . . . , k K be sampled uniformly at random from Z n 2 , where F ( k i ) = 0 , ∀ i ∈ [ K ] , but F ( k ) = 0 for all other k ∈ Z n 2 . Construct disjoint sets I c ⊂ [ n ] for c = 1 , . . . , C , and the corresponding matrix H c according to Definition C.2. Let A c ( j ) correspond to the aliasing sets after sampling with respect to matrix H c . Now define:

Then if b = O (log( K )) , K = O (2 n/C ) , in the limit as n → ∞ with C = O (1) , j c i are mutually independent and uniformly distributed over Z b 2 .

̸

The proof is given in Appendix C.6.1, and follows directly from the form of the aliasing sets A c ( j ) . Corollary C.3 means that using H as constructed in Definition C.2 ensures that we all k with F ( k ) = 0 are uniformly distributed over the aliasing sets, which maximizes the number of singletons. This result, however, hinges on the fact that the non-zero coefficients are uniformly distributed. We are also interested in the case where the non-zero coefficients are all low degree. In order to induce a uniform distribution in this case, we need to exploit a group testing matrix.

̸

Lemma C.4. (low degree interactions) Let k 1 , . . . , k K be sampled uniformly at random from { k : | k | ≤ t, k ∈ Z n 2 } , where F ( k i ) = 0 , ∀ i ∈ [ K ] , but F ( k ) = 0 for all other k ∈ Z n 2 . By constructing C matrices H c , c = 1 , . . . , C from rows of a near constant column weight group testing matrix, and sampling as in (23) , if t = Θ( n α ) for α < 0 . 409 , and b = O (log( K )) , K = O ( n t ) , in the limit as n →∞ , j c i as defined in (27) are mutually independent and uniformly distributed over Z b 2 .

The proof is given in Appendix C.6.2. It relies on an information theoretic argument, exploiting a result from optimal group testing [53].

C.3 Proof of Lemma C.1

Proof. Taking the Mobius transform of u gives us:

Now let's just focus on the term in the parenthesis for now, which we have called I ( r ) .

Case 1: Hr = k

First note that under this condition, ℓ = k = ⇒ r ≤ ⊙ i : ℓ i =0 h i . To see this, note that k j = 0 = ⇒ r ≤ h j . Since this holds for all j such that k j = 0 , we have the previously mentioned implication.

Conversely, if ℓ j < k j (this means ℓ j = 0 AND k j = 1 ) for some j , then r and h j must overlap. Thus,

We can split I ( r ) into two parts, the part where ℓ = k and the part where ℓ < k :

̸

̸

Finally, we have the case where k ′ < k . First, if there is a coordinate j such that 0 = ℓ j < k ′ j = 1 , we know that 1 { r ≤ h j } = 0 so we have 1 { r ≤ ⊙ i : ℓ i =0 h i } = 0 ∀ ℓ s.t. ∃ j, ℓ j < k ′ j . The only ℓ that remain are those such that k ′ ≤ ℓ ≤ k . It is easy to see that this is a sufficient condition for 1 { r ≤ ⊙ i : ℓ i =0 h i } = 1 .

Case 2: Hr = k Let Hr = k ′ = k . This case itself will be broken into two parts. First let's say there is some j such that k j = 0 and k ′ j = 1 . Since k ′ j = 1 we know that 1 { r ≤ h j } = 0 . Furthermore, since ∀ ℓ ∈ { ℓ : ℓ ≤ k } we have ℓ j = 0 . Then by a similar argument to our previous one, we have 1 { r ≤ ⊙ i : ℓ i =0 h i } = 0 ∀ ℓ ≤ k . It follows immediately that I ( r ) = 0 in this case.

Where the final sum is zero because exactly half of the ℓ have even and odd parity respectively.

Thus, the subsampling pattern becomes:

C.4 Proof of Section 4

C.5 Proof of Main Theorems

Theorem 5.1. (Recovery with K Uniform Interactions) Let f satisfy Assumption 2.1 for some K = O (2 nδ ) with δ ≤ 1 3 and let the non-zero coefficients of F be drawn from an absolutely continuous distribution. For { H c } C c =1 chosen as in Lemma C.3 with b = O (log( K )) , C = 3 and D c = I , Algorithm 1 exactly computes the transform F in O ( Kn ) samples and O ( Kn 2 ) time complexity with probability at least 1 -O (1 /K ) .

Theorem 5.2. (Noise-Robust Recovery with K t -Degree Interactions) Let f satisfy Assumption 2.2 for K = O (poly( n )) and t = Θ( n α ) with α ≤ 0 . 409 . Assume either:

The non-zero coefficients of F are drawn from an arbitrary continuous distribution, or

U c,p is corrupted by noise as in (10) and let non-zero coefficients satisfy | F ( k ) | = ρ .

Then, for { H c } C c =1 chosen as in Lemma C.4 with b = O (log( K )) , C = 3 , and D c chosen as a suitable group testing matrix, Algorithm 1 exactly computes the transform F in O ( Kt log( n )) samples and O ( K poly( n )) time complexity with probability at least 1 -O (1 /K ) in both the noiseless case (1) and noisy case (2).

Proof. The first step for proving both Theorem 5.1 and Theorem 5.2 is to show that Algorithm 1 can successfully recover all Mobius coefficients with probability 1 -O (1 /K ) under the assumption that we have access to a Detect ( U c ( j )) function that can output the type Type( U c ( j )) for any aliasing set U c ( j ) . Under this assumption, we use density evolution proof techniques to obtain Theorem C.5 and conclude both theorems.

Then, to remove this assumption, we need to show that we can process each aliasing set U c ( j ) correctly, meaning that each bin is correctly identified as a zeroton, singleton, or multiton. Define E as the error

event where the detector makes a mistake in O ( K ) peeling iterations. If the error probability satisfies Pr( E ) ≤ O (1 /K ) , the probability of failure of the algorithm satisfies

̸

̸

̸

In the following, we describe how we achieve Pr( E ) ≤ O (1 /K ) under different scenarios.

In the case of uniformly distributed interactions without noise, singleton identification and detection can be performed without error as described in Section C.7.1. In the case of interactions with low degree and without noise, singleton identification and detection can be performed with vanishing error as described in Section C.7.2. Lastly, we can perform noisy singleton identification and detection with vanishing error for low degree interactions as described in Section C.7.2.

C.6 Density Evolution Proofs

The density evolution proof is generally separated into two parts.

We show that with high probability, nearly all of the variable nodes will be resolved.

We show that with high probability, the graph is a good expander , which ensures that if only a small number are unresolved, the remaining variable nodes will be resolved.

Whether the decoding succeeds or fails depends entirely on the graph (or rather distribution over graphs) that is induced by the algorithm. The graph ensemble is parameterized as G ( D , { M c } c ∈ [ C ] ) . D { M } ∼ D

is the support distribution. The set of non-zero Mobius coefficients r : [ f ]( r ) = 0 is drawn from this distribution. In [13], using the arguments above it is shown that if the following conditions hold, the peeling message passing successfully resolves all variable nodes:

̸

In the limit as n →∞ asymptotic check-node degree distribution from an edge perspective converges to that of independent an identically distributed Poisson distribution (shifted by 1).

The variable nodes have a constant degree C ≥ 3 (This is needed for the expander property).

The number of check nodes b in each of the C sampling group is such that 2 b = O ( K ) .

Theorem C.5 ([13]) . If the above three conditions hold, the peeling decoder recovers all Mobius coefficients with probability 1 -O (1 /K ) .

In the following section, we show that for suitable choice of sampling matrix, these conditions are satisfied, both in the case of uniformly distributed and low degree Mobius coefficients.

C.6.1 Uniform Distribution

In order to satisfy the conditions for the case of a uniform distribution of we use the matrix in Corollary C.3. We select C = 3 different I 1 , I 2 , I 3 such that I i ∩ I j = ∅ ∀ i = j ∈ { 1 , 2 , 3 } . Note that this satisfies condition (2) above. Furthermore, we let k scale as O (2 nδ ) . In order to satisfy condition (3), we must have δ < 1 3 , since each I i can consist of at most 1 3 of all the coordinates.

̸

We now introduce some notation. Let g j ( · ) represent the hash function , that maps a frequency r to a check node index k in each subsampling group j = 1 , . . . , C , i.e., g j ( r ) = H j r . Per our assumption, we have K non-zero variable notes r (1) , . . . , r ( K ) chosen uniformly at random. Technically, we are sampling without replacement, however, since K 2 n → 0 , the probability of selecting a previously selected r ( i ) vanishes. Going forward in this subsection, we will assume that each r i is sampled with replacement for a more brief solution. A more careful analysis that deals with sampling with replacement before taking limits yields an identical result.

First, let's consider the marginal distribution of g j ( r ( i ) ) for some arbitrary j ∈ [ C ] and i ∈ [ K ] . Assuming sampling with replacement, we have:

Thus, we have established that the our approach induces a uniform marginal distribution over the 2 b check nodes. Next, we consider the independence of our bins. By assuming sampling with replacement, we can immediately factor our probability mass function.

Furthermore, since we carefully chose the I i such that they are pairwise disjoint, we have

establishing independence. Let's define an inverse load factor η = 2 b K . From a edge perspective, sampling with replacement with independent uniformly distributed gives us:

For fixed η , asymptotically as K →∞ this converges to:

C.6.2 Low Degree Distribution

For this proof, we take an entirely different approach to the uniform case. We instead exploit the results of Theorem F.1, which is about asymptotically exactly optimal group testing, and then make an information-theoretic argument. Let X n be a group testing matrix (constructed either by an i.i.d. Bernoulli design or a constant column weight design using the parameters required for the given n ). We don't explicitly write the dependence of X n on t , since by invoking Theorem F.1, we assume some implicit relationship where t = Θ( n θ ) for θ satisfying the theorem conditions. Now consider some r n chosen uniformly at random from the ( n t ) weight t binary vectors. Note that in this work we actually use what is known as the 'i.i.d prior' as opposed to the 'combinatorial prior' that we have just defined. As noted in [15], these are actually equivalent, so we can arbitrarily choose to work with one, and the result holds for the other as well. We define:

Furthermore, we define the decoding function Dec n ( · ) , which represents the deterministic procedure that successfully recovers r with vanishing error probability. We have the following bounds on the entropy of Y n :

where we have used the fact that binary random variables have a maximum entropy of 1 . Furthermore, by the properties of entropy we also have H ( Y n ) ≥ H (Dec( Y n , X n ) | X n ) . Dividing through by T , we have:

̸

Let Dec n ( Y n , X n )) = r n +err n ( Y n , X n ) . It is known (see [15]) that Pr(err n ( Y n , X n ) = 0) = O (poly( T ) e -T ) . Thus, we can bound the left-hand side as:

Where in (45) we have used the bound H ( A + B ) ≥ H ( A ) -H ( B ) and the fact that X n and r n are independent, and in (46) we have used the fact that conditioning only decreases entropy. By the continuity of entropy and Theorem F.1, we have that:

This establishes that:

Unfortunately, this does not immediately imply that all of the summands have a limit of 1 , however, it does mean that the fraction of total summands that are less than one goes to zero (it grows as o ( T ( n )) ). Let G ⊂ N correspond to the set containing all the indicies i of the summands that are equal to 1 . By using the fact that conditioning only reduces entropy, we have

Now we define the countable sequence of random variables:

H ( ¯ Y i | ¯ Y S i ) = 1 , (51) Noting that conditioning only decreases entropy, we immediately have that ¯ Y i ∼ Bern( 1 2 ) . Now consider some arbitrary finite set S ∗ ⊂ G . We will now prove that { ¯ Y i , i ∈ S ∗ } is mutually independent.

By continuity of entropy, and the above limit and definition, we have:

Proof. Let i 1 < i 2 < . . . < i | S ∗ | be an ordered indexing of the elements of S ∗ . Furthermore, let Q j = { i q | 1 ≤ q ≤ j } . Assume the set { ¯ Y i , i ∈ Q j } is mutually independent, and use the notation Y Q j to represent a vector containing all of these entries. We have:

However, by using the fact that conditioning only decreases entropy we have:

thus,

This leads to the following chain of implications:

From this, and the initial inductive assumption, we can conclude that { ¯ Y i , i ∈ Q j +1 } is mutually independent. The base case of j = 1 follows from the fact that a set containing just one single random variable is mutually independent. Since Q | S ∗ | = S ∗ the proof is complete.

Now let L ( n ) = | G ∩ [ n ] | we know L = Θ( T ( n )) , which follows from the stronger result that lim n L ( n ) T ( n ) = 1 . Take b L ( n ) C By leveraging the above results, we can select our subsampling

→∞ ≤ matrices { H i } C i =1 from suitable rows of X n . Let S ( n ) 1 , . . . , S ( n ) C ⊂ G ∩ [ n ] , ∣ ∣ ∣ S ( n ) i ∣ ∣ ∣ = b and S ( n ) i ∩ S ( n ) j = ∅ . Then take n

Due to the independence result proved above, the asymptotic degree distribution is:

C.7 Singleton Detection and Identification

In this section, we prove the main results about singleton detection and identification. For the noiseless case, we introduce another assumption:

Assumption C.6. (No Cancellation) Suppose the non-zero values F ( k 1 ) , ... , F ( k K ) are sampled from a joint distribution P that satisfies the following condition:

̸

̸

This assumption is quite mild, and there are many classes of P that satisfy this assumption.

Lemma C.7. Any absolutely continuous P satisfies Assumption C.6 a.s..

Proof. For simplicity let F represent a K dimensional random vector containing F ( k i ) at index i . Let the set R ( S, α ) = { F : ∑ i ∈ S F ( k i ) = α } . Since P is absolutely continuous, a density p exists such that:

However, dim( R ( S, α )) = K -| S | . Thus for S = ∅ , R ( S, α ) has Lebesgue measure zero, thus

̸

C.7.1 Uniform Interactions Singleton Identification and Detection without Noise

For singleton identification, we observe that in the case of a singleton, y c = k ∗ , thus, in the case of a singleton k ∗ can be recovered. We still need to show that the Detect function correctly identifies the type. We separate the proof into three parts:

Prove Detect( U c ( j )) = H Z = ⇒ Type( U c ( j )) = H Z .

Prove Detect( U c ( j )) = H M = ⇒ Type( U c ( j )) = H M .

Prove Detect( U c ( j )) = H S = ⇒ Type( U c ( j )) = H S .

Consider the subsampling group U c ( j ) for some fixed c, j . We first consider the case where | k | is not restricted, and denote the set of non-zero indices k 1 , . . . , k K as N .

Proof of (1) Let Detect( U c ( j )) = H Z , and for contradiction's sake assume Type( U c ( j )) = H Z .

̸

̸

Since Type( U c ( j )) = H Z , we have that N ∩ { k : H c k = j } ̸ = ∅ . Thus, if we consider the above implication for the case of p = 0 and noting that d 0 = 0 , we have:

But considering the no cancellation condition, this is impossible, thus proving (1).

Proof of (2) Note that the converse of (1) is true (the proof is immediate). Thus, proving Detect( U c ( j )) = H S = ⇒ ¬ (Type( U c ( j )) = H M ) is the same as proving (2). We will again use the method of contradiction, and assume Detect( U c ( j )) = H S and Type( U c ( j )) = H M .

̸

Note that by our assumption, U c, 0 ( j ) = 0 . By our assumption that Type( U c ( j )) = H M , we must have |N ∩ { k : H c k = j } ̸ = ∅| ≥ 2 . Choose k 1 , k 2 ∈ N ∩ { k : H c k = j } . Given our choice of d p

∃ p ∗ > 1 s.t. only *one* of k 1 or k 2 ≤ ¯ d p ∗ . Without loss of generality we will assume k 2 ≤ ¯ d p ∗ and k 1 ≰ ¯ d p ∗ . Now, define the following sets:

We know k 2 ∈ J p ∗ and k 1 ∈ J 0 \ J p ∗ , thus |J p ∗ | ≥ 1 and |J 0 \ J p ∗ | ≥ 1 . With this, we can show that the implication above cannot be satisfied.

Case 1: U c,p ∗ ( j ) = 0 .

Since J ∗ p is not empty, from our distributional assumption, the above sum cannot be 0 .

Since J 0 \ J ∗ p is not empty, from our distributional assumption, the above sum cannot be 0 . This implies that Type( U c ( j )) = H M must be false, thus proving (2).

Proof of (3): Since we have a converse for (1), a converse for (2) suffices to prove (3). The converse follows below:

̸

Since F ( k ∗ ) = 0 , we have U c, 0 ( j ) = 0 and all entries of U c ( j ) are either F ( k ∗ ) or 0 . Thus, Detect( U c ( j )) = H S .

̸

C.7.2 Low Degree Singleton Identification and Detection without Noise

Here we include a sketch of the argument for the noiseless part of Theorem 2, which focuses on the case | k | ≤ t (the rest of Theorem 2 is unchanged).

Proof of (1) The proof is identical to above.

Proof of (2+3) The proof is the same, with one notable exception. In the low degree case, p ∗ may not always exist. In this case, we can simply rely on the result of [54]. Since Pr( ˆ k = k ∗ ) ≤ n -β , we correctly recover each k ∗ in the limit, meaning we must have Pr( D c k 1 = D c k 2 ) → 1 (this is equivalent to the existence of p ∗ ). Thus, by the same argument this implies that Detect ( U c ( j )) has vanishing error in the limit. The complete argument requires us to union bound over all of the singleton identifications success, but since T is linear in β , so long as K = poly( n ) , constant β suffices for vanishing error.

C.7.3 Singleton Identification in i.i.d. Spectral Noise

In this section, we discuss how to ensure that we can detect the true non-zero index r ∗ from the delayed samples, under the i.i.d. noise assumption. We first discuss the delay matrix itself, D ∈ Z P 1 × n 2 . As in the noiseless case, we want to choose this matrix to be a group testing matrix. For the purposes of theory, we will choose D such that each element is drawn i.i.d. as a Bern ( ν t ) for some ν = Θ(1) . We denote the i th row of D as d i . Each group test is derived from one of the delayed samples. Under the i.i.d. spectral noise model, this means each sample has the form:

̸

̸

where Z i ( k ) ∼ N ( 0 , σ 2 ) . Essentially, we can view this as a hypothesis testing problem, where we have one sample X , and hypothesis and the alternative are:

Furthermore, lets say the magnitude of | F [ k ] | = ρ is known. We construct a threshold test:

With such a test, we can compute the cross-over probabilities:

For the sake of simplicity, we will make the choice to choose γ such that p 10 = p 01 . In that case, we can numerically solve for the cross-over probability which is fixed for a given signal-to-noise ratio.

10

-

1

10

0

10

1

ρ/σ

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

Crossover Probability

̸

Lemma C.8. For any fixed SNR, taking D c such that each element is Bern ( ν t ) , and t = Θ( n α ) for α ∈ (0 , 1) , taking P 1 = O ( t log( n )) suffices to ensure that we can achieve error of bin identification failure with probability of error O (1 /K 3 ) .

Now we can use [54], to prove our desired result. let q be the resulting cross-over probability for a given ρ/σ . The probability that all of our Singleton identifications succeed can computed by a union bound on P e = Pr ( ˆ k = k ) ≤ n -β . If K = poly( n ) , a constant β suffices us to drive the union bound to zero.

C.7.4 Singleton Detection in i.i.d. Spectral Noise

We note that the general flow of this proof follows [40], but there are several fundamental differences that make this proof overall quite different. We define E b as the error event where a bin k is decoded wrongly, and then using a union bound over different bins and different iterations, the probability of the algorithm making a mistake in bin identification satisfies

The number of bins is at most ηK for some constant η and the number of iterations is at most CK (at least one edge is peeled off at each iteration in the worst case). Hence, Pr( E ) ≤ ηCK 2 Pr( E b ) . In order to satisfy Pr( E ) ≤ O (1 /K ) , we need to show that Pr( E b ) ≤ O (1 /K 3 ) .

We already showed in Lemma C.8 that we can achieve singleton identification under noise with vanishing error O (1 /K 3 ) with a delay matrix D ∈ Z P 1 × n 2 .

To achieve type detection, we construct another pair of delay matrices D 1 ∈ Z P 2 × n 2 and D 2 ∈ Z P 2 × n 2 . We will choose D 1 and D 2 such that each element is drawn i.i.d. as a Bern ( (1 / 2) 1 /t ) . We denote

the i th row of D 1 as d 1 i and denote the i th row of D 2 as d 2 i . Then, with these delay matrices, we can obtain observations of the form

Note that we can represent these observations as

with W 1 , W 2 ∼ N (0 , σ 2 I ) , a α vector with entries F ( r ) for coefficients in the set and binary signature matrices S 1 , S 2 with entries indicating the subsets of coefficients included in each sum.

Then, we subtract these observations to obtain a single observation U = U 1 -U 2 which can be written as

with W ∼ N (0 , 2 σ 2 I ) and S = S 1 -S 2 . This construction allows us to show that the columns of S are sufficiently incoherent and hence we can correctly perform identification.

Lemma C.9. For any fixed SNR, taking D 1 c and D 2 c such that each element is Bern ( (1 / 2) 1 /t ) , and t = Θ( n α ) for α ∈ (0 , 1 / 2) and taking P 2 = O ( t log( n )) suffices to ensure that the probability Pr( E b ) for an arbitrary bin can be upper bounded as Pr( E b ) ≤ O (1 /K 3 ) .

Proof. In the following, we prove that Pr( E b ) ≤ O (1 /K 3 ) holds using the observation model. We consider separate cases where the bin in consideration is fixed as a zeroton, singleton, or multiton.

The error probability Pr( E b ) for an arbitrary bin can be upper bounded as above, each of these events should be read as:

{F ← H S ( r , F ( r )) } : missed verification in which the singleton verification fails when the ground truth is in fact a singleton.

{H S ( ̂ r , ̂ F ( r )) ←F} : false verification in which the singleton verification is passed when the ground truth is not a singleton.

{H S ( ̂ r , ̂ F ( r )) ← H S ( r , F ( r )) } : crossed verification in which a singleton with a wrong index-value pair passes the singleton verification when the ground truth is another singleton pair.

We can upper-bound each of these error terms using Propositions C.10, C.11, and C.12. Note that all upper-bound terms decay exponentially with P 2 except for the term Pr( r = r ) ≤ O (1 /K 3 ) .

̸

̸

̂ We use Theorem F.3 to show that we can achieve Pr( ̂ r = r ) ≤ O (1 /K 3 ) if we choose P 1 = O ( t log n ) . Since all other error probabilities decay exponentially with P 2 , it is clear that if P 2 is chosen as P 2 = O ( t log n ) , the error probability can be bounded as Pr( E b ) ≤ O (1 /K 3 ) .

Proposition C.10 (False Verification Rate) . For 0 < γ < η 4 SNR , the false verification rate for each bin hypothesis satisfies:

where P 2 is the number of the random offsets.

Proof. The probability of detecting a zeroton as a singleton can be upper-bounded by the probability of a zeroton failing the zeroton verification. This means

by noting that W ∼ N (0 , ν 2 I ) and applying Lemma C.13.

On the other hand, given some multiton observation U = S α + W , the probability of detecting it as a singleton with index-value pair ( r , F ( r )) can be written as

To upper bound the first term, we use Lemma C.13. Note that the first term is conditioned on the event ∥ g ∥ 2 /P 2 ≥ 2 γν 2 , thus the normalized non-centrality parameter satisfies θ 0 ≥ 2 γ . As a result, we can use Lemma C.13 by letting τ 2 = (1 + γ ) ν 2 . Then, the first term is upper bounded by exp {-( P 2 γ 2 ) / (4(1 + 4 γ )) } . To analyze the second term, we let β = α -̂ F ( ̂ r ) e ̂ r and write g = S β . Denoting its support as L := supp( β ) , we can further write S β = S L β L where S L is the sub-matrix of S consisting of the columns in L and β L is the sub-vector consisting of the elements in L . Then, we consider two scenarios:

The multiton size is a constant, i.e., |L| = L = O (1) . In this case, we have

Using ∥ β L ∥ 2 ≥ Lρ 2 , the probability can be bounded as

On the other hand, using Lemma C.14 with the selection β = 1 / 2 and η = 1 1+2 L ( 1 2 -2 γν 2 Lρ 2 ) , we have

which holds as long as γ < Lρ 2 / (4 ν 2 ) = Lη 4 SNR .

The multiton size grows asymptotically with respect to K , i.e., |L| = L = ω (1) . As a result, the vector of random variables g = S L β L becomes asymptotically Gaussian due to the central limit theorem with zero mean and a covariance

which holds as long as γ < Lρ 2 /ν 2 = Lη SNR .

By combining the results from both cases, there exists some absolute constant ϵ > 0 such that

as long as γ < ρ 2 / (4 ν 2 ) = η 4 SNR .

Proposition C.11 (Missed Verification Rate) . For 0 < γ < η 2 SNR , the missed verification rate for each bin hypothesis satisfies

̸

where P 2 is the number of the random offsets.

Proof. The probability of detecting a singleton as a zeroton can be upper bounded by the probability of a singleton passing the zeroton verification. Hence, by noting that W ∼ N (0 , ν 2 I ) and applying Lemma C.13,

which holds as long as γ < ρ 2 /ν 2 = η SNR .

On the other hand, the probability of detecting a singleton as a multiton can be written as the probability of failing the singleton verification step for some index-value pair ( ̂ r , ̂ F [ ̂ r ]) . Hence, we can write

̸

̸

On the other hand, the second term can be bounded as

̸

̸

̸

̸

̸

̸

̸

̸

̸

̸

̸

The first term is the error probability of a BPSK signal with amplitude ρ , and it can be bounded as

̸

Proposition C.12 (Crossed Verification Rate) . For 0 < γ < η 2 SNR , the crossed verification rate for each bin hypothesis satisfies

Pr( H S ( ̂ r , ̂ F [ ̂ r ]) ←H S ( r , F [ r ])) ≤ e -P 2 γ 2 4(1+4 γ ) + Ke -ϵ ( 1 -4 γν 2 ρ 2 ) 2 P 2 + K 2 e -ϵ ( 1 -4 γν 2 ρ 2 ) 2 P 2 2 /t . where P 2 is the number of the random offsets.

Proof. This error event can only occur if a singleton with index-value pair ( r , F [ r ]) passes the singleton verification step for some index-value pair ( r , F [ r ]) such that r = r . Hence,

̸

where β is a 2 -sparse vector with non-zero entries from { ρ, -ρ } . Using Lemma C.13, the first term is upper-bounded as

By Lemma C.14, the second term is upper bounded as

which holds as long as γ < ρ 2 / (2 ν 2 ) = η 2 SNR .

Lemma C.13 (Non-central Tail Bounds (Lemma 11 in [13])) . Given any g ∈ R P and a Gaussian vector v ∼ N (0 , ν 2 I ) , the following tail bounds hold:

for any τ 1 and τ 2 that satisfy τ 1 ≥ ν 2 (1 + θ 0 ) ≥ τ 2 where

is the normalized non-centrality parameter.

Lemma C.14. Suppose β = Θ(1) , η = Ω(1) , and t = Θ( n α ) for some α ∈ (0 , 1 / 2) . Then, there exists some n 0 such that for all n ≥ n 0 , we have

Proof. For any r sampled uniformly from vectors up to degree t , the probability that it will have degree 0 ≤ k ≤ t can be written as

We know that the entries of s r are given as ( s 1 r ) i = 1 { r ≤ ¯ d 1 i } and ( s 2 r ) i = 1 { r ≤ ¯ d 2 i } . Therefore,

With β = Θ(1) and t = Θ( n α ) for α ∈ (0 , 1 / 2) , we can show that lim n →∞ g ( t, n ) = β . Therefore, there exists some n 0 such that | Pr ( ( s 1 r ) i = 1 ) -β | ≤ η for all n ≥ n 0 . For the rest of the proof, let g = Pr ( ( s 1 r ) i = 1 ) and assume | g -β | ≤ η .

Then, recalling ( s r ) i = ( s 1 r ) i -( s 2 r ) i , the distribution for each entry of s r can be written as

Hence, using Hoeffding's inequality, we obtain

Furthermore, the conditional probability of another vector m = r being included in test i is given by

̸

With β = Θ(1) and t = Θ( n α ) for α ∈ (0 , 1) , for any k ≤ t , we can show that lim n →∞ f ( t, n, k ) = β . Therefore, there exists some n 0 such that | Pr ( ( s 1 m ) i = 1 | ( s 1 r ) i = 1 ) -β | ≤ η for all n ≥ n 0 . For the rest of the proof, let f = Pr ( ( s 1 m ) i = 1 | ( s 1 r ) i = 1 ) and assume | f -β | ≤ η . On the other hand,

Since lim n →∞ E [( s m ) i ( s r ) i ] = 0 , there exists some n 0 such that -η ≤ E [( s m ) i ( s r ) i ] ≤ η for all n ≥ n 0 . For the rest of the proof assume -η ≤ E [( s m ) i ( s r ) i ] ≤ η . As a result, we can write

As a result, we have

By Gershgorin Circle Theorem, the minimum eigenvalue of 1 P 2 S ⊤ L S L is lower bounded as

̸

Lastly, we apply a union bound over all ( r , m ) pairs to obtain

D Worst-Case Time Complexity

In this section, we discuss the computational complexity of Algorithm 1, which is broken down into the following parts:

Computing Samples Computing samples for one sapling matrix requires computing the row-span of H c , which can be computed in n 2 b operations. Then for each sample, we must take the bit-wise and with each row of the delay matrix, so the total complexity is: Cn 2 b P .

Taking Small Mobius Transform Computing the Mobius transform for each of the CP subsampled functions is CPb 2 b .

Singleton Detection To detect each singleton requires computing y . This requires P divisions for each of the C 2 b bins, for a total of CP 2 b operations.

Singleton Identification To identify each singleton requires different complexity for our different assumptions.

In the case of uniformly distributed interactions, singleton detection is O (1) , since y = k ∗ immediately, so doing this for each singleton makes the total complexity CK .

In the noiseless low degree case decoding k ∗ from y is poly( n ) , so for each singleton the complexity is CK poly( n )

Message Passing In the worst case, we peel exactly one singleton per iteration, resulting in CK subtractions (the above singleton identification bounds already take into account the need to re-do singleton identification).

Thus in the case of uniformly distributed and low degree interactions respectively, the complexity is:

Uniform distributed noiseless time complexity

= O ( CPn 2 b + CPb 2 b + CK )

=

O

(

CPnK

)

= O ( n 2 K ) .

Low degree (noisy) time complexity

=

O

(

CPn

2

b

+

CPb

2

b

+

CK

poly(

n

))

= O ( CP poly( n ) K )

= O (poly( n ) K ) .

E Additional Simulations

In this section, we present some additional simulations that did not fit in the body of the manuscript. Fig 15 and 16. Plot the runtime of SMT vs. n under both of our assumptions. In both cases we observe excellent scaling with n . We note that our low degree setting has a higher fixed cost since we

are using linear programming to solve our group testing problem and the solver appears to have some non-trivial fixed time cost.

Fig. 17 plots the perfect reconstruction percentage against n and sample complexity. We also observe a phase transition, however, the phase threshold appears very insensitive to n , as expected, since our sample complexity requirement is growing like log( n ) , and we are already plotting on a log scale.

2

100

2

200

2

300

2

400

2

500

2

600

2

700

2

800

2

900

2

1000

100

200

300

400

500

600

700

800

900

1000

n

N

10

-

3

10

-

2

10

-

1

10

0

10

1

Runtime Complexity (sec)

SMT

2

100

2

200

2

300

2

400

2

500

2

600

2

700

2

800

2

900

2

1000

100

200

300

400

500

600

700

800

900

1000

n

N

10

-

3

10

-

2

10

-

1

10

0

10

1

Runtime Complexity (sec)

SMT

2

100

2

200

2

300

2

400

2

500

2

600

2

700

2

800

2

900

2

1000

100

200

300

400

500

600

700

800

900 1000

n

N

10

4

10

5

Sample Complexity

0%

20%

40%

60%

80%

100%

Perfect Reconstruction %

F Group Testing

F.1 Group Testing Achievability Results From Literature

Theorem F.1 (Part of Theorem 4.1 and 4.2 in [15]) . Asymptotic Rate 1 Noiseless Group Testing: Consider a noiseless group testing problem with t = Θ( n θ ) defects out of n elements. We define the rate of a group testing procedure as:

where T is the number of tests performed by the group testing procedure. For an i.i.d. Bernoulli design matrix, for θ ∈ [0 , 1 / 3] , in the limit as n →∞ , a rate R ∗ BERN = 1 is achievable with vanishing error. Furthermore, for the constant column-weight design matrix, for θ ∈ [0 , 0 . 409] a rate R ∗ CCW = 1 is achievable with vanishing error.

Theorem F.2 ([42, 54]) . Noiseless Group Testing : Consider the noiseless non-adaptive group testing setup with t = | k | defects out of n items, with t scaling arbitrarily in n . Let ˆ k be the output

of a group testing decoder and let T ∗ = Θ(min { t log( n ) , n } ) . Then there exists a strategy using T ≤ (1 + ϵ ) T ∗ such that in the limit as n →∞ we have:

̸

Furthermore, there is a poly( n ) algorithm for computing ˆ k . From [54], for t = o ( n ) we can achieve:

̸

with number of tests T = O ((1 + δ ) t log( n )) .

Note that the above error rate is not a state-of-art result, but suffices in this case for our proof, and is very convenient in its form.

Theorem F.3 ([43]) . Noisy Group Testing Under General Binary Noise : Consider the general binary noisy group testing setup with crossover probabilities p 10 and p 01 . We use i.i.d Bernoulli testing with parameter ν > 0 . There are a total of | k | = t = Θ( n θ ) defects, where θ ∈ (0 , 1) . Let T ∗ = max { T ( D ) 1 , T ( ND ) 1 , T ( D ) 2 , T ( ND ) 2 } , where we have

where D ( x ) = x log( x ) -x +1 , and w = (1 -p 01 ) e -ν + p 10 (1 -e -ν ) . For any α ∈ ( p 10 , 1 -p 01 ) , β ∈ ( p 01 , 1 -p 10 ) , there exist some number of tests T < (1 + ϵ ) T ∗ where the Noisy DD algorithm produces ˆ k such that in the limit as n →∞ we have:

̸

The above result is state-of-art for noisy group testing and could be of interest generally for proving the type of results we have here, however, but for simplicity, we state a similar more compact result that suffices for our proofs in this paper.

Theorem F.4 ([54]) . Let | k | = t = o ( n ) , and consider an i.i.d. Bernoulli design group testing matrix. Further consider the binary symmetric noise model with crossover probability q . If we construct ˆ k via the noisy column matching algorithm, we achieve:

̸

with number of tests where γ is a constant that depends on β .

F.2 Group Testing Implementation

We implement group testing via linear programming. As noted in [15], linear programming generally outperforms most other group testing algorithms in both the noisy and noiseless case. We use the

following linear program, to implement group testing.

G Impact Statement

Rigorous tools for understanding models can potentially profoundly increase trust in deep learning systems. If we can understand and reason for ourselves why a model is making a decision, we can put greater trust into those decisions. Furthermore, if we understand why a model is doing something that we believe is incorrect, we can better steer it towards doing what we believe is correct. This 'steering' of model behavior is sometimes described as alignment , and is a critical task for addressing things like incorrect or misleading information generated by a model, or for address any undesirable biases. In terms of concerns, it is important to not misinterpret or over-interpret the interaction indices that come out of SMT. It could be the case that looking over some selection of interactions doesn't reveal the full picture, and leads one down an incorrect line of reasoning.

NeurIPS Paper Checklist

The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and precede the (optional) supplemental material. The checklist does NOT count towards the page limit.

Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:

You should answer [Yes] , [No] , or [NA] .

[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.

Please provide a short (1-2 sentence) justification right after your answer (even for NA).

The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.

The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation. While '[Yes] ' is generally preferable to '[No] ', it is perfectly acceptable to answer '[No] ' provided a proper justification is given (e.g., 'error bars are not reported because it would be too computationally expensive' or 'we were unable to find the license for the dataset we used'). In general, answering '[No] ' or '[NA] ' is not grounds for rejection. While the questions are phrased in a binary way, we acknowledge that the true answer is often more nuanced, so please just use your best judgment and write a justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.

IMPORTANT, please:

Delete this instruction block, but keep the section heading 'NeurIPS paper checklist' ,

Keep the checklist subsection headings, questions/answers and guidelines below.

Do not modify the questions and only use the provided macros for your answers .

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction explicitly state the main theoretical contributions of this work. The main contributions section highlight this, and specifically call out the theoretical nature of this work. The introduction contains information about the practical applications and motivations. A detailed descriptions of assumptions is left to Section 2, to avoid an overly technical introduction, but it is made clear that there are limiting assumptions

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: There is a limitations section, and limitations are mentioned periodically when relevant.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate 'Limitations' section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The proofs are included in the Appendix, and are complete and correct. Section 3 and 4 are effectively a proof sketch explaining how and why the algorithm works.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: The algorithm is exactly stated in the paper, and implementation details (specifically related to group testing) are included in the appendix. A reader would be able to exactly reproduce our code with the given information. We plan to release the code with the camera ready version.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: Code is included.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: The algorithms are relatively simple and complicated implementation details are in the appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: Error bars are included in the main paper.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer 'Yes' if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Experiments are not run at very large scale and runtime numbers are included only for relative comparison.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: Ethical concerns are satisfied.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: Though this work is primarily theoretical, there is a clear application in mind, and thus is discussed in appendix section G.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: The details of the code are included in the main body of the paper.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper present a way to compute high-order interactions using Mobius transformation.\n\nSTRENGTHS:\nThe paper shows some interesting results on the sampling requirement for recovering the interactions among features, specially tailored for explainability with boolean functions (i.e., explanation game)\n\nWEAKNESSES:\n1. The paper is difficult to follow and does not have a clear storyline; most importantly, it is not clear how the significant interactions could be recovered at the end after obtaining the required samples\n\n2. Aside from the clarification, though having an innovative approach and some interesting results, the method seems to have little applicability in real-world applications; one reason for doing so is one of the main assumptions of the method of having only K nonzero interactions (= K nonzero Mobius coefficient). For explainability, as defined, for instance, in SHAP, it is typically the case that we have many nonzero interactions since it is rare that the interactions among a feature subset will go to zero (given the way we compute the interactions). This implies that in most cases, we have a very large K of nonzero interactions, and this would make the sparse Mobius transformation impractical.\n\n3. That being said, it is not clear how exactly the interactions are obtained; as the authors stated in the paper, the naive computation of all Mobius coefficients includes exponentially many parameters; given the samples, what interactions should be included as significant?\n\n4. Last but not least, in explainability, we usually seek the most influential features and the most important interactions (rather than all feature importance or significant interactions); the paper seems to merely focus on computing all the significant interactions rather than the most important ones, and in explainability, this is not essentially sought and seems impractical due to many nonzero interactions.\n\nMinor: \nSome nations need more explanation as they are borrowed from other disciplines (as the authors also stated in the Introduction, the paper is multidisciplinary); For instance, the functional subsampling and aliasing that are the backbone of the sampling strategies, required more explanation to average reader of the paper, IMHO.\n\nQUESTIONS:\nhow can the results of the paper be generalized to non-boolean functions? this would generalize the results from detecting interactions for explainability to a general case of interaction detection for any arbitrary dataset.\n\nLIMITATIONS:\nThe assumption of having only K non-zero interactions, while in practice K is typically large for explainability.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis study proposes a new efficient method to compute the Möbius Transform. To understand the behavior of large complex machine learning models, various studies have used game theory concepts such as the Shapley value to measure the impact of input variables on the model's outputs. While Shapley values only focus on individual impacts of input variables, Möbius Transform can score any subset of input variables, which can take into account higher-order interactions between input variables. One of the fundamental challenges of computing Möbius Transform is its exponential computational cost (i.e., requiring $2^n$ inferences with $n$ input variables). This study reduces its cost; with a sparsity assumption, the computation runs with $O(n)$ number of samples, and with a low-degree assumption, it further reduces to $O(\\log(n))$. The proposed method (Sparse Möbius Transformer; SMT) exploits a connection between the Möbius Transform in the original $n$-dimensional space (say, F-space) and that in a lower $b$-dimentional space (say, U-space). If the *aliasing set* is a singleton, then the Möbius Transform in F-space can be recovered from that in U-space. The authors proposed methods to detect singletons and turn mulitons into singletons using graph peeling. The experimental results justify the efficiency and the high faithfulness recovered by SMT.\n\nSTRENGTHS:\n- This study offers an efficient computational method of Möbius Transform, which encompasses widely used game-theoretic metrics such as Shapley values and Benzhaf values, and the accelaration from baselines is significant.\n- The proposed method is theoretically supported.\n- While the paper is densely theoretical, the paper writing is generally easy to follow. \n- While not strongly practical and large scale, the assumptions and claims are justified by numerical experiments at key points.\n\nWEAKNESSES:\n**Major comments**\n- The proposed algorithm works on a $b$-dimensional space, but I was not able to find out how $b$ is set in the experiments and ablation experiments on this hyper-parameter. While the theory suggests $b = O(\\log(K))$, this is not very suggestful due to the lack of the constant factor. \n- The baselines in Fig. 6 are all first-order metrics. In machine learning and computer vision, second-order metrics (e.g., Interactions) are also popular. Why were those methods not included in the experiments? \n\n**Minor comments**\n- It'd be better to clarify several notations, such as $|\\mathbf{k}|$ and $[K]$.\n- To better convince readers of the connection between Shapley value and Möbius basis, it'd be better to provide a deriviation of the left equality of Eq. (2). \n- In the main text, $C$, the number of samplings (?), appears a bit out of sudden at line 193.\n\nQUESTIONS:\nI'd like the Authors to answer the weaknesses raised above. Plus, \n- At Limitations, the Authors mentioned that \"we may be more interested in taking a sparse projection onto a subset of low-order terms\", but I'm not fully following this. Can't we resolve it simply by zeroing out the coefficients of high-order terms (just as \"a low-pass filter\")?\n\nLIMITATIONS:\nLimitations are adequately presented. For the potential improvements, see Weakness and Questions.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes an algorithm to efficiently compute Mobius transform under the assumption that the function to be transformed is composed of sparse and low-degree interaction terms. The paper also provides an asymptotic analysis of the sample complexity, time complexity, and accuracy of the algorithm.\n\nSTRENGTHS:\n1.\tThis paper focuses on an important problem in Explainable AI, i.e., how to efficiently compute the Mobius transform (or equivalently, the Harsanyi dividend [cite1]) of a function to obtain the interaction between different input variables. The computation requires inferences on $2^n$ different subsets of input variables, which is intractable in general. The paper proposes an algorithm that leverages the sparse and low-degree structure of the interactions observed in experiments, thus reducing the computational cost.\n\n2.\tThe algorithm is presented in full detail, with straightforward examples to illustrate each step.\n\nWEAKNESSES:\nI have been studying explainable AI (XAI) for years, with a special interest in the Shapley value, the Shapley interaction index, and the Mobius transform. I am impressed by this work and greatly appreciate the progress in the efficient computation of Mobius transform. The sparsity and low-degree structure of interactions and the proposed algorithm can be of great interest to the XAI community, because they help explain a neural network’s decision-making logic into sparse symbolic interactions.\n\nHowever, I find that the claim and corresponding proof on the singleton detection algorithm are incorrect, and the literature review is insufficient. Nevertheless, **I would like to significantly raise my score to acceptance if these concerns are appropriately addressed.**\n\n1. The most important part of the proposed method is the singleton detection algorithm (in Section 4 and Appendix C.7). However, I find that the claim “singleton identification and detection can be performed without error in the noiseless setting” and the corresponding proof (in Appendix C.7.1) are incorrect. To be specific, the proposed method uses $U(j)$ to efficiently compute of the Mobius transform if $U(j)$ passes the singleton detection, but I find that the singleton detection algorithm in the paper cannot guarantee that $U(j)$ *is indeed a singleton*, although the paper claims that it can.\n\n To this end, we can easily find counterexamples on which the singleton detection algorithm fails. In summary, we can prove that the algorithm will fail under the following conditions. Given the number of input variables $n$ and the subsampling parameter $b<n$, the interactions satisfy (1) $\\forall k\\in \\\\{0,1\\\\}^n$ s.t. the first $b$ elements of $k$ are all 1’s, $F(k)=-\\delta$ if $|k|=n$, $F(k)=\\frac{\\delta}{n-1-b}$ if $|k|=n-1$, and $F(k)=0$ if $|k|<n-1$, where $\\delta \\in \\mathbb{R}$ is an *arbitrary* scalar (2) $\\forall k\\in \\\\{0,1\\\\}^n$ s.t.  at least one of the first $b$ elements of $k$ is 0, the value of $F(k)$ can be *arbitrary*.\n\n**Counterexample:** Let us consider $n=6$ input variables and set $b=2$ for subsampling, which follows the setting in Figure 3. Suppose there are five non-zero interactions in total: $k_1=110111, k_2=111011, k_3=111101, k_4=111110, k_5=111111$. And $ F(k_1)= F(k_2)= F(k_3)= F(k_4)=1/3$, $F(k_5)=-1$. In the $c$-th subsampling run, let the subsampling matrix $H_c=[[1,0,0,0,0,0],[0,1,0,0,0,0]]$. Then, we will obtain $u_c(00)=f(001111)$, $u_c(01)=f(011111)$, $u_c(10)=f(101111)$, $u_c(11)=f(111111)$, and also $U_c(00)=0$, $U_c(01)=0$, $U_c(10)=0$, $U_c(11)=F(k_1)+ F(k_2) + F(k_3) + F(k_4) + F(k_5)$. \nThe next step in the proposed method is to detect if each $U_c(j)$ is a zeroton, a singleton, or a multiton. This is done by adding different unit vectors $d_{c,p}=e_p, p=1,…,n$, and obtain $u_{c,p}(l)=f\\left(\\overline{H_c^\\top \\bar{l} + d_{c,p}}\\right) \\rightarrow U_{c,p}(j)=\\sum_{k\\le \\bar{d} \\ s.t. Hk=j} F(k)$. We list the results of $U_{c,p}(j)$ for $j=11$, in the following table:\n\n|              | p=0    | p=1    | p=2    | p=3    | p=4    | p=5    | p=6    |\n| ------------ | ------ | ------ | ------ | ------ | ------ | ------ | ------ |\n| $d_{c,p}$            | 000000 | 100000 | 010000 | 001000 | 000100 | 000010 | 000001 |\n| $U_{c,p}(11)$ | 1/3    | 0      | 0      | 1/3    | 1/3    | 1/3    | 1/3    |\n\n**The above counterexample passes the singleton detection algorithm, but $U_c(11)= F(k_1)+ F(k_2) + F(k_3) + F(k_4) + F(k_5)$ contains five terms and is apparently not a singleton.** To be more specific, we can see that the metric $y_{c,p}=1-\\frac{U_{c,p}(11)}{U_{c,0}(11)}$ is either 0 or 1, for $p=1,…,n$. Therefore, according to the criteria in Equation (9), the multiton $U_c(11)= F(k_1)+ F(k_2) + F(k_3) + F(k_4) + F(k_5)$ will be mistakenly judged as a singleton by the algorithm. As a result, the algorithm will yield an incorrect output that contains only one interaction term, while the ground truth contains five interaction terms.\n\nI suggest the authors either revise the claim and the proof in Appendix C.7.1 to be in an asymptotic or probabilistic manner, or provide detailed discussion to the failure cases of the singleton detection algorithm. Even though the algorithm cannot handle all scenarios, I’m still willing to raise my score if proper discussion on the failure cases is added.\n\n---\n\n2. The literature review is insufficient. The core background behind the proposed method is the sparsity of Mobius transform. In fact, the sparsity of Mobius transform (or equivalently, the Harsanyi dividend) has been well studied by many previous works, but these works are not properly discussed in the paper. For example, [cite2, cite3] observed the sparsity of Mobius transform on a wide range of network architectures (e.g., MLPs, CNNs, transformers, PointNet) trained on various tasks (e.g., tabular data classification, image recognition, sentiment analysis, pointcloud classification). [cite4] explored the source of the sparsity and even proved the sparsity of Mobius transform under three sufficient conditions. The authors are expected to include these previous works in the Related Work section or in the discussion of the sparsity assumption in Section 2.\n\n---\n\n3. The paper reports the relationship between Mobius transform and the Shapley value in Equation (2) and Appendix A. However, this relationship has already been discussed in the original paper of Harsanyi dividend [cite1]. Therefore, I encourage the authors to cite paper [cite1] as a proper reference.\n\n\n[cite1] John C. Harsanyi. A simplified bargaining model for the n-person cooperative game. International Economic Review, 4(2):194–220, 1963.\n\n[cite2] Ren et al. Defining and Quantifying the Emergence of Sparse Concepts in DNNs. CVPR, 2023.\n\n[cite3] Li and Zhang. Does a Neural Network Really Encode Symbolic Concepts? ICML, 2023.\n\n[cite4] Ren et al. Where We Have Arrived in Proving the Emergence of Sparse Interaction Primitives in DNNs. ICLR, 2024.\n\nQUESTIONS:\nSee weaknesses above\n\n---\n\nAfter rebuttal: The authors' response mostly addresses my concern about the singleton detection algorithm. The authors also commit to incorporating the relevant literature into their paper. Therefore, I have raised my score to acceptance.\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}