Original: List-Decodable Sparse Mean Estimation
Revised: List-Decodable Sparse Mean Estimation

Original: Abstract
Revised: Abstract

Original: Robust mean estimation is one of the most important problems in statistics: given a set of samples in R d where an α fraction are drawn from some distribution D and the rest are adversarially corrupted, we aim to estimate the mean of D .
Revised: Robust mean estimation is one of the most important problems in statistics: given a set of samples in R d where an α fraction are drawn from some distribution D and the rest are adversarially corrupted, we aim to estimate the mean of D .

Original: A surge of recent research interest has been focusing on the list-decodable setting where α ∈ (0 , 12 ] , and the goal is to output a ﬁnite number of estimates among which at least one approximates the target mean.
Revised: A surge of recent research interest has been focusing on the list-decodable setting where α ∈ (0 , 12 ] , and the goal is to output a ﬁnite number of estimates among which at least one approximates the target mean.

Original: In this paper, we consider that the underlying distribution D is Gaussian with k -sparse mean.
Revised: In this paper, we consider that the underlying distribution D is Gaussian with k -sparse mean.

Original: Our main contribution is the ﬁrst polynomial-time algorithm that enjoys sample complexity O (cid:0) poly( k, log d ) (cid:1) , i.e. poly-logarithmic in the dimension.
Revised: Our main contribution is the ﬁrst polynomial-time algorithm that enjoys sample complexity O (cid:0) poly( k, log d ) (cid:1) , i.e. poly-logarithmic in the dimension.

Original: One of our core algorithmic ingredients is using low-degree sparse polynomials to ﬁlter outliers, which may ﬁnd more applications.
Revised: One of our core algorithmic ingredients is using low-degree sparse polynomials to ﬁlter outliers, which may ﬁnd more applications.

Original:  Introduction
Revised:  Introduction

Original: Mean estimation is arguably a fundamental inference task in statistics and machine learning.
Revised: Mean estimation is arguably a fundamental inference task in statistics and machine learning.

Original: Given a set of samples { x 1 , . . .
Revised: Given a set of samples { x 1 , . . .

Original: , x n } ⊂ R d where an α fraction are drawn from some well-behaved (e g.
Revised: , x n } ⊂ R d where an α fraction are drawn from some well-behaved (e g.

Original: Gaussian) distribution D and the rest are adversarially corrupted, the goal is to estimate the mean of
Revised: Gaussian) distribution D and the rest are adversarially corrupted, the goal is to estimate the mean of

Original: D .
Revised: D .

Original: In the noiseless case where α = 1 , the problem can be easily solved in view of the concentration of measure phenomenon [LT91].
Revised: In the noiseless case where α = 1 , the problem can be easily solved in view of the concentration of measure phenomenon [LT91].

Original: However, this is rarely the case as modern data sets are often contaminated by random noise or even by adversarial corruptions.
Revised: However, this is rarely the case as modern data sets are often contaminated by random noise or even by adversarial corruptions.

Original: Thus, a great deal of recent efforts are focused on efﬁciently and robustly estimating the target mean in the presence of outliers.
Revised: Thus, a great deal of recent efforts are focused on efﬁciently and robustly estimating the target mean in the presence of outliers.

Original: Generally speaking, there is a phase transition between α > 1 / 2 and 0 < α ≤ 1 / 2 , and solving either problem in a computationally efﬁcient manner is highly nontrivial.
Revised: Generally speaking, there is a phase transition between α > 1 / 2 and 0 < α ≤ 1 / 2 , and solving either problem in a computationally efﬁcient manner is highly nontrivial.

Original: The problem that most of the samples are uncorrupted, i.e. α > 1 / 2 , has a very long history dating back to the 1960s [Tuk60,
Revised: The problem that most of the samples are uncorrupted, i.e. α > 1 / 2 , has a very long history dating back to the 1960s [Tuk60,

Original: Hub64], yet only until recently have computationally efﬁcient algorithms been established [DKK + 16 ,
Revised: Hub64], yet only until recently have computationally efﬁcient algorithms been established [DKK + 16 ,

Original: LRV16].
Revised: LRV16].

Original: The other yet more challenging regime concerns that an overwhelming fraction of the samples are corrupted, i.e. α ≤ 1 / 2 , which even renders estimation impossible.
Revised: The other yet more challenging regime concerns that an overwhelming fraction of the samples are corrupted, i.e. α ≤ 1 / 2 , which even renders estimation impossible.

Original: This motivates a line of research on list-decodable mean estimation [CSV17], where in place of outputting one single estimate, the algorithm is allowed to generate a ﬁnite list of candidates and is considered to be successful if there exists at least one candidate in the list that is sufﬁciently close to the target mean.
Revised: This motivates a line of research on list-decodable mean estimation [CSV17], where in place of outputting one single estimate, the algorithm is allowed to generate a ﬁnite list of candidates and is considered to be successful if there exists at least one candidate in the list that is sufﬁciently close to the target mean.

Original: In this work, we investigate the problem of list-decodable mean estimation, for which there have been a plethora of elegant results established in recent years.
Revised: In this work, we investigate the problem of list-decodable mean estimation, for which there have been a plethora of elegant results established in recent years.

Original: From a high level, most of them concern error guarantees and running time.
Revised: From a high level, most of them concern error guarantees and running time.

Original: For example, [CSV17] proposed the ﬁrst tractable algorithm based on semideﬁnite programming, which runs in polynomial time and achieves optimal error rate for variance-bounded distributions.
Revised: For example, [CSV17] proposed the ﬁrst tractable algorithm based on semideﬁnite programming, which runs in polynomial time and achieves optimal error rate for variance-bounded distributions.

Original: [DKS18b] developed a multi-ﬁltering scheme and showed that the error rate can be improved by using high degree polynomials if the underlying distribution is Gaussian.
Revised: [DKS18b] developed a multi-ﬁltering scheme and showed that the error rate can be improved by using high degree polynomials if the underlying distribution is Gaussian.

Original: The more recent works [CMY20, DKK + 21a] further addressed the computational efﬁciency of this task and achieved almost linear running time in certain regimes.
Revised: The more recent works [CMY20, DKK + 21a] further addressed the computational efﬁciency of this task and achieved almost linear running time in certain regimes.

Original: Although all of these algorithms exhibit near-optimal guarantees on either error rate or computational complexity, it turns out that less is explored to improve another yet important metric: the sample complexity.
Revised: Although all of these algorithms exhibit near-optimal guarantees on either error rate or computational complexity, it turns out that less is explored to improve another yet important metric: the sample complexity.

Original: In particular, the sample complexity of all these algorithms is O (poly( d )) , hence they quickly break down for data-demanding applications such as healthcare where the number of available samples is typically orders of magnitude less than the dimension d
Revised: In particular, the sample complexity of all these algorithms is O (poly( d )) , hence they quickly break down for data-demanding applications such as healthcare where the number of available samples is typically orders of magnitude less than the dimension d

Original: [Wai19].
Revised: [Wai19].

Original: Therefore, a pressing question that needs to be addressed in such a high-dimensional regime is the following:
Revised: Therefore, a pressing question that needs to be addressed in such a high-dimensional regime is the following:

Original: Does
Revised: Does

Original: there exist a provably robust algorithm for list-decodable mean estimation
Revised: there exist a provably robust algorithm for list-decodable mean estimation

Original: that
Revised: that

Original: runs in polynomial time and enjoys a sample complexity bound of O (polylog( d ))
Revised: runs in polynomial time and enjoys a sample complexity bound of O (polylog( d ))

Original: In this paper, we answer the question in the afﬁrmative by showing that when the target mean is k -sparse, i.e. it has at most k non-zero elements, it is attribute-efﬁciently list-decodable.
Revised: In this paper, we answer the question in the afﬁrmative by showing that when the target mean is k -sparse, i.e. it has at most k non-zero elements, it is attribute-efﬁciently list-decodable.

Original: Theorem 1 (Main result) .
Revised: Theorem 1 (Main result) .

Original: Given noise rate α ∈ (0 , 12 ] , failure probability τ ∈ (0 , 1) , a natural number (cid:96) ≥ 1 , and a set T of Ω (cid:0) (cid:96) 7 · k 14 (cid:96) α· log 8 (cid:96) ( (cid:96)dατ ) (cid:1) samples in R d , of which at least a (2 α ) -fraction are independent draws from the Gaussian distribution N ( µ, I d ) where (cid:107) µ (cid:107) 0 ≤ k , there exists an algorithm that runs in time poly (cid:0)
Revised: Given noise rate α ∈ (0 , 12 ] , failure probability τ ∈ (0 , 1) , a natural number (cid:96) ≥ 1 , and a set T of Ω (cid:0) (cid:96) 7 · k 14 (cid:96) α· log 8 (cid:96) ( (cid:96)dατ ) (cid:1) samples in R d , of which at least a (2 α ) -fraction are independent draws from the Gaussian distribution N ( µ, I d ) where (cid:107) µ (cid:107) 0 ≤ k , there exists an algorithm that runs in time poly (cid:0)

Original: | T | , d (cid:96) , 1 α (cid:1) , uses polynomials of degree at most 2 (cid:96) , and returns a list of O (1 /α ) number of k -sparse vectors such that with probability 1 − τ , the list contains at least one ˆ µ ∈ R d with (cid:107) ˆ µ − µ (cid:107) 

Revised: | T | , d (cid:96) , 1 α (cid:1) , uses polynomials of degree at most 2 (cid:96) , and returns a list of O (1 /α ) number of k -sparse vectors such that with probability 1 − τ , the list contains at least one ˆ µ ∈ R d with (cid:107) ˆ µ − µ (cid:107)

Original: = ˜ O (cid:0) α − 12 (cid:96) · √ (cid:96) ( (cid:96) + log 1 α ) (cid:1) , where ˜ O ( · ) hides poly-logarithmic factors.
Revised: = ˜ O (cid:0) α − 12 (cid:96) · √ (cid:96) ( (cid:96) + log 1 α ) (cid:1) , where ˜ O ( · ) hides poly-logarithmic factors.

Original: Remark 2.
Revised: Remark 2.

Original: The key message of the theorem is that when the true mean is k -sparse, it is possible to efﬁciently approximate it with O (polylog( d )) samples.
Revised: The key message of the theorem is that when the true mean is k -sparse, it is possible to efﬁciently approximate it with O (polylog( d )) samples.

Original: This is in stark contrast to existing listdecodable results [CSV17, DKS18b, CMY20, DKK20a, DKK + 21a] where the sample complexity is
Revised: This is in stark contrast to existing listdecodable results [CSV17, DKS18b, CMY20, DKK20a, DKK + 21a] where the sample complexity is

Original: O (poly( d )) .
Revised: O (poly( d )) .

Original: The only attribute-efﬁcient robust mean estimators are [BDLS17, DKK + 19, CDK + 21], but their results hold only for the mild corruption regime where α > 1 / 2 .
Revised: The only attribute-efﬁcient robust mean estimators are [BDLS17, DKK + 19, CDK + 21], but their results hold only for the mild corruption regime where α > 1 / 2 .

Original: Remark 3.
Revised: Remark 3.

Original: Our algorithm and analysis hold for any degree (cid:96) ≥ 1 .
Revised: Our algorithm and analysis hold for any degree (cid:96) ≥ 1 .

Original: When (cid:96) = 1 , the sample complexity reads as ˜ O
Revised: When (cid:96) = 1 , the sample complexity reads as ˜ O

Original: ( α − 7 k 14 log 8 d ) and the algorithm achieves error
Revised: ( α − 7 k 14 log 8 d ) and the algorithm achieves error

Original: ˜ O
Revised: ˜ O

Original: ( α − 12 ) .
Revised: ( α − 12 ) .

Original: As opposed to an
Revised: As opposed to an

Original: O (1 − α ) error rate obtained for α > 1 / 2 , the (non-vanishing) error rate ˜ O
Revised: O (1 − α ) error rate obtained for α > 1 / 2 , the (non-vanishing) error rate ˜ O

Original: ( α − 12 ) is typically what one can expect for list-decodable mean estimation under bounded second order moment condition, in light of the lower bounds in [DKS18b].
Revised: ( α − 12 ) is typically what one can expect for list-decodable mean estimation under bounded second order moment condition, in light of the lower bounds in [DKS18b].

Original: When leveraging degree- 2 (cid:96) polynomials into algorithmic design, we obtain the improved ˜ O (cid:0) α − 12 (cid:96) √ (cid:96) ( (cid:96) + log 1 α ) (cid:1) error guarantee.
Revised: When leveraging degree- 2 (cid:96) polynomials into algorithmic design, we obtain the improved ˜ O (cid:0) α − 12 (cid:96) √ (cid:96) ( (cid:96) + log 1 α ) (cid:1) error guarantee.

Original: Specially, when taking (cid:96) =
Revised: Specially, when taking (cid:96) =

Original: Θ (cid:0) log 1 α (cid:1) , our algorithm achieves error rate of ˜ O (cid:0) log 32 ( 1 α ) (cid:1) in quasi-polynomial time.
Revised: Θ (cid:0) log 1 α (cid:1) , our algorithm achieves error rate of ˜ O (cid:0) log 32 ( 1 α ) (cid:1) in quasi-polynomial time.

Original: This is very close to the minimax error rate of Θ(log 12 ( 1 α )) established in [DKS18b].
Revised: This is very close to the minimax error rate of Θ(log 12 ( 1 α )) established in [DKS18b].

Original: Remark 4.
Revised: Remark 4.

Original: If we further increase the sample size with an (cid:96) (cid:96) multiplicative factor with (cid:96) = Θ(log 1 α ) , our algorithm will achieve an ˜ O (cid:0) log 12 ( 1 α ) (cid:1) error guarantee, which matches the minimax lower bound.
Revised: If we further increase the sample size with an (cid:96) (cid:96) multiplicative factor with (cid:96) = Θ(log 1 α ) , our algorithm will achieve an ˜ O (cid:0) log 12 ( 1 α ) (cid:1) error guarantee, which matches the minimax lower bound.

Original: The proof follows the same pipeline and we leave it to interested readers.
Revised: The proof follows the same pipeline and we leave it to interested readers.

Original: 1.1 Overview of Our Techniques
Revised: 1.1 Overview of Our Techniques

Original: Our main algorithm is inspired by the multiﬁltering framework of [DKS18b], where the primary idea is to construct a sequence of polynomials to test the concentration of the samples to Gaussian so that the algorithm either certiﬁes that the sample set behaves like Gaussian, or sanitizes it by removing a sufﬁcient amount of outliers.
Revised: Our main algorithm is inspired by the multiﬁltering framework of [DKS18b], where the primary idea is to construct a sequence of polynomials to test the concentration of the samples to Gaussian so that the algorithm either certiﬁes that the sample set behaves like Gaussian, or sanitizes it by removing a sufﬁcient amount of outliers.

Original: Our key technical contribution lies into a new design of sparse polynomials , and new ﬁltering rules tailored to the sparse polynomials.
Revised: Our key technical contribution lies into a new design of sparse polynomials , and new ﬁltering rules tailored to the sparse polynomials.

Original: Sparse polynomials and sparsity-induced ﬁlters.
Revised: Sparse polynomials and sparsity-induced ﬁlters.

Original: To ensure that our algorithm is attribute-efﬁcient, we will only control the maximum eigenvalue of the sample covariance matrix on sparse directions.
Revised: To ensure that our algorithm is attribute-efﬁcient, we will only control the maximum eigenvalue of the sample covariance matrix on sparse directions.

Original: Since such computation is NP-hard in general, we ﬁrst consider a sufﬁcient condition which tests the maximum Frobenuis norm under a cardinality constraint, similar to the idea of [DKK + 19].
Revised: Since such computation is NP-hard in general, we ﬁrst consider a sufﬁcient condition which tests the maximum Frobenuis norm under a cardinality constraint, similar to the idea of [DKK + 19].

Original: If such Frobenuis norm is small, it implies a small restricted eigenvalue and hence the sample mean is returned.
Revised: If such Frobenuis norm is small, it implies a small restricted eigenvalue and hence the sample mean is returned.

Original: Otherwise, we construct sparse polynomials in the sense that they can be represented by a set of O ( (cid:96) 2 k 4 (cid:96) ) basis polynomials and O ( (cid:96)k 2 (cid:96) ) coordinates of the samples (see Deﬁnition 7), and measure the concentration of these sparse polynomials to the Gaussian.
Revised: Otherwise, we construct sparse polynomials in the sense that they can be represented by a set of O ( (cid:96) 2 k 4 (cid:96) ) basis polynomials and O ( (cid:96)k 2 (cid:96) ) coordinates of the samples (see Deﬁnition 7), and measure the concentration of these sparse polynomials to the Gaussian.

Original: Now as the underlying polynomials are sparse, we also design new sparsity-induced ﬁlters to certify the sample set, as otherwise a large amount of clean samples will be removed.
Revised: Now as the underlying polynomials are sparse, we also design new sparsity-induced ﬁlters to certify the sample set, as otherwise a large amount of clean samples will be removed.

Original: See Algorithm 3 and Algorithm 4.
Revised: See Algorithm 3 and Algorithm 4.

Original: Clustering by L ∞ -norm.
Revised: Clustering by L ∞ -norm.

Original: Technically, the success of our attribute-efﬁcient multiﬁltering approach hinges on a condition that all the samples lie within a small L ∞ -norm ball.
Revised: Technically, the success of our attribute-efﬁcient multiﬁltering approach hinges on a condition that all the samples lie within a small L ∞ -norm ball.

Original: It is not hard to see that all the Gaussian samples satisfy such condition, and we show that there is a simple scheme which can simultaneously prune and cluster the given samples into O (1 /α ) groups, such that the retained samples are close under the L ∞ -norm and at least one group contains most of the Gaussian samples.
Revised: It is not hard to see that all the Gaussian samples satisfy such condition, and we show that there is a simple scheme which can simultaneously prune and cluster the given samples into O (1 /α ) groups, such that the retained samples are close under the L ∞ -norm and at least one group contains most of the Gaussian samples.

Original: We note that the use of the L ∞ -norm as our metric ensures attribute efﬁciency of this step.
Revised: We note that the use of the L ∞ -norm as our metric ensures attribute efﬁciency of this step.

Original: An immediate implication of this clustering step is that the polynomials of Gaussian samples will be close enough, which facilitates the analysis of the performance of our ﬁlters.
Revised: An immediate implication of this clustering step is that the polynomials of Gaussian samples will be close enough, which facilitates the analysis of the performance of our ﬁlters.

Original: See Section 2.3.
Revised: See Section 2.3.

Original: 1.2 Related Works
Revised: 1.2 Related Works

Original: Breaking the barrier of the typical O (poly( d )) sample complexity bound is one of the central problems across many ﬁelds of science and engineering.
Revised: Breaking the barrier of the typical O (poly( d )) sample complexity bound is one of the central problems across many ﬁelds of science and engineering.

Original: Motivated by real-world applications, a property termed sparsity is often assumed for this end, meaning that only k out of the d number of attributes contribute to the underlying inference problem.
Revised: Motivated by real-world applications, a property termed sparsity is often assumed for this end, meaning that only k out of the d number of attributes contribute to the underlying inference problem.

Original: In this way, an improved bound of
Revised: In this way, an improved bound of

Original: O (poly( k, log d )) can be obtained in many inference paradigms such as linear regression [CDS98,
Revised: O (poly( k, log d )) can be obtained in many inference paradigms such as linear regression [CDS98,

Original: Tib96, CT05, Don06, PV13, PVY17, SL17a, SL17b, SL18], learning of threshold functions [Lit87,
Revised: Tib96, CT05, Don06, PV13, PVY17, SL17a, SL17b, SL18], learning of threshold functions [Lit87,

Original: BHL95, STT12, ABHZ16, ZSA20, SZ21], principal component analysis [Ma13, DKK + 19], and mean estimation [BDLS17, DKK + 19, CDK + 21].
Revised: BHL95, STT12, ABHZ16, ZSA20, SZ21], principal component analysis [Ma13, DKK + 19], and mean estimation [BDLS17, DKK + 19, CDK + 21].

Original: Unfortunately, the success of all these attributeefﬁcient algorithms hinges on the presumption that the majority of the data are uncorrupted.
Revised: Unfortunately, the success of all these attributeefﬁcient algorithms hinges on the presumption that the majority of the data are uncorrupted.

Original: Learning with mild corruption ( α > 1 / 2 ).
Revised: Learning with mild corruption ( α > 1 / 2 ).

Original: Learning in the presence of noise has been extensively studied in a broad context.
Revised: Learning in the presence of noise has been extensively studied in a broad context.

Original: In supervised learning where a sample consists of an instance (i.e. feature vector) and a label, lots of research efforts were dedicated to robust algorithms under label noise [AL87, Slo88, MN06].
Revised: In supervised learning where a sample consists of an instance (i.e. feature vector) and a label, lots of research efforts were dedicated to robust algorithms under label noise [AL87, Slo88, MN06].

Original: Recent years have witnessed signiﬁcant progress towards optimal algorithms in the presence of label noise, see for example, [KKMS05, ABL17, DKTZ20, ZSA20,
Revised: Recent years have witnessed signiﬁcant progress towards optimal algorithms in the presence of label noise, see for example, [KKMS05, ABL17, DKTZ20, ZSA20,

Original: DKK + 20b, ZS22] and the references therein.
Revised: DKK + 20b, ZS22] and the references therein.

Original: The regime that both instances and labels are corrupted turns out to be signiﬁcantly more challenging.
Revised: The regime that both instances and labels are corrupted turns out to be signiﬁcantly more challenging.

Original: The problem of learning halfspaces under such setting was put forward in the 1980s [Val85, KL88], yet only until recently have efﬁcient algorithms been established with near-optimal noise tolerance [ABL17, DKS18a, She21, SZ21].
Revised: The problem of learning halfspaces under such setting was put forward in the 1980s [Val85, KL88], yet only until recently have efﬁcient algorithms been established with near-optimal noise tolerance [ABL17, DKS18a, She21, SZ21].

Original: In addition, [BJK15,
Revised: In addition, [BJK15,

Original: KKM18, LSLC20] studied robust linear regression and [BDLS17] presented a set of interesting results under various statistical models.
Revised: KKM18, LSLC20] studied robust linear regression and [BDLS17] presented a set of interesting results under various statistical models.

Original: More in line with this work is the problem of robust mean estimation, see the breakthrough works of [DKK + 16, LRV16] and many follow-up works [DKK + 17,
Revised: More in line with this work is the problem of robust mean estimation, see the breakthrough works of [DKK + 16, LRV16] and many follow-up works [DKK + 17,

Original: BDLS17, DKS17, SCV18, KSS18, DKK + 19, HLZ20, CDK + 21].
Revised: BDLS17, DKS17, SCV18, KSS18, DKK + 19, HLZ20, CDK + 21].

Original: Learning with overwhelming corruption ( α ≤ 1 / 2 ).
Revised: Learning with overwhelming corruption ( α ≤ 1 / 2 ).

Original: The agnostic label noise of [Hau92, KSS92] seems the earliest model that allows the adversary to arbitrarily corrupt any fraction of the data (say 70% ), though it can only corrupt labels.
Revised: The agnostic label noise of [Hau92, KSS92] seems the earliest model that allows the adversary to arbitrarily corrupt any fraction of the data (say 70% ), though it can only corrupt labels.

Original: Following [CSV17], a considerate number of of recent works have studied the scenario that both instances and labels are grossly corrupted, and the goal is to output a ﬁnite list of candidate parameters among which at least one is a good approximation to the target.
Revised: Following [CSV17], a considerate number of of recent works have studied the scenario that both instances and labels are grossly corrupted, and the goal is to output a ﬁnite list of candidate parameters among which at least one is a good approximation to the target.

Original: This includes list-decodable learning of mixture models [DKS18b, DKK + 21b], regression [KKK19,
Revised: This includes list-decodable learning of mixture models [DKS18b, DKK + 21b], regression [KKK19,

Original: RY20a], and subspace recovery [RY20b, BK21].
Revised: RY20a], and subspace recovery [RY20b, BK21].

Original: Interestingly, there are some works studying the problem under crowdsourcing models, where the samples are collected from crowd workers and most of them behave adversarially [SVC16, ABHM17, MV18, ZS21].
Revised: Interestingly, there are some works studying the problem under crowdsourcing models, where the samples are collected from crowd workers and most of them behave adversarially [SVC16, ABHM17, MV18, ZS21].

Original: It is worth noting that [DKK + 22] concurrently and independently developed a polynomial-time algorithm to solve the same problem, with an interesting difference-of-pairs metric to ﬁlter outliers.
Revised: It is worth noting that [DKK + 22] concurrently and independently developed a polynomial-time algorithm to solve the same problem, with an interesting difference-of-pairs metric to ﬁlter outliers.

Original: 1.3 Roadmap
Revised: 1.3 Roadmap

Original: We collect useful notations, deﬁnitions, and some preliminary results in Section 2.
Revised: We collect useful notations, deﬁnitions, and some preliminary results in Section 2.

Original: Our main algorithms are described in Section 3 along with performance guarantees.
Revised: Our main algorithms are described in Section 3 along with performance guarantees.

Original: We conclude the work in
Revised: We conclude the work in

Original: Section 4, and defer all proof details to the appendix.
Revised: Section 4, and defer all proof details to the appendix.

Original:  Preliminaries
Revised:  Preliminaries

Original: Vector, matrix, and tensor.
Revised: Vector, matrix, and tensor.

Original: For a d -dimensional vector v = ( v 1 , . . .
Revised: For a d -dimensional vector v = ( v 1 , . . .

Original: , v d ) , denote by (cid:107) v (cid:107) 

Revised: , v d ) , denote by (cid:107) v (cid:107)

Original: its
Revised: its

Original: L 2 -norm, (cid:107) v (cid:107) 1 its L 1 -norm, (cid:107) v (cid:107) 

Revised: L 2 -norm, (cid:107) v (cid:107) 1 its L 1 -norm, (cid:107) v (cid:107)

Original: its L 0 -“norm” that counts the number of non-zeros, and (cid:107) v (cid:107) ∞ its inﬁnity norm.
Revised: its L 0 -“norm” that counts the number of non-zeros, and (cid:107) v (cid:107) ∞ its inﬁnity norm.

Original: The hard thresholding operator trim k : R d → R d keeps the k largest elements (in magnitude) of a vector and sets the remaining to zero.
Revised: The hard thresholding operator trim k : R d → R d keeps the k largest elements (in magnitude) of a vector and sets the remaining to zero.

Original: Let [ d ] := { 1 , 2 , . . .
Revised: Let [ d ] := { 1 , 2 , . . .

Original: , d } for some natural number d > 0 .
Revised: , d } for some natural number d > 0 .

Original: For an index set Ω ⊆
Revised: For an index set Ω ⊆

Original: [ d ] , v Ω is the vector of v restricted on Ω .
Revised: [ d ] , v Ω is the vector of v restricted on Ω .

Original: We say a vector is k -sparse if it has at most k non-zero elements, and likewise for matrices and tensors.
Revised: We say a vector is k -sparse if it has at most k non-zero elements, and likewise for matrices and tensors.

Original: For a matrix M of size d 1 × d 2 , denote by (cid:107) M (cid:107) F its Frobenius norm and by (cid:107) M (cid:107) ∗ its nuclear norm.
Revised: For a matrix M of size d 1 × d 2 , denote by (cid:107) M (cid:107) F its Frobenius norm and by (cid:107) M (cid:107) ∗ its nuclear norm.

Original: For
Revised: For

Original: U ⊆
Revised: U ⊆

Original: [ d 1 ] × [ d 2 ] , denote by M U the submatrix of M with entries restricted to U .
Revised: [ d 1 ] × [ d 2 ] , denote by M U the submatrix of M with entries restricted to U .

Original: We also use tensors in our algorithms to ease expressions.
Revised: We also use tensors in our algorithms to ease expressions.

Original: Note that vectors and matrices can be seen as order- 1 and order- 2 tensors respectively.
Revised: Note that vectors and matrices can be seen as order- 1 and order- 2 tensors respectively.

Original: We say that an order- l tensor A is symmetric if
Revised: We say that an order- l tensor A is symmetric if

Original: A i 1 ,...,i l =
Revised: A i 1 ,...,i l =

Original: A π ( i 1 ,...,i l ) for all permutations π .
Revised: A π ( i 1 ,...,i l ) for all permutations π .

Original: Given two tensors A and B , denote by A ⊗ B the outer product (or tensor product) of A and B .
Revised: Given two tensors A and B , denote by A ⊗ B the outer product (or tensor product) of A and B .

Original: We will slightly abuse (cid:107) A (cid:107) 2 to denote the L 2 -norm of a tensor A by seeing it as a long vector.
Revised: We will slightly abuse (cid:107) A (cid:107) 2 to denote the L 2 -norm of a tensor A by seeing it as a long vector.

Original: Probability.
Revised: Probability.

Original: We reserve the capital letter G for a random draw from N ( µ, I d ) , i.e. G ∼ N ( µ, I d ) , where µ ∈ R d is the target mean that we aim to estimate which is assumed to be k -sparse.
Revised: We reserve the capital letter G for a random draw from N ( µ, I d ) , i.e. G ∼ N ( µ, I d ) , where µ ∈ R d is the target mean that we aim to estimate which is assumed to be k -sparse.

Original: Suppose that T is a ﬁnite sample set.
Revised: Suppose that T is a ﬁnite sample set.

Original: We use µ T to denote the sample mean of T , i.e. µ T = 1 | T | (cid:80) x ∈ T x , and use p ( T ) to denote the random variable p ( x ) where x is drawn uniformly from T .
Revised: We use µ T to denote the sample mean of T , i.e. µ T = 1 | T | (cid:80) x ∈ T x , and use p ( T ) to denote the random variable p ( x ) where x is drawn uniformly from T .

Original: Constants.
Revised: Constants.

Original: The capital letter C and its subscript variants such as C 1 , C 2 are used to denote positive absolute constants.
Revised: The capital letter C and its subscript variants such as C 1 , C 2 are used to denote positive absolute constants.

Original: However, their values may change from appearance to appearance.
Revised: However, their values may change from appearance to appearance.

Original: 2.1 Polynomials
Revised: 2.1 Polynomials

Original: Let x = ( x 1 , . . .
Revised: Let x = ( x 1 , . . .

Original: , x d ) be a d -dimensional vector in R d , and let a = ( a 1 , . .
Revised: , x d ) be a d -dimensional vector in R d , and let a = ( a 1 , . .

Original: . , a d ) ∈ N d be a ddimensional multi-index.
Revised: . , a d ) ∈ N d be a ddimensional multi-index.

Original: A monomial of x is a product of powers of the coordinates of x with natural exponents, written as x a := (cid:81) dj
Revised: A monomial of x is a product of powers of the coordinates of x with natural exponents, written as x a := (cid:81) dj

Original: =1 x a j j .
Revised: =1 x a j j .

Original: A polynomial of x , p ( x ) , is a ﬁnite sum of its monomials multiplied by real coefﬁcients; that is, p ( x ) =
Revised: A polynomial of x , p ( x ) , is a ﬁnite sum of its monomials multiplied by real coefﬁcients; that is, p ( x ) =

Original: (cid:80) a ∈A c a x a where A ⊂ N d is a ﬁnite set of multiindices and the c
Revised: (cid:80) a ∈A c a x a where A ⊂ N d is a ﬁnite set of multiindices and the c

Original: a ’s are real coefﬁcients.
Revised: a ’s are real coefﬁcients.

Original: Note that the degree of p ( x ) is given by max a ∈A (cid:107) a (cid:107) 1 .
Revised: Note that the degree of p ( x ) is given by max a ∈A (cid:107) a (cid:107) 1 .

Original: We denote by P ( R d , l ) the class of polynomials on R d with degree at most l .
Revised: We denote by P ( R d , l ) the class of polynomials on R d with degree at most l .

Original: We will often use the probabilist’s Hermite polynomials that form a complete orthogonal basis with respect to N (0 , I d ) .
Revised: We will often use the probabilist’s Hermite polynomials that form a complete orthogonal basis with respect to N (0 , I d ) .

Original: Deﬁnition 5 (Hermite polynomials) .
Revised: Deﬁnition 5 (Hermite polynomials) .

Original: Let x ∈ R be a variate.
Revised: Let x ∈ R be a variate.

Original: For any natural number l ∈ N , the degree- l Hermite polynomial is deﬁned as He l ( x ) =
Revised: For any natural number l ∈ N , the degree- l Hermite polynomial is deﬁned as He l ( x ) =

Original: ( − 1) l e x 2 2 d l dx l e − x 2 2 .
Revised: ( − 1) l e x 2 2 d l dx l e − x 2 2 .

Original: For a ∈ N d and x ∈ R d , the d -variate Hermite polynomial is given by He a ( x ) :
Revised: For a ∈ N d and x ∈ R d , the d -variate Hermite polynomial is given by He a ( x ) :

Original: = (cid:81) di
Revised: = (cid:81) di

Original: =

Revised: =

Original: He a i ( x i ) , which is of degree (cid:107) a (cid:107) 1 .
Revised: He a i ( x i ) , which is of degree (cid:107) a (cid:107) 1 .

Original: Harmonic and homogeneous polynomials.
Revised: Harmonic and homogeneous polynomials.

Original: A polynomial h ( x ) ∈ P ( R d , l ) is called harmonic if it can be written as a linear combination of degree- l Hermite polynomials.
Revised: A polynomial h ( x ) ∈ P ( R d , l ) is called harmonic if it can be written as a linear combination of degree- l Hermite polynomials.

Original: A polynomial Hom( x ) ∈ P ( R d , l ) is called homogeneous if all of its monomials have degree exactly l .
Revised: A polynomial Hom( x ) ∈ P ( R d , l ) is called homogeneous if all of its monomials have degree exactly l .

Original: Fact 6.
Revised: Fact 6.

Original: If a polynomial is degree- l harmonic or homogeneous, then there is a one-to-one mapping between it and an order- l symmetric tensor.
Revised: If a polynomial is degree- l harmonic or homogeneous, then there is a one-to-one mapping between it and an order- l symmetric tensor.

Original: To see this, we may deﬁne an operation “ ◦ ” such that He l ( x i ) ◦
Revised: To see this, we may deﬁne an operation “ ◦ ” such that He l ( x i ) ◦

Original: He l ( x j ) =
Revised: He l ( x j ) =

Original: He l ( x i ) ·
Revised: He l ( x i ) ·

Original: He l ( x j ) if i (cid:54) =
Revised: He l ( x j ) if i (cid:54) =

Original: j and equals He 2 l ( x i ) otherwise.
Revised: j and equals He 2 l ( x i ) otherwise.

Original: Then any degree- l Hermite polynomial can be written as
Revised: Then any degree- l Hermite polynomial can be written as

Original: He 1 ( x i 1 ) ◦
Revised: He 1 ( x i 1 ) ◦

Original: He 1 ( x i 2 ) · · · ◦ He 1 ( x i l ) where all the indices i t ∈ [ d ] .
Revised: He 1 ( x i 2 ) · · · ◦ He 1 ( x i l ) where all the indices i t ∈ [ d ] .

Original: We will consider that one such sequence ( i 1 , . . .
Revised: We will consider that one such sequence ( i 1 , . . .

Original: , i l ) exactly corresponds to one degree- l Hermite polynomial on R d , and there are d l number of such sequences that form all degree- l Hermite polynomials.
Revised: , i l ) exactly corresponds to one degree- l Hermite polynomial on R d , and there are d l number of such sequences that form all degree- l Hermite polynomials.

Original: In this sense, any harmonic polynomial h ( x ) can be written as h ( x ) =
Revised: In this sense, any harmonic polynomial h ( x ) can be written as h ( x ) =

Original: (cid:80) i 1 ,...,
Revised: (cid:80) i 1 ,...,

Original: i l A
Revised: i l A

Original: i
Revised: i

Original: [Equation]
Revised: [Equation]

Original: where A i 1 ,...,i l ’s are the coefﬁcients which form an order- l tensor.
Revised: where A i 1 ,...,i l ’s are the coefﬁcients which form an order- l tensor.

Original: If we choose A as symmetric, it is easy to see that A fully represents h ( x ) .
Revised: If we choose A as symmetric, it is easy to see that A fully represents h ( x ) .

Original: Then, we can convert “ ◦ ” back to the regular product by counting the number of times a particular index j appearing in ( i 1 , . . .
Revised: Then, we can convert “ ◦ ” back to the regular product by counting the number of times a particular index j appearing in ( i 1 , . . .

Original: , i l ) .
Revised: , i l ) .

Original: If we denote this number as c j ( i 1 , . .
Revised: If we denote this number as c j ( i 1 , . .

Original: , i l ) , we have
Revised: , i l ) , we have

Original: [Equation]
Revised: [Equation]

Original: where the factor 1 / √ l ! is only used to normalize the magnitude of A to ease our analysis.
Revised: where the factor 1 / √ l ! is only used to normalize the magnitude of A to ease our analysis.

Original: Likewise, any homogeneous polynomial takes the form
Revised: Likewise, any homogeneous polynomial takes the form

Original: [Equation]
Revised: [Equation]

Original: Sparse polynomials.
Revised: Sparse polynomials.

Original: In order to deﬁne sparse polynomials, we will ﬁrst specify a set of basis polynomials { b 1 , . . .
Revised: In order to deﬁne sparse polynomials, we will ﬁrst specify a set of basis polynomials { b 1 , . . .

Original: , b d l } ⊂ P ( R d , l ) .
Revised: , b d l } ⊂ P ( R d , l ) .

Original: In this paper, we will either choose such set as all degree- l monomials or all degree- l Hermite polynomials.
Revised: In this paper, we will either choose such set as all degree- l monomials or all degree- l Hermite polynomials.

Original: Deﬁnition 7 ( ( κ, ψ ) -sparse polynomials) .
Revised: Deﬁnition 7 ( ( κ, ψ ) -sparse polynomials) .

Original: We say that a polynomial p ∈ P ( R d , l ) is ( κ, ψ ) -sparse if it can be represented by at most κ number of basis polynomials and ψ coordinates of the input vector.
Revised: We say that a polynomial p ∈ P ( R d , l ) is ( κ, ψ ) -sparse if it can be represented by at most κ number of basis polynomials and ψ coordinates of the input vector.

Original: We denote by P ( R d , l, κ, ψ ) the class of ( κ, ψ ) -sparse polynomials.
Revised: We denote by P ( R d , l, κ, ψ ) the class of ( κ, ψ ) -sparse polynomials.

Original: Note that when κ and l are ﬁxed, p ( x ) will depend on at most κ · l coordinates.
Revised: Note that when κ and l are ﬁxed, p ( x ) will depend on at most κ · l coordinates.

Original: Thus, the introduction of the parameter ψ makes sense only when ψ ≤ κ · l .
Revised: Thus, the introduction of the parameter ψ makes sense only when ψ ≤ κ · l .

Original: In our algorithm, we will always have l ≤ 2 (cid:96) , κ = 4 (cid:96) 2 k 4 (cid:96) , and ψ = 2 (cid:96)k 2 (cid:96) for some natural number (cid:96) ≥ 1 .
Revised: In our algorithm, we will always have l ≤ 2 (cid:96) , κ = 4 (cid:96) 2 k 4 (cid:96) , and ψ = 2 (cid:96)k 2 (cid:96) for some natural number (cid:96) ≥ 1 .

Original: 2.2 Representative Set and Good Set
Revised: 2.2 Representative Set and Good Set

Original: To ease our analysis, we will need a deterministic condition on the set of uncorrupted samples.
Revised: To ease our analysis, we will need a deterministic condition on the set of uncorrupted samples.

Original: Deﬁnition 8 (Representative set) .
Revised: Deﬁnition 8 (Representative set) .

Original: Given α ∈ (0 , 12 ] and τ ∈ (0 , 1) , we say that a sample set S G ⊂ R d is representative with respect to P := P ( R d , 2 (cid:96), 4 (cid:96) 2 k 4 (cid:96) , 2 (cid:96)k 2 (cid:96) )
Revised: Given α ∈ (0 , 12 ] and τ ∈ (0 , 1) , we say that a sample set S G ⊂ R d is representative with respect to P := P ( R d , 2 (cid:96), 4 (cid:96) 2 k 4 (cid:96) , 2 (cid:96)k 2 (cid:96) )

Original: if the following holds:
Revised: if the following holds:

Original: [Equation]
Revised: [Equation]

Original: , where
Revised: , where

Original: [Equation]
Revised: [Equation]

Original: We show that a sufﬁciently large set drawn independently from N ( µ, I d ) is representative.
Revised: We show that a sufﬁciently large set drawn independently from N ( µ, I d ) is representative.

Original: The proof follows from the classic VC theory, and is deferred to Appendix A.1.
Revised: The proof follows from the classic VC theory, and is deferred to Appendix A.1.

Original: Proposition 9 (Sample complexity) .
Revised: Proposition 9 (Sample complexity) .

Original: Given α ∈ (0 , 12 ] and τ ∈ (0 , 1) , let S G be a set consisting of
Revised: Given α ∈ (0 , 12 ] and τ ∈ (0 , 1) , let S G be a set consisting of

Original: [Equation]
Revised: [Equation]

Original: independent samples from N ( µ, I d ) where C > 0 is a sufﬁciently large absolute constant.
Revised: independent samples from N ( µ, I d ) where C > 0 is a sufﬁciently large absolute constant.

Original: Then, with probability 1 − τ ,
Revised: Then, with probability 1 − τ ,

Original: [Equation]
Revised: [Equation]

Original: In particular, when l = 2 (cid:96) , κ = 4 (cid:96) 2 k 4 (cid:96) , ψ = 2 (cid:96)k 2 (cid:96) , and (cid:15) =
Revised: In particular, when l = 2 (cid:96) , κ = 4 (cid:96) 2 k 4 (cid:96) , ψ = 2 (cid:96)k 2 (cid:96) , and (cid:15) =

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: log
Revised: log

Original: [Equation]
Revised: [Equation]

Original: ( (cid:96)dατ ) for some natural number (cid:96) ≥ 1 , it sufﬁces to pick | S G | =
Revised: ( (cid:96)dατ ) for some natural number (cid:96) ≥ 1 , it sufﬁces to pick | S G | =

Original: C (cid:48)
Revised: C (cid:48)

Original: [Equation]
Revised: [Equation]

Original: ) for some sufﬁciently large constant
Revised: ) for some sufﬁciently large constant

Original: C (cid:48) so that S G is a representative set.
Revised: C (cid:48) so that S G is a representative set.

Original: Our algorithm will progressively remove samples from T , and a key property that ensures the success of the algorithm is that most corrupted samples are eliminated while almost all uncorrupted samples are retained.
Revised: Our algorithm will progressively remove samples from T , and a key property that ensures the success of the algorithm is that most corrupted samples are eliminated while almost all uncorrupted samples are retained.

Original: Alternatively, we hope that T contains a representative set that contributes to a nontrivial fraction.
Revised: Alternatively, we hope that T contains a representative set that contributes to a nontrivial fraction.

Original: For technical reasons, we also require that all samples in T lie in a small L ∞ -ball.
Revised: For technical reasons, we also require that all samples in T lie in a small L ∞ -ball.

Original: Deﬁnition 10 ( α -good set) .
Revised: Deﬁnition 10 ( α -good set) .

Original: A multiset T ⊂ R
Revised: A multiset T ⊂ R

Original: d is α -good if the following holds:
Revised: d is α -good if the following holds:

Original: There exists a set S G which is representative and satisﬁes | S G ∩ T | ≥ max { (1 − α/ 6) | S G | , α | T |} .
Revised: There exists a set S G which is representative and satisﬁes | S G ∩ T | ≥ max { (1 − α/ 6) | S G | , α | T |} .

Original: max x,y ∈ T (cid:107) x − y (cid:107) ∞ ≤ C · (cid:112) log( d | S G | /τ ) for some constant C > 0 .
Revised: max x,y ∈ T (cid:107) x − y (cid:107) ∞ ≤ C · (cid:112) log( d | S G | /τ ) for some constant C > 0 .

Original: It is not hard to verify that the initial sample set T satisﬁes the ﬁrst condition, and will also fulﬁll the second one with a simple data pre-processing, as stated in the next section.
Revised: It is not hard to verify that the initial sample set T satisﬁes the ﬁrst condition, and will also fulﬁll the second one with a simple data pre-processing, as stated in the next section.

Original: 2.3 Clustering for the Initial List
Revised: 2.3 Clustering for the Initial List

Original: Since the corrupted samples may behave adversarially, we will perform a preliminary step of clustering which splits T into an initial list of subsets, among which at least one is α -good in the sense of
Revised: Since the corrupted samples may behave adversarially, we will perform a preliminary step of clustering which splits T into an initial list of subsets, among which at least one is α -good in the sense of

Original: Deﬁnition 10.
Revised: Deﬁnition 10.

Original: We ﬁrst show that all Gaussian samples have bounded L ∞ -norm with high probability, which simply follows from the Gaussian tail bound.
Revised: We ﬁrst show that all Gaussian samples have bounded L ∞ -norm with high probability, which simply follows from the Gaussian tail bound.

Original: Lemma 11.
Revised: Lemma 11.

Original: Given τ ∈ (0 , 1) , with probability 1 − τ , we have max x ∈ S G (cid:107) x − µ (cid:107) ∞ ≤ (cid:113) 2 log d | S G | τ , where S G is a set of samples drawn independently from N ( µ, I d ) .
Revised: Given τ ∈ (0 , 1) , with probability 1 − τ , we have max x ∈ S G (cid:107) x − µ (cid:107) ∞ ≤ (cid:113) 2 log d | S G | τ , where S G is a set of samples drawn independently from N ( µ, I d ) .

Original: The above observation implies that for any x, y ∈ S G , their distance under the L ∞ -norm metric is at most 2 (cid:112) 2 log( d | S G | /τ )
Revised: The above observation implies that for any x, y ∈ S G , their distance under the L ∞ -norm metric is at most 2 (cid:112) 2 log( d | S G | /τ )

Original: ≤ O (cid:0)(cid:113) (cid:96) · log (cid:96)dατ (cid:1) as far as the size of S G has the same order with the one in Proposition 9.
Revised: ≤ O (cid:0)(cid:113) (cid:96) · log (cid:96)dατ (cid:1) as far as the size of S G has the same order with the one in Proposition 9.

Original: To guarantee the existence of such S G , it sufﬁces to draw a corrupted sample set T that is 1 /α times larger than | S G | .
Revised: To guarantee the existence of such S G , it sufﬁces to draw a corrupted sample set T that is 1 /α times larger than | S G | .

Original: The lemma below further shows that this is sufﬁcient to guarantee the existence of an α -good subset of T .
Revised: The lemma below further shows that this is sufﬁcient to guarantee the existence of an α -good subset of T .

Original: Lemma 12 (C LUSTER ) .
Revised: Lemma 12 (C LUSTER ) .

Original: Given α ∈ (0 , 12 ] and τ ∈ (0 , 1) , let T be the sample set given to the learner.
Revised: Given α ∈ (0 , 12 ] and τ ∈ (0 , 1) , let T be the sample set given to the learner.

Original: If | T | =
Revised: If | T | =

Original: C · (cid:96) 7 · k 14 (cid:96) α· log 8 (cid:96) ( (cid:96)dατ ) and a (2 α ) -fraction are independent samples from N ( µ, I d ) ,
Revised: C · (cid:96) 7 · k 14 (cid:96) α· log 8 (cid:96) ( (cid:96)dατ ) and a (2 α ) -fraction are independent samples from N ( µ, I d ) ,

Original: Algorithm 1 returns a list of at most 1 /α many subsets of T , such that with probability at least 1 − τ , at least one of them is an α -good set.
Revised: Algorithm 1 returns a list of at most 1 /α many subsets of T , such that with probability at least 1 − τ , at least one of them is an α -good set.

Original: As will be clear in our analysis, the motivation of bounding the L ∞ -distance is to make sure that the function value of any p ( x ) =
Revised: As will be clear in our analysis, the motivation of bounding the L ∞ -distance is to make sure that the function value of any p ( x ) =

Original: h A ( x − µ T ) ∈ P ( R d , l, κ, ψ ) is bounded for samples in the α -good subset T i .
Revised: h A ( x − µ T ) ∈ P ( R d , l, κ, ψ ) is bounded for samples in the α -good subset T i .

Original: This is because when there exist a signiﬁcant fraction of good samples in T i , we want to efﬁciently distinguish the corrupted and uncorrupted ones.
Revised: This is because when there exist a signiﬁcant fraction of good samples in T i , we want to efﬁciently distinguish the corrupted and uncorrupted ones.

Original: A value-bounded polynomial function will facilitate our analysis on the function variance.
Revised: A value-bounded polynomial function will facilitate our analysis on the function variance.

Original: Lemma 13.
Revised: Lemma 13.

Original: Suppose that T is α -good and a polynomial p ∈ P ( R d , l, 4 (cid:96) 2 k 4 (cid:96) , 2 (cid:96)k 2 (cid:96) ) satisﬁes the following: there exists a symmetric order- l tensor A such that (cid:107) A (cid:107) 2 ≤ 1 and p ( x ) =
Revised: Suppose that T is α -good and a polynomial p ∈ P ( R d , l, 4 (cid:96) 2 k 4 (cid:96) , 2 (cid:96)k 2 (cid:96) ) satisﬁes the following: there exists a symmetric order- l tensor A such that (cid:107) A (cid:107) 2 ≤ 1 and p ( x ) =

Original: h A ( x − µ T ) .
Revised: h A ( x − µ T ) .

Original: Then, it holds that max x,y ∈ T | p ( x ) − p ( y ) | ≤ 2 k (cid:96) · γ l , where γ = C 0 · (cid:113) (cid:96) · log( (cid:96)dατ ) .
Revised: Then, it holds that max x,y ∈ T | p ( x ) − p ( y ) | ≤ 2 k (cid:96) · γ l , where γ = C 0 · (cid:113) (cid:96) · log( (cid:96)dατ ) .

Original:  Main Algorithms and Performance Guarantees
Revised:  Main Algorithms and Performance Guarantees

Original: We start with a review of the multiﬁltering framework that has been broadly used in prior works [DKS18b, DKK20a, DKK + 21b], followed by a highlight of our new techniques.
Revised: We start with a review of the multiﬁltering framework that has been broadly used in prior works [DKS18b, DKK20a, DKK + 21b], followed by a highlight of our new techniques.

Original: The multiﬁltering framework, i.e. Algorithm 2, includes three major steps.
Revised: The multiﬁltering framework, i.e. Algorithm 2, includes three major steps.

Original: The ﬁrst step is to invoke
Revised: The ﬁrst step is to invoke

Original: C LUSTER (Algorithm 1) to generate an initial list L which guarantees the existence of an α -good subset of T (see Lemma 12).
Revised: C LUSTER (Algorithm 1) to generate an initial list L which guarantees the existence of an α -good subset of T (see Lemma 12).

Original: We then imagine that there is a tree with root being the original contaminated sample set T and each child node of the root represents a member in L .
Revised: We then imagine that there is a tree with root being the original contaminated sample set T and each child node of the root represents a member in L .

Original: The algorithm iterates through these child nodes and performs one of the following: (1) creating a leaf node which is an estimate of the target mean; (2) creating one or two child nodes where are subsets of the parent node; (3) certifying that the set cannot be α -good and delete branch.
Revised: The algorithm iterates through these child nodes and performs one of the following: (1) creating a leaf node which is an estimate of the target mean; (2) creating one or two child nodes where are subsets of the parent node; (3) certifying that the set cannot be α -good and delete branch.

Original: In the end, if all leaves of the tree cannot be further split or deleted, the mean of the subsets on leaf nodes will be collected as a list M .
Revised: In the end, if all leaves of the tree cannot be further split or deleted, the mean of the subsets on leaf nodes will be collected as a list M .

Original: It is worth noting that the goal of algorithmic design is to guarantee that there always exists a branch that includes only α -good subsets.
Revised: It is worth noting that the goal of algorithmic design is to guarantee that there always exists a branch that includes only α -good subsets.

Original: In other words, at any level of the algorithm, at least one of the subsets of T is α -good, which ensures the existence of a good estimation in the returned list M .
Revised: In other words, at any level of the algorithm, at least one of the subsets of T is α -good, which ensures the existence of a good estimation in the returned list M .

Original: The ﬁnal step is a black-box algorithm that reduces the size of M from O (poly(1 /α )) to O (1 /α ) , which is due to [DKS18b].
Revised: The ﬁnal step is a black-box algorithm that reduces the size of M from O (poly(1 /α )) to O (1 /α ) , which is due to [DKS18b].

Original: Our technical contributions lie into an attribute-efﬁcient implementation of the ﬁrst and second steps.
Revised: Our technical contributions lie into an attribute-efﬁcient implementation of the ﬁrst and second steps.

Original: In this section, we elaborate on the second step, i.e. the
Revised: In this section, we elaborate on the second step, i.e. the

Original: A TTRIBUTE - EFFICIENT -M ULTIFILTER algorithm.
Revised: A TTRIBUTE - EFFICIENT -M ULTIFILTER algorithm.

Original: 3.1 Overview of Attribute-Efﬁcient Multiﬁltering
Revised: 3.1 Overview of Attribute-Efﬁcient Multiﬁltering

Original: The A TTRIBUTE - EFFICIENT -M ULTIFILTER algorithm is presented in Algorithm 3.
Revised: The A TTRIBUTE - EFFICIENT -M ULTIFILTER algorithm is presented in Algorithm 3.

Original: The starting point of the algorithm is a well-known fact that if the adversary were to signiﬁcantly deteriorate our estimate on µ , the spectral norm of a certain sample covariance matrix ˜Σ would become large [DKK + 16 , LRV16].
Revised: The starting point of the algorithm is a well-known fact that if the adversary were to signiﬁcantly deteriorate our estimate on µ , the spectral norm of a certain sample covariance matrix ˜Σ would become large [DKK + 16 , LRV16].

Original: In order to achieve attribute-efﬁcient sample complexity O (poly( k, log d )) , it is however vital to control the spectral norm only on k (cid:96) -sparse directions for some pre-speciﬁed polynomial degree (cid:96) ≥ 1 , which can further be certiﬁed by a small Frobenius norm restricted on the largest k 2 (cid:96) entries.
Revised: In order to achieve attribute-efﬁcient sample complexity O (poly( k, log d )) , it is however vital to control the spectral norm only on k (cid:96) -sparse directions for some pre-speciﬁed polynomial degree (cid:96) ≥ 1 , which can further be certiﬁed by a small Frobenius norm restricted on the largest k 2 (cid:96) entries.

Original: If the restricted Frobenius norm is sufﬁciently small, it implies that the sample covariance matrix behaves as a Gaussian one, and the algorithm returns the empirical mean truncated to be k -sparse
Revised: If the restricted Frobenius norm is sufﬁciently small, it implies that the sample covariance matrix behaves as a Gaussian one, and the algorithm returns the empirical mean truncated to be k -sparse

Original: (see Step 4).
Revised: (see Step 4).

Original: Otherwise, the algorithm will invoke either B ASIC MF (i.e. Algorithm 4)
Revised: Otherwise, the algorithm will invoke either B ASIC MF (i.e. Algorithm 4)

Original: Require: A multiset of samples T ⊂ R d , noise rate α ∈ (0 , 1 / 2] , failure probability τ ∈ (0 , 1) , degree of polynomials (cid:96) ≥ 1 .
Revised: Require: A multiset of samples T ⊂ R d , noise rate α ∈ (0 , 1 / 2] , failure probability τ ∈ (0 , 1) , degree of polynomials (cid:96) ≥ 1 .

Original: 1: { T 1 , . . .
Revised: 1: { T 1 , . . .

Original: , T m } ← C LUSTER ( T, α, τ, (cid:96) ) , L ← { ( T 1 , α/ 2) , . . .
Revised: , T m } ← C LUSTER ( T, α, τ, (cid:96) ) , L ← { ( T 1 , α/ 2) , . . .

Original: , ( T m , α/ 2) } , M ← ∅ .
Revised: , ( T m , α/ 2) } , M ← ∅ .

Original: 2: while L (cid:54) = ∅ do 3: ( T (cid:48) , α (cid:48) ) ← an element in L , L ← L\{ ( T (cid:48) , α (cid:48) ) } .
Revised: 2: while L (cid:54) = ∅ do 3: ( T (cid:48) , α (cid:48) ) ← an element in L , L ← L\{ ( T (cid:48) , α (cid:48) ) } .

Original: ANS ← A TTRIBUTE - EFFICIENT -M ULTIFILTER ( T (cid:48) , α (cid:48) , τ/ | T | , (cid:96) ) .
Revised: ANS ← A TTRIBUTE - EFFICIENT -M ULTIFILTER ( T (cid:48) , α (cid:48) , τ/ | T | , (cid:96) ) .

Original: 5: end while 6: return L IST R EDUCTION ( T
Revised: 5: end while 6: return L IST R EDUCTION ( T

Original: , M ) .
Revised: , M ) .

Original: Require: A multiset of samples T ⊂ R d , noise rate α ∈ (0 , 1 / 2] , failure probability τ ∈ (0 , 1) , degree of polynomials (cid:96) ≥ 1 .
Revised: Require: A multiset of samples T ⊂ R d , noise rate α ∈ (0 , 1 / 2] , failure probability τ ∈ (0 , 1) , degree of polynomials (cid:96) ≥ 1 .

Original: 1: ˜Σ ← E [ P d,(cid:96)
Revised: 1: ˜Σ ← E [ P d,(cid:96)

Original: ( T − µ T ) · P d,(cid:96) ( T − µ T ) (cid:62) ] , and P d,(cid:96) ( x ) is the column vector of all degree- (cid:96) Hermite polynomials of x .
Revised: ( T − µ T ) · P d,(cid:96) ( T − µ T ) (cid:62) ] , and P d,(cid:96) ( x ) is the column vector of all degree- (cid:96) Hermite polynomials of x .

Original: ≥← index set of the k (cid:96) diagonal entries and 12 ( k 2 (cid:96) − k (cid:96) )
Revised: ≥← index set of the k (cid:96) diagonal entries and 12 ( k 2 (cid:96) − k (cid:96) )

Original: entries above the main diagonal of ˜Σ with largest magnitude.
Revised: entries above the main diagonal of ˜Σ with largest magnitude.

Original: U ← { ( i t , j t ) } t ≥ 1 ∪ { ( j t , i t ) } t ≥ 1 , U (cid:48)
Revised: U ← { ( i t , j t ) } t ≥ 1 ∪ { ( j t , i t ) } t ≥ 1 , U (cid:48)

Original: ← I × I , with I =
Revised: ← I × I , with I =

Original: { i t } t ≥ 1 ∪ { j t } t ≥ 1 .
Revised: { i t } t ≥ 1 ∪ { j t } t ≥ 1 .

Original: [Table]
Revised: [Table]

Original: large enough constant C the largest eigenvalue and eigenvector of then 7: if (cid:96) =
Revised: large enough constant C the largest eigenvalue and eigenvector of then 7: if (cid:96) =

Original: 1 then
Revised: 1 then

Original: ←
Revised: ←

Original: ASIC MF ( T, α, τ, p 1 ) else ANS ← H ARMONIC MF ( T, α, τ, p 1 )
Revised: ASIC MF ( T, α, τ, p 1 ) else ANS ← H ARMONIC MF ( T, α, τ, p 1 )

Original: where p 1 ( x ) :
Revised: where p 1 ( x ) :

Original: = v ∗ · P d,(cid:96) ( x − µ T ) .
Revised: = v ∗ · P d,(cid:96) ( x − µ T ) .

Original: 8: else 9: p
Revised: 8: else 9: p

Original: ←
Revised: ←

Original: [Table]
Revised: [Table]

Original: A
Revised: A

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 11: end if 12: return or H ARMONIC MF (i.e. Algortihm 5) to examine the concentration of a polynomial of the empirical data to that of Gaussian.
Revised: 11: end if 12: return or H ARMONIC MF (i.e. Algortihm 5) to examine the concentration of a polynomial of the empirical data to that of Gaussian.

Original: Both algorithms will either assert that the current sample set does not contain a sufﬁciently large amount of Gaussian samples, or will prune many corrupted samples to increase the fraction of Gaussian ones.
Revised: Both algorithms will either assert that the current sample set does not contain a sufﬁciently large amount of Gaussian samples, or will prune many corrupted samples to increase the fraction of Gaussian ones.

Original: A more detailed description of the two algorithms can be found in
Revised: A more detailed description of the two algorithms can be found in

Original: Section 3.2.1 and Section 3.2.2 respectively.
Revised: Section 3.2.1 and Section 3.2.2 respectively.

Original: What is subtle in Algorithm 3 is that we will check the maximum eigenvalue λ ∗ of the empirical covariance matrix ˜Σ restricted on a carefully chosen subset U (cid:48) , which corresponds to the maximum eigenvalue on a certain (2 k 2 (cid:96) )
Revised: What is subtle in Algorithm 3 is that we will check the maximum eigenvalue λ ∗ of the empirical covariance matrix ˜Σ restricted on a carefully chosen subset U (cid:48) , which corresponds to the maximum eigenvalue on a certain (2 k 2 (cid:96) )

Original: -sparse direction.
Revised: -sparse direction.

Original: If λ ∗ is too large, this indicates an easy problem since it must be the case that the adversary corrupted the samples in an aggressively way.
Revised: If λ ∗ is too large, this indicates an easy problem since it must be the case that the adversary corrupted the samples in an aggressively way.

Original: Therefore, it sufﬁces to prune outliers using a degree- (cid:96) polynomial p 1 which is simply the projection of P d,(cid:96) ( x − µ T ) onto the span of the maximum eigenvector; see
Revised: Therefore, it sufﬁces to prune outliers using a degree- (cid:96) polynomial p 1 which is simply the projection of P d,(cid:96) ( x − µ T ) onto the span of the maximum eigenvector; see

Original: Step 7 in Algorithm 3.
Revised: Step 7 in Algorithm 3.

Original: On the other hand, if λ ∗ is on a moderate scale, it indicates that the adversary corrupted the samples in a very delicate way so that it passes the tests of both Frobenius norm and spectral norm.
Revised: On the other hand, if λ ∗ is on a moderate scale, it indicates that the adversary corrupted the samples in a very delicate way so that it passes the tests of both Frobenius norm and spectral norm.

Original: Now the main idea is to check the concentration of higher degree polynomials induced by the sample set; we show that it sufﬁces to construct a degree- 2 (cid:96) harmonic polynomial; see Step 10.
Revised: Now the main idea is to check the concentration of higher degree polynomials induced by the sample set; we show that it sufﬁces to construct a degree- 2 (cid:96) harmonic polynomial; see Step 10.

Original: While sparse mean estimation has been studied in [DKK + 19] and the idea of using restricted
Revised: While sparse mean estimation has been studied in [DKK + 19] and the idea of using restricted

Original: Frobenius norm and ﬁltering was also developed, we note that their analysis only holds in the mild corruption regime where α > 1 / 2 .
Revised: Frobenius norm and ﬁltering was also developed, we note that their analysis only holds in the mild corruption regime where α > 1 / 2 .

Original: To establish the main results, we will leverage the tools from [DKS18b], with a speciﬁc treatment on the fact that µ is k -sparse, to ensure an attribute-efﬁcient sample complexity bound.
Revised: To establish the main results, we will leverage the tools from [DKS18b], with a speciﬁc treatment on the fact that µ is k -sparse, to ensure an attribute-efﬁcient sample complexity bound.

Original: As we will show later, a key idea to this end is to utilize a sequence of carefully chosen sparse polynomials in the sense of Deﬁnition 7 along with sparsity-induced ﬁlters.
Revised: As we will show later, a key idea to this end is to utilize a sequence of carefully chosen sparse polynomials in the sense of Deﬁnition 7 along with sparsity-induced ﬁlters.

Original: The performance guarantee of A TTRIBUTE - EFFICIENT -M ULTIFILTER is as follows.
Revised: The performance guarantee of A TTRIBUTE - EFFICIENT -M ULTIFILTER is as follows.

Original: Theorem 14 (Algorithm 3) .
Revised: Theorem 14 (Algorithm 3) .

Original: Consider Algorithm 3 and denote by ANS its output.
Revised: Consider Algorithm 3 and denote by ANS its output.

Original: With probability 1 − τ , the following holds.
Revised: With probability 1 − τ , the following holds.

Original: ANS cannot be TBD .
Revised: ANS cannot be TBD .

Original: If ANS is a k -sparse vector and if T is α -good, then (cid:107) µ − ˆ µ (cid:107) 2 ≤ ˜ O
Revised: If ANS is a k -sparse vector and if T is α -good, then (cid:107) µ − ˆ µ (cid:107) 2 ≤ ˜ O

Original: (cid:0) α − 12 (cid:96) √ (cid:96) ( (cid:96) + log 1 α ) (cid:1) .
Revised: (cid:0) α − 12 (cid:96) √ (cid:96) ( (cid:96) + log 1 α ) (cid:1) .

Original: If ANS =
Revised: If ANS =

Original: NO , then T is not α -good.
Revised: NO , then T is not α -good.

Original: If ANS =
Revised: If ANS =

Original: ( T i , α i ) } mi =1 for some m ≤ 2 , then T i ⊂ T for all i ∈ [ m ] and (cid:80) mi =1 1 α 2 i ≤ 1 α 2 ; if additionally T is α -good, then at least one T i is α i -good.
Revised: ( T i , α i ) } mi =1 for some m ≤ 2 , then T i ⊂ T for all i ∈ [ m ] and (cid:80) mi =1 1 α 2 i ≤ 1 α 2 ; if additionally T is α -good, then at least one T i is α i -good.

Original: Finally, the algorithm runs in time O (cid:0) poly( | T | , d (cid:96) ) (cid:1) .
Revised: Finally, the algorithm runs in time O (cid:0) poly( | T | , d (cid:96) ) (cid:1) .

Original: 3.2 Analysis of A TTRIBUTE - EFFICIENT -M ULTIFILTER
Revised: 3.2 Analysis of A TTRIBUTE - EFFICIENT -M ULTIFILTER

Original: We ﬁrst show that if the restricted Frobenius norm of the sample covariance matrix is small, then the sample mean is a good estimate of the target mean.
Revised: We ﬁrst show that if the restricted Frobenius norm of the sample covariance matrix is small, then the sample mean is a good estimate of the target mean.

Original: Lemma 15.
Revised: Lemma 15.

Original: Consider Algorithm 3.
Revised: Consider Algorithm 3.

Original: If the algorithm returns a vector ˆ µ at Step 4 and if T is α -good, we have that (cid:107) ˆ µ − µ (cid:107)
Revised: If the algorithm returns a vector ˆ µ at Step 4 and if T is α -good, we have that (cid:107) ˆ µ − µ (cid:107)

Original: [Equation]
Revised: [Equation]

Original: O (cid:0)
Revised: O (cid:0)

Original: [Equation]
Revised: [Equation]

Original: √ (cid:96)
Revised: √ (cid:96)

Original: [Equation]
Revised: [Equation]

Original: ( (cid:96)
Revised: ( (cid:96)

Original: +
Revised: +

Original: [Equation]
Revised: [Equation]

Original: · log
Revised: · log

Original: [Equation]
Revised: [Equation]

Original: (2 + log
Revised: (2 + log

Original: [Equation]
Revised: [Equation]

Original: ) (cid:1) .
Revised: ) (cid:1) .

Original: Next, we give performance guarantees on the remaining steps of Algorithm 3, where we consider the case that the algorithm does not return at Step 4.
Revised: Next, we give performance guarantees on the remaining steps of Algorithm 3, where we consider the case that the algorithm does not return at Step 4.

Original: Namely, the algorithm will either reach at Stepor Step 10, and will return the ANS obtained thereof.
Revised: Namely, the algorithm will either reach at Stepor Step 10, and will return the ANS obtained thereof.

Original: These two steps will invoke B ASIC MF or
Revised: These two steps will invoke B ASIC MF or

Original: H ARMONIC MF on different sparse polynomials.
Revised: H ARMONIC MF on different sparse polynomials.

Original: Observe that both algorithms may return 1) “ NO ”, which certiﬁes that the current input set T is not α -good; 2) a list of subsets { ( T i , α i ) } mi =1 for some m ≤ 2 , on which Algorithm 3 will be called in a recursive manner; or 3) TBD , which indicates that the algorithm is uncertain on T being α -good.
Revised: Observe that both algorithms may return 1) “ NO ”, which certiﬁes that the current input set T is not α -good; 2) a list of subsets { ( T i , α i ) } mi =1 for some m ≤ 2 , on which Algorithm 3 will be called in a recursive manner; or 3) TBD , which indicates that the algorithm is uncertain on T being α -good.

Original: In the following, we prove that the way that we invoke
Revised: In the following, we prove that the way that we invoke

Original: B ASIC MF and H ARMONIC MF ensures that they will never return TBD when being called within
Revised: B ASIC MF and H ARMONIC MF ensures that they will never return TBD when being called within

Original: Algorithm 3.
Revised: Algorithm 3.

Original: We then give performance guarantees on these two ﬁltering algorithms when they return “ NO ” or { ( T i , α i ) } mi =1 , thus establishing Theorem 14.
Revised: We then give performance guarantees on these two ﬁltering algorithms when they return “ NO ” or { ( T i , α i ) } mi =1 , thus establishing Theorem 14.

Original: Let us consider that the algorithm reaches Step 7, i.e. the largest eigenvalue on one sparse direction is larger than the threshold λ ∗ sparse .
Revised: Let us consider that the algorithm reaches Step 7, i.e. the largest eigenvalue on one sparse direction is larger than the threshold λ ∗ sparse .

Original: It is easy to see that when (cid:96) = 1 , ANS cannot be TBD since the only way that B ASIC MF returns TBD is when Var[ p ( T )] is not too large, but this would violate the condition that λ ∗ > λ ∗ sparse in view of our setting on λ ∗ sparse .
Revised: It is easy to see that when (cid:96) = 1 , ANS cannot be TBD since the only way that B ASIC MF returns TBD is when Var[ p ( T )] is not too large, but this would violate the condition that λ ∗ > λ ∗ sparse in view of our setting on λ ∗ sparse .

Original: Similarly, we show that under the large λ ∗ regime, H ARMONIC MF will not return TBD either.
Revised: Similarly, we show that under the large λ ∗ regime, H ARMONIC MF will not return TBD either.

Original: Thus, we have the following lemma.
Revised: Thus, we have the following lemma.

Original: Lemma 16.
Revised: Lemma 16.

Original: Consider Algorithm 3.
Revised: Consider Algorithm 3.

Original: If it reaches Step 7, then ANS (cid:54) = TBD .
Revised: If it reaches Step 7, then ANS (cid:54) = TBD .

Original: Now it remains to consider the case that the algorithm reaches Step 10, which is more subtle since the evidence from the magnitude of the largest restricted eigenvalue is not so strong to prune outliers.
Revised: Now it remains to consider the case that the algorithm reaches Step 10, which is more subtle since the evidence from the magnitude of the largest restricted eigenvalue is not so strong to prune outliers.

Original: Note that this could happen even when T contains many outliers, since λ ∗ is not the maximum eigenvalue on all sparse directions but on a submatrix indexed by U (cid:48) .
Revised: Note that this could happen even when T contains many outliers, since λ ∗ is not the maximum eigenvalue on all sparse directions but on a submatrix indexed by U (cid:48) .

Original: Fortunately, if λ ∗ is not large, we show that the algorithm can still make progress by calling H ARMONIC MF on degree- 2 (cid:96) sparse polynomials.
Revised: Fortunately, if λ ∗ is not large, we show that the algorithm can still make progress by calling H ARMONIC MF on degree- 2 (cid:96) sparse polynomials.

Original: This is because higher-degree polynomials are more sensitive to outliers than low-degree polynomials, as far as we can certify the concentration of high-degree polynomials on clean samples.
Revised: This is because higher-degree polynomials are more sensitive to outliers than low-degree polynomials, as far as we can certify the concentration of high-degree polynomials on clean samples.

Original: As a result, we will have the following guarantee.
Revised: As a result, we will have the following guarantee.

Original: Lemma 17.
Revised: Lemma 17.

Original: Consider Algorithm 3.
Revised: Consider Algorithm 3.

Original: If it reaches Step 10, then ANS (cid:54) = TBD .
Revised: If it reaches Step 10, then ANS (cid:54) = TBD .

Original: 3.2.1 Basic Multiﬁlter for Sparse Polynomials
Revised: 3.2.1 Basic Multiﬁlter for Sparse Polynomials

Original: The B ASIC MF algorithm (Algorithm 4) is a key ingredient in the multiﬁltering framework.
Revised: The B ASIC MF algorithm (Algorithm 4) is a key ingredient in the multiﬁltering framework.

Original: It takes as input a sparse polynomial p and uses it to certify whether T is α -good and sufﬁciently concentrated.
Revised: It takes as input a sparse polynomial p and uses it to certify whether T is α -good and sufﬁciently concentrated.

Original: The central idea is to measure how p ( T ) distributed and compare it to that of the distribution of p ( G ) .
Revised: The central idea is to measure how p ( T ) distributed and compare it to that of the distribution of p ( G ) .

Original: We require the input p has certiﬁable variance on G , i.e. Var[ p ( G )] ≤ 1 , as otherwise, it could ﬁlter away a large number of the good samples.
Revised: We require the input p has certiﬁable variance on G , i.e. Var[ p ( G )] ≤ 1 , as otherwise, it could ﬁlter away a large number of the good samples.

Original: We note that the bounded variance condition is always satisﬁed for degree- 1 Hermite polynomials under proper normalization, while for high-degree polynomials, one cannot invoke B ASIC MF directly (see Section 3.2.2 for a remedy).
Revised: We note that the bounded variance condition is always satisﬁed for degree- 1 Hermite polynomials under proper normalization, while for high-degree polynomials, one cannot invoke B ASIC MF directly (see Section 3.2.2 for a remedy).

Original: The way that B ASIC MF certiﬁes the input sample set T not being α -good is quite simple: if not all samples lie in a small L ∞ -ball, it returns “ NO ” at Step 2, in that this contradicts Lemma 13.
Revised: The way that B ASIC MF certiﬁes the input sample set T not being α -good is quite simple: if not all samples lie in a small L ∞ -ball, it returns “ NO ” at Step 2, in that this contradicts Lemma 13.

Original: Otherwise, the algorithm will attempt to search for a ﬁner interval [ a, b ] such that it includes most of the samples.
Revised: Otherwise, the algorithm will attempt to search for a ﬁner interval [ a, b ] such that it includes most of the samples.

Original: If such interval exists, then either the adversary corrupted the samples such that the sample variance is as small as that of Gaussian while the sample mean may deviate far from the target, in which case B ASIC MF returns TBD at Step 5; or the sample variance is large, in which case
Revised: If such interval exists, then either the adversary corrupted the samples such that the sample variance is as small as that of Gaussian while the sample mean may deviate far from the target, in which case B ASIC MF returns TBD at Step 5; or the sample variance is large, in which case

Original: Require: A multiset of samples T ⊂ R d , noise rate α ∈ (0 , 1 / 2] , failure probability τ ∈ (0 , 1) , a polynomial p ∈ P ( R d , l, 4 (cid:96) 2 k 4 (cid:96) , 2 (cid:96)k 2 (cid:96) ) such that l ≤ 2 (cid:96) , Var[ p ( G )] ≤ 1 , and p ( x ) =
Revised: Require: A multiset of samples T ⊂ R d , noise rate α ∈ (0 , 1 / 2] , failure probability τ ∈ (0 , 1) , a polynomial p ∈ P ( R d , l, 4 (cid:96) 2 k 4 (cid:96) , 2 (cid:96)k 2 (cid:96) ) such that l ≤ 2 (cid:96) , Var[ p ( G )] ≤ 1 , and p ( x ) =

Original: h A ( x − µ T ) .
Revised: h A ( x − µ T ) .

Original: 1: R ← ( C 1 · log 1 α ) l/ 2 , γ ← C 0 · (cid:113) (cid:96) · log (cid:96)dατ .
Revised: 1: R ← ( C 1 · log 1 α ) l/ 2 , γ ← C 0 · (cid:113) (cid:96) · log (cid:96)dατ .

Original: 2: if max x,y ∈ T | p ( x ) − p ( y ) | > 2 k (cid:96) · γ l then return “ NO ”.
Revised: 2: if max x,y ∈ T | p ( x ) − p ( y ) | > 2 k (cid:96) · γ l then return “ NO ”.

Original: 3: if there is an interval [ a, b ] of length C 1 · R · log(2 + log 1 α ) that contains at least (1 − α 2 ) -fraction of samples in { p ( x ) : x ∈ T } then 4: if Var[ p ( T )] ≤ C 1 · (cid:0) (cid:96) + C 1 log 1 α (cid:1) l · log 2 (2 + log 1 α ) then 5: return “ TBD ”.
Revised: 3: if there is an interval [ a, b ] of length C 1 · R · log(2 + log 1 α ) that contains at least (1 − α 2 ) -fraction of samples in { p ( x ) : x ∈ T } then 4: if Var[ p ( T )] ≤ C 1 · (cid:0) (cid:96) + C 1 log 1 α (cid:1) l · log 2 (2 + log 1 α ) then 5: return “ TBD ”.

Original: 6: else 7:
Revised: 6: else 7:

Original: Find a threshold t > 2 R such that
Revised: Find a threshold t > 2 R such that

Original: Pr x ∼ T (cid:2) min {| p ( x ) − a | , | p ( x ) − b |} ≥ t (cid:3) > 32 α exp( − ( t − 2 R ) 2 /l )
Revised: Pr x ∼ T (cid:2) min {| p ( x ) − a | , | p ( x ) − b |} ≥ t (cid:3) > 32 α exp( − ( t − 2 R ) 2 /l )

Original: + 2 αk 2 (cid:96) log l ( (cid:96)dατ ) .
Revised: + 2 αk 2 (cid:96) log l ( (cid:96)dατ ) .

Original: T (cid:48) ← { x ∈ T : min {| p ( x ) − a | , | p ( x ) − b |} ≤ t } , α (cid:48) ← α · (cid:16) (1 − α/ 8) | T | | T (cid:48) | + α 8 (cid:17) .
Revised: T (cid:48) ← { x ∈ T : min {| p ( x ) − a | , | p ( x ) − b |} ≤ t } , α (cid:48) ← α · (cid:16) (1 − α/ 8) | T | | T (cid:48) | + α 8 (cid:17) .

Original: 9: return { ( T (cid:48) , α (cid:48) ) } .
Revised: 9: return { ( T (cid:48) , α (cid:48) ) } .

Original: 10: end if 11: else 12:
Revised: 10: end if 11: else 12:

Original: Find t ∈ R , R (cid:48) > 0 such that the sets T 1 := { x ∈ T : p ( x ) > t − R (cid:48) } and T 2 :=
Revised: Find t ∈ R , R (cid:48) > 0 such that the sets T 1 := { x ∈ T : p ( x ) > t − R (cid:48) } and T 2 :=

Original: { x ∈ T : p ( x ) <
Revised: { x ∈ T : p ( x ) <

Original: t + R (cid:48) } satisfy | T 1 | 2 + | T 2 | 2 ≤ | T | 2 (1 − α/ 100) 2 and | T | − max( | T 1 | , | T 2 | ) ≥
Revised: t + R (cid:48) } satisfy | T 1 | 2 + | T 2 | 2 ≤ | T | 2 (1 − α/ 100) 2 and | T | − max( | T 1 | , | T 2 | ) ≥

Original: α | T | / 4 .
Revised: α | T | / 4 .

Original: 13: α i ← α · (1 − α 2 / 100) · | T | / | T i | , for i = 1 , 2 .
Revised: 13: α i ← α · (1 − α 2 / 100) · | T | / | T i | , for i = 1 , 2 .

Original: 14: return { ( T 1 , α 1 ) , ( T 2 , α 2 ) } .
Revised: 14: return { ( T 1 , α 1 ) , ( T 2 , α 2 ) } .

Original: 15: end if it is possible to construct a sparsity-induced ﬁlter to prune outliers (see Steps 7 and 8).
Revised: 15: end if it is possible to construct a sparsity-induced ﬁlter to prune outliers (see Steps 7 and 8).

Original: We note that in Step 7, the ﬁrst term on the right-hand side is derived from Chernoff bound for degree- l Gaussian polynomials and the second term is due to concentration of empirical samples to Gaussian (see
Revised: We note that in Step 7, the ﬁrst term on the right-hand side is derived from Chernoff bound for degree- l Gaussian polynomials and the second term is due to concentration of empirical samples to Gaussian (see

Original: Deﬁnition 8), both of which are scaled by a factor 8 /α so that the number of the samples removed from T is 8 /α times more than that of the good samples in the representative set S G ⊂ T , which means most of the removed samples are outliers.
Revised: Deﬁnition 8), both of which are scaled by a factor 8 /α so that the number of the samples removed from T is 8 /α times more than that of the good samples in the representative set S G ⊂ T , which means most of the removed samples are outliers.

Original: We show by contradiction the existence of the threshold t (see Lemma 26).
Revised: We show by contradiction the existence of the threshold t (see Lemma 26).

Original: In fact, had such threshold t not existed, the set T must be sufﬁciently concentrated such that the algorithm would have returned at Step 5.
Revised: In fact, had such threshold t not existed, the set T must be sufﬁciently concentrated such that the algorithm would have returned at Step 5.

Original: This essentially relies on our result of the initial clustering of Algorithm 1, which guarantees that each subset T is bounded in a small L ∞ -ball and the function value of p on the α -good T does not change drastically (Lemma 13).
Revised: This essentially relies on our result of the initial clustering of Algorithm 1, which guarantees that each subset T is bounded in a small L ∞ -ball and the function value of p on the α -good T does not change drastically (Lemma 13).

Original: We then show that equipped with such threshold t , T (cid:48) is a subset of T and it is α (cid:48)
Revised: We then show that equipped with such threshold t , T (cid:48) is a subset of T and it is α (cid:48)

Original: -good if T is α -good (Lemma 28).
Revised: -good if T is α -good (Lemma 28).

Original: When there is no such short interval [ a, b ] , the algorithm splits T into two overlapping subsets { T 1 , T 2 } such that T 1 ∩ T 2 is large enough to contain most of the samples in S G .
Revised: When there is no such short interval [ a, b ] , the algorithm splits T into two overlapping subsets { T 1 , T 2 } such that T 1 ∩ T 2 is large enough to contain most of the samples in S G .

Original: This guarantees that most of the samples in S G (if T is α -good) are always contained in one subset and thus there always exists an α -good subset of T .
Revised: This guarantees that most of the samples in S G (if T is α -good) are always contained in one subset and thus there always exists an α -good subset of T .

Original: We show that an appropriate threshold t can also be found at
Revised: We show that an appropriate threshold t can also be found at

Original: Step 12 (Lemma 30), and at least one T i is α i -good
Revised: Step 12 (Lemma 30), and at least one T i is α i -good

Original: if T is α -good.
Revised: if T is α -good.

Original: As a result, we have the following guarantees for Algorithm 4; see Appendix B for the full proof.
Revised: As a result, we have the following guarantees for Algorithm 4; see Appendix B for the full proof.

Original: Theorem 18 (B ASIC MF) .
Revised: Theorem 18 (B ASIC MF) .

Original: Consider Algorithm 4.
Revised: Consider Algorithm 4.

Original: Denote by ANS its return.
Revised: Denote by ANS its return.

Original: Suppose that T being α -good implies Var[ p ( G )] ≤ 1 .
Revised: Suppose that T being α -good implies Var[ p ( G )] ≤ 1 .

Original: Then with probability 1 − τ , the following holds.
Revised: Then with probability 1 − τ , the following holds.

Original: ANS is either “ NO ”, “ TBD ”, or a list of { ( T i , α i ) } mi =1 with m ≤ 2 . 1)
Revised: ANS is either “ NO ”, “ TBD ”, or a list of { ( T i , α i ) } mi =1 with m ≤ 2 . 1)

Original: If ANS =
Revised: If ANS =

Original: NO , then T is not α -good.
Revised: NO , then T is not α -good.

Original: If
Revised: If

Original: ANS =
Revised: ANS =

Original: TBD , then Var[ p ( T )] ≤ O (cid:0) ( (cid:96)
Revised: TBD , then Var[ p ( T )] ≤ O (cid:0) ( (cid:96)

Original: + log 1 α ) l · log 2 (2+log 1 α ) (cid:1) ; and if additionally T is α -good, then | E [ p ( G )] − E
Revised: + log 1 α ) l · log 2 (2+log 1 α ) (cid:1) ; and if additionally T is α -good, then | E [ p ( G )] − E

Original: [ p ( T )]
Revised: [ p ( T )]

Original: | ≤ O (cid:0) ( (cid:96) + log 1 α ) l 2 · log(2 + log 1 α ) (cid:1) .
Revised: | ≤ O (cid:0) ( (cid:96) + log 1 α ) l 2 · log(2 + log 1 α ) (cid:1) .

Original: If ANS =
Revised: If ANS =

Original: ( T i , α i ) } mi =1 , then
Revised: ( T i , α i ) } mi =1 , then

Original: T i ⊂ T and (cid:80)
Revised: T i ⊂ T and (cid:80)

Original: i 1 α 2 i ≤ 1 α 2 for all i ∈ [ m ] ; if additionally T is α -good, then at least one T i is α i -good.
Revised: i 1 α 2 i ≤ 1 α 2 for all i ∈ [ m ] ; if additionally T is α -good, then at least one T i is α i -good.

Original: Algorithm
Revised: Algorithm

Original: Require: A multiset of samples T ⊂ R d , noise rate α ∈ (0 , 1 / 2] , failure probability τ ∈ (0 , 1) , a polynomial p ∈ P ( R d , l, 2 (cid:96)k 2 (cid:96) , 2 (cid:96)k 2 (cid:96) ) such that p ( x ) =
Revised: Require: A multiset of samples T ⊂ R d , noise rate α ∈ (0 , 1 / 2] , failure probability τ ∈ (0 , 1) , a polynomial p ∈ P ( R d , l, 2 (cid:96)k 2 (cid:96) , 2 (cid:96)k 2 (cid:96) ) such that p ( x ) =

Original: h A ( x − µ T ) and (cid:107) A (cid:107) 2 = 1 .
Revised: h A ( x − µ T ) and (cid:107) A (cid:107) 2 = 1 .

Original: 1: for l (cid:48)
Revised: 1: for l (cid:48)

Original: = 0 , 1 , . .
Revised: = 0 , 1 , . .

Original: , l do 2:
Revised: , l do 2:

Original: Let B ( l (cid:48) ) be an order- 2 l (cid:48) tensor with
Revised: Let B ( l (cid:48) ) be an order- 2 l (cid:48) tensor with

Original: B ( l (cid:48) )
Revised: B ( l (cid:48) )

Original: i 1 ,...,i l (cid:48) ,j 1 ,...,j l (cid:48) =
Revised: i 1 ,...,i l (cid:48) ,j 1 ,...,j l (cid:48) =

Original: (cid:88) k l (cid:48)
Revised: (cid:88) k l (cid:48)

Original: +1 ,...,k l
Revised: +1 ,...,k l

Original: A i 1 ...,i l (cid:48) ,k l (cid:48) +1 ,...,k l A j 1 ...,j l (cid:48) ,k l (cid:48) +1 ,...,k l .
Revised: A i 1 ...,i l (cid:48) ,k l (cid:48) +1 ,...,k l A j 1 ...,j l (cid:48) ,k l (cid:48) +1 ,...,k l .

Original: Consider B ( l (cid:48) ) as a d l (cid:48) ⊗ d l (cid:48) symmetric matrix by grouping each of the i 1 , . .
Revised: Consider B ( l (cid:48) ) as a d l (cid:48) ⊗ d l (cid:48) symmetric matrix by grouping each of the i 1 , . .

Original: , i l (cid:48) and j 1 , . . .
Revised: , i l (cid:48) and j 1 , . . .

Original: , j l (cid:48) coordinates together.
Revised: , j l (cid:48) coordinates together.

Original: Apply eigenvalue decomposition on B ( l (cid:48) ) to obtain B ( l (cid:48) )
Revised: Apply eigenvalue decomposition on B ( l (cid:48) ) to obtain B ( l (cid:48) )

Original: = (cid:80) i λ
Revised: = (cid:80) i λ

Original: i V i ⊗ V i .
Revised: i V i ⊗ V i .

Original: ANS i ← M ULTILINEAR MF ( T, V i , l (cid:48) , α, τ/ ( ld l )) for every V i .
Revised: ANS i ← M ULTILINEAR MF ( T, V i , l (cid:48) , α, τ/ ( ld l )) for every V i .

Original: If ANS i
Revised: If ANS i

Original: =
Revised: =

Original: NO or a list of { ( T j , α j ) } for some i , then return ANS i .
Revised: NO or a list of { ( T j , α j ) } for some i , then return ANS i .

Original: If ANS i =
Revised: If ANS i =

Original: TBD , continue .
Revised: TBD , continue .

Original: 5: end for 6: ANS ← B ASIC MF ( T, α, τ, 1 β h
Revised: 5: end for 6: ANS ← B ASIC MF ( T, α, τ, 1 β h

Original: A ( x − µ T )) with β := (cid:0) C 1 · (1 + log 1 α ) · log 2 (2 + log 1 α ) (cid:1) l 2 .
Revised: A ( x − µ T )) with β := (cid:0) C 1 · (1 + log 1 α ) · log 2 (2 + log 1 α ) (cid:1) l 2 .

Original: If ANS =
Revised: If ANS =

Original: NO or a list of ( T j , α j ) , return ANS .
Revised: NO or a list of ( T j , α j ) , return ANS .

Original: If ANS =
Revised: If ANS =

Original: TBD , still return “ NO ”.
Revised: TBD , still return “ NO ”.

Original: 3.2.2 Harmonic Multiﬁlter with Hermite Polynomials
Revised: 3.2.2 Harmonic Multiﬁlter with Hermite Polynomials

Original: Recall that applying B ASIC MF (Algorithm 4) on a polynomial p requires Var[ p ( G )] ≤ 1 .
Revised: Recall that applying B ASIC MF (Algorithm 4) on a polynomial p requires Var[ p ( G )] ≤ 1 .

Original: It is nontrivial to verify this condition for a high-degree polynomial p , as the variance of high-degree
Revised: It is nontrivial to verify this condition for a high-degree polynomial p , as the variance of high-degree

Original: Gaussian polynomials depends on the distribution mean, i.e. µ − µ T in this case, which is unfortunately unknown.
Revised: Gaussian polynomials depends on the distribution mean, i.e. µ − µ T in this case, which is unfortunately unknown.

Original: As a remedy, notice that for any harmonic polynomial h A ( x ) , E x ∼ N ( µ (cid:48) , I ) [ h A ( x ) 2 ] equals the summation of homogeneous polynomials of µ (cid:48) , which can also be seen as the expectation of multilinear polynomials over independent variables
Revised: As a remedy, notice that for any harmonic polynomial h A ( x ) , E x ∼ N ( µ (cid:48) , I ) [ h A ( x ) 2 ] equals the summation of homogeneous polynomials of µ (cid:48) , which can also be seen as the expectation of multilinear polynomials over independent variables

Original: X ( i ) ∼ N ( µ (cid:48) , I d ) .
Revised: X ( i ) ∼ N ( µ (cid:48) , I d ) .

Original: Thus, we only need to verify the expectation of these corresponding multilinear polynomials, whose variance on G does not hinge on µ (cid:48) .
Revised: Thus, we only need to verify the expectation of these corresponding multilinear polynomials, whose variance on G does not hinge on µ (cid:48) .

Original: The harmonic multiﬁlter is presented in Algorithm 5, where the subroutine M ULTILINEAR MF can be found in Appendix D.
Revised: The harmonic multiﬁlter is presented in Algorithm 5, where the subroutine M ULTILINEAR MF can be found in Appendix D.

Original: We ﬁrst present the guarantee when Algorithm 5 returns all TBD at
Revised: We ﬁrst present the guarantee when Algorithm 5 returns all TBD at

Original: Step 4 and reaches Step 6, where we can certify a bounded variance for h A ( x − µ T ) on G .
Revised: Step 4 and reaches Step 6, where we can certify a bounded variance for h A ( x − µ T ) on G .

Original: Lemma 19 (Variance of p ) .
Revised: Lemma 19 (Variance of p ) .

Original: Consider Algorithm 5.
Revised: Consider Algorithm 5.

Original: If it reaches Step 6 and T is α -good, then we have E [ h A ( G − µ T ) 2 ] ≤ β 2 .
Revised: If it reaches Step 6 and T is α -good, then we have E [ h A ( G − µ T ) 2 ] ≤ β 2 .

Original: Based on Lemma 19, we have that Var[ h A ( G − µ T ) /β ] ≤ 1 , for which we can invoke B ASIC MF on h A ( x − µ T ) /β
Revised: Based on Lemma 19, we have that Var[ h A ( G − µ T ) /β ] ≤ 1 , for which we can invoke B ASIC MF on h A ( x − µ T ) /β

Original: and Theorem 18 can be applied immediately.
Revised: and Theorem 18 can be applied immediately.

Original: We are ready to elaborate the proof ideas for Lemma 16 and 17.
Revised: We are ready to elaborate the proof ideas for Lemma 16 and 17.

Original: First, observe that B ASIC MF returns “ TBD ” at Step 6 if and only if
Revised: First, observe that B ASIC MF returns “ TBD ” at Step 6 if and only if

Original: Var[ h A ( T − µ T ) /β ] ≤ C 1 · (cid:0) (cid:96) + C 1 log 1 α (cid:1) l · log 2 (2 + log 1 α ) .
Revised: Var[ h A ( T − µ T ) /β ] ≤ C 1 · (cid:0) (cid:96) + C 1 log 1 α (cid:1) l · log 2 (2 + log 1 α ) .

Original: Now return to Algorithm 3.
Revised: Now return to Algorithm 3.

Original: When h A ( x − µ T ) =
Revised: When h A ( x − µ T ) =

Original: p 1 ( x ) , this could not happen because Var[ p 1 ( T )] =
Revised: p 1 ( x ) , this could not happen because Var[ p 1 ( T )] =

Original: Var[ v ∗ · P d,(cid:96) ( T − µ T )] ≥ λ ∗ ≥ λ ∗ sparse = (cid:2) C 1 · ( (cid:96) + C 1 log 1 α ) · log 2 (2 + log 1 α ) (cid:3) 2 (cid:96) ≥ β 2 · C 1 · (cid:0) (cid:96) + C 1 log 1 α (cid:1) (cid:96) · log 2 (2 + log 1 α ) .
Revised: Var[ v ∗ · P d,(cid:96) ( T − µ T )] ≥ λ ∗ ≥ λ ∗ sparse = (cid:2) C 1 · ( (cid:96) + C 1 log 1 α ) · log 2 (2 + log 1 α ) (cid:3) 2 (cid:96) ≥ β 2 · C 1 · (cid:0) (cid:96) + C 1 log 1 α (cid:1) (cid:96) · log 2 (2 + log 1 α ) .

Original: A contradiction that implies Lemma 16.
Revised: A contradiction that implies Lemma 16.

Original: When h A ( x − µ T ) = p 2 ( x ) , the case is more delicate.
Revised: When h A ( x − µ T ) = p 2 ( x ) , the case is more delicate.

Original: Here, we instead show that T must not be α -good and H ARMONIC MF will return “ NO ” correctly.
Revised: Here, we instead show that T must not be α -good and H ARMONIC MF will return “ NO ” correctly.

Original: This is because if T is α -good, Proposition 18 implies that E [ p 2 ( G )] is close to E [ p 2 ( T )] , and together with
Revised: This is because if T is α -good, Proposition 18 implies that E [ p 2 ( G )] is close to E [ p 2 ( T )] , and together with

Original: Lemma 19 we can show that E [ p 2 ( T )] is small.
Revised: Lemma 19 we can show that E [ p 2 ( T )] is small.

Original: However, by construction (cid:13)(cid:13) (˜Σ) U (cid:13)(cid:13) =
Revised: However, by construction (cid:13)(cid:13) (˜Σ) U (cid:13)(cid:13) =

Original: E [ p 2 ( T )] ≥ λ ∗ sparse , a contradiction that gives Lemma 17.
Revised: E [ p 2 ( T )] ≥ λ ∗ sparse , a contradiction that gives Lemma 17.

Original: The detailed proof can be found in Appendix B.3.
Revised: The detailed proof can be found in Appendix B.3.

Original:  Conclusion and Future Work
Revised:  Conclusion and Future Work

Original: In this paper, we developed an attribute-efﬁcient mean estimation algorithm which achieves sample complexity poly-logarithmic in the dimension with low-degree sparse polynomials under the listdecodable setting.
Revised: In this paper, we developed an attribute-efﬁcient mean estimation algorithm which achieves sample complexity poly-logarithmic in the dimension with low-degree sparse polynomials under the listdecodable setting.

Original: A natural question is whether the current techniques could be utilized to attributeefﬁciently solve the other list-decodable problems, such as learning of halfspaces and linear regression.
Revised: A natural question is whether the current techniques could be utilized to attributeefﬁciently solve the other list-decodable problems, such as learning of halfspaces and linear regression.

Original: Acknowledgments and Disclosure of Funding
Revised: Acknowledgments and Disclosure of Funding

Original: We thank the anonymous reviewers and meta-reviewer for valuable discussions.
Revised: We thank the anonymous reviewers and meta-reviewer for valuable discussions.

Original: This work is supported by NSF-IIS-1948133 and the startup funding from Stevens Institute of Technology.
Revised: This work is supported by NSF-IIS-1948133 and the startup funding from Stevens Institute of Technology.

Original: A Omitted Proofs from SectionA.1 Proof of Proposition
Revised: A Omitted Proofs from SectionA.1 Proof of Proposition

Original: Proof.
Revised: Proof.

Original: Fix a subset Ω ⊂ [ d ] with size ψ , and then ﬁx a set of κ monomials on Ω with degree at most l , denoted by M (Ω , l ) .
Revised: Fix a subset Ω ⊂ [ d ] with size ψ , and then ﬁx a set of κ monomials on Ω with degree at most l , denoted by M (Ω , l ) .

Original: Let P ( R d , M (Ω , l ) , Ω) be the induced class of polynomials.
Revised: Let P ( R d , M (Ω , l ) , Ω) be the induced class of polynomials.

Original: Note that
Revised: Note that

Original: P ( R d , l, κ, ψ ) = ∪ Ω ∪ M (Ω ,l ) P ( R d , M (Ω , l ) , Ω) .
Revised: P ( R d , l, κ, ψ ) = ∪ Ω ∪ M (Ω ,l ) P ( R d , M (Ω , l ) , Ω) .

Original: It is easy to see that for any p ∈ P ( R d , M (Ω , l ) , Ω) , it can be represented by a linear combinations of the κ monomials.
Revised: It is easy to see that for any p ∈ P ( R d , M (Ω , l ) , Ω) , it can be represented by a linear combinations of the κ monomials.

Original: Thus, the VC dimension of this class equals κ + 1 .
Revised: Thus, the VC dimension of this class equals κ + 1 .

Original: Then, we note that there are (cid:80) ψj =0 (cid:0) dj (cid:1) choices of Ω , and for any given Ω , there are (cid:80) κj =0 (cid:0) 2 d l j (cid:1) choices of M (Ω , l ) .
Revised: Then, we note that there are (cid:80) ψj =0 (cid:0) dj (cid:1) choices of Ω , and for any given Ω , there are (cid:80) κj =0 (cid:0) 2 d l j (cid:1) choices of M (Ω , l ) .

Original: Therefore, the total number of the subclass P ( R d , M (Ω , l ) , Ω) is at most
Revised: Therefore, the total number of the subclass P ( R d , M (Ω , l ) , Ω) is at most

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: The concept class union argument states that for H = ∪ mi =1 H i , the VC dimension of H is upper bounded by O (max { V, log m + V log log mV } ) or loosely O ( V log m ) , where V is an upper bound on the VC dimension of all H i .
Revised: The concept class union argument states that for H = ∪ mi =1 H i , the VC dimension of H is upper bounded by O (max { V, log m + V log log mV } ) or loosely O ( V log m ) , where V is an upper bound on the VC dimension of all H i .

Original: In our case, we have V = κ + 1 and m ≤ (cid:16) edψ (cid:17) ψ · (cid:16) 2 ed l κ (cid:17) κ .
Revised: In our case, we have V = κ + 1 and m ≤ (cid:16) edψ (cid:17) ψ · (cid:16) 2 ed l κ (cid:17) κ .

Original: By calculation, we can show that the VC dimension of P ( R d , l, κ, ψ ) is upper bounded by
Revised: By calculation, we can show that the VC dimension of P ( R d , l, κ, ψ ) is upper bounded by

Original: [Equation]
Revised: [Equation]

Original: Recall that the VC theory states that for any (cid:15), τ ∈ (0 , 1) , as long as | S G | ≥ C (cid:16)
Revised: Recall that the VC theory states that for any (cid:15), τ ∈ (0 , 1) , as long as | S G | ≥ C (cid:16)

Original: [Equation]
Revised: [Equation]

Original: log
Revised: log

Original: [Equation]
Revised: [Equation]

Original: +
Revised: +

Original: [Equation]
Revised: [Equation]

Original: 1 τ
Revised: 1 τ

Original: [Equation]
Revised: [Equation]

Original: for some absolute constant C > 0 , the following holds with probability 1 − τ :
Revised: for some absolute constant C > 0 , the following holds with probability 1 − τ :

Original: [Equation]
Revised: [Equation]

Original: With the expression of d (cid:48) in (A.2), it is not hard to see that we can set | S G | = C ·
Revised: With the expression of d (cid:48) in (A.2), it is not hard to see that we can set | S G | = C ·

Original: [Equation]
Revised: [Equation]

Original: for some absolute constant C > 0 to ensure that the above holds.
Revised: for some absolute constant C > 0 to ensure that the above holds.

Original: When l = 2 (cid:96) , κ = 4 (cid:96) 2 k 4 (cid:96) , ψ = 2 (cid:96)k 2 (cid:96) , and (cid:15) =
Revised: When l = 2 (cid:96) , κ = 4 (cid:96) 2 k 4 (cid:96) , ψ = 2 (cid:96)k 2 (cid:96) , and (cid:15) =

Original: [Equation]
Revised: [Equation]

Original: k
Revised: k

Original: [Equation]
Revised: [Equation]

Original: log
Revised: log

Original: [Equation]
Revised: [Equation]

Original: ( (cid:96)dατ ) for some natural number (cid:96) ≥ 1 , by algebraic calculation, it sufﬁces to pick | S G | =
Revised: ( (cid:96)dατ ) for some natural number (cid:96) ≥ 1 , by algebraic calculation, it sufﬁces to pick | S G | =

Original: C (cid:48) ·
Revised: C (cid:48) ·

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: ( (cid:96)dατ ) for some sufﬁciently large constant C (cid:48) .
Revised: ( (cid:96)dατ ) for some sufﬁciently large constant C (cid:48) .

Original: This completes the proof.
Revised: This completes the proof.

Original: A.2 Proof of Lemma
Revised: A.2 Proof of Lemma

Original: Proof.
Revised: Proof.

Original: By the standard tail bound of Gaussian distribution, for any x drawn from N ( µ, I d ) , it holds that for any given index i ∈
Revised: By the standard tail bound of Gaussian distribution, for any x drawn from N ( µ, I d ) , it holds that for any given index i ∈

Original: [ d ] , Pr[ | x i − µ i | ≥ t ] ≤ 2 exp( − t 2 / 2) .
Revised: [ d ] , Pr[ | x i − µ i | ≥ t ] ≤ 2 exp( − t 2 / 2) .

Original: By taking union bound over both index i and sample x ∈ S G , we have Pr[max x ∈ S G max i ∈
Revised: By taking union bound over both index i and sample x ∈ S G , we have Pr[max x ∈ S G max i ∈

Original: [ d ] | x i − µ i | ≥ t ] ≤ 2 d | S G | exp( − t 2 / 2) .
Revised: [ d ] | x i − µ i | ≥ t ] ≤ 2 d | S G | exp( − t 2 / 2) .

Original: Choosing t = (cid:112) 2 log( d | S G | /τ ) completes the proof.
Revised: Choosing t = (cid:112) 2 log( d | S G | /τ ) completes the proof.

Original: A.3 Proof of Lemma
Revised: A.3 Proof of Lemma

Original: Proof.
Revised: Proof.

Original: Let S G be the subset of T containing the samples drawn i.i.d.
Revised: Let S G be the subset of T containing the samples drawn i.i.d.

Original: from N ( µ, I d ) .
Revised: from N ( µ, I d ) .

Original: Since | S G | = 2 α · | T | , we know that S G is a representative set with probability at least 1 − τ in light of Prop. 9.
Revised: Since | S G | = 2 α · | T | , we know that S G is a representative set with probability at least 1 − τ in light of Prop. 9.

Original: Consider Algorithm 1.
Revised: Consider Algorithm 1.

Original: If for all x, y ∈ T , we have
Revised: If for all x, y ∈ T , we have

Original: [Equation]
Revised: [Equation]

Original: then the algorithm returns only one cluster and the lemma follows immediately.
Revised: then the algorithm returns only one cluster and the lemma follows immediately.

Original: If that is not the case, we ﬁrst note that, with probability at least 1 − τ all of the samples in S G satisfy
Revised: If that is not the case, we ﬁrst note that, with probability at least 1 − τ all of the samples in S G satisfy

Original: Eq.
Revised: Eq.

Original: (A.4) due to Lemma 11.
Revised: (A.4) due to Lemma 11.

Original: Let us condition on this event occurs from now on.
Revised: Let us condition on this event occurs from now on.

Original: Algorithm 1 constructs a set of disjoint L ∞ -balls of radius 2 γ , of which each is centered at one sample in T and contains at least an α -fraction of samples in T .
Revised: Algorithm 1 constructs a set of disjoint L ∞ -balls of radius 2 γ , of which each is centered at one sample in T and contains at least an α -fraction of samples in T .

Original: Therefore, the number of such balls is at most m = (cid:98) 1 /α (cid:99) .
Revised: Therefore, the number of such balls is at most m = (cid:98) 1 /α (cid:99) .

Original: Denote the set by { B 1 , . . .
Revised: Denote the set by { B 1 , . . .

Original: , B m } .
Revised: , B m } .

Original: Let B (cid:48)
Revised: Let B (cid:48)

Original: i be the ball that has the same center as B i but with (cid:96) ∞ -radius of 6 γ .
Revised: i be the ball that has the same center as B i but with (cid:96) ∞ -radius of 6 γ .

Original: In the following, we show that there exists i ∈ [ m ] , such that
Revised: In the following, we show that there exists i ∈ [ m ] , such that

Original: T i = T ∩ B (cid:48)
Revised: T i = T ∩ B (cid:48)

Original: i is α -good.
Revised: i is α -good.

Original: Consider a sample x ∈ S G , for which we know that (cid:107) x − µ (cid:107) ∞ ≤ γ .
Revised: Consider a sample x ∈ S G , for which we know that (cid:107) x − µ (cid:107) ∞ ≤ γ .

Original: Then, for the L ∞ -ball
Revised: Then, for the L ∞ -ball

Original: B x := { y ∈ R
Revised: B x := { y ∈ R

Original: d : (cid:107) y − x (cid:107) ∞ ≤ 2 γ } , all of the samples in S G will be contained in B x .
Revised: d : (cid:107) y − x (cid:107) ∞ ≤ 2 γ } , all of the samples in S G will be contained in B x .

Original: In addition, there must exist one B i that intersects B x , as otherwise B x will be in the set { B 1 , . . .
Revised: In addition, there must exist one B i that intersects B x , as otherwise B x will be in the set { B 1 , . . .

Original: , B m } .
Revised: , B m } .

Original: That is, ∃ z ∈ T, z ∈ B x ∩ B i .
Revised: That is, ∃ z ∈ T, z ∈ B x ∩ B i .

Original: By construction, B x must be containted in B (cid:48)
Revised: By construction, B x must be containted in B (cid:48)

Original: i .
Revised: i .

Original: Therefore, all samples of S G must be included in T i and T
Revised: Therefore, all samples of S G must be included in T i and T

Original: i is α -good.
Revised: i is α -good.

Original: A.4 Proof of Lemma
Revised: A.4 Proof of Lemma

Original: Proof.
Revised: Proof.

Original: Recall that after running Algorithm 1, every subset T i is contained in an L ∞ -ball of radius 6 γ .
Revised: Recall that after running Algorithm 1, every subset T i is contained in an L ∞ -ball of radius 6 γ .

Original: By Jensen’s inequality and the convexity of the L ∞ -norm, we have for all x ∈ T , (cid:107) x − µ T (cid:107) ∞ ≤ 6 γ .
Revised: By Jensen’s inequality and the convexity of the L ∞ -norm, we have for all x ∈ T , (cid:107) x − µ T (cid:107) ∞ ≤ 6 γ .

Original: Recall that we assumed p ( x ) =
Revised: Recall that we assumed p ( x ) =

Original: h A ( x − µ T ) .
Revised: h A ( x − µ T ) .

Original: Thus E x ∼ N ( µ T , I ) [ p ( x )]
Revised: Thus E x ∼ N ( µ T , I ) [ p ( x )]

Original: = 

Revised: =

Original: due to the deﬁnition of harmonic polynomials.
Revised: due to the deﬁnition of harmonic polynomials.

Original: Thus, Var x ∼ N ( µ T , I ) [ p ( x )]
Revised: Thus, Var x ∼ N ( µ T , I ) [ p ( x )]

Original: = (cid:107) A (cid:107) 22 .
Revised: = (cid:107) A (cid:107) 22 .

Original: Denote z = x − µ T .
Revised: Denote z = x − µ T .

Original: Then, | p ( x ) | = (cid:12)(cid:12) (cid:12) (cid:12) (cid:12)(cid:12) (cid:88)
Revised: Then, | p ( x ) | = (cid:12)(cid:12) (cid:12) (cid:12) (cid:12)(cid:12) (cid:88)

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: ≤
Revised: ≤

Original: [Table]
Revised: [Table]

Original: He
Revised: He

Original: [Table]
Revised: [Table]

Original: ) where a ( j ) is a d -dimensional multi-index for the j -th monomial, and c a ( j ) denotes its coefﬁcient.
Revised: ) where a ( j ) is a d -dimensional multi-index for the j -th monomial, and c a ( j ) denotes its coefﬁcient.

Original: Observe that in the ﬁrst step, p ( x ) is written as a linear combination of k 2 (cid:96) Hermite polynomials, since we are considering p ∈ P ( R d , l, k 2 (cid:96) , 2 (cid:96)k 2 (cid:96) ) .
Revised: Observe that in the ﬁrst step, p ( x ) is written as a linear combination of k 2 (cid:96) Hermite polynomials, since we are considering p ∈ P ( R d , l, k 2 (cid:96) , 2 (cid:96)k 2 (cid:96) ) .

Original: Note also that (cid:80) j ∈
Revised: Note also that (cid:80) j ∈

Original: [ k 2 (cid:96) ]
Revised: [ k 2 (cid:96) ]

Original: c 2 a ( j ) =
Revised: c 2 a ( j ) =

Original: (cid:107) A (cid:107) 22 ≤ 1 .
Revised: (cid:107) A (cid:107) 22 ≤ 1 .

Original: To bound the second factor on the right-hand side of (A.5), we use Mehler’s formula, which shows that for any u with | u | < 1 and any natural number a , ∞ (cid:88)
Revised: To bound the second factor on the right-hand side of (A.5), we use Mehler’s formula, which shows that for any u with | u | < 1 and any natural number a , ∞ (cid:88)

Original: a
Revised: a

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Since each He a ( j ) ( z ) has degree at most l , it can be decomposed as a product of at most l univariate
Revised: Since each He a ( j ) ( z ) has degree at most l , it can be decomposed as a product of at most l univariate

Original: Hermite polynomials.
Revised: Hermite polynomials.

Original: Thus, we take such product and sum over j ∈ [ k 2 (cid:96) ] to obtain
Revised: Thus, we take such product and sum over j ∈ [ k 2 (cid:96) ] to obtain

Original: [Equation]
Revised: [Equation]

Original: To simplify the above expression, observe that (cid:81) a ( j ) i (cid:54) =

Revised: To simplify the above expression, observe that (cid:81) a ( j ) i (cid:54) =

Original: u a ( j ) i = u
Revised: u a ( j ) i = u

Original: [Equation]
Revised: [Equation]

Original: a
Revised: a

Original: [Equation]
Revised: [Equation]

Original: j ) (cid:107) 1 ≥ u l .
Revised: j ) (cid:107) 1 ≥ u l .

Original: In addition, (cid:107) trim l ( z ) (cid:107) 22 ≤ l · (cid:107) z (cid:107) 2 ∞
Revised: In addition, (cid:107) trim l ( z ) (cid:107) 22 ≤ l · (cid:107) z (cid:107) 2 ∞

Original: ≤ 36 lγ 2 .
Revised: ≤ 36 lγ 2 .

Original: Lastly, by algebra, (1 − u 2 ) − l 2 ≤ e u 2 l 2 .
Revised: Lastly, by algebra, (1 − u 2 ) − l 2 ≤ e u 2 l 2 .

Original: Putting all pieces together gives (cid:88)
Revised: Putting all pieces together gives (cid:88)

Original: [Equation]
Revised: [Equation]

Original: We set u = 1 γ ; this is possible as γ > 1 .
Revised: We set u = 1 γ ; this is possible as γ > 1 .

Original: Then the exponent u 2 l 2 + 36 lγ 2 u 1+ u = l 2 γ 

Revised: Then the exponent u 2 l 2 + 36 lγ 2 u 1+ u = l 2 γ

Original: +
Revised: +

Original: [Equation]
Revised: [Equation]

Original: ≤
Revised: ≤

Original: [Equation]
Revised: [Equation]

Original: Without loss of generality, we may assume that γ > e 37 ; in fact, we can always ensure this by setting γ
Revised: Without loss of generality, we may assume that γ > e 37 ; in fact, we can always ensure this by setting γ

Original: = ( C 0 + e 37 ) · (cid:113) (cid:96) · log (cid:96)dατ where C 0 is the constant given in Algorithm 1.
Revised: = ( C 0 + e 37 ) · (cid:113) (cid:96) · log (cid:96)dατ where C 0 is the constant given in Algorithm 1.

Original: Thus, it follows that
Revised: Thus, it follows that

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: Plugging it into (A.5) completes the proof.
Revised: Plugging it into (A.5) completes the proof.

Original: B Analysis of A TTRIBUTE - EFFICIENT -M ULTIFILTER
Revised: B Analysis of A TTRIBUTE - EFFICIENT -M ULTIFILTER

Original: We collect a few useful facts about Hermite polynomials.
Revised: We collect a few useful facts about Hermite polynomials.

Original: Recall that for an order- l tensor A ∈ R d l , (cid:107) A (cid:107) 2 denotes its L 2 norm by seeing it as a long vector, and for a polynomial p : R d → R , (cid:107) p (cid:107) 2 :=
Revised: Recall that for an order- l tensor A ∈ R d l , (cid:107) A (cid:107) 2 denotes its L 2 norm by seeing it as a long vector, and for a polynomial p : R d → R , (cid:107) p (cid:107) 2 :=

Original: E x ∼ N (0 , I d ) [ p 2 ( x )]) 1 / 2 .
Revised: E x ∼ N (0 , I d ) [ p 2 ( x )]) 1 / 2 .

Original: The following can be easily seen from the deﬁnition of harmonic polynomials.
Revised: The following can be easily seen from the deﬁnition of harmonic polynomials.

Original: Fact 20.
Revised: Fact 20.

Original: For all order- l symmetric tensors A and its corresponding harmonic polynomial h A , we have that (cid:107) h A (cid:107) 

Revised: For all order- l symmetric tensors A and its corresponding harmonic polynomial h A , we have that (cid:107) h A (cid:107)

Original: = (cid:107) A (cid:107) 2 .
Revised: = (cid:107) A (cid:107) 2 .

Original: Moreover, if l > 0 , then E x ∼ N (0 , I d ) [ h A ( x )] = 0 .
Revised: Moreover, if l > 0 , then E x ∼ N (0 , I d ) [ h A ( x )] = 0 .

Original: Claim 21.
Revised: Claim 21.

Original: Let v ∈ R
Revised: Let v ∈ R

Original: d be a unit vector.
Revised: d be a unit vector.

Original: For x ∈ R d , the polynomial p ( x ) =
Revised: For x ∈ R d , the polynomial p ( x ) =

Original: He l ( v · x ) is harmonic with respect to x with degree l .
Revised: He l ( v · x ) is harmonic with respect to x with degree l .

Original: That is, there exists a tensor A = tensor( p ) which is symmetric and with order l .
Revised: That is, there exists a tensor A = tensor( p ) which is symmetric and with order l .

Original: B.1 Proof of Lemma
Revised: B.1 Proof of Lemma

Original: Proof.
Revised: Proof.

Original: Recall that we denoted λ ∗ sparse =
Revised: Recall that we denoted λ ∗ sparse =

Original: C 1 · (cid:2) ( (cid:96) + C 1 log 1 α ) · log 2 (2 + log 1 α ) (cid:3) 2 (cid:96) in Algorithm 3.
Revised: C 1 · (cid:2) ( (cid:96) + C 1 log 1 α ) · log 2 (2 + log 1 α ) (cid:3) 2 (cid:96) in Algorithm 3.

Original: Observe that if (cid:13)(cid:13)(cid:13) ˜Σ U (cid:13)(cid:13)(cid:13) F ≤ λ ∗ sparse , then for any index set Ω ⊂ [ d (cid:96) ] with | Ω | ≤ k (cid:96) , we have
Revised: Observe that if (cid:13)(cid:13)(cid:13) ˜Σ U (cid:13)(cid:13)(cid:13) F ≤ λ ∗ sparse , then for any index set Ω ⊂ [ d (cid:96) ] with | Ω | ≤ k (cid:96) , we have

Original: [Equation]
Revised: [Equation]

Original: where λ max ( · ) denotes the maximum eigenvalue and the second step follows from our choice of U which maximizes the restricted Frobenius norm.
Revised: where λ max ( · ) denotes the maximum eigenvalue and the second step follows from our choice of U which maximizes the restricted Frobenius norm.

Original: Thus, for any u ∈ R d (cid:96) with (cid:107) u (cid:107) 0 ≤ k (cid:96) ,
Revised: Thus, for any u ∈ R d (cid:96) with (cid:107) u (cid:107) 0 ≤ k (cid:96) ,

Original: [Equation]
Revised: [Equation]

Original: Let v be a k -sparse unit vector in R d .
Revised: Let v be a k -sparse unit vector in R d .

Original: That is, v ∈ R d , (cid:107) v (cid:107) 0 ≤ k, (cid:107) v (cid:107) 2 = 1 .
Revised: That is, v ∈ R d , (cid:107) v (cid:107) 0 ≤ k, (cid:107) v (cid:107) 2 = 1 .

Original: Consider some symmetric order- (cid:96) tensor B such that He (cid:96) ( v · ( x − µ T ))
Revised: Consider some symmetric order- (cid:96) tensor B such that He (cid:96) ( v · ( x − µ T ))

Original: =
Revised: =

Original: h B ( x − µ T ) (Claim 21).
Revised: h B ( x − µ T ) (Claim 21).

Original: Due to the sparsity of v , we know that B is an outer product of (cid:96) number of k -sparse vectors; hence (cid:107) B (cid:107) 0 ≤ k (cid:96) .
Revised: Due to the sparsity of v , we know that B is an outer product of (cid:96) number of k -sparse vectors; hence (cid:107) B (cid:107) 0 ≤ k (cid:96) .

Original: As h B ( x − µ T ) is a degree- (cid:96) harmonic polynomial and the vector P d,(cid:96) ( x − µ T ) includes all Hermite polynomials with degree exactly (cid:96) , we know that we can write h B ( x − µ T ) =
Revised: As h B ( x − µ T ) is a degree- (cid:96) harmonic polynomial and the vector P d,(cid:96) ( x − µ T ) includes all Hermite polynomials with degree exactly (cid:96) , we know that we can write h B ( x − µ T ) =

Original: u B · P d,(cid:96) ( x − µ T ) for some u B ∈ R d (cid:96) , (cid:107) u B (cid:107) 

Revised: u B · P d,(cid:96) ( x − µ T ) for some u B ∈ R d (cid:96) , (cid:107) u B (cid:107)

Original: ≤ k (cid:96) .
Revised: ≤ k (cid:96) .

Original: Thus, we have that
Revised: Thus, we have that

Original: E
Revised: E

Original: [Equation]
Revised: [Equation]

Original: By Fact 20, observe that (cid:107) B (cid:107) 

Revised: By Fact 20, observe that (cid:107) B (cid:107)

Original: = E x ∼ N ( µ T , I d ) [ h B ( x − µ T ) 2 ] = (cid:96) ! , and thus we have E
Revised: = E x ∼ N ( µ T , I d ) [ h B ( x − µ T ) 2 ] = (cid:96) ! , and thus we have E

Original: [He (cid:96) ( v · ( T − µ T )) 2 ] =
Revised: [He (cid:96) ( v · ( T − µ T )) 2 ] =

Original: E [ h B ( T − µ T ) 2 ] ≤ λ ∗ sparse (cid:96) ! .
Revised: E [ h B ( T − µ T ) 2 ] ≤ λ ∗ sparse (cid:96) ! .

Original: As a result, we have for any k -sparse unit vector v ∈ R d that
Revised: As a result, we have for any k -sparse unit vector v ∈ R d that

Original: [Equation]
Revised: [Equation]

Original: where the ﬁrst inequality follows from the condition that T is α -good, which, by Deﬁnition 10, implies | S G ∩ T | / | T | ≥ α .
Revised: where the ﬁrst inequality follows from the condition that T is α -good, which, by Deﬁnition 10, implies | S G ∩ T | / | T | ≥ α .

Original: The remaining analysis borrows the proof strategy from [DKS18b].
Revised: The remaining analysis borrows the proof strategy from [DKS18b].

Original: In particular, we will need the following lemma.
Revised: In particular, we will need the following lemma.

Original: Lemma 22 (Lemma 3.34 of [DKS18b]) .
Revised: Lemma 22 (Lemma 3.34 of [DKS18b]) .

Original: For any v ∈ R d , the polynomial He l ( v · ( G − µ T )) has mean ( v · ( µ − µ T )) l and variance at most 2 max( l, v · ( µ − µ T )) 2( l − 1) .
Revised: For any v ∈ R d , the polynomial He l ( v · ( G − µ T )) has mean ( v · ( µ − µ T )) l and variance at most 2 max( l, v · ( µ − µ T )) 2( l − 1) .

Original: Now to ease the notation, write θ := v · ( µ − µ T ) .
Revised: Now to ease the notation, write θ := v · ( µ − µ T ) .

Original: By Cantelli’s inequality we have
Revised: By Cantelli’s inequality we have

Original: [Equation]
Revised: [Equation]

Original: Since S G is representative, by Deﬁnition
Revised: Since S G is representative, by Deﬁnition

Original: [Equation]
Revised: [Equation]

Original: Since T is α -good, due to Deﬁnition 10, | S G ∩ T | / | S G | ≥ 1 − α 6 ≥ 80100 , we have that
Revised: Since T is α -good, due to Deﬁnition 10, | S G ∩ T | / | S G | ≥ 1 − α 6 ≥ 80100 , we have that

Original: [Equation]
Revised: [Equation]

Original: On the other hand, due to Eq.
Revised: On the other hand, due to Eq.

Original: (B.2), applying Markov’s inequality gives that for any k -sparse unit vector v ,
Revised: (B.2), applying Markov’s inequality gives that for any k -sparse unit vector v ,

Original: [Equation]
Revised: [Equation]

Original: Recall that θ = v · ( µ − µ T ) .
Revised: Recall that θ = v · ( µ − µ T ) .

Original: From Eq.
Revised: From Eq.

Original: (B.1) and (B.3), we have that for any k -sparse unit vector v ∈ R d ,
Revised: (B.1) and (B.3), we have that for any k -sparse unit vector v ∈ R d ,

Original: [Equation]
Revised: [Equation]

Original: Note that θ (cid:96) ≤ √ 2 max( (cid:96), θ ) ( (cid:96) − 1) only when θ ≤ 2 (cid:96) , and so we have that for any k -sparse unit vector v ∈ R d ,
Revised: Note that θ (cid:96) ≤ √ 2 max( (cid:96), θ ) ( (cid:96) − 1) only when θ ≤ 2 (cid:96) , and so we have that for any k -sparse unit vector v ∈ R d ,

Original: [Equation]
Revised: [Equation]

Original: By choosing v = trim k ( µ − µ T ) and combining the above with Lemma 39, we complete the proof.
Revised: By choosing v = trim k ( µ − µ T ) and combining the above with Lemma 39, we complete the proof.

Original: B.2 Analysis of B ASIC MF
Revised: B.2 Analysis of B ASIC MF

Original: Recall the notations in B ASIC MF (Algorithm 4): R =
Revised: Recall the notations in B ASIC MF (Algorithm 4): R =

Original: ( C 1 · log( 1 α )) l/ 2 , γ = C 0 · (cid:113) (cid:96) · log( (cid:96)dατ ) , and the length of the interval [ a, b ] , i.e. b − a , equals C 1 · R · log(2 + log 1 α ) .
Revised: ( C 1 · log( 1 α )) l/ 2 , γ = C 0 · (cid:113) (cid:96) · log( (cid:96)dατ ) , and the length of the interval [ a, b ] , i.e. b − a , equals C 1 · R · log(2 + log 1 α ) .

Original: We will need a series of results to prove Theorem 18.
Revised: We will need a series of results to prove Theorem 18.

Original: First, we note that if B ASIC MF returns at Step 2, then T must not be α -good in view of Lemma 11.
Revised: First, we note that if B ASIC MF returns at Step 2, then T must not be α -good in view of Lemma 11.

Original: Thus we only need to consider the remaining steps.
Revised: Thus we only need to consider the remaining steps.

Original: In particular, we divide the output of B ASIC MF into three cases:
Revised: In particular, we divide the output of B ASIC MF into three cases:

Original: • C ASE 1: it returns TBD at Step 5.
Revised: • C ASE 1: it returns TBD at Step 5.

Original: • C ASE 2: it returns one subset { ( T (cid:48) , α (cid:48) ) } at Step 9.
Revised: • C ASE 2: it returns one subset { ( T (cid:48) , α (cid:48) ) } at Step 9.

Original: • C ASE 3: it returns two subsets { ( T
Revised: • C ASE 3: it returns two subsets { ( T

Original: 1 , α 1 ) , ( T 2 , α 2 ) } at Step 14.
Revised: 1 , α 1 ) , ( T 2 , α 2 ) } at Step 14.

Original: We analyze the performance for each case in the following.
Revised: We analyze the performance for each case in the following.

Original: B.2.1 Analysis of C ASE
Revised: B.2.1 Analysis of C ASE

Original: Proposition 23.
Revised: Proposition 23.

Original: Consider Algorithm 4.
Revised: Consider Algorithm 4.

Original: If it returns TBD and if T is an α -good set, then | E [ p ( G )] − E [ p ( T )] | ≤ O (cid:0) ( (cid:96) + log 1 α ) l 2 · log(2 + log 1 α ) (cid:1) .
Revised: If it returns TBD and if T is an α -good set, then | E [ p ( G )] − E [ p ( T )] | ≤ O (cid:0) ( (cid:96) + log 1 α ) l 2 · log(2 + log 1 α ) (cid:1) .

Original: Proof.
Revised: Proof.

Original: We ﬁrst argue that most of the good samples in T have p ( x ) value close to E [ p ( G )] .
Revised: We ﬁrst argue that most of the good samples in T have p ( x ) value close to E [ p ( G )] .

Original: Claim 24.
Revised: Claim 24.

Original: If T is α -good, then the samples x ∈ T ∩ S G that satisfy | p ( x ) − E [ p ( G )]
Revised: If T is α -good, then the samples x ∈ T ∩ S G that satisfy | p ( x ) − E [ p ( G )]

Original: | < R constitute at least an (cid:16) α − α 3 100 (cid:17) -fraction of T and an (1 − α 6 − α 3 100 ) -fraction of S G .
Revised: | < R constitute at least an (cid:16) α − α 3 100 (cid:17) -fraction of T and an (1 − α 6 − α 3 100 ) -fraction of S G .

Original: Next, we claim that if there exists an appropriate interval [ a, b ] in Step 3, then the mean of p ( G ) is in the interval [ a − R, b + R ] .
Revised: Next, we claim that if there exists an appropriate interval [ a, b ] in Step 3, then the mean of p ( G ) is in the interval [ a − R, b + R ] .

Original: Claim 25.
Revised: Claim 25.

Original: If T is α -good, and the interval [ a, b ] contains at least (1 − α 2 ) -fraction of values of p ( x ) for x ∈ T , then E [ p ( G )] ∈ [ a − R, b + R ] .
Revised: If T is α -good, and the interval [ a, b ] contains at least (1 − α 2 ) -fraction of values of p ( x ) for x ∈ T , then E [ p ( G )] ∈ [ a − R, b + R ] .

Original: Now by construction, if Algorithm 4 returns TBD , then
Revised: Now by construction, if Algorithm 4 returns TBD , then

Original: [Equation]
Revised: [Equation]

Original: On the other hand, the interval [ a, b ] contains at least (1 − α 2 ) fraction of values of p ( x ) for x ∈ T .
Revised: On the other hand, the interval [ a, b ] contains at least (1 − α 2 ) fraction of values of p ( x ) for x ∈ T .

Original: Therefore, the contribution of the samples in [ a, b ] to the variance gives
Revised: Therefore, the contribution of the samples in [ a, b ] to the variance gives

Original: [Equation]
Revised: [Equation]

Original: To see this, note that a + b 2 is the midpoint and b − a 2 is the length of interval [ a, b ] .
Revised: To see this, note that a + b 2 is the midpoint and b − a 2 is the length of interval [ a, b ] .

Original: When E [ p ( T )] is inside the interval,
Revised: When E [ p ( T )] is inside the interval,

Original: (cid:12)(cid:12) E [ p ( T )] − a + b 2 (cid:12)(cid:12) − b − a 2 < 0 and the variance is lowered bounded by 0 .
Revised: (cid:12)(cid:12) E [ p ( T )] − a + b 2 (cid:12)(cid:12) − b − a 2 < 0 and the variance is lowered bounded by 0 .

Original: Otherwise, when E [ p ( T )] is outside the interval, the distance from any sample in [ a, b ] to E [ p ( T )] is
Revised: Otherwise, when E [ p ( T )] is outside the interval, the distance from any sample in [ a, b ] to E [ p ( T )] is

Original: at least (cid:12)(cid:12) E
Revised: at least (cid:12)(cid:12) E

Original: [ p ( T )] − a + b 2 (cid:12)(cid:12) − b − a 2 ≥ 0 .
Revised: [ p ( T )] − a + b 2 (cid:12)(cid:12) − b − a 2 ≥ 0 .

Original: Moreover, since b − a ≤ O ((log(1 /α )) l/ 2 · log(2 + log(1 /α ))) , | E [ p
Revised: Moreover, since b − a ≤ O ((log(1 /α )) l/ 2 · log(2 + log(1 /α ))) , | E [ p

Original: [Equation]
Revised: [Equation]

Original: From the Claim 25, we also have
Revised: From the Claim 25, we also have

Original: [Equation]
Revised: [Equation]

Original: By the triangle inequality, we have that | E [ p ( G )] − E
Revised: By the triangle inequality, we have that | E [ p ( G )] − E

Original: [ p ( T )]
Revised: [ p ( T )]

Original: | =
Revised: | =

Original: O (( (cid:96) + C log(1 /α )) l/ 2 log(2 + log(1 /α ))) .
Revised: O (( (cid:96) + C log(1 /α )) l/ 2 log(2 + log(1 /α ))) .

Original: Proof of Claim 24.
Revised: Proof of Claim 24.

Original: Since T is α -good, and Var[ p ( G )] ≤ 1 .
Revised: Since T is α -good, and Var[ p ( G )] ≤ 1 .

Original: By degree- l Chernoff bound (Lemma 40) and deﬁnition of representative set (Deﬁnition 8), for R =
Revised: By degree- l Chernoff bound (Lemma 40) and deﬁnition of representative set (Deﬁnition 8), for R =

Original: ( C 1 · log(1 /α )) l/
Revised: ( C 1 · log(1 /α )) l/

Original: [Equation]
Revised: [Equation]

Original: for large enough constant C > 0 .
Revised: for large enough constant C > 0 .

Original: Proof of Claim 25.
Revised: Proof of Claim 25.

Original: From Claim 24, at least an ( α − α 3 / 100) -fraction of T is R -close to E [ p ( G )] .
Revised: From Claim 24, at least an ( α − α 3 / 100) -fraction of T is R -close to E [ p ( G )] .

Original: Also we know that at most an α 2 -fraction of T are not in [ a, b ] by the deﬁnition of the interval [ a, b ] .
Revised: Also we know that at most an α 2 -fraction of T are not in [ a, b ] by the deﬁnition of the interval [ a, b ] .

Original: Then, there must be at least
Revised: Then, there must be at least

Original: [Equation]
Revised: [Equation]

Original: fraction of samples in T that are in [ a, b ] and R close to E [ p ( G )] .
Revised: fraction of samples in T that are in [ a, b ] and R close to E [ p ( G )] .

Original: Therefore, E [ p ( G )] must be in [ a − R, b + R ] .
Revised: Therefore, E [ p ( G )] must be in [ a − R, b + R ] .

Original: Lemma 26.
Revised: Lemma 26.

Original: Consider Algorithm 4.
Revised: Consider Algorithm 4.

Original: If it reaches Step 7, there must exist a threshold t > 2 R satisfying the inequality thereof.
Revised: If it reaches Step 7, there must exist a threshold t > 2 R satisfying the inequality thereof.

Original: Proof.
Revised: Proof.

Original: We will prove this lemma by contradiction.
Revised: We will prove this lemma by contradiction.

Original: Assume that Algorithm 4 reaches Step 7, but for all t > 2 R , we have
Revised: Assume that Algorithm 4 reaches Step 7, but for all t > 2 R , we have

Original: [Equation]
Revised: [Equation]

Original: By change of variables, we have that for any t > 2 R + b − a 2 ,
Revised: By change of variables, we have that for any t > 2 R + b − a 2 ,

Original: [Equation]
Revised: [Equation]

Original: Note that this inequality only holds non-trivially when t ≥ t 0 where t 0 = 2 R + b − a 2 + (log 32 α ) l/ 2 ; namely, if t < t 0 , the right-hand side is at least 1 .
Revised: Note that this inequality only holds non-trivially when t ≥ t 0 where t 0 = 2 R + b − a 2 + (log 32 α ) l/ 2 ; namely, if t < t 0 , the right-hand side is at least 1 .

Original: By Lemma 13, we have max x,y ∈ T | p ( x ) − p ( y ) | ≤ 2 k (cid:96) · γ l , where γ = C 0 · (cid:113) (cid:96) · log( (cid:96)dατ ) .
Revised: By Lemma 13, we have max x,y ∈ T | p ( x ) − p ( y ) | ≤ 2 k (cid:96) · γ l , where γ = C 0 · (cid:113) (cid:96) · log( (cid:96)dατ ) .

Original: Also note that the size of the interval [ a, b ] equals C 1 · R · log(2 + log 1 α ) which is less than k (cid:96) · γ l .
Revised: Also note that the size of the interval [ a, b ] equals C 1 · R · log(2 + log 1 α ) which is less than k (cid:96) · γ l .

Original: Therefore,
Revised: Therefore,

Original: [Equation]
Revised: [Equation]

Original: B
Revised: B

Original: [Equation]
Revised: [Equation]

Original: Then, we have that
Revised: Then, we have that

Original: Var
Revised: Var

Original: [Equation]
Revised: [Equation]

Original: p
Revised: p

Original: [Equation]
Revised: [Equation]

Original: where ζ 1 holds in view of (B.8), and where ζ 2 follows since
Revised: where ζ 1 holds in view of (B.8), and where ζ 2 follows since

Original: [Equation]
Revised: [Equation]

Original: where ζ 3 is due to the incomplete gamma function (see Claim 3.11 of [DKS18b]),
Revised: where ζ 3 is due to the incomplete gamma function (see Claim 3.11 of [DKS18b]),

Original: i.e. (cid:82) ∞ x e − t · t s − 1 dt ≤ e − x ( x + s ) s − 1 , for s ≥ 1 , x ≥ 0 .
Revised: i.e. (cid:82) ∞ x e − t · t s − 1 dt ≤ e − x ( x + s ) s − 1 , for s ≥ 1 , x ≥ 0 .

Original: In other words, had we not found an appropriate threshold t > 2 R at Step 7, Algorithm 4 would have returned at Step 5, which is a contradiction.
Revised: In other words, had we not found an appropriate threshold t > 2 R at Step 7, Algorithm 4 would have returned at Step 5, which is a contradiction.

Original: This completes the proof.
Revised: This completes the proof.

Original: Once we have veriﬁed the existence of such threshold t , it is easy to see that the resultant T (cid:48) is a subset of T , and α (cid:48) ≥
Revised: Once we have veriﬁed the existence of such threshold t , it is easy to see that the resultant T (cid:48) is a subset of T , and α (cid:48) ≥

Original: α by algebraic calculation.
Revised: α by algebraic calculation.

Original: This has been already shown in [DKS18b].
Revised: This has been already shown in [DKS18b].

Original: Lemma 27 (Lemma 3.13 of [DKS18b]) .
Revised: Lemma 27 (Lemma 3.13 of [DKS18b]) .

Original: Consider Algorithm 4.
Revised: Consider Algorithm 4.

Original: If it reaches Step 9, then the output { ( T (cid:48) , α (cid:48) ) } is such that T (cid:48)
Revised: If it reaches Step 9, then the output { ( T (cid:48) , α (cid:48) ) } is such that T (cid:48)

Original: ⊂ T and α (cid:48) > α .
Revised: ⊂ T and α (cid:48) > α .

Original: Next, we show that B ASIC MF sanitizes the sample set, i.e. it removes more corrupted samples than the uncorrupted ones.
Revised: Next, we show that B ASIC MF sanitizes the sample set, i.e. it removes more corrupted samples than the uncorrupted ones.

Original: Lemma 28.
Revised: Lemma 28.

Original: Consider Algorithm 4.
Revised: Consider Algorithm 4.

Original: If it reaches Step 9, and if T is α -good and Var[ p ( G )] ≤ 1 , then the output { ( T (cid:48) , α (cid:48) ) } is such that T (cid:48) is α (cid:48) -good.
Revised: If it reaches Step 9, and if T is α -good and Var[ p ( G )] ≤ 1 , then the output { ( T (cid:48) , α (cid:48) ) } is such that T (cid:48) is α (cid:48) -good.

Original: Proof.
Revised: Proof.

Original: Due to Algorithm 1, the (cid:96) ∞ -distance among all pairs of the samples are bounded.
Revised: Due to Algorithm 1, the (cid:96) ∞ -distance among all pairs of the samples are bounded.

Original: It remains to show | S G ∩ T (cid:48) | / | T (cid:48) | ≥ α (cid:48) and | S G ∩ T (cid:48) | / | S G | ≥ 1 − α (cid:48) / 6 .
Revised: It remains to show | S G ∩ T (cid:48) | / | T (cid:48) | ≥ α (cid:48) and | S G ∩ T (cid:48) | / | S G | ≥ 1 − α (cid:48) / 6 .

Original: We claim that for any t > 2 R , the following holds:
Revised: We claim that for any t > 2 R , the following holds:

Original: [Equation]
Revised: [Equation]

Original: To see the rationale, we note that by Claim 25, we have E [ p ( G )]
Revised: To see the rationale, we note that by Claim 25, we have E [ p ( G )]

Original: ∈ [ a − R, b + R ] .
Revised: ∈ [ a − R, b + R ] .

Original: Since E [ p ( G )] − R ≤ b , we have
Revised: Since E [ p ( G )] − R ≤ b , we have

Original: [Equation]
Revised: [Equation]

Original: where in the third step, we used the fact that S G is representative (see Deﬁnition 8), and in the last step we applied Lemma 40.
Revised: where in the third step, we used the fact that S G is representative (see Deﬁnition 8), and in the last step we applied Lemma 40.

Original: The inequality (B.9) follows since min {| p ( S G ) − a | , | p ( S G ) − b |} ≥ t is a subevent of | p ( S G ) − b | > t .
Revised: The inequality (B.9) follows since min {| p ( S G ) − a | , | p ( S G ) − b |} ≥ t is a subevent of | p ( S G ) − b | > t .

Original: Since T is α -good, we know that a 1 − α 6 ≥ 12 fraction of the samples in S G is in S G ∩ T .
Revised: Since T is α -good, we know that a 1 − α 6 ≥ 12 fraction of the samples in S G is in S G ∩ T .

Original: Therefore,
Revised: Therefore,

Original: [Equation]
Revised: [Equation]

Original: Due to the inequality of Step 7 in Algorithm 4, we know that the above probability is at least 8 /α times larger for the samples in T .
Revised: Due to the inequality of Step 7 in Algorithm 4, we know that the above probability is at least 8 /α times larger for the samples in T .

Original: Therefore,
Revised: Therefore,

Original: [Equation]
Revised: [Equation]

Original: meaning that the remaining fraction of good samples in T (cid:48) is at least α (cid:48) .
Revised: meaning that the remaining fraction of good samples in T (cid:48) is at least α (cid:48) .

Original: On the other hand, since | S G ∩ T | / | S G | ≥ 1 − α/ 6 and (cid:16) 1 − α 8 · (cid:16) 1 − | T
Revised: On the other hand, since | S G ∩ T | / | S G | ≥ 1 − α/ 6 and (cid:16) 1 − α 8 · (cid:16) 1 − | T

Original: [Equation]
Revised: [Equation]

Original: | | T | (cid:17)(cid:17) α
Revised: | | T | (cid:17)(cid:17) α

Original: = α (cid:48) | T (cid:48) | / | T | , we have
Revised: = α (cid:48) | T (cid:48) | / | T | , we have

Original: [Equation]
Revised: [Equation]

Original: thus,
Revised: thus,

Original: [Equation]
Revised: [Equation]

Original: This proves that T (cid:48) is α (cid:48) -good.
Revised: This proves that T (cid:48) is α (cid:48) -good.

Original: We summarize the performance of B ASIC MF in C ASE 2 in the following proposition, which is an immediate combination of Lemma 26, Lemma 27, and Lemma 28.
Revised: We summarize the performance of B ASIC MF in C ASE 2 in the following proposition, which is an immediate combination of Lemma 26, Lemma 27, and Lemma 28.

Original: Proposition 29.
Revised: Proposition 29.

Original: Consider Algorithm 4.
Revised: Consider Algorithm 4.

Original: If it reaches Step 7, there must exist t > 2 R that satisﬁes the inequality of this step, and the algorithm will output { ( T (cid:48) , α (cid:48) ) } with T (cid:48)
Revised: If it reaches Step 7, there must exist t > 2 R that satisﬁes the inequality of this step, and the algorithm will output { ( T (cid:48) , α (cid:48) ) } with T (cid:48)

Original: ⊂ T and α (cid:48) ≥ α .
Revised: ⊂ T and α (cid:48) ≥ α .

Original: If, in addition, T is α -good and Var[ p ( G )] ≤ 1 , then T (cid:48) is α (cid:48) -good.
Revised: If, in addition, T is α -good and Var[ p ( G )] ≤ 1 , then T (cid:48) is α (cid:48) -good.

Original: B.2.3 Analysis of C ASE
Revised: B.2.3 Analysis of C ASE

Original: Lemma 30 (Lemma 3.12 of [DKS18b]) .
Revised: Lemma 30 (Lemma 3.12 of [DKS18b]) .

Original: Consider Algorithm 4.
Revised: Consider Algorithm 4.

Original: If it reaches Step 12, there must exist a threshold t that satisfy the conditions thereof.
Revised: If it reaches Step 12, there must exist a threshold t that satisfy the conditions thereof.

Original: Lemma 31 (Lemma 3.14 of [DKS18b]) .
Revised: Lemma 31 (Lemma 3.14 of [DKS18b]) .

Original: Consider Algorithm 4.
Revised: Consider Algorithm 4.

Original: If it reaches Step 12, then the output { ( T 1 , α 1 ) , ( T 2 , α 2 ) } is such that T 1 ⊂ T , T 2 ⊂ T , and 1 α 21 + 1 α 22 ≤ 1 α 2 .
Revised: If it reaches Step 12, then the output { ( T 1 , α 1 ) , ( T 2 , α 2 ) } is such that T 1 ⊂ T , T 2 ⊂ T , and 1 α 21 + 1 α 22 ≤ 1 α 2 .

Original: Lemma 32.
Revised: Lemma 32.

Original: Consider Algorithm 4.
Revised: Consider Algorithm 4.

Original: If it reaches Step 12, and if T is α -good, then the output { ( T 1 , α 1 ) , ( T 2 , α 2 ) } is such that T i is α
Revised: If it reaches Step 12, and if T is α -good, then the output { ( T 1 , α 1 ) , ( T 2 , α 2 ) } is such that T i is α

Original: i -good for some i ∈ { 1 , 2 } .
Revised: i -good for some i ∈ { 1 , 2 } .

Original: Proof.
Revised: Proof.

Original: Recall that Claim 24 lower bounds the fraction of the good samples (i.e. x ∈ S G ∩ T ) that satisfy | p ( x ) − E
Revised: Recall that Claim 24 lower bounds the fraction of the good samples (i.e. x ∈ S G ∩ T ) that satisfy | p ( x ) − E

Original: [ p ( G )]
Revised: [ p ( G )]

Original: | < R .
Revised: | < R .

Original: Since T 1 and T 2 overlap in an interval of length at least 2 R , the good samples must be contained in either one of both two clusters.
Revised: Since T 1 and T 2 overlap in an interval of length at least 2 R , the good samples must be contained in either one of both two clusters.

Original: We will show that the T i with these good samples (in interval of length 2 R ) is α i -good.
Revised: We will show that the T i with these good samples (in interval of length 2 R ) is α i -good.

Original: Since T is α -good, we have | S G ∩ T | / | T | ≥ α and | S G ∩ T | / | S G | ≥ (1 − α/ 6) .
Revised: Since T is α -good, we have | S G ∩ T | / | T | ≥ α and | S G ∩ T | / | S G | ≥ (1 − α/ 6) .

Original: We want to show that (i) | S G ∩ T i | / | T i | ≥ α i and (ii) | S G ∩ T i | / | S G | ≥ (1 − α i / 6) .
Revised: We want to show that (i) | S G ∩ T i | / | T i | ≥ α i and (ii) | S G ∩ T i | / | S G | ≥ (1 − α i / 6) .

Original: To show (i), note that | S G ∩ T i | ≥ (cid:16) α − α 3 100 (cid:17) | T | due to Claim 24.
Revised: To show (i), note that | S G ∩ T i | ≥ (cid:16) α − α 3 100 (cid:17) | T | due to Claim 24.

Original: Thus,
Revised: Thus,

Original: [Equation]
Revised: [Equation]

Original: where the last transition is by deﬁnition.
Revised: where the last transition is by deﬁnition.

Original: To show (ii), we only have to show that α i / 6 ≥ α/ 6 + α 3 / 100 , i.e. α i ≥ α + 3 α 3 / 50 .
Revised: To show (ii), we only have to show that α i / 6 ≥ α/ 6 + α 3 / 100 , i.e. α i ≥ α + 3 α 3 / 50 .

Original: Note that | T | − | T i | ≥ α 4 | T | , ∀ i .
Revised: Note that | T | − | T i | ≥ α 4 | T | , ∀ i .

Original: Thus, | T | / | T i | ≥ 1 1 − α/ 4 and we can show that
Revised: Thus, | T | / | T i | ≥ 1 1 − α/ 4 and we can show that

Original: [Equation]
Revised: [Equation]

Original: This completes the proof.
Revised: This completes the proof.

Original: Combining Lemma 30, Lemma 31, and Lemma 32, we immediately have the following.
Revised: Combining Lemma 30, Lemma 31, and Lemma 32, we immediately have the following.

Original: Proposition 33.
Revised: Proposition 33.

Original: Consider Algorithm 4.
Revised: Consider Algorithm 4.

Original: If it reaches Step 12, then there must exist a threshold t that satisﬁes the conditions in this step.
Revised: If it reaches Step 12, then there must exist a threshold t that satisﬁes the conditions in this step.

Original: Moreover, the output { ( T 1 , α 1 ) , ( T 2 , α 2 ) } is such that T 1 ⊂ T ,
Revised: Moreover, the output { ( T 1 , α 1 ) , ( T 2 , α 2 ) } is such that T 1 ⊂ T ,

Original: T 2 ⊂ T , and 1 α 21 + 1 α 22 ≤ 1 α 2 .
Revised: T 2 ⊂ T , and 1 α 21 + 1 α 22 ≤ 1 α 2 .

Original: If, in addition, T is α -good, then T i is α
Revised: If, in addition, T is α -good, then T i is α

Original: i -good for some i ∈ { 1 , 2 } .
Revised: i -good for some i ∈ { 1 , 2 } .

Original: B.2.4 Proof of Theorem
Revised: B.2.4 Proof of Theorem

Original: Proof.
Revised: Proof.

Original: Observe that now Theorem 18 is an immediate result by combining Proposition 23, Proposition 29, and Proposition 33.
Revised: Observe that now Theorem 18 is an immediate result by combining Proposition 23, Proposition 29, and Proposition 33.

Original: B.3 Analysis of H ARMONIC MF B.3.1 Certifying the varaince of p on G
Revised: B.3 Analysis of H ARMONIC MF B.3.1 Certifying the varaince of p on G

Original: Proof of Lemma 19.
Revised: Proof of Lemma 19.

Original: The proof follows directly from Lemma 3.31 of [DKS18b].
Revised: The proof follows directly from Lemma 3.31 of [DKS18b].

Original: B.3.2 Analysis for p
Revised: B.3.2 Analysis for p

Original: Proof of Lemma 16.
Revised: Proof of Lemma 16.

Original: First, if at any subroutine of H ARMONIC MF, it returns “ NO ” or a list of pairs { ( T i , α i ) } , ANS (cid:54) =
Revised: First, if at any subroutine of H ARMONIC MF, it returns “ NO ” or a list of pairs { ( T i , α i ) } , ANS (cid:54) =

Original: TBD .
Revised: TBD .

Original: If that is not the case, it means H ARMONIC MF reaches Step 6 and
Revised: If that is not the case, it means H ARMONIC MF reaches Step 6 and

Original: B ASIC MF returns “ TBD ”, then we have
Revised: B ASIC MF returns “ TBD ”, then we have

Original: [Equation]
Revised: [Equation]

Original: because p 1 is of degree (cid:96) .
Revised: because p 1 is of degree (cid:96) .

Original: However, recall that the condition of Step 6 in Algorithm 3 is satisﬁed, thus
Revised: However, recall that the condition of Step 6 in Algorithm 3 is satisﬁed, thus

Original: Var[ p 1 ( T )] =
Revised: Var[ p 1 ( T )] =

Original: Var[ v ∗ · P d,(cid:96)
Revised: Var[ v ∗ · P d,(cid:96)

Original: [Equation]
Revised: [Equation]

Original: )] ≥ λ ∗ ≥ λ ∗ sparse
Revised: )] ≥ λ ∗ ≥ λ ∗ sparse

Original: [Equation]
Revised: [Equation]

Original: which induces a contradition.
Revised: which induces a contradition.

Original: We conclude that B ASIC MF will not return “ TBD ” at Step 6, which completes the proof.
Revised: We conclude that B ASIC MF will not return “ TBD ” at Step 6, which completes the proof.

Original: B.3.3 Analysis for p
Revised: B.3.3 Analysis for p

Original: Proof of Lemma 17.
Revised: Proof of Lemma 17.

Original: We see that the lemma holds as long as H ARMONIC MF returns either “ NO ” or a list of ( T i , α i ) for p 2 correctly.
Revised: We see that the lemma holds as long as H ARMONIC MF returns either “ NO ” or a list of ( T i , α i ) for p 2 correctly.

Original: First, we claim that p 2 ( x ) is harmonic such that M ULTILINEAR MF multiﬁlters correctly at Step 4.
Revised: First, we claim that p 2 ( x ) is harmonic such that M ULTILINEAR MF multiﬁlters correctly at Step 4.

Original: To prove the claim, simply note that p 2 is of degree 2 (cid:96) and consists of a set of k 2 (cid:96) Hermite polynomials.
Revised: To prove the claim, simply note that p 2 is of degree 2 (cid:96) and consists of a set of k 2 (cid:96) Hermite polynomials.

Original: In addition, p 2 ( x ) only applies on a set of 2 (cid:96)k 2 (cid:96) coordinates.
Revised: In addition, p 2 ( x ) only applies on a set of 2 (cid:96)k 2 (cid:96) coordinates.

Original: Based on the correctness of M ULTILINEAR MF, it remains to show that if every subroutine of
Revised: Based on the correctness of M ULTILINEAR MF, it remains to show that if every subroutine of

Original: H ARMONIC MF returns “ TBD ”, then T must not be α -good.
Revised: H ARMONIC MF returns “ TBD ”, then T must not be α -good.

Original: Consider that Algorithm 5 reaches
Revised: Consider that Algorithm 5 reaches

Original: Step 6, and B ASIC MF returns “ TBD ”.
Revised: Step 6, and B ASIC MF returns “ TBD ”.

Original: Applying Lemma 19, we know that E [ p 2 ( G ) 2 ] ≤ β 2 = (cid:0) C 1 · (1 + log( 1 α )) · log 2 (2 + log( 1 α ))
Revised: Applying Lemma 19, we know that E [ p 2 ( G ) 2 ] ≤ β 2 = (cid:0) C 1 · (1 + log( 1 α )) · log 2 (2 + log( 1 α ))

Original: (cid:1) 2 (cid:96) .
Revised: (cid:1) 2 (cid:96) .

Original: Therefore, Var[ 1 β · p 2 ( G )] ≤ 1 β 2 · E [ p 2 ( G ) 2 ] ≤ 1 and thus satisﬁes the preconditions of B ASIC MF.
Revised: Therefore, Var[ 1 β · p 2 ( G )] ≤ 1 β 2 · E [ p 2 ( G ) 2 ] ≤ 1 and thus satisﬁes the preconditions of B ASIC MF.

Original: Then, if B ASIC MF also returns “ TBD ”, we can show that Var[ 1 β · p 2 ( T )]
Revised: Then, if B ASIC MF also returns “ TBD ”, we can show that Var[ 1 β · p 2 ( T )]

Original: = O (cid:0) ( (cid:96) + log( 1 α )) 2 (cid:96) · log 2 (2 + log( 1 α ))
Revised: = O (cid:0) ( (cid:96) + log( 1 α )) 2 (cid:96) · log 2 (2 + log( 1 α ))

Original: (cid:1) according to Theorem 18.
Revised: (cid:1) according to Theorem 18.

Original: Thus,
Revised: Thus,

Original: [Equation]
Revised: [Equation]

Original: We then show by contradiction.
Revised: We then show by contradiction.

Original: Assume the above holds and T is α -good.
Revised: Assume the above holds and T is α -good.

Original: Due to Theorem 18, we can show that
Revised: Due to Theorem 18, we can show that

Original: [Equation]
Revised: [Equation]

Original: Additionally, since
Revised: Additionally, since

Original: [Equation]
Revised: [Equation]

Original: Therefore, by Cauchy-Schwarz inequality, we conclude that | E [ p 2 ( T )] | ≤ O (( (cid:96) + log( 1 α )) log 2 (2 + log( 1 α )))
Revised: Therefore, by Cauchy-Schwarz inequality, we conclude that | E [ p 2 ( T )] | ≤ O (( (cid:96) + log( 1 α )) log 2 (2 + log( 1 α )))

Original: 2 (cid:96) .
Revised: 2 (cid:96) .

Original: However, by construction, we have
Revised: However, by construction, we have

Original: [Equation]
Revised: [Equation]

Original: where ζ 4 is due to the condition in Step 4 of Algorithm 3.
Revised: where ζ 4 is due to the condition in Step 4 of Algorithm 3.

Original: This is a contradiction.
Revised: This is a contradiction.

Original: Hence, we conclude that T cannot be α -good
Revised: Hence, we conclude that T cannot be α -good

Original: and we remove it from the list.
Revised: and we remove it from the list.

Original: Moreover, if B ASIC MF returns NO or a list { ( T i , α i ) } , the guarantees follow from Theorem 18.
Revised: Moreover, if B ASIC MF returns NO or a list { ( T i , α i ) } , the guarantees follow from Theorem 18.

Original: The proof is complete
Revised: The proof is complete

Original: [Equation]
Revised: [Equation]

Original: Lemma 34 (Algorithm 5) .
Revised: Lemma 34 (Algorithm 5) .

Original: Consider Algorithm 5 with input polynomial being p 1 or p 2 in view of Algorithm 3, and denote by ANS its output.
Revised: Consider Algorithm 5 with input polynomial being p 1 or p 2 in view of Algorithm 3, and denote by ANS its output.

Original: With probability 1 − τ , the following holds.
Revised: With probability 1 − τ , the following holds.

Original: If
Revised: If

Original: ANS =
Revised: ANS =

Original: NO , then T is not α -good.
Revised: NO , then T is not α -good.

Original: If ANS =
Revised: If ANS =

Original: ( T i , α i ) } mi =1 for some m ≤ 2 , then T i ⊂ T for all i ∈ [ m ] and (cid:80) m i
Revised: ( T i , α i ) } mi =1 for some m ≤ 2 , then T i ⊂ T for all i ∈ [ m ] and (cid:80) m i

Original: =1 1 α 2 i ≤ 1 α 2 ; if additionally T is α -good, then at least one T i is α i -good.
Revised: =1 1 α 2 i ≤ 1 α 2 ; if additionally T is α -good, then at least one T i is α i -good.

Original: Proof.
Revised: Proof.

Original: Inside any subroutine of B ASIC MF or M ULTILINEAR MF called by H ARMONIC MF, if ANS is assigned “ NO ” or a list of pairs { ( T i , α i ) } , the guarantees are ensured by Lemma 19, Theoremand Lemma 36.
Revised: Inside any subroutine of B ASIC MF or M ULTILINEAR MF called by H ARMONIC MF, if ANS is assigned “ NO ” or a list of pairs { ( T i , α i ) } , the guarantees are ensured by Lemma 19, Theoremand Lemma 36.

Original: It remains to show the correctness of the algorithm returning “ NO ” at Step 6 when
Revised: It remains to show the correctness of the algorithm returning “ NO ” at Step 6 when

Original: B ASIC MF returns TBD , which is implied by Lemma 17.
Revised: B ASIC MF returns TBD , which is implied by Lemma 17.

Original: B.4 Proof of Theorem
Revised: B.4 Proof of Theorem

Original: Proof.
Revised: Proof.

Original: The theorem follows from Lemma 15, Lemma 16, Lemma 17, Theorem 18 and Lemma 34.
Revised: The theorem follows from Lemma 15, Lemma 16, Lemma 17, Theorem 18 and Lemma 34.

Original: C Proof of Theorem
Revised: C Proof of Theorem

Original: Theorem 1 directly follows from the guarantees of our initial clustering step (Lemma 12), the main subroutine (Theorem 14), and the black-box list reduction algorithm (Proposition 37).
Revised: Theorem 1 directly follows from the guarantees of our initial clustering step (Lemma 12), the main subroutine (Theorem 14), and the black-box list reduction algorithm (Proposition 37).

Original: Proof of Theorem 1.
Revised: Proof of Theorem 1.

Original: Consider Algorithm 2.
Revised: Consider Algorithm 2.

Original: By Lemma 12, T will be divided into at most 12 α number of subsets, at least one of which is α 2 -good.
Revised: By Lemma 12, T will be divided into at most 12 α number of subsets, at least one of which is α 2 -good.

Original: Algorithm 2 then maintains a list L of pairs { ( T i , α i ) } on which Algorithm 3 is called repetitively until the list becomes empty.
Revised: Algorithm 2 then maintains a list L of pairs { ( T i , α i ) } on which Algorithm 3 is called repetitively until the list becomes empty.

Original: Theorem 14 implies that when
Revised: Theorem 14 implies that when

Original: Algorithm 3 is called on some T i ∈
Revised: Algorithm 3 is called on some T i ∈

Original: L which is α i -good, if a list of pairs { ( T j , α j ) } is returned, then at least one of { T j } is α j -good ( α j > α i ).
Revised: L which is α i -good, if a list of pairs { ( T j , α j ) } is returned, then at least one of { T j } is α j -good ( α j > α i ).

Original: This ensures that there always exists an α 2 -good subset T i in list L , except that a leaf node has been created for this branch and the empirical mean of an α 2 -good data set is returnd.
Revised: This ensures that there always exists an α 2 -good subset T i in list L , except that a leaf node has been created for this branch and the empirical mean of an α 2 -good data set is returnd.

Original: We then argue that Algorithm 2 eventually returns an estimated mean at the branch that includes only α 2 -good subsets.
Revised: We then argue that Algorithm 2 eventually returns an estimated mean at the branch that includes only α 2 -good subsets.

Original: Since the subsets are α 2 -good, ANS never equals to NO .
Revised: Since the subsets are α 2 -good, ANS never equals to NO .

Original: In addition, the branch will not create child nodes forever: note that the true multiﬁltering step is in
Revised: In addition, the branch will not create child nodes forever: note that the true multiﬁltering step is in

Original: B ASIC MF, and both Step 8 and 12 reduce the subset size | T i | by at least 1 ; since α i is non-decreasing, the algorithm can’t remove only inliers; by Deﬁnition 10, | S G ∩ T i | ≥ (1 − α i / 6) | S G | ≥ 12 | S G | .
Revised: B ASIC MF, and both Step 8 and 12 reduce the subset size | T i | by at least 1 ; since α i is non-decreasing, the algorithm cannot remove only inliers; by Deﬁnition 10, | S G ∩ T i | ≥ (1 − α i / 6) | S G | ≥ 12 | S G | .

Original: Therefore, the algorithm must return an estimated mean when there is no outliers to ﬁlter.
Revised: Therefore, the algorithm must return an estimated mean when there is no outliers to ﬁlter.

Original: We then bound the list size of the returned list of estimated means.
Revised: We then bound the list size of the returned list of estimated means.

Original: Since during the process of multiﬁltering, (cid:80) i
Revised: Since during the process of multiﬁltering, (cid:80) i

Original: α − 2 i is non-increasing, we have that (cid:80) |L| i =1 α − 2 i ≤ 12 α · α − 2 at any point of
Revised: α − 2 i is non-increasing, we have that (cid:80) |L| i =1 α − 2 i ≤ 12 α · α − 2 at any point of

Original: Algorithm 2.
Revised: Algorithm 2.

Original: In addition, α i ≤ 1 , ∀ i , meaning that the list size will never be larger than O ( α − 3 ) .
Revised: In addition, α i ≤ 1 , ∀ i , meaning that the list size will never be larger than O ( α − 3 ) .

Original: So does the size of M .
Revised: So does the size of M .

Original: Then, by applying L IST R EDUCTION on M with | M | ≤ O ( α − 3 ) , the list size can be reduced to O ( α − 1 ) in view of Proposition 37.
Revised: Then, by applying L IST R EDUCTION on M with | M | ≤ O ( α − 3 ) , the list size can be reduced to O ( α − 1 ) in view of Proposition 37.

Original: Finally, note that C LUSTER runs in time O (cid:0) poly( | T | , d ) (cid:1) , A TTRIBUTE - EFFICIENT -M ULTIFILTER runs in time O (cid:0) poly( | T | , d (cid:96) ) (cid:1) in view of Theorem 14, and L IST R EDUCTION runs in time
Revised: Finally, note that C LUSTER runs in time O (cid:0) poly( | T | , d ) (cid:1) , A TTRIBUTE - EFFICIENT -M ULTIFILTER runs in time O (cid:0) poly( | T | , d (cid:96) ) (cid:1) in view of Theorem 14, and L IST R EDUCTION runs in time

Original: O (cid:0) poly( | T | , d ) (cid:1) .
Revised: O (cid:0) poly( | T | , d ) (cid:1) .

Original: Moreover, there are at most O ( | T | /α 3 ) number of calls to A TTRIBUTE - EFFICIENTM ULTIFILTER , and only one call to C LUSTER and one call to L IST R EDUCTION , we conclude that the time complexity of Algorithm 2 is O (cid:0) poly (cid:0) | T | , d (cid:96) , 1 α (cid:1)(cid:1) .
Revised: Moreover, there are at most O ( | T | /α 3 ) number of calls to A TTRIBUTE - EFFICIENTM ULTIFILTER , and only one call to C LUSTER and one call to L IST R EDUCTION , we conclude that the time complexity of Algorithm 2 is O (cid:0) poly (cid:0) | T | , d (cid:96) , 1 α (cid:1)(cid:1) .

Original: D Omitted Algorithms
Revised: D Omitted Algorithms

Original: In the following, we present the omitted algorithms.
Revised: In the following, we present the omitted algorithms.

Original: In particular, M ULTILINEAR MF (Algorithms 6) is an important component of H ARMONIC MF, for which we tailor the algorithms in [DKS18b] to our sparse setting.
Revised: In particular, M ULTILINEAR MF (Algorithms 6) is an important component of H ARMONIC MF, for which we tailor the algorithms in [DKS18b] to our sparse setting.

Original: M ULTILINEAR MF will further invoke D EGREE 2H OMOGENEOUS (Algorithm 7).
Revised: M ULTILINEAR MF will further invoke D EGREE 2H OMOGENEOUS (Algorithm 7).

Original: Algorithm 8, due to [DKS18b], is the black-box list reduction approach that was invoked in
Revised: Algorithm 8, due to [DKS18b], is the black-box list reduction approach that was invoked in

Original: Algorithm 2.
Revised: Algorithm 2.

Original: D. M ULTILINEAR MF
Revised: D. M ULTILINEAR MF

Original: We introduce useful facts about multilinear polynomial here.
Revised: We introduce useful facts about multilinear polynomial here.

Original: For d, l ∈ N , a polynomial p ( x 1 , . . .
Revised: For d, l ∈ N , a polynomial p ( x 1 , . . .

Original: , x l ) : R dl → R , where x i ∈ R d , is called multilinear if it is linear in each of its l arguments, i.e. if holds that p ( a · x 1 + b · x (cid:48) 1 , x 2 , . . .
Revised: , x l ) : R dl → R , where x i ∈ R d , is called multilinear if it is linear in each of its l arguments, i.e. if holds that p ( a · x 1 + b · x (cid:48) 1 , x 2 , . . .

Original: , x l ) = a · p ( x 1 , x 2 , . . .
Revised: , x l ) = a · p ( x 1 , x 2 , . . .

Original: , x l ) + b · p ( x 1 , . . .
Revised: , x l ) + b · p ( x 1 , . . .

Original: , x l ) , for all a, b ∈ R and x i , x (cid:48)
Revised: , x l ) , for all a, b ∈ R and x i , x (cid:48)

Original: i ∈ R d , and similarly for all the other arguments.
Revised: i ∈ R d , and similarly for all the other arguments.

Original: Moreover, a polynomial p is called symmetric if p ( x 1 , . . .
Revised: Moreover, a polynomial p is called symmetric if p ( x 1 , . . .

Original: , x l ) = p ( x π (1) , . . .
Revised: , x l ) = p ( x π (1) , . . .

Original: , x π ( l ) ) for any permutation π : [ l ] → [ l ] .
Revised: , x π ( l ) ) for any permutation π : [ l ] → [ l ] .

Original: Any degree- l multilinear polynomial p : R dl
Revised: Any degree- l multilinear polynomial p : R dl

Original: → R can be expressed as A ( x 1 , . . .
Revised: → R can be expressed as A ( x 1 , . . .

Original: , x l ) for an order- l tensor
Revised: , x l ) for an order- l tensor

Original: A over R d .
Revised: A over R d .

Original: Moreover, A is symmetric if p is symmetric.
Revised: Moreover, A is symmetric if p is symmetric.

Original: Require: A multiset of samples T ⊂ R d , noise rate α ∈ (0 , 1 / 2] , failure probability τ ∈ (0 , 1) , a degree- l multilinear polynomialV ( x 1 , . . .
Revised: Require: A multiset of samples T ⊂ R d , noise rate α ∈ (0 , 1 / 2] , failure probability τ ∈ (0 , 1) , a degree - l multilinear polynomial

Original:  , x l ) over R dl with (cid:107) V (cid:107) 2 ≤ 1 , where V isthe outer product of l number of ψ -sparse vectors.
Revised: V ( x 1 , . . . , x l ) over R dl with (cid:107) V (cid:107) 2 ≤ 1 , where V is

Revised: the

Revised: outer product of l number of ψ -sparse vectors.

Original: 1: If l = 1 , run B ASIC MF on V ( x − µ T ) , and return its output.
Revised: 1: If l = 1 , run B ASIC MF on V ( x − µ T ) , and return its output.

Original: 2: Compute the quadratic polynomial q ( x ) = (cid:107) V ( x − µ T ) (cid:107) 22 , where x ∈ R d and V x is an order- ( l − 1) tensor with ( V x ) i 2 ,...,i l = (cid:80) i 1 x i 1 V i 1 ,...,i l . 3:
Revised: 2: Compute the quadratic polynomial q ( x ) = (cid:107) V ( x − µ T ) (cid:107) 22 , where x ∈ R d and V x is an order- ( l − 1) tensor with ( V x ) i 2 ,...,i l = (cid:80) i 1 x i 1 V i 1 ,...,i l . 3:

Original: Run D EGREE 2H OMOGENEOUS on q ( x ) .
Revised: Run D EGREE 2H OMOGENEOUS on q ( x ) .

Original: If it returns NO or a list { T i , α i } , then return the same result.
Revised: If it returns NO or a list { T i , α i } , then return the same result.

Original: 4: Sample a set Φ of m = 200 · α − 1 log(4 /τ ) instances uniformly at random from T . 5: ∀ x ∈ Φ , let V x = 1 √ q ( x ) · V ( x − µ T ) .
Revised: 4: Sample a set Φ of m = 200 · α − 1 log(4 /τ ) instances uniformly at random from T . 5: ∀ x ∈ Φ , let V x = 1 √ q ( x ) · V ( x − µ T ) .

Original: ANS ← M ULTILINEAR MF on ( T, V x, l − 1 , α, τ/ 2) .
Revised: ANS ← M ULTILINEAR MF on ( T, V x, l − 1 , α, τ/ 2) .

Original: If it returns NO or a list { ( T i , α i ) } , then return the same result.
Revised: If it returns NO or a list { ( T i , α i ) } , then return the same result.

Original: 6: Otherwise, return TBD .
Revised: 6: Otherwise, return TBD .

Original: Require: A multiset of samples T ⊂ R d , noise rate α ∈ (0 , 1 / 2] , failure probability τ ∈ (0 , 1) , homogeneous polynomial
Revised: Require: A multiset of samples T ⊂ R d , noise rate α ∈ (0 , 1 / 2] , failure probability τ ∈ (0 , 1) , homogeneous polynomial

Original: A is a d × d matrix
Revised: A is a d × d matrix

Original: (cid:107)
Revised: (cid:107)

Original: A
Revised: A

Original: ∗ ≤ 1 .
Revised: ∗ ≤ 1 .

Original: The M ULTILINEAR MF works in the following way: Given degree- l multilinear polynomial V ( x 1 − µ T , . . .
Revised: The M ULTILINEAR MF works in the following way: Given degree- l multilinear polynomial V ( x 1 − µ T , . . .

Original: , x l − µ T ) , where V is an order- l symmetric tensor and x i ’s are l number of independent variables.
Revised: , x l − µ T ) , where V is an order- l symmetric tensor and x i ’s are l number of independent variables.

Original: The goal is to show that the polynomial has small absolute expectation over l number of i.i.d.
Revised: The goal is to show that the polynomial has small absolute expectation over l number of i.i.d.

Original: draws from G ∼ N ( µ, I d ) if the algorithm does not ﬁlter any samples and returns “ TBD ”; and otherwise, the algorithm multiﬁlters correctly.
Revised: draws from G ∼ N ( µ, I d ) if the algorithm does not ﬁlter any samples and returns “ TBD ”; and otherwise, the algorithm multiﬁlters correctly.

Original: Since all subroutine of M ULTILINEAR MF multiﬁlter the data set by calling B ASIC MF on linear polynomials, we know that if it returns “ NO ” or a list of pairs { ( T i , α i ) } , the correctness is guaranteed by Theorem 18.
Revised: Since all subroutine of M ULTILINEAR MF multiﬁlter the data set by calling B ASIC MF on linear polynomials, we know that if it returns “ NO ” or a list of pairs { ( T i , α i ) } , the correctness is guaranteed by Theorem 18.

Original: It remains to bound the expected value of the multilinear polynomial when all subroutines return “ TBD ” and T is α -good.
Revised: It remains to bound the expected value of the multilinear polynomial when all subroutines return “ TBD ” and T is α -good.

Original: The idea is to sub-sample a large enough sample Φ from T .
Revised: The idea is to sub-sample a large enough sample Φ from T .

Original: If T is α -good, then with sufﬁciently high probability, ∃ x ∈ Φ that is from G .
Revised: If T is α -good, then with sufﬁciently high probability, ∃ x ∈ Φ that is from G .

Original: By recursively doing this, with sufﬁciently high probability, we construct a multilinear polynomial V ( G 1 − µ T , G 2 − µ T , . . .
Revised: By recursively doing this, with sufﬁciently high probability, we construct a multilinear polynomial V ( G 1 − µ T , G 2 − µ T , . . .

Original: , G l − µ T ) , the expectation of which is what we concerned about.
Revised: , G l − µ T ) , the expectation of which is what we concerned about.

Original: The upper bound is then shown by induction.
Revised: The upper bound is then shown by induction.

Original: When the polynomial is linear, | E [ V x l − 1 ( G − µ T )] | ≤ O ( (cid:112) 1 + log(1 /α ) · log(2 + log(1 /α )) .
Revised: When the polynomial is linear, | E [ V x l − 1 ( G − µ T )] | ≤ O ( (cid:112) 1 + log(1 /α ) · log(2 + log(1 /α )) .

Original: Here, we use V x i to denote taking i times of inner product between tensor V and a vector x .
Revised: Here, we use V x i to denote taking i times of inner product between tensor V and a vector x .

Original: Note that x can be different in each time of the inner product.
Revised: Note that x can be different in each time of the inner product.

Original: Then, without loss of generality, assume that for order- ( l − 1) tensor V x , | E [ V x ( G 1 − µ T , . .
Revised: Then, without loss of generality, assume that for order- ( l − 1) tensor V x , | E [ V x ( G 1 − µ T , . .

Original: , G l − 1 − µ T )] | ≤ f ( l − 1 , α ) , we can show that | E [ V ( G 1 − µ T , . . .
Revised: , G l − 1 − µ T )] | ≤ f ( l − 1 , α ) , we can show that | E [ V ( G 1 − µ T , . . .

Original: , G l − µ T )] | ≤ f ( l, α ) , provided that S G is sufﬁciently epresentative with respect to G on any linear polynomial V x l − 1 ( x − µ T ) and any quadratic polynomial q ( x ) = (cid:107)
Revised: , G l − µ T )] | ≤ f ( l, α ) , provided that S G is sufﬁciently epresentative with respect to G on any linear polynomial V x l − 1 ( x − µ T ) and any quadratic polynomial q ( x ) = (cid:107)

Original: V x i ( x − µ T ) (cid:107) 22 , ∀ i ∈
Revised: V x i ( x − µ T ) (cid:107) 22 , ∀ i ∈

Original: [ l ] .
Revised: [ l ] .

Original: In this analysis, the only difference between our setting and that of [DKS18b] is the deﬁnition of representative set S G (Deﬁnition 8).
Revised: In this analysis, the only difference between our setting and that of [DKS18b] is the deﬁnition of representative set S G (Deﬁnition 8).

Original: Fortunately, since all polynomials in our algorithm apply to at most ψ = 2 (cid:96)k 2 (cid:96) coordinates, the linear polynomials must be in P ( R d , 1 , 2 (cid:96)k 2 (cid:96) , 2 (cid:96)k 2 (cid:96) ) , and the quadratic polynomials must be in
Revised: Fortunately, since all polynomials in our algorithm apply to at most ψ = 2 (cid:96)k 2 (cid:96) coordinates, the linear polynomials must be in P ( R d , 1 , 2 (cid:96)k 2 (cid:96) , 2 (cid:96)k 2 (cid:96) ) , and the quadratic polynomials must be in

Original: P ( R d , 2 , 4 (cid:96) 2 k 4 (cid:96) , 2 (cid:96)k 2 (cid:96) ) .
Revised: P ( R d , 2 , 4 (cid:96) 2 k 4 (cid:96) , 2 (cid:96)k 2 (cid:96) ) .

Original: Therefore, our deﬁnition of representative set sufﬁcies.
Revised: Therefore, our deﬁnition of representative set sufﬁcies.

Original: The proof follows the same pipeline as that of Lemma 3.27 in [DKS18b].
Revised: The proof follows the same pipeline as that of Lemma 3.27 in [DKS18b].

Original: As a result, it can be shown that f ( l, α ) =
Revised: As a result, it can be shown that f ( l, α ) =

Original: f ( l − 1 , α ) · O ( (cid:112) 1 + log(1 /α ) · log(2+log(1 /α )) , which renders | E [ V ( G 1 − µ T , . .
Revised: f ( l − 1 , α ) · O ( (cid:112) 1 + log(1 /α ) · log(2+log(1 /α )) , which renders | E [ V ( G 1 − µ T , . .

Original: , G l − µ T )] | ≤
Revised: , G l − µ T )] | ≤

Original: O (cid:0) (1 + log(1 /α )) l/ 2 · log l (2 + log(1 /α ) (cid:1) .
Revised: O (cid:0) (1 + log(1 /α )) l/ 2 · log l (2 + log(1 /α ) (cid:1) .

Original: Deﬁnition 35 (Multiﬁlter condition) .
Revised: Deﬁnition 35 (Multiﬁlter condition) .

Original: We say that a list of pairs { ( T i , α i ) } , where T i ⊂ T and α i ∈ (0 , 1) , satisﬁes the multiﬁlter condition for ( T, α ) if the following hold:
Revised: We say that a list of pairs { ( T i , α i ) } , where T i ⊂ T and α i ∈ (0 , 1) , satisﬁes the multiﬁlter condition for ( T, α ) if the following hold:

Original: (cid:80) i 1 α 2 i ≤ 1 α 2 , and 2.
Revised: (cid:80) i 1 α 2 i ≤ 1 α 2 , and 2.

Original: If T is α -good, then at least one T i is α i -good.
Revised: If T is α -good, then at least one T i is α i -good.

Original: Lemma 36 (M ULTILINEAR MF, Lemma 3.27 of [DKS18b]) .
Revised: Lemma 36 (M ULTILINEAR MF, Lemma 3.27 of [DKS18b]) .

Original: Given α ∈ (0 , 12 ] and τ ∈ (0 , 1) , let T be the input sample set, and a degree- l multilinear polynomial V ( x 1 , . . .
Revised: Given α ∈ (0 , 12 ] and τ ∈ (0 , 1) , let T be the input sample set, and a degree- l multilinear polynomial V ( x 1 , . . .

Original: , x l ) over R dl with (cid:107) V (cid:107) 2 = 1 .
Revised: , x l ) over R dl with (cid:107) V (cid:107) 2 = 1 .

Original: Algorithm 6 returns one of the following with guarantees: (1) TBD , and we have that, if T is α -good, then with probability 1 − τ , | E [ V ( G 1 − µ T , . . .
Revised: Algorithm 6 returns one of the following with guarantees: (1) TBD , and we have that, if T is α -good, then with probability 1 − τ , | E [ V ( G 1 − µ T , . . .

Original: , G l − µ T )]
Revised: , G l − µ T )]

Original: | =
Revised: | =

Original: O (cid:0) (1+log( 1 α ) log 2 (2+log( 1 α ))) (cid:1) l/ 2 , where G i are independent copies of G .
Revised: O (cid:0) (1+log( 1 α ) log 2 (2+log( 1 α ))) (cid:1) l/ 2 , where G i are independent copies of G .

Original: NO , then T is not α -good.
Revised: NO , then T is not α -good.

Original: (3) A list of pairs { ( T i , α i ) } ,
Revised: (3) A list of pairs { ( T i , α i ) } ,

Original: T i ⊂ T , satisfying the multiﬁlter condition for ( T, α ) .
Revised: T i ⊂ T , satisfying the multiﬁlter condition for ( T, α ) .

Original: D. L IST R EDUCTION
Revised: D. L IST R EDUCTION

Original: Proposition 37 (L IST R EDUCTION , Proposition B.1 of [DKS18b]) .
Revised: Proposition 37 (L IST R EDUCTION , Proposition B.1 of [DKS18b]) .

Original: Fix α, β, δ, t > 0 and let µ ∗ ∈ R
Revised: Fix α, β, δ, t > 0 and let µ ∗ ∈ R

Original: d be ﬁnite, and let S ⊆ T be so that (i) | S | / | T | ≥ α , and (ii) for all unit vectors v ∈ R d , we have
Revised: d be ﬁnite, and let S ⊆ T be so that (i) | S | / | T | ≥ α , and (ii) for all unit vectors v ∈ R d , we have

Original: Pr[ v · ( S − µ ∗ ) > t ] < δ .
Revised: Pr[ v · ( S − µ ∗ ) > t ] < δ .

Original: Then, given M = { µ 1 , . . . , µ n } ⊂ R d so that δn =
Revised: Then, given M = { µ 1 , . . . , µ n } ⊂ R d so that δn =

Original: o
Revised: o

Original: (1) and there
Revised: (1) and there

Original: [Table] [Table]
Revised: [Table] [Table]

Original: is some i so that (cid:107) µ i − µ ∗ (cid:107) 2 ≤ β for some µ i ∈ M , Algorithm 8 outputs M (cid:48) ⊆
Revised: is some i so that (cid:107) µ i − µ ∗ (cid:107) 2 ≤ β for some µ i ∈ M , Algorithm 8 outputs M (cid:48) ⊆

Original: M so that | M (cid:48) | ≤ 1 α (1 + O ( δn )) and (cid:107) µ (cid:48) − µ ∗ (cid:107) 2 ≤ 3( β + t ) for some µ (cid:48)
Revised: M so that | M (cid:48) | ≤ 1 α (1 + O ( δn )) and (cid:107) µ (cid:48) − µ ∗ (cid:107) 2 ≤ 3( β + t ) for some µ (cid:48)

Original: ∈ M (cid:48) .
Revised: ∈ M (cid:48) .

Original: Remark 38.
Revised: Remark 38.

Original: Under the setting of Algorithm 8, we have that n =
Revised: Under the setting of Algorithm 8, we have that n =

Original: O ( α − 3 ) .
Revised: O ( α − 3 ) .

Original: This combined with the parameter settings in L IST R EDUCTION shows that the size of M (cid:48) is O (1 /α ) and there is at least one µ i ∈ M (cid:48) that has comparable error guarantee to those in M .
Revised: This combined with the parameter settings in L IST R EDUCTION shows that the size of M (cid:48) is O (1 /α ) and there is at least one µ i ∈ M (cid:48) that has comparable error guarantee to those in M .

Original: E Useful Lemmas
Revised: E Useful Lemmas

Original: Lemma 39 (Lemma 3.2 of [CDK + 21]) .
Revised: Lemma 39 (Lemma 3.2 of [CDK + 21]) .

Original: Fix two vectors x, y with (cid:107) x (cid:107) 0 ≤ k and (cid:107)
Revised: Fix two vectors x, y with (cid:107) x (cid:107) 0 ≤ k and (cid:107)

Original: trim k ( x − y )
Revised: trim k ( x − y )

Original: (cid:107) 2 ≤ δ .
Revised: (cid:107) 2 ≤ δ .

Original: We have that (cid:107) x − trim k ( y ) (cid:107) 2 ≤ √ 5 δ .
Revised: We have that (cid:107) x − trim k ( y ) (cid:107) 2 ≤ √ 5 δ .

Original: Lemma 40 (degree- l Chernoff bound, Fact 2.8 of [DKS18b]) .
Revised: Lemma 40 (degree- l Chernoff bound, Fact 2.8 of [DKS18b]) .

Original: Let G ∼ N ( µ, I d ) , µ ∈ R d .
Revised: Let G ∼ N ( µ, I d ) , µ ∈ R d .

Original: Let p : R d → R be a degree- l polynomial.
Revised: Let p : R d → R be a degree- l polynomial.

Original: For any t > 0 , we have that Pr (cid:2)
Revised: For any t > 0 , we have that Pr (cid:2)

Original: | p ( G ) − E
Revised: | p ( G ) − E

Original: [ p ( G )]
Revised: [ p ( G )]

Original: | ≥ t · (cid:112) Var[ p ( G )] (cid:3) ≤ exp (cid:0) − Ω( t 2 /l ) (cid:1) .
Revised: | ≥ t · (cid:112) Var[ p ( G )] (cid:3) ≤ exp (cid:0) − Ω( t 2 /l ) (cid:1) .

Original: Lemma 41 (Harmonic and multilinear polynomials, Lemma 3.24 of [DKS18b]) .
Revised: Lemma 41 (Harmonic and multilinear polynomials, Lemma 3.24 of [DKS18b]) .

Original: Let
Revised: Let

Original: X, X (1) , . . .
Revised: X, X (1) , . . .

Original: , X ( (cid:96) ) be i.i.d random variables distributed as N ( µ, I ) for some µ ∈ R d .
Revised: , X ( (cid:96) ) be i.i.d random variables distributed as N ( µ, I ) for some µ ∈ R d .

Original: Then, for any symmetric matrix A , we have
Revised: Then, for any symmetric matrix A , we have

Original: [Equation]
Revised: [Equation]

Original: , and
Revised: , and

Original: [Equation]
Revised: [Equation]

Original: where B ( (cid:96) (cid:48) ) is the order- 2 (cid:96) (cid:48) tensor with
Revised: where B ( (cid:96) (cid:48) ) is the order- 2 (cid:96) (cid:48) tensor with

Original: B ( (cid:96) (cid:48) )
Revised: B ( (cid:96) (cid:48) )

Original: i 1 ,...,
Revised: i 1 ,...,

Original: i
Revised: i

Original: [Table]
Revised: [Table]