This paper is concerned with computationally efficient learning of homogeneous sparse halfspaces in %DIFDELCMD < \Rd %%%  under noise. Though recent works have established attribute-efficient learning algorithms under various types of label noise (e.g. bounded noise), it remains an open question of when and how s-sparse halfspaces can be efficiently learned under the challenging {\em malicious noise} model, where an adversary may corrupt both the unlabeled data distribution and the labels. We answer this question in the affirmative by designing a computationally efficient algorithm with near-optimal label complexity  O\big( s \log^3 d \cdot \log ^4\frac{1{\epsilon}} {\epsilon}}} \big) and noise tolerance \eta = \Omega(\epsilon), where \epsilon \in (0, 1) is the target error rate. Our main techniques include attribute-efficient paradigms for instance reweighting and for empirical risk minimization, and a new analysis of uniform concentration for unbounded data~--~all of them crucially take the structure of the underlying halfspace into account. To the best of our knowledge, this is the first near-optimal result in the setting. As a byproduct of our analysis, we resolve a long-standing problem in statistics and machine learning: we show that a global optimum of  sparse principal component analysis can be found in polynomial time without any statistical assumption on the data . This result might be of independent interest to both communities .
This paper is concerned with computationally efficient learning of homogeneous sparse halfspaces in %DIFDELCMD < \Rd %%% \mathbb{R under noise. Though recent works have established attribute-efficient learning algorithms under various types of label noise (e.g. bounded noise), it remains an open question of when and how s-sparse halfspaces can be efficiently learned under the challenging {\em malicious noise} model, where an adversary may corrupt both the unlabeled data distribution and the labels. We answer this question in the affirmative by designing a computationally efficient algorithm with near-optimal label complexity of O\big( s \log^3 d \cdot \log {\epsilon}} ^3\frac{1{\epsilon}}} \big) and noise tolerance \eta = \Omega(\epsilon), where \epsilon \in (0, 1) is the target error rate. Our main techniques include attribute-efficient paradigms for instance reweighting and for empirical risk minimization, and a new analysis of uniform concentration for unbounded data~--~all of them crucially take the structure of the underlying halfspace into account. To the best of our knowledge, this is the first near-optimal result in the setting. As a byproduct of our analysis, we show that by applying a result of Bienstock and Michalka (2014), a global optimum of a form of sparse principal component analysis can be found in polynomial time without any statistical assumption on the data , which has not been recognized before and might be of independent interest .
A
\mathbb{R
A
of
D
^4\frac{1
A
^3\frac{1
R
resolve a long-standing problem in statistics and machine learning: we show that a
show that by applying a result of Bienstock and Michalka (2014), a
A
a form of
R
. This result
, which has not been recognized before and
D
to both communities