This paper is concerned with computationally efficient learning of homogeneous sparse halfspaces in R^d under noise. Though recent works have established attribute-efficient learning algorithms under various types of label noise (e.g. bounded noise), it remains an open question of when and how s-sparse halfspaces can be efficiently learned under the challenging %DIFDELCMD < {\em %%% malicious noise model, where an adversary may corrupt both the unlabeled data distribution and the labels. We answer this question in the affirmative by designing a computationally efficient  algorithm with near-optimal label complexity of O\big({s \log ^3 d \cdot \log^3\frac{1{\epsilon}}  } \big) and noise tolerance \eta = \Omega(\epsilon), where \epsilon \in (0, 1) is the target error rate . OurO\big({} \big)  main techniques include attribute-efficient paradigms for instance reweighting and for empirical risk minimization, and a new analysis of uniform concentration for unbounded data ~ -- ~ all of them crucially take the structure of the underlying halfspace into account . To the best of our knowledge, this is the first near-optimal result in the setting. As a byproduct of our analysis, we show that by applying a result of Bienstock and Michalka (2014), a global optimum of a form of sparse principal component analysis can be found in polynomial time without any statistical assumption on the data, which has not been recognized before and might be of independent interest .
This paper is concerned with computationally efficient learning of homogeneous sparse halfspaces in R^d under noise. Though recent works have established attribute-efficient learning algorithms under various types of label noise (e.g. bounded noise), it remains an open question when and how s-sparse halfspaces can be efficiently learned under the challenging %DIFDELCMD < {\em %%% malicious noise model, where an adversary may corrupt both the unlabeled examples and the labels. We answer this question in the affirmative by designing a computationally efficient active learning algorithm with near-optimal label complexity of O\big({s \log {\epsilon}} ^4 \frac d \epsilon } \big) and noise tolerance \eta = \Omega(\epsilon), where \epsilon \in (0, 1) is the target error rate , under the assumption that the distribution over (uncorrupted) unlabeled examples is isotropic log-concave. Our algorithm can be straightforwardly tailored to the passive learning setting, and we show that the sample complexity isO\big({\frac 1 \epsilon s^2 \log^5 d} \big) which also enjoys the attribute efficiency. Our main techniques include attribute-efficient paradigms for instance reweighting and for empirical risk minimization, and a new analysis of uniform concentration for unbounded data -- all of them crucially take the structure of the underlying halfspace into account .
D
of
R
malicious noise
malicious noise
R
data distribution
examples
A
active learning
D
^3 d \cdot \log^3\frac{1
A
^4 \frac d \epsilon
R
. Our
, under the assumption that the distribution over (uncorrupted) unlabeled examples is isotropic log-concave. Our algorithm can be straightforwardly tailored to the passive learning setting, and we show that the sample complexity is
A
\frac 1 \epsilon s^2 \log^5 d
A
which also enjoys the attribute efficiency. Our
D
~
D
~
D
. To the best of our knowledge, this is the first near-optimal result in the setting. As a byproduct of our analysis, we show that by applying a result of Bienstock and Michalka (2014), a global optimum of a form of sparse principal component analysis can be found in polynomial time without any statistical assumption on the data, which has not been recognized before and might be of independent interest