  Adversarially robust learning aims to design algorithms that are robust to
small adversarial perturbations on input variables. Beyond the existing studies
on the predictive performance to adversarial samples, our goal is to understand
statistical properties of adversarially robust estimates and analyze
adversarial risk in the setup of linear regression models. By discovering the
statistical minimax rate of convergence of adversarially robust estimators, we
emphasize the importance of incorporating model information, e.g., sparsity, in
adversarially robust learning. Further, we reveal an explicit connection of
adversarial and standard estimates, and propose a straightforward two-stage
adversarial learning framework, which facilitates to utilize model structure
information to improve adversarial robustness. In theory, the consistency of
the adversarially robust estimator is proven and its Bahadur representation is
also developed for the statistical inference purpose. The proposed estimator
converges in a sharp rate under either low-dimensional or sparse scenario.
Moreover, our theory confirms two phenomena in adversarially robust learning:
adversarial robustness hurts generalization, and unlabeled data help improve
the generalization. In the end, we conduct numerical simulations to verify our
theory.
