  We study the problem of agnostically learning halfspaces under the Gaussian
distribution. Our main result is the {\em first proper} learning algorithm for
this problem whose sample complexity and computational complexity qualitatively
match those of the best known improper agnostic learner. Building on this
result, we also obtain the first proper polynomial-time approximation scheme
(PTAS) for agnostically learning homogeneous halfspaces. Our techniques
naturally extend to agnostically learning linear models with respect to other
non-linear activations, yielding in particular the first proper agnostic
algorithm for ReLU regression.
