  This paper serves as a survey of recent advances in large margin training and
its theoretical foundations, mostly for (nonlinear) deep neural networks (DNNs)
that are probably the most prominent machine learning models for large-scale
data in the community over the past decade. We generalize the formulation of
classification margins from classical research to latest DNNs, summarize
theoretical connections between the margin, network generalization, and
robustness, and introduce recent efforts in enlarging the margins for DNNs
comprehensively. Since the viewpoint of different methods is discrepant, we
categorize them into groups for ease of comparison and discussion in the paper.
Hopefully, our discussions and overview inspire new research work in the
community that aim to improve the performance of DNNs, and we also point to
directions where the large margin principle can be verified to provide
theoretical evidence why certain regularizations for DNNs function well in
practice. We managed to shorten the paper such that the crucial spirit of large
margin learning and related methods are better emphasized.
