# Double Additive Margin Softmax Loss for Face Recognition
## 3 Double Additive Margin Softmax Loss
### Geometric Interpretation  
Our double additive margin has a more explicit geometric interpretation on the hypersphere manifold. To simplify the geometric interpretation, we project the features onto two dimensional space and discuss the binary classification case on the hypersphere manifold where there are only \(\mathbf{w}_{1}\) and \(\mathbf{w}_{2}\), and \(\|\mathbf{w}_{1}\|=\|\mathbf{w}_{2}\|=1\). Thus, the classification performance depends totally on the angles \(\theta_{1}\) between \(\mathbf{f}\) and \(\mathbf{w}_{1}\), and \(\theta_{2}\) between \(\mathbf{f}\) and \(\mathbf{w}_{2}\).  
**Classification Stringency**. In order to correctly classify the learned feature vector \(\mathbf{f}\) as class 1, the traditional softmax loss requires \(\theta_{1}<\theta_{2}\) which easily has \(\theta_{1}-\theta_{2}<0\), the AM-Softmax loss requires \(cos(\theta_{1})-m<cos(\theta_{2})\) which can be rewritten as \(cos(\theta_{1})-cos(\theta_{2})<m\), while our DAM-Softmax loss needs \(cos(\theta_{1})-m<cos(\theta_{2})+m\) which can be reformulated into \(cos(\theta_{1})-cos(\theta_{2})<2m\). It is obvious to see that the DAM-Softmax loss is more stringent than the orginal softmax loss and AM-Softmax loss for satisfying the corresponding classification criteria.  
**Classification Boundary**. In Figure 1, we draw a schematic diagram to show the classification boundary of the classical softmax loss, AM-Softmax loss and the proposed DAM-Softmax loss. The classification boundary of the traditional softmax loss is denoted as the vector \(\mathbf{p}_{0}\). In this case, we have \(\mathbf{w}_{1}^{\top}\mathbf{p}_{0}=\mathbf{w}_{2}^{\top}\mathbf{p}_{0}\) at the decision boundary \(\mathbf{p}_{0}\) ( \(\mathbf{w}_{1}\in class1,\mathbf{w}_{2}\in class2\) ). For the AM-Softmax, the boundary becomes a marginal region instead of a single vector. At the new boundary \(\mathbf{p}_{1}\) for class 1, one has \(\mathbf{w}_{1}^{\top}\mathbf{p}_{1}-m=\mathbf{w}_{2}^{\top}\mathbf{p}_{1}\), which gives \(m=(\mathbf{w}_{1}^{\top}-\mathbf{w}_{2}^{\top})\mathbf{p}_{1}=cos(\theta_{ \mathbf{w}_{1},\mathbf{p}_{1}})-cos(\theta_{\mathbf{w}_{2},\mathbf{p}_{1}})\). If we further assume that all the classes have the same intra-class variance and the boundary for class 2 is at \(\mathbf{p}_{2}\), we can get \(cos(\theta_{\mathbf{w}_{2},\mathbf{p}_{1}})=cos(\theta_{\mathbf{w}_{1},\mathbf{ p}_{2}})\). Thus, \(m=cos(\theta_{\mathbf{w}_{1},\mathbf{p}_{1}})-cos(\theta_{\mathbf{w}_{1}, \mathbf{p}_{2}})\), which is the difference of the cosine scores for class 1 between the two sides of the margin region. For our DAM-Softmax loss, the boundary becomes a wider marginal region than the one of AM-Softmax loss. At the new boundary \(\mathbf{p}_{3}\) for class 1, one has \(\mathbf{w}_{3}^{\top}\mathbf{p}_{3}-m=\mathbf{w}_{4}^{\top}\mathbf{p}_{3}+m\) ( \(\mathbf{w}_{3}\in class1,\mathbf{w}_{4}\in class2\) ), which gives \(2m=(\mathbf{w}_{3}^{\top}-\mathbf{w}_{4}^{\top})\mathbf{p}_{3}=cos(\theta_{ \mathbf{w}_{3},\mathbf{p}_{3}})-cos(\theta_{\mathbf{w}_{4},\mathbf{p}_{3}})\). If we further assume that all the classes have the same intra-class variance and the boundary for class 2 is