# Additive Orthant Loss for Deep Face Recognition
## 3 Orthant Loss
To avoid a premature decision of weight vectors, we apply the Orthant loss after the learning progresses sufficiently. In this paper, we apply the Orthant loss from 20,000 th iteration. Figure 3 displays 2D and 3D plots of features for the ArcFace and the ArcOrthFace, respectively. We use the LResNet18E-IR [33], where \(a=2\), \(r=30\), and \(N=128\). In addition, we use \(s_{y_{i}}=s_{j}=4\), \(m_{\theta}=0.1\) and \(m_{c}=1/\sqrt{2}\) for 2D features of Figure 3a,b and \(s_{y_{i}}=s_{j}=8\), \(m_{\theta}=0.2\) and \(m_{c}=1/\sqrt{3}\) for 3D features of Figure 3c,d, respectively. For training and testing, we use a total of 2907 images of top four classes and a total of 4186 images of the top six classes sorted by the number of images in the CASIA-Webface dataset [47] for 2D and 3D features, respectively. We assume that there are \(d=4\) and \(d=6\) identities for 2D and 3D features, respectively. Here, we choose the number of identities by \(d\leq 2^{l}\), which is valid in practice. In Figure 3, we project features outside the unit sphere onto the sphere, where the features do not cause inter-class ambiguity due to the sufficiently high magnitude. In this example, Figure 3a,b have 2907 features, respectively, while Figure 3c,d have 4186 features, respectively. Figure 2: Geometric interpretation of feature space learned by a softmax-based loss function (**a**) without and (**b**) with Orthant loss, respectively. In Figure 3b,d, we can see that ArcOrthFace has fewer features in the unit sphere by attempting to push features away from the origin, compared to ArcFace. It suggests that ArcOrthFace has improved inter-class separability compared to ArcFace, since the fewer features cause less inter-class ambiguity around the origin. In specific, for 2D and 3D features, we can observe that ArcOrthFace reduces the number of features inside the unit sphere to 278 and 100, while 604 and 344 features remain by ArcFace, respectively. As a result, ArcOrthFace, which learns the distributions of features away from the origin, shows the improved inter-class separability, compared to ArcFace. Figure 3b,d also show that ArcOrthFace has the better intra-class compactness than ArcFace by compressing feature spaces further at the center of each class through 1-bit quantization and element-wise operation. The feature spaces learned by ArcOrthFace are distributed around 1-bit quantized vectors, i.e., \(\{-1,+1\}^{l}\), which demonstrates the improved intra-class compactness, compared to ArcFace. **Training. ** Table 2 presents common training setting and hyperparameters. In order to fairly compare verification and identification accuracies that depend only on loss functions, we fix the network structure and training dataset. The recognition accuracies for the methods of comparison may be different from those in the original papers, which used their own training datasets and network structures. The learning rate starts at 0.1 and is changed to 0.01 at 20,000 th iteration and 0.001 at 28,000 th iteration, respectively. We use a horizontal random flip for data augmentation. For the hyperparameters of Center loss, SphereFace, ArcFace and MagFace not shown in Table 2, we use the recommended values of [25; 28; 33; 35], respectively. In Tables 3 and 4, the loss function replacing the suffix'max' or 'Face' with 'OrthFace', respectively, indicates a combined one with the Orthant loss. In addition, the loss function replacing the suffix 'Face' in ArcFace and MagFace with 'CentFace' indicates a combined one with the Center loss. \begin{table}
\begin{tabular}{l l l l} \hline \hline
**Datasets** & **Number of Identities** & **Number of Images** & **Types** \\ \hline CASIA-Webface [47] & 10 K & 0.5 M & Training \\ LFW [49] & 5749 & 13,233 & Validation \\ CFP-FP [40] & 500 & 7000 & Validation \\ AgeDB-30 [41] & 568 & 16,488 & Validation \\ MegaFace(pro.) [42] & 530 & 3530 & Testing \\ MegaFace(dis.) [42] & 690 K & 1 M & Testing \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Datasets for Training and Testing.