# Additive Orthant Loss for Deep Face Recognition
## 3 Orthant Loss
\begin{table}
\begin{tabular}{l l} \hline \hline Network model & LResNet50E-IR [33] \\ \hline Batch size \(N\) & 512 \\ \hline Feature length \(l\) & 512 \\ \hline Optimizer & SGD Optimizer \\ \hline Momentum & 0.9 \\ \hline Weight decay & 0.0005 \\ \hline Total iteration & \(32,000\) \\ \hline Scaling constant \(s\) & 64 \\ \hline Margin size \(m_{c}\) & \(1/\sqrt{l}\) \\ \hline Slope control factor \(r\) & 30 \\ \hline Margin control factor \(a\) & 2 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Training Setting and Hyperparamters. \begin{table}
\begin{tabular}{l l l l} \hline \hline
**Loss Function** & **LFW** & **CFP-FP** & **AgeDB-30** \\ \hline Softmax & 99.27 & 94.67 & 92.90 \\ SoftOrthFace & 99.30 & 94.94 & 92.60 \\ N-Softmax & 98.47 & 89.81 & 89.12 \\ N-SoftOrthFace & 98.75 & 92.96 & 90.62 \\ ArcFace [33] & 99.43 & 95.57 & 94.95 \\ ArcCentFace & **99.45** & 95.41 & 95.13 \\ ArcOrthFace & 99.42 & **95.73** & **95.18** \\ MagFace [35] & 99.18 & 94.97 & 94.62 \\ MagCentFace & 99.32 & 95.33 & 94.88 \\ MagOrthFace & 99.42 & 94.80 & 94.67 \\ Center loss [25] & 99.28 & 94.84 & 92.05 \\ SphereFace [28] & 99.12 & 94.49 & 92.68 \\ \hline \hline \end{tabular} Note: The accuracies in bold represent the highest ones among the listed loss functions.

\end{table} 
Table 3: Verification Accuracy (%) on Several Benchmarks with Various Loss Functions. **Testing. ** In the testing phase, features are extracted using the trained DCNN model, as shown in Figure 1. To make a feature from a testing image, we add the features obtained by original and flipped images and then normalize the combined feature. To evaluate the accuracy for LFW [49], CFP-FP [40] and AgeDB-30 [41], we measure the mean accuracy of 10 subsets of each dataset using the leave-one-out cross validation [50], following the unrestricted protocol with labeled outside data [49]. For the MegaFace dataset, we investigate the true positive rate (TPR) at \(10^{-6}\) false positive rate (FPR) for face verification and rank-1 accuracy [3] for face identification with \(10^{6}\) distractors, respectively. Figure 4 shows the training loss of the new loss functions combined with the proposed Orthant loss function. We can observe that the new combined loss functions reduce the training loss to make a good-fitting model as training progresses. In particular, at the 20,000 th and 28,000 th iterations, we use a learning rate decay method to improve the training speed and model accuracy. In addition, it can be confirmed that the training loss does not diverge or converge to zero to avoid underfitting and overfitting, respectively, since we use dropout and weight decay techniques. Table 3 shows the verification accuracy of various loss functions for LFW, CFP-FP and AgeDB-30, respectively. Table 3 demonstrates that the loss functions using Orthant loss improve the accuracy over the original ones in most benchmarks. This means that Orthant loss can improve the inter-class separability and the intra-class compactness by combining it with softmax-based loss functions. Additionally, ArcOrthFace, which combines ArcFace with Orthant loss, shows the highest accuracy on CFP-FP and AgeDB-30. In addition, it \begin{table}
\begin{tabular}{l c c} \hline \hline
**Loss Function** & **Rank1 Accuracy** & **Verification Accuracy** \\ \hline ArcFace & 91.70 & 93.95 \\ ArcCentFace & 91.39 & 93.85 \\ ArcOrthFace & **91.77** & **94.32** \\ MagFace & 89.54 & 92.57 \\ MagCentFace & 91.22 & 93.96 \\ MagOrthFace & 89.79 & 92.83 \\ Center loss & 79.96 & 83.45 \\ SphereFace & 81.95 & 88.06 \\ \hline \hline \end{tabular} Note: The accuracies in bold represent the highest ones among the listed loss functions.

\end{table} 
Table 4: Identification and Verification Results (%) on MegaFace with Various Loss Functions. Figure 4: Training loss of the new loss functions combined with the Orthant loss function. shows that SoftOrthFace has higher accuracy than Center loss, which demonstrates that Orthant loss has more discriminative capability than the additive term of Center loss.