# Additive Orthant Loss for Deep Face Recognition
## 1 Introduction  
Face recognition (FR) [1; 2] based on deep convolutional neural networks (DCNN) [3; 4; 5; 6; 7] is one of the fields in computer vision that found great success in the 2010s, and it has been actively studied as a biometric authentication [8; 9; 10] technique for face verification and identification. In particular, deep FR has better recognition accuracy for images with large variations such as pose, illumination, age, and expression compared to FR without using DCCNs. The good recognition accuracy of the deep FR comes from the fact that each DCNN layer is trained on a face image from various perspectives. It is important to design an efficient loss function that can distinguish identities well using a training dataset with a limited number of images and identities. To improve the recognition accuracy, various loss functions [11; 12; 13; 14; 15; 16; 17] as well as network architectures [18; 19; 20; 21] have been studied. Many loss functions for FR have been studied based on the softmax function [22] for image classification problems. In [23; 24], the authors first trained a deep face verification model using the softmax function. [PERSON] et al. [25] proposed the Center loss that uses both the softmax function and an additive term to improve the intra-class compactness. [PERSON] et al. [26] proposed a multiplicative angular margin to increase the cosine similarity of each class. In [27], [PERSON] et al. showed that the cosine similarity can be optimized using a new scaling constant replacing the magnitudes of the inner product between weight and feature vectors. [PERSON] et al. [28] proposed the angular softmax function that moves to the hypersphere manifold through weight normalization. In [29], [PERSON] et al. proposed the Margin loss that combines an additive term with the Center loss to enlarge the distances between centers of each class. [PERSON] et al. [30] proposed the Center invariant loss combined with the center loss to solve a bias caused by imbalanced image data between identities. In [31; 32], the authors proposed a loss function by introducing an additive angular margin to increase the cosine similarity of each class. [PERSON] et al. [33] proposed the ArcFace function that performs feature and weight normalizations in addition to the use of an additive angular margin. In [34], [PERSON] et al. proposed the LinCos-Softmax that replaces the cosine similarity between feature and weight vectors with an approximated linear angleto prevent the angle saturation caused by the nonlinearity of cosine similarity. In [35], [PERSON] et al. proposed an adaptive loss function that efficiently learns the distributions of features using the magnitudes of features. [PERSON] et al. [36] proposed the FCGFace loss that combines the ArcFace with two additive terms to compactly guide profile face features to each center of the frontal face features. In [37], [PERSON] et al. proposed the RVFace that drops the noisy feature vectors and adaptively learn the distributions of features using semi-hard feature vectors to improve the feature discriminative capability. For surveys on more network structures and loss functions for deep FR, readers are referred to [38; 39]. Many recent loss functions [31; 32; 33; 34] only use the cosine similarity between weight and feature vectors in the hypersphere manifold without feature magnitudes. Then, even if feature vectors have high cosine similarities with weight vectors of their corresponding class, the feature magnitudes may be small. In this case, the feature vectors have a problem that the cosine similarity can be changed significantly when the signs of features generated from images with large variation are changed due to their small magnitudes. On the other hand, in the property of monotonicity of [35], it has been proven that feature magnitudes increase with the cosine similarity. It has also been shown that an optimal feature magnitude exists for a feature by the property of convergence of [35].