# Large Margin Cotangent Loss for Deep Similarity Learning

Anh-Kiet Duong, Hoang-Lan Nguyen, Toan-Thinh Truong

► To cite this version:

Anh-Kiet Duong, Hoang-Lan Nguyen, Toan-Thinh Truong. Large Margin Cotangent Loss for Deep Similarity Learning. 2022 International Conference on Advanced Computing and Analytics (ACOMPA), Nov 2022, Ho Chi Minh City, France. pp.40-47, 10.1109/ACOMPA57018.2022.00013. hal-04996818

HAL Id: hal-04996818

https://hal.science/hal-04996818v1

Submitted on 19 Mar 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Large Margin Cotangent Loss for Deep Similarity Learning

Anh-Kiet Duong<fnref n="1" />,<fnref n="2" />†, Hoang-Lan Nguyen<fnref n="1" />,<fnref n="2" />†, Toan-Thinh Truong<fnref n="1" />,<fnref n="2" />*

{18120046,18120051}@student.hcmus.edu.vn, ttthinh@fit.hcmus.edu.vn

**Abstract**—Deep Convolutional Neural Networks (DCNN) models have become popular in feature extraction tasks. One of the best approaches to effectively classify the features is to utilize the loss function. Softmax loss is one of the losses initiated from One-Shot Learning method; however, using Softmax loss is considered impractical for real-world tasks since the number of labels can change frequently and require re-train when new labels emerge. In contrast, many types of loss function from Similarity Learning method such as Intra loss, Inter loss, Triplet loss, and Margin loss can tackle those drawbacks. Among them, the Margin-Loss-based losses have recently claimed to be the most effective strategies; such as CosFace, by adding a fixed parameter as a cosine margin to eliminate the dependence of the normal distribution to the cosine of weights and feature vectors; as well as ArcFace, by adding addictive angular penalty margins in the cosine function to simultaneously enhance class compactness and class differentiation. However, we evaluated that cosine functions used from these strategies do not fully represent the angle when they get smaller due to the limitation on the range of values. We propose Large Margin Cotangent Loss (LMCot), which uses cotangent function instead of cosine function since cotangent perform better by the unlimited on the range of values. As a result, our proposed method has great potential to improve performance on verification and identification tasks. Furthermore, we also experimented on various datasets from accredited competitions, the results of LMCot advanced the state-of-the-art in some measurements.

**Index Terms**—cotangent, one-shot learning, similarity learning

## I. INTRODUCTION

Learning similarities have gained more popularity in deep learning, especially in face verification and face identification tasks. Instead of predicting a probability distribution to find the label that fits best for the input image, the algorithm compares the distance between the input image with all the remaining images. Since this method doesn't depend on the number of labels, there is no need to re-train when new labels appear and still find similar images. The main point is to build a good DCNN model to project the images onto an n-dimensional Euclidean space, the distance will be used to decide the labels afterward. There are two main ways to train the DCNNs more effectively: train a classification-based model by using Softmax loss [1, 2, 3, 4, 5, 6] or optimize the distance directly from the vector embeddings during training

[7], [8]. Although both methods have shown decent results, they still have some shortcomings. Since triplet loss compares three samples at a time, with the increase in the amount of data, the number of triplets will also increase exponentially resulting in a significant increase in the number of iterations. The method that is said to be optimal when training with triplet loss is semi-hard sample training, which is quite difficult to train effectively. On the other hand, there are several variants of Softmax loss have been proposed and proven to carry some different drawbacks. For instance, the traditional Softmax loss has its boundary dependent on both the magnitude of weights and the angles, therefore the boundary overlaps on the decision area in cosine space; Normalized Version of Softmax Loss (NSL) performs weaker with noise; A-Softmax [1] loss reduces its boundary when the angles decrease and may completely vanish when the angles come to zero. Some improved versions of Softmax loss have been proposed, such as CosFace, ArcFacem, and ElasticFace. However, they all shared a common of applying cosine loss to calculate the angles between the feature vectors and corresponding weights.

In this work, we propose Large Margin Cotangent Loss by building it on top of ArcFace. We explain the advancement of cotangent compared to cosine, followed by some detailed pseudo code implementations. We also suggest the ensemble technique with the existing loss functions to increase performance. Afterward, several experiments are explicitly clarified, and a specific comparative discussion of the achieved results with those obtained from other losses on many datasets and competitions. The advantages of the proposed LMCot can be summarized as follows:

**Engaging.** LMCot optimizes the angle of weights and feature vectors by using cotangent instead of cosine. The cosine value is limited on range [-1, +1], whereas the cotangent value is unlimited on range (-∞, +∞). This helps the cotangent perform better than the cosine when the angle is too small or too large.

**Effective.** LMCot achieves state-of-the-art on all benchmarks that we tested. Using the same backbone, the model trained by LMCot has better results than the model trained by ArcFace.

**Efficient.** We propose two pseudo code versions of LMCot. Among these two versions, we provide a solution that skips the arc-cosine calculating steps to improve the computation efficiency.

<footnote n="1">¹University of Science, Ho Chi Minh City, Vietnam</footnote>

<footnote n="2">²Vietnam National University, Ho Chi Minh City, Vietnam</footnote>

<footnote>†These authors share the equal contribution</footnote>

<footnote>*Corresponding author. Email: ttthinh@fit.hcmus.edu.vn</footnote>
## II. RELATED WORK

We first inspect the traditional Softmax loss, an ordinary loss function used in classification-based models.

$$L_{\text{Softmax}} = -\frac{1}{N} \sum_{i=1}^{N} \log \frac{e^{W_{y_i}^T x_i + b_{y_i}}}{\sum_{j=1}^{n} e^{W_j^T x_i + b_j}}, \quad (1)$$

where $y_i$ is the class to which the data $i^{th}$ belongs, $n$ is the number of labels, and $N$ is the number of samples. However, model using the Softmax loss function can only train with a fixed number of classes. It is obligatory to re-train the model if we want to add a new label. Siamese Network [9] is one of the architectures that can work with new labels apart from the training set rather than re-train itself. Instead of classifying each input, we extract a feature vector, then compare it with the existing ones in the database to give decision. Thus, when adding a new label, we only need to add the vector of that label to the database.

<figure><img src="image_2.png" /><figcaption>Fig. 1. Illustrate how the siamese model works. The image is taken from the CelebA [10] dataset. The similarity measure could be cosine, euclidean, manhattan, etc.</figcaption></figure>

Next, we transform the output logit [11] as $W_j^T x_i = \|W_j\| \|x_i\| \cos(\theta_{ji})$, where $\theta_{ji}$ is the angle between the weight $W_j^T$ and the feature $x_i$. By $l_2$ normalization [12], we set $\|W_j\| = \|x_i\| = 1$, $b_j = 0$, and having $s$ as a scale parameter, the original function (1) becomes:

$$L = -\frac{1}{N} \sum_{i=1}^{N} \log \frac{e^{s \cos \theta_{y_i}}}{e^{s \cos \theta_{y_i}} + \sum_{j=1, j \neq y_i}^{n} e^{s \cos \theta_{ji}}}. \quad (2)$$

Established on the above function, some improved methods using margin value $m$ to increase performance.

**Liu et al.** [1]

$$L_{\text{SphereFace}} = -\frac{1}{N} \sum_{i=1}^{N} \log \frac{e^{s \cos(m*\theta_{y_i})}}{e^{s \cos(m*\theta_{y_i})} + \sum_{j=1, j \neq y_i}^{n} e^{s \cos \theta_{ji}}}, \quad (3)$$

**Wang et al.** [3]

$$L_{\text{CosFace}} = -\frac{1}{N} \sum_{i=1}^{N} \log \frac{e^{s(\cos(\theta_{y_i}) - m)}}{e^{s(\cos(\theta_{y_i}) - m)} + \sum_{j=1, j \neq y_i}^{n} e^{s \cos \theta_{ji}}}, \quad (4)$$

**Deng et al.** [4]

$$L_{\text{ArcFace}} = -\frac{1}{N} \sum_{i=1}^{N} \log \frac{e^{s(\cos(\theta_{y_i} + m))}}{e^{s(\cos(\theta_{y_i} + m))} + \sum_{j=1, j \neq y_i}^{n} e^{s \cos \theta_{ji}}}, \quad (5)$$

and **Boutros et al.** [6] $L_{\text{ElasticFace}}$, which its margin is a random value of the normal distribution.

Moreover, some other loss functions such as **Pairwise confusion loss** [7], **Triplet loss** [8] use Siamese Networks to extract features and optimize the distance between the outputs.

## III. PROPOSED APPROACH

### A. Large Margin Cotangent Loss (LMCot)

Large Margin Loss is generally very common in learning similarity, Ex: SphereFace [1], SV-AM-Softmax [2], CosFace [3], ArcFace [4], AdaCos [5], etc. The most common function in both competitions and applications is ArcFace, as they are effective and easy to implement.

ElasticFace-Arc [6] is a method inherited from ArcFace [4]. The main idea is to use random margin values drawn from a normal distribution in each training iteration. This method advanced state-of-the-art on seven mainstream face recognition benchmarks. Therefore, we also investigate the effects of using cosine and cotangent by implementing cotangent loss on top of ArcFace. We examine the ArcFace loss function:

$$L = -\frac{1}{N} \sum_{i=1}^{N} \log \frac{e^{s(\cos(\theta_{y_i} + m))}}{e^{s(\cos(\theta_{y_i} + m))} + \sum_{j=1, j \neq y_i}^{n} e^{s \cos \theta_{ji}}}, \quad (6)$$

where $\theta_{ji}$ is the angle between the weight $W_j^T$ and the feature $x_i$. The label of $i^{th}$ is $y$, $\theta_{y_i}$ is the angle between the feature $x_i$ and the feature of class $y$ stored in weight $W$. The margin $m$ is the penalty, and $s$ is a scale parameter.

The problem with ArcFace is that the cosine function only provides value within [-1, 1]. Hence, it is not fully representing the degrees between the angles as much as the cotangent, where the value obtained within (-∞, +∞). For instance, by $j \neq y$ with small $\theta_{ij}$ and smaller $\theta_{iy}$, the cosine function will not be large enough. When the initialized weights are bad, the $\theta_{iy}$ is closer to $\pi$. As a result, the cotangent function will receive a larger loss to make the model optimize faster. This is more satisfactory for tasks that need high top-1 accuracy. LMCot is presented as follows:

$$L = -\frac{1}{N} \sum_{i=1}^{N} \log \frac{e^{s(\cot(\theta_{y_i} + m))}}{e^{s(\cot(\theta_{y_i} + m))} + \sum_{j=1, j \neq y_i}^{n} e^{s \cot \theta_{ji}}}. \quad (7)$$

In bad weight initialization scenario, the cotangent function will perform better since cot($\theta_{ij}$) will be very large. Other variables remain the same as those in ArcFace, we only replace the cosine function with the cotangent function. Furthermore, when the angle is enough (not too large and not too small), the cosine and cotangent functions would start getting closer to each other.
<figure><img src="image_3.png" /><figcaption>Fig. 2. DCNN model trained by Large Margin Cotangent Loss based on ArcFace [4]. The $cos(\theta_j)$ (logit) for each class is obtained from the normalization of the feature $x_i$ and weight $W$, presented as $W_j^T x_i$. The angle between the feature $x_i$ and the weight $W$ is computed by $arccos(\theta_{yi})$. After calculating $cot(\theta_{yi} + m)$, all logits are multiplied by the scale $s$ to go through the softmax and the cross-entropy loss.</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Fig. 3. The graph represents the function $y = \cos(x)$ and the function $y = \cot(x)$ respectively. In the area of the red circle, $\cos(x)$ are close to $\cot(x)$. By convention, the angle of the two feature vectors $x \in [0, \pi)$.</figcaption></figure>

Pseudo-code of LMCot in Siamese Network with backbone $B$, weight $W$, and $\epsilon$ is the smallest value that can become the denominator.

**Input:** data $x$, scale $s$, margin $m$, epsilon $\epsilon$, class number $n$, Ground-Truth ID $gt$.

1) $f = \|B(x)\|$

2) $W = \|W\|$

3) $\cos_{\theta} = Wf$

4) $\theta = \arccos(\cos_{\theta})$

5) $\cot_{t} = \cos_{\theta}/\max(\sin(\theta), \epsilon)$

6) $\cot_{t-m} = \cos(\theta + m)/\max(\sin(\theta + m), \epsilon)$

7) $one\_hot = one\_hot(\text{gt})$

8) $\text{logit} = one\_hot * \cot_{t-m} + (1 - one\_hot) * \cot_{t}$

9) $\text{logit} = s * \text{logit}$

**Output:** Class-wise affinity score $\text{logit}$

In the circumstance that the arc-cosine computation becomes complicated, we can use the following pseudo-code to skip the theta computing step.

**Input:** data $x$, scale $s$, margin $m$, epsilon $\epsilon$, class number $n$, Ground-Truth ID $gt$.

1) $f = \|B(x)\|$

2) $W = \|W\|$

3) $\cos_{\theta} = Wf$

4) $\sin_{\theta} = \sqrt{1 - \cos_{\theta}}$

5) $\cot_{\theta} = \cos_{\theta}/\max(\sin_{\theta}, \epsilon)$

6) $\cos_{\theta} = \cos_{\theta} * \cos(m) - \sin_{\theta} * \sin(m)$

7) $\sin_{\theta} = \sin_{\theta} * \cos(m) + \cos_{\theta} * \sin(m)$

8) $\cot_{\theta} = \cos_{\theta}/\max(\sin_{\theta}, \epsilon)$

9) $one\_hot = one\_hot(\text{gt})$

10) $\text{logit} = one\_hot * \cot_{\theta} + (1 - one\_hot) * \cot_{\theta}$

11) $\text{logit} = s * \text{logit}$

**Output:** Class-wise affinity score $\text{logit}$

## B. Ensemble methods

We present the idea of integrating SphereFace [1], CosFace [3], ArcFace [4], ElasticFace [6] methods and use cotangent instead of cosine.

$$L = -\frac{1}{N} \sum_{i=1}^{N} \log \frac{e^{s(\cot(E(m_1, \sigma_1)\theta_{yi} + E(m_2, \sigma_2)) + E(m_3, \sigma_3))}}{e^{s(\cot(E(m_1, \sigma_1)\theta_{yi} + E(m_2, \sigma_2)) + E(m_3, \sigma_3))} + I}$$

$$I = \sum_{j=1, j \neq y_i}^{n} e^{s \cot \theta_{ji}}, \qquad (8)$$

where $E(\bar{x}, \sigma)$ is the normal function that returns a random value from a Gaussian distribution with the mean $\bar{x}$ and the standard deviation $\sigma$.
In addition, we present the method of combining cotangent and cosine functions.

$$L = \alpha L_{\text{cot}} + \beta L_{\text{cos}},$$
$$L_{\text{cot}} = -\frac{1}{N} \sum_{i=1}^{N} \log \frac{e^{s(\cot(E(m_1, \sigma_1)\theta_{yi} + E(m_2, \sigma_2)) + E(m_3, \sigma_3))}}{e^{s(\cot(E(m_1, \sigma_1)\theta_{yi} + E(m_2, \sigma_2)) + E(m_3, \sigma_3))} + I},$$
$$L_{\text{cos}} = -\frac{1}{N} \sum_{i=1}^{N} \log \frac{e^{s(\cos(E(m_1, \sigma_1)\theta_{yi} + E(m_2, \sigma_2)) + E(m_3, \sigma_3))}}{e^{s(\cos(E(m_1, \sigma_1)\theta_{yi} + E(m_2, \sigma_2)) + E(m_3, \sigma_3))} + I},$$
$$\text{and } I = \sum_{j=1, j \neq y_i}^{n} e^{s \cot \theta_{ji}}. \qquad (9)$$

Finally, we divide the training process into several stages. In each stage, we use a different loss function.

## IV. EXPERIMENTS

### A. Face Datasets

To compare with other loss functions, we trained a model with backbone of ResNet50 [14], [15] on CASIA [16] dataset and tested on LFW [17, 18] dataset.

<table><caption>TABLE I
RESULTS ON LFW</caption><thead><tr><th>Loss Functions</th><th>Accuracy on LFW</th></tr></thead><tbody><tr><td>LMCot(0.5)(ArcFace-based)</td><td>99.58</td></tr><tr><td>ArcFace(0.5)</td><td>99.53</td></tr><tr><td>ArcFace(0.4)</td><td>99.53</td></tr><tr><td>CosFace</td><td>99.51</td></tr><tr><td>SphereFace</td><td>99.42</td></tr><tr><td>Softmax</td><td>99.08</td></tr><tr><td>Triplet(0.35)</td><td>98.98</td></tr></tbody></table>

Verification results (%) of different loss functions ([CASIA, ResNet50, loss*])

Large-scale CelebFaces Attributes (CelebA) Dataset [10] compares more than 10k identities, 200k images of the size 178 × 218, each with 40 attributes. This is an ideal dataset to compare loss functions. The dataset is divided into 3 parts: training, validation, and testing. In addition, by the attributes, we can check whether our model is effective in extracting features or not. All models have same backbone Efficient-NetV2S [19, 20], same learning rate with Adam optimizer and all configurations. The test set consists of 19962 images, in a total of $C_{1992}^2 = 199230741$ pairs. Among these pairs, a quantity of 229587 pairs belongs to the same person. We calculate the distance of each pair and the Equal Error Rate (ERR) on the corresponding distance.

Cosine distances distribution of 100k random positive pairs and 100k random negative pairs. After training, a part of positive pairs is more separate from a part of negative pairs than it was at the beginning. LMCot has a lower intersection than ArcFace, but actually, it looks pretty similar. We can see it more clearly by the number in table II, the histograms of arcFace and LMCot are really hard to distinguish. This proves

<table><caption>TABLE II
RESULTS ON CELEBA</caption><thead><tr><th></th><th>EER on CelebA</th></tr></thead><tbody><tr><td>LMCot (ArcFace-based)</td><td>7.3%</td></tr><tr><td>ArcFace</td><td>7.35%</td></tr><tr><td>CosFace</td><td>7.33%</td></tr><tr><td>SphereFace</td><td>7.42%</td></tr></tbody></table>

Equal Error Rate (%) of different loss functions ([CelebA, EfficientNetv2S, loss*])

<figure><img src="image_5.png" /><figcaption>Fig. 4. Cosine distances distribution of 100k random positive pairs and 100k random negative pairs from the CelebA test set. The blue part is positive pairs distance and the orange part is negative pairs distance. The red line shows the value at which the False Acceptance Rate (FAR) equals the False Rejection Rate (FRR). Figure (4a), (4b), and (4c) illustrate the cosine distances distribution from the pretrained-weight on ImageNet, from a model trained with ArcFace and LMCot respectively.</figcaption></figure>
<figure><img src="image_6.png" /><figcaption>(a) Result from pretrained-weight on ImageNet</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>(b) Result from ArcFace</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Fig. 5. We use PCA method to decompose feature into two components and plot with 40 attributes from CelebA test set. The red points represent the positive side of the feature and the blue points represent the negative side of the feature. Figure (5a), (5b), and (5c) illustrate the distribution of 40 attributes from pretrained-weight on ImageNet, on model trained with ArcFace and LMCot respectively.</figcaption></figure>

that the cotangent is quite similar to the cosine in most parts, where the $\theta$ angle is not too large and not too small.

To check the feature extraction's performance of the model by the attributes, we use the Principal Component Analysis [21] (PCA) method to decompose the feature into two components. After training the model, it is possible to temporarily separate the attributes of the two opposite parts, such as male or female, black hair or not, double chin or not, etc.

## B. Speaker Verification (VLSP2021)

The VLSP2021 Speaker Verification competition was held at the 8th International Workshop on Vietnamese Language and Speech Processing. Focus on developing SV models with limited data. The provided training set has over 1000 speaker identities, and the competition consists of two relevant tasks:

* SV-T1: Test set combines in-domain speakers and out-domain speakers.

* SV-T2: All speakers are external to the training dataset.

The performance of the models will be evaluated by the Equal Error Rate (EER) where the False Acceptance Rate (FAR) equals the False Rejection Rate (FRR). The evaluation metric is cosine similarity of the enrollment utterance and the test utterance.

In this competition, each participant has five submissions for each task. We experimented ResNet50 [14], EfficientNetB0 [19] as a backbone model, then add padding to the recordings to match the length of the longest recording. We used librosa [22] to plot Mel Spectrograms of these audios with a default parameter, then fed it into the backbone model. Training data is split into five folds, four for training and one for validation. In the end, the model weight that has the best ERR on the validation set will be chosen. Finally, we ensemble all these weights by a mean score of each sample on the test set. By using this simple method, we achieved the following results:

<table><caption>TABLE III
RESULTS ON VLSP2021</caption><thead><tr><th></th><th>SV-T1</th><th>SV-T2</th></tr></thead><tbody><tr><td>R50 + ArcFace</td><td>10.520</td><td>19.850</td></tr><tr><td>B0 + ArcFace</td><td>10.365</td><td>12.665</td></tr><tr><td>R50 + LMCot (ArcFace-based)</td><td>10.270</td><td>12.840</td></tr><tr><td><b>B0 + LMCot (ArcFace-based)</b></td><td><b>9.215</b></td><td><b>12.535</b></td></tr><tr><td>Ensemble all</td><td>8.805</td><td>11.605</td></tr></tbody></table>

Equal Error Rate (%) on Speaker Verification (VLSP2021) competition. R50 is ResNet50 and B0 is EfficientNetB0. In the last row, we ensembled all 4 above results by mean with the same weight.

## C. Google Landmark Retrieval 2021

The 4th Landmark Retrieval competition is one of the Instance-Level Recognition (ILR) workshop challenges at the International Conference on Computer Vision 2021 (ICCV 2021). In this competition, the developed models are expected to retrieve relevant database images to a given query image.

In this competition we use Google Landmark Dataset v2 [23] (gldv2) as dataset. These names are used for the following subsets of gldv2 [24]:

* gldv2: 5M images of 200k classes.

* gldv2c: cleaned subset (1.6M images of 81k classes).

* gldv2x: non-cleaned subset restricted to only 81k classes, same as gldv2c (3.2M images).

Submissions are evaluated according to the mean Average Precision at 100, commonly referred to as (mAP@100):

$$mAP@100 = \frac{1}{Q} \sum_{q=1}^{Q} \frac{1}{\min(m_q, 100)} \sum_{k=1}^{\min(n_q, 100)} P_q(k) rel_q(k) \quad (10)$$

where Q is the number of query images, $m_q$ is the number of index images containing a landmark in common with the query image q. $n_q$ is the number of predictions made by the system for query q, $P_q(k)$ is the precision at rank k for the
q-th query and $rel_q(k)$ is 1 if the k-th prediction is correct, and is 0 otherwise.

We won the silver medal in this competition by using LMCot, despite the limited resources on hardware.

<table><caption>TABLE IV
RESULTS ON GOOGLE LANDMARK RETRIEVAL 2021</caption><thead><tr><th></th><th>Public</th><th>Private</th></tr></thead><tbody><tr><td>DELG (Baseline)</td><td>0.21480</td><td>0.22152</td></tr><tr><td>4*B0-ArcFace (Public)</td><td>0.24890</td><td>0.26164</td></tr><tr><td><strong>4*B0-LMCot (Ours)</strong></td><td><strong>0.26146</strong></td><td><strong>0.28338</strong></td></tr><tr><td>4*B7-ArcFace (Public)</td><td>0.31940</td><td>0.32955</td></tr><tr><td><strong>4*B7-LMCot (Ours)</strong></td><td><strong>0.32476</strong></td><td><strong>0.34691</strong></td></tr><tr><td>4*V2M-ArcFace (Ours)</td><td>0.32554</td><td>0.33973</td></tr><tr><td><strong>4*V2M-LMCot (Ours)</strong></td><td><strong>0.34933</strong></td><td><strong>0.36649</strong></td></tr><tr><td>ViT-ArcFace (Ours)</td><td>0.35127</td><td>0.36569</td></tr><tr><td><strong>ViT-LMCot (Ours)</strong></td><td><strong>0.35720</strong></td><td><strong>0.36723</strong></td></tr><tr><td>ViT-ArcFace + ViT-LMCot + B7-ArcFace + B7-LMCot + V2M-LMCot + 2*V2S-LMCot</td><td>0.37283</td><td>0.38798</td></tr></tbody></table>

Results in Google Landmark Retrieval 2021 competition. B0 is EfficientNetB0[19], B7 is Efficient-NetB7[19], ViT is Vision Transformer [25], V2S is EfficientNetV2S[20], V2M is EfficientNetV2M[20]. Submissions are evaluated according to the mean Average Precision at 100 (equation (10)). All LMCot solutions we used are built upon ArcFace.

With a similar backbone model and training seed, LMCot gives better results than ArcFace. All results were trained using Google Colab Tensor Processing Unit (TPU) [26]. Hence, the results are reproducible. For models using ArcFace, to be most objective, we use results from public notebooks of other participants in this competition (if possible). All models are trained and inference on the same image size [256,256,3]. And all models trained by gldv2c are packed in TFRecord format.

Dieter's 1st place solution [21] is very complex, combining many large models. Consequently, the result is outstanding by 0.51779 for the public score and 0.53751 for the private score. They trained 10 epochs on gldv2c (image size 224x224). After that, they trained 30-40 epochs on gldv2x (image size 512x512). Lastly, a few epochs on gldv2x of size 768x768. The models used are:

* DOLG-EfficientNet-B5 (768x768) [27]

* DOLG-EfficientNet-B6 (768x768) [27]

* DOLG-EfficientNet-B7 (448x448) stride 1 [27]

* Hybrid SwinBase224-EfficientNet-B5 (896x896) [28]

* Hybrid SwinBase224-EfficientNet-B3 (896x896) [28]

* Hybrid SwinBase384-EfficientNet-B6 (384,384) stride 1 [28]

* 3rd place GLRec2020 EfficientNet-B6 (512x512) [29]

* 2nd place GLRec2020 EfficientNet-B5 (768x768) [30]

Models used by top teams are also very complicated and expensive which require a tremendous amount of GPU resources, such as from 2nd place team [31] and 3rd place team [32].

## D. Google Landmark Recognition 2021

This is also one of the Instance-Level Recognition (ILR) workshop challenges at the International Conference on Computer Vision 2021 (ICCV 2021). This competition requires Kagglers to build correct landmark recognition models (if any) to evaluate challenging test images. Not surprisingly, cotangent consistently outperformed cosine in all tests.

In this competition, gldv2 is also used [23] as the training dataset. And submissions are evaluated by Global Average Precision (GAP) at (k), where (k=1)

$$ GAP = \frac{1}{M} \sum_{i=1}^{N} P(i) rel(i) \quad (11) $$

where *N* is the total number of predictions returned from the judging system across all queries. At least one landmark in the total number of queries from the training set visible in it is *M*, some queries may not depict landmarks. *P* and *rel* are the same as Google Landmark Retrieval 2021 metric.

Despite the limited resources on hardware, we also won the silver medal in this competition.

<table><caption>TABLE V
RESULTS ON GOOGLE LANDMARK RECOGNITION 2021</caption><thead><tr><th></th><th>Public</th><th>Private</th></tr></thead><tbody><tr><td>DELG (Baseline)</td><td>0.20245</td><td>0.19975</td></tr><tr><td>4*B0-ArcFace (Public)</td><td>0.21898</td><td>0.21468</td></tr><tr><td><strong>4*B0-LMCot (Ours)</strong></td><td><strong>0.22460</strong></td><td><strong>0.22242</strong></td></tr><tr><td>4*B7-ArcFace (Public)</td><td>0.27760</td><td>0.26292</td></tr><tr><td><strong>4*B7-LMCot (Ours)</strong></td><td><strong>0.29926</strong></td><td><strong>0.29038</strong></td></tr><tr><td>4*V2M-ArcFace (Ours)</td><td>0.28593</td><td>0.29134</td></tr><tr><td><strong>4*V2M-LMCot (Ours)</strong></td><td><strong>0.29963</strong></td><td><strong>0.29353</strong></td></tr><tr><td>ViT-ArcFace (Ours)</td><td>0.30309</td><td>0.30480</td></tr><tr><td><strong>ViT-LMCot (Ours)</strong></td><td><strong>0.30225</strong></td><td><strong>0.30605</strong></td></tr><tr><td>ViT-ArcFace + ViT-LMCot + B7-ArcFace + B7-LMCot + V2M-LMCot + 2*V2S-LMCot</td><td>0.30900</td><td>0.31503</td></tr></tbody></table>

Results in Google Landmark Recognition 2021 competition. B0 is EfficientNetB0 [19], B7 is Efficient-NetB7 [19], ViT is Vision Transformer [25], V2S is EfficientNetV2S [20], V2M is EfficientNetV2M [20]. Submissions are evaluated according to the mean Global Average Precision at *k* = 1 (equation (11)). All LMCot we used is built on top of ArcFace, replacing cosine with cotangent.

Due to the restricted hardware with GPU-configuration limitation, we could not experiment larger models, in contrast to some complex solutions from 1st place team [24], 2nd place team [33] and 3rd place team [34].

## V. CONCLUSION

In this paper, we propose Large Margin Cotangent Loss for similarity learning. We examine why cotangent is more effective than the cosine function in small angles (near to 0) and large angles (near to $\pi$) situations. We implement the loss based on ArcFace, which is evaluated on several datasets as well as competitions. In face datasets, we tested the ResNet50 model on the LFW dataset along with the EfficientNetV2S model on the CelebA validation dataset. Furthermore, we experimented with the speaker verification problem from VLSP2021. Lastly, we examined LMCot on Google Landmark competitions: Google Landmark Retrieval
and Google Landmark Recognition. By experimentally replac-
ing the cosine function with the cotangent function in ArcFace,
LMCot is proved to advance the conventional ArcFace in some
measurements.

Nevertheless, applying the cotangent function could cause
a small increase in computation complexity. In addition, we
need to set epsilon ε to ensure the value of the cotangent value
does not exceed the limit. In the future, we will find ways to
improve those limitations of LMCot.

ACKNOWLEDGMENT

This study was funded by Vietnam National University, Ho
Chi Minh City (VNU-HCM) under grant number C2021-18-
21.

REFERENCES

[1] W. Liu, Y. Wen, Z. Yu, M. Li, B. Raj, and L. Song, "Sphereface: Deep hypersphere embedding for face recognition," in *Proceedings of the IEEE conference on computer vision and pattern recognition*, 2017, pp. 212–220.

[2] X. Wang, S. Wang, S. Zhang, T. Fu, H. Shi, and T. Mei, "Support vector guided softmax loss for face recognition," *arXiv preprint arXiv:1812.11317*, 2018.

[3] H. Wang et al., "Cosface: Large margin cosine loss for deep face recognition," in *Proceedings of the IEEE conference on computer vision and pattern recognition*, 2018, pp. 5265–5274.

[4] J. Deng, J. Guo, N. Xue, and S. Zafeiriou, “Arcface: Additive angular margin loss for deep face recognition,” in *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, 2019, pp. 4690–4699.

[5] X. Zhang, R. Zhao, Y. Qiao, X. Wang, and H. Li, "Adacos: Adaptively scaling cosine logits for effectively learning deep face representations," in *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, 2019, pp. 10823–10832.

[6] F. Boutros, N. Damer, F. Kirchbuchner, and A. Kuijper, "Elasticface: Elastic margin loss for deep face recognition," in *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, 2022, pp. 1578–1587.

[7] A. Dubey, O. Gupta, P. Guo, R. Raskar, R. Farrell, and N. Naik, "Pairwise confusion for fine-grained visual classification," in *Proceedings of the European conference on computer vision (ECCV)*, 2018, pp. 70–86.

[8] F. Schroff, D. Kalenichenko, and J. Philbin, “Facenet: A unified embedding for face recognition and clustering,” in *Proceedings of the IEEE conference on computer vision and pattern recognition*, 2015, pp. 815–823.

[9] J. Bromley, I. Guyon, Y. LeCun, E. Säckinger, and R. Shah, “Signature verification using a” siamese” time delay neural network,” *Advances in neural information processing systems*, vol. 6, 1993.

[10] Z. Liu, P. Luo, X. Wang, and X. Tang, “Deep learning face attributes in the wild,” in *Proceedings of International Conference on Computer Vision (ICCV)*, Dec. 2015.

[11] G. Pereyra, G. Tucker, J. Chorowski, Ł. Kaiser, and G. Hinton, “Regularizing neural networks by penalizing confident output distributions,” *arXiv preprint arXiv:1701.06548*, 2017.

[12] F. Wang, X. Xiang, J. Cheng, and A. L. Yuille, “Normface: L2 hypersphere embedding for face verification,” in *Proceedings of the 25th ACM international conference on Multimedia*, 2017, pp. 1041–1049.

[13] M. Hohenwarter and M. Hohenwarter, “Geogebra,” *Available on-line at http://www.geogebra.org/cms/en*, 2002.

[14] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” *2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, pp. 770–778, 2016.

[15] D. Han, J. Kim, and J. Kim, “Deep pyramidal residual networks,” *2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, pp. 6307–6315, 2017.

[16] D. Yi, Z. Lei, S. Liao, and S. Li, “Learning face representation from scratch,” *ArXiv*, vol. abs/1411.7923, 2014.

[17] G. B. Huang, M. Ramesh, T. Berg, and E. Learned-Miller, “Labeled faces in the wild: A database for studying face recognition in unconstrained environments,” University of Massachusetts, Amherst, Tech. Rep. 07-49, Oct. 2007.

[18] G. B. H. E. Learned-Miller, “Labeled faces in the wild: Updates and new reporting procedures,” University of Massachusetts, Amherst, Tech. Rep. UM-CS-2014-003, May 2014.

[19] M. Tan and Q. Le, “Efficientnet: Rethinking model scaling for convolutional neural networks,” in *International conference on machine learning*, PMLR, 2019, pp. 6105–6114.

[20] ——, “Efficientnetv2: Smaller models and faster training,” in *International Conference on Machine Learning*, PMLR, 2021, pp. 10906–10910.

[21] S. Wold, K. Esbensen, and P. Geladi, “Principal component analysis,” *Chemometrics and intelligent laboratory systems*, vol. 2, no. 1-3, pp. 37–52, 1987.

[22] B. McFee et al., “Librosa: Audio and music signal analysis in python,” in *Proceedings of the 14th python in science conference*, vol. 8, 2015, pp. 18–25.

[23] T. Weyand, A. Araujo, B. Cao, and J. Sim, “Google landmarks dataset v2-a large-scale benchmark for instance-level recognition and retrieval,” in *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, 2020, pp. 2575–2584.

[24] C. Henkel, “Efficient large-scale image retrieval with deep feature orthogonality and hybrid-swin-transformers,” *arXiv preprint arXiv:2110.03786*, 2021.
[25] A. Dosovitskiy et al., "An image is worth 16x16 words: Transformers for image recognition at scale," *arXiv preprint arXiv:2010.11929*, 2020.

[26] N. P. Jouppi et al., "In-datacenter performance analysis of a tensor processing unit," in *Proceedings of the 44th annual international symposium on computer architecture*, 2017, pp. 1-12.

[27] M. Yang et al., "Dolg: Single-stage image retrieval with deep orthogonal fusion of local and global features," in *Proceedings of the IEEE/CVF International Conference on Computer Vision*, 2021, pp. 11772-11781.

[28] Z. Liu et al., "Swin transformer: Hierarchical vision transformer using shifted windows," in *Proceedings of the IEEE/CVF International Conference on Computer Vision*, 2021, pp. 10012-10022.

[29] Q. Ha, B. Liu, F. Liu, and P. Liao, "Google landmark recognition 2020 competition third place solution," *arXiv preprint arXiv:2010.05350*, 2020.

[30] S. Dai, "Google landmark recognition 2020 competition third place solution," 2020.

[31] Z. Yuqi et al., "2nd place solution to google landmark retrieval 2021," *arXiv preprint arXiv:2110.04294*, 2021.

[32] Q. Ha, B. Liu, and H. Zhang, "Google landmark retrieval 2021 competition third place solution," *arXiv preprint arXiv:2110.04619*, 2021.

[33] S. Dai, "2nd place solution to google landmark recognition competition 2021," *arXiv preprint arXiv:2110.02638*, 2021.

[34] C. Xu et al., "3rd place solution to google landmark recognition competition 2021," *arXiv preprint arXiv:2110.02794*, 2021.