SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} = \frac {1}{N - 1} \left(\sum_ {i = 1} ^ {N} \sum_ {\substack {j = 1 \\ j \neq i}} ^ {N} \gamma g \left(\boldsymbol {x} ^ {(i)}, \boldsymbol {x} _ {*} ^ {(j)}\right) - \sum_ {i = 1} ^ {N} \sum_ {\substack {j = 1 \\ j \neq i}} ^ {N} \gamma g \left(\boldsymbol {x} ^ {(j)}, \boldsymbol {x} _ {*} ^ {(j)}\right)\right) (47) \\ = \frac {1}{N - 1} \sum_ {i = 1} ^ {N} \sum_ {\substack {j = 1 \\ j \neq i}} ^ {N} \left(\gamma g \left(\boldsymbol {x} ^ {(i)}, \boldsymbol {x} _ {*} ^ {(j)}\right) - \gamma g \left(\boldsymbol {x} ^ {(j)}, \boldsymbol {x} _ {*} ^ {(j)}\right)\right) (48) \\ = \frac {1}{N - 1} \sum_ {i = 1} ^ {N} \sum_ {\substack {j = 1 \\ j \neq i}} ^ {N} \left(\gamma g \left(\boldsymbol {x} ^ {(i)}, \boldsymbol {x} _ {*} ^ {(j)}\right) + \gamma g \left(\boldsymbol {x} ^ {(j)}, \boldsymbol {x} _ {*} ^ {(j)}\right) - 2 \gamma g \left(\boldsymbol {x} ^ {(j)}, \boldsymbol {x} _ {*} ^ {(j)}\right)\right) (49) \\ = \frac {2}{N - 1} \sum_ {i = 1} ^ {N} \sum_ {\substack {j = 1 \\ j \neq i}} ^ {N} \left(\log \exp \frac {\gamma g \left(\boldsymbol {x} ^ {(i)} , \boldsymbol {x} _ {*} ^ {(j)}\right) + \gamma g \left(\boldsymbol {x} ^ {(j)} , \boldsymbol {x} _ {*} ^ {(j)}\right)}{2} - \log \exp \left(\gamma g \left(\boldsymbol {x} ^ {(j)}, \boldsymbol {x} _ {*} ^ {(j)}\right)\right)\right) (50) \\ \leq \frac {2}{N - 1} \sum_ {i = 1} ^ {N} \sum_ {\substack {j = 1 \\ j \neq i}} ^ {N} \left(\log \frac {\mathrm {e} ^ {\gamma g \left(\boldsymbol {x} ^ {(i)} , \boldsymbol {x} _ {*} ^ {(j)}\right)} + \mathrm {e} ^ {\gamma g \left(\boldsymbol {x} ^ {(j)}, \boldsymbol {x} _ {*} ^ {(j)}\right)}}{2} - \log \exp \left(\gamma g \left(\boldsymbol {x} ^ {(j)}, \boldsymbol {x} _ {*} ^ {(j)}\right)\right)\right) (51) \\ = \frac {2}{N - 1} \sum_ {i = 1} ^ {N} \sum_ {\substack {j = 1 \\ j \neq i}} ^ {N} \left[ \log \left(\mathrm {e} ^ {\gamma g \left(\boldsymbol {x} ^ {(i)}, \boldsymbol {x} _ {*} ^ {(j)}\right)} + \mathrm {e} ^ {\gamma g \left(\boldsymbol {x} ^ {(j)}, \boldsymbol {x} _ {*} ^ {(j)}\right)}\right) - \log \exp \left(\gamma g \left(\boldsymbol {x} ^ {(j)}, \boldsymbol {x} _ {*} ^ {(j)}\right)\right) \right] - 2 N \log 2 (52) \\ = \frac {2}{N - 1} \sum_ {i = 1} ^ {N} \sum_ {\substack {j = 1 \\ j \neq i}} ^ {N} \log \left(1 + \frac {\mathrm {e} ^ {\gamma g \left(\boldsymbol {x} ^ {(i)} , \boldsymbol {x} _ {*} ^ {(j)}\right)}}{\mathrm {e} ^ {\gamma g \left(\boldsymbol {x} ^ {(j)}, \boldsymbol {x} _ {*} ^ {(j)}\right)}}\right) - 2 N \log 2. (53) \\ \end{array}
$$
B Marginal Sample-to-Sample Based Losses
We have derived three sample-to-sample based losses in the manuscript, i.e., USS loss, sample-to-sample based softmax, and BCE losses. We hereby present their respective marginal versions:
$$
L _ {\mathrm {u s s - m}} \left(\boldsymbol {X} ^ {(i)}\right) = \log \left(1 + \mathrm {e} ^ {- \gamma (g \left(\boldsymbol {x} ^ {(i)}, \boldsymbol {x} _ {*} ^ {(i)}\right) - m) + b}\right) + \sum_ {\substack {j = 1 \\ j \neq i}} ^ {N} \log \left(1 + \mathrm {e} ^ {\gamma g \left(\boldsymbol {x} ^ {(i)}, \boldsymbol {x} _ {*} ^ {(j)}\right) - b}\right), \tag{54}
$$
$$
L _ {\text {s o f t - m}} \left(\boldsymbol {X} ^ {(i)}\right) = = - \log \frac {\mathrm {e} ^ {\gamma \left(g \left(\boldsymbol {x} ^ {(i)} , \boldsymbol {x} _ {*} ^ {(i)}\right) - m\right)}}{\sum_ {j = 1} ^ {N} \mathrm {e} ^ {\gamma g \left(\boldsymbol {x} ^ {(i)} , \boldsymbol {x} _ {*} ^ {(j)}\right)}}, \tag {55}
$$
$$
L _ {\text {b c e - m}} \left(\boldsymbol {X} ^ {(i)}\right) = \log \left(1 + \mathrm {e} ^ {- \gamma \left(g \left(\boldsymbol {x} ^ {(i)}, \boldsymbol {x} _ {*} ^ {(i)}\right) - m\right) + b _ {i}}\right) + \sum_ {\substack {j = 1 \\ j \neq i}} ^ {N} \log \left(1 + \mathrm {e} ^ {\gamma g \left(\boldsymbol {x} ^ {(i)}, \boldsymbol {x} _ {*} ^ {(j)}\right) - b _ {j}}\right). \tag{56}
$$
The experimental evaluations of such marginal losses have been included in Sec. 4.3 of the manuscript.
In our work, we choose the cosine function to represent the similarity of two features, i.e.,
$$
g \left(\boldsymbol {x}, \boldsymbol {x} _ {*}\right) = \cos \left(\boldsymbol {x}, \boldsymbol {x} _ {*}\right) = \frac {\left\langle \boldsymbol {x} , \boldsymbol {x} _ {*} \right\rangle}{\left\| \boldsymbol {x} \right\| \left\| \boldsymbol {x} _ {*} \right\|}, \quad \text {f o r} \forall \boldsymbol {x}, \boldsymbol {x} _ {*} \in \mathcal {F}. \tag {57}
$$
Following (30; 7), we use customized ResNets (such as ResNet-50, ResNet-100, and ResNet-200) as our backbone networks. We implement all models using Pytorch and train them using the SGD optimizer with a weight decay of 5e-4 and momentum of 0.9. We use  $\gamma = 64$  for  $L_{\mathrm{uss}}$ ,  $L_{\mathrm{soft}}$ , and  $L_{\mathrm{bce}}$  in all experiments. Note that, we use a combination of CosFace  $(m = 0.4)$  and our  $\mathrm{USS}(m = 0.1)$  as UniTSFace in Tables 4 and 5.
For the face models (using ResNet-50 as the backbone) on CASIA-WebFace, we train them over 28 epochs with a batch size of 512. The learning rate starts at 0.1 and is reduced by a factor of 10 at the  $16^{th}$  and  $24^{th}$  epoch. All models in ablation and parameter study were trained on CASIA-WebFace.
In Table 1, the margin  $m$  of  $L_{\mathrm{soft - m}}$ ,  $L_{\mathrm{bce - m}}$  and  $L_{\mathrm{uss - m}}$  are set to be 0.1. In Table 3, the margin  $m$  of ArcFace and CosFace are set to be 0.5 and 0.4 respectively. The UniTSFace under the 'Small' protocol of MegaFace Challenge 1 in Table 4 and the models re-implemented in MFR Ongoing (the first ten rows in Table 5) were also trained on CASIA-WebFace.
For Glint360K, we train the models(ResNet-100) for 20 epochs using a batch size of 1024. Initially, the learning rate was set at 0.1, and a polynomial decay strategy (power=2) was applied to the learning rate schedule. The UniTSFace under the 'Large' protocol of MegaFace Challenge 1 (as shown in Table 4) was trained on Glint360K.
For WebFace4M, we train the models(ResNet-50) for 20 epochs using a batch size of 1024. The learning rate was initially set at 0.1, while a polynomial decay strategy (power=2) was applied to the learning rate schedule. The UniTSFace at the  $12^{th}$  row of Table 5 was trained on WebFace4M.
In the case of WebFace42M, we train the models(ResNet-200) for 20 epochs, using a larger batch size of 4096. The learning rate linearly warmed up from 0 to 0.4 during the first epoch, followed by a polynomial decay (power=2) for the remaining 19 epochs. The UniTSFace at the last row of Table 5 was trained on WebFace42M.
D Megaface Challenge 1
We notice that the official MegaFace challenge website has been decommissioned and MegaFace data are no longer being distributed. However, despite these changes, we can still utilize the previously released data and development kit to evaluate the performance of trained models. Specifically, we have adopted the MegaFace testsuite provided by InsightFace (https://github.com/deepinsight/insightface/tree/master/recognition/_evaluation_/megaface), which also includes the official devkit.