## 2 Materials and Methods
### Deep Metric Embeddings  
In its general form, the metric learning problem considers similarities between different pairs or triplets of samples coming from the dataset \(X=\left(x_{1},x_{2},\ldots,x_{N}\right)\), where \(x_{i}\in\mathcal{R}^{n}\) is \(i\)-th training sample and \(N\) is the total number of samples in the dataset. As was reported in [32], these similarities are determined by the sets \(S,D,R\) given by  
\[S=\left\{\left(x_{i},x_{j}\right)\in X\times X:x_{i}\text{ and }x_{j}\text{ are similar}\right\},\]  
\[D=\left\{\left(x_{i},x_{j}\right)\in X\times X:x_{i}\text{ and }x_{j}\text{ are not similar}\right\},\]  
\[R=\left\{\left(x_{i},x_{j},x_{k}\right)\in X\times X:x_{i}\text{ is more similar to }x_{j}\text{ than to }x_{k}\right\}.\]  
The problem then consists of finding a distance \(d\in\mathcal{D}\) that best fits the criteria specified by the similarity sets \(S,D,R\). With respect to a certain loss function \(\mathcal{L}\), the distance is found by optimising the problem  
\[\min_{d\in\mathcal{D}}\mathcal{L}\left(d,S,D,R\right)\,,\]  
where \(\mathcal{D}\) is a family of distances. Under the supervised learning setting with the true labels known \(\left(y_{1},y_{2},\ldots,y_{N}\right)\), the similarity sets are reformulated as follows:  
\[S=\left\{\left(x_{i},x_{j}\right)\in X\times X:y_{i}=y_{j}\right\}\]  
\[D=\left\{\left(x_{i},x_{j}\right)\in X\times X:y_{i}\
eq y_{j}\right\},\]  
\[R=\left\{\left(x_{i},x_{j},x_{k}\right)\in X\times X:y_{i}=y_{j}\
eq y_{k} \right\}.\]  
Loss function is one of the most important functions in deep feature learning. In general, the models presented in the the domain of deep metric learning can be categorised as minimising either the contrastive loss or the triplet loss [33; 34]. If the contrastive loss \(L_{\text{C}}\)[35] function is selected, the model is penalised differently based on whether the classes of the samples are the same or different. Specifically, if the classes agree, the loss function encourages the network to output feature vectors that are more similar, whereas if the classes differ, the loss function encourages the models to output feature vectors that are less similar. As such, contrastive loss \(L_{\text{C}}\) is given by  
\[L_{\text{C}}=\frac{1}{2}\left(1-Y\right)D_{W}^{2}+\frac{1}{2}Y\left\{max(0,m- D_{W})\right\}^{2},\]  
where \(m\) is a margin value, which has to be selected manually; \(Y\) indicates whether those two features come from the same class or not; \(D_{W}(G(X_{1}),G(X_{2}))\) is a distance metric for a pair of input samples, where \(X_{1}\) and \(X_{2}\) is a pair of inputs in the training set, while \(G(X_{1})\) and \(G(X_{2})\) are two vectors generated as a new representation of paired samples in the embedding space. In the current paper, the distance metric \(D_{W}\) is specified with a Euclidean metric for further analysis. Optionally, a more general metric, Mahalanobis distance, can be also considered. The function \(G\) denotes the results of a next-to-last convolutional layer. For the case considered in this paper, this function refers to the results of a 12 th layer.  
Another metric considered in the current work is the triplet loss with semi-hard negative mining [20; 34]. Instead of calculating the loss based on two examples, triplet loss \(L_{\text{T}}\) involves three inputs: an anchor example \(X\), similar to anchor input \(X^{S}\) and dissimilar to anchor input \(X^{D}\). As such, triplet loss \(L_{T}\) is given by  
\[L_{T}=\max{(0,D_{W_{1}}^{2}-D_{W_{2}}^{2}+\alpha)},\]  
where \(\alpha\) is the margin value; \(D_{W_{1}}=D_{W}(G(X),G(X^{S}))\) and \(D_{W_{2}}=D_{W}(G(X),G(X^{D}))\) are distance metrics for two vectors of new representations \(G(X)\), \(G(X^{S})\) and \(G(X^{D}))\), respectively. As one can see, triplet loss function penalises the model such that the distance between the matching examples is reduced and the distance between the nonmatching examples is increased.