# Metric Embedding Learning on Multi-Directional Projections
## 2 Related Work  
Since the introduction of Convolutional Neural Nets [20], the technique has undergone revolutionary transformations. Novel architectures on a large number of hidden, deep layers have been introduced to improve representational learning [21; 22; 23], while fundamental design patterns [24] and best practices [25] have been presented. In metric learning, the Siamese structure was the first CNN-based approach to measure the similarity of images [3]. In the following years, novel ideas and applications have emerged [4; 26; 27]. Applications on facial recognition [28] gained popularity, as error rate dropped below a level where production-level implementation became possible. The basis of the Siamese structure (visualized on Figure 1) is that a model with two consecutive sub-networks is applied. The two identical models are responsible for feature extraction on each of the inputs, and the output is given as the measured distance between the two encoded representations. The output is binary and represents the similarity of the two. The loss is defined as contrastive loss [29], where the distance of the encodings of true pairs are reduced, and at the same time, negative pairs are separated and distance is increased. The contrastive loss function is defined as:  
\[\mathcal{L}_{\mathtt{contrastive}}=(1-Y)\frac{1}{2}\mathtt{max}(0,(m-d(x_{1}, x_{2}))^{2})+Y\frac{1}{2}d(x_{1},x_{2})^{2}, \tag{1}\]  
where \(x_{1}\) and \(x_{2}\) are the two inputs, \(Y\) represents the target label, \(m\) stands for the margin, and \(d(a,b)\) defines the distance between given parameters. As the expected labels are in codomain \(\{0,1\}\) and the two parts of the equation are distinguished by the \(Y\) and \(1-Y\) multipliers. If \(x_{1}\) and \(x_{2}\) are different objects, \(Y=0\), then the loss is defined as the non-negative squared distance between the two and is easy by a margin value. If the two elements are similar, \(Y=1\), then the loss is given by the squared distance. This constant reduction of positive distances results in small clusters. This equation also gives, that for negative pairs, the loss is only positive when the distance is lower than the value defined as the margin. Therefore, the margin stops the enlargement of cluster distances. The pairwise loss is referred to as contrastive as optimization is done based on positive or negative pairs. To increase performance, a novel loss function was introduced with FaceNet [5]. Triplet loss is based on three samples: an anchor, a positive, and a negative pair. The objective is to push negative pairs compared to positive pairs. Formally, triplet loss can be given as:  
\[\mathcal{L}_{\mathtt{triplet}}=\mathtt{max}(0,m+d(x_{a},x_{p})-d(x_{a},x_{n})), \tag{2}\]  
where \(x_{a}\) represents the anchor and \(x_{p}\) and \(x_{n}\) stand for the positive and negative pairs, respectively. As the loss is given by the difference between the negative distance and the positive distance with margin included, every case is penalized where \(m+d(x_{a},x_{p})>d(x_{a},x_{n})\). In other cases, loss is 0. As a general result of this definition, positive pairs are kept together, while negative pairs are pushed away, within the frames defined in margin \(m\). The architecture of the triplet loss based neural network is different from the two-headed Siamese architecture, as the output is the encoding of the given input. Therefore, it is an encoder-type structure, where the training is done by measuring the distances of the encoding vectors, and classification is done by finding the closest neighbours in the embedded space. A diagram on the structure is shown on Figure 2.  
Figure 1: The architecture of the Siamese Neural Network. The underlying backbone network is shared, the output of the structure is a similarity score. During training, the samples given to the network should be chosen wisely. In ideal cases, for every sample, the farthest positive and the closest negative pair should be considered to have significant loss in order to find the best representation.