# Anomaly Detection of CAN Bus Messages Using a Deep Neural Network for Autonomous Vehicles
## 3 Proposed Method
### The Triplet Loss Network  
The triple loss consists of a randomly selected sample from the training dataset, which is called Anchor, and then a sample of the same class as the anchor and a different sample randomly selected from Anchor. The corresponding samples are called Positive and Negative. We define \(Y=f(x)\in R^{d}\) as the extracted data feature vector from original CAN data sequences, and then the input of the triplet loss network can be represented as \(\left(f\big{(}X_{i}^{a}\big{)},f\big{(}X_{i}^{n}\big{)}f\big{(}X_{i}^{p} \big{)}\right)\). Referring to the idea of learning to rank, in this paper we utilize triplet loss to rank the similarity score of a batch data feature vector. As shown in Figure 4, the aim of the triplet loss is to close the distance between the anchor and positive and extend the distance between the anchor and negative. Unlike face recognition, we utilize a triplet loss network to find abnormal data sequences while face recognition uses it to find images belonging to the same face. The embedding is represented by \(f(x)\in R^{d}\); it embeds a data sequence into a d-dimensional Euclidean space. Furthermore, we constrain this embedding to live on the d-dimensional hypersphere [14], i.e., \(\left\|f(x)\right\|_{2}=1\). The function of triplet loss is to ensure that a data sequence \(X_{i}^{n}\) (anchor) of a special CAN bus data sequence is more similar to \(X_{i}^{p}\) (positive) and dissimilarity to \(X_{i}^{n}\) (negative). Then the formula becomes:  
\[\left\|f\left(x_{i}^{a}\right)-f\left(x_{i}^{p}\right)\right\|_{2}^{2}+\alpha< \left\|f\left(x_{i}^{a}\right)-f\left(x_{i}^{n}\right)\right\|_{2}^{2} \tag{6}\]  
\[\forall\left(f\left(x_{i}^{a}\right),f\left(x_{i}^{p}\right),f\left(x_{i}^{n} \right)\right)\in\Gamma \tag{7}\]  
where \(\alpha\) is a margin that is enforced between the positive and negative pair. \(\Gamma\) is the set of triplets in the training data sequences. Then we minimize the loss by utilizing the formula:  
\[L=\sum_{i}^{N}\left[\left\|f\left(x_{i}^{a}\right)-f\left(x_{i}^{p}\right) \right\|_{2}^{2}-\left\|f\left(x_{i}^{a}\right)-f\left(x_{i}^{n}\right)\right\| _{2}^{2}+\alpha\right]_{+} \tag{8}\]  
The purpose of the loss function is to make the reduction of loss in the training iteration as small as possible. In other words, it should be as close as possible to positive as the anchor, and as far as possible between the anchor and negative. Based on the above, we analyze the value of the margin next. The smaller the margin value, the easier it is for loss to approach zero. Therefore, both the anchor and the positive need not be pulled too close, and the anchor and the negative need not be pulled too far, so that loss can quickly approach zero. As a result of such training, it cannot distinguish similar data well. When the margin is larger, it is necessary for the network parameters to narrow the distance between the positive and the anchor and pull the distance between the negative and the anchor. If the margin value is set too high, it is likely that the value of loss will be large, which makes it difficult to approach zero. Therefore, it is critical to set a reasonable margin value, which is an important indicator of similarity. In fact, to generate all possible triples will result in many easily satisfied triples, which will not affect the operation of training, and lead to a slower convergence rate. The key of the triplet loss network is to choose hard triplets, which are positive, and can therefore help improve the model. However, the online generation and use of large mini-batches in the order of a few thousand exemplars only compute the \(\operatorname*{argmax}_{x_{i}^{n}}\left\|f\left(x_{i}^{a}\right)-f\left(x_{i} ^{n}\right)\right\|_{2}^{2}\) and \(\operatorname*{argmax}_{i}^{p}\left\|f\left(x_{i}^{a}\right)-f\left(x_{i}^{p} \right)\right\|_{2}^{2}\) within a mini-batch. In addition, to have a meaningful representation of anchors, it is necessary to ensure that a minimum number of exemplars of any one identifier appear in each mini-batch.