# A Deep Similarity Metric Method Based on Incomplete Data for Traffic Anomaly Detection in IoT
## 3 Methodology
### Deep Transfer Learning Method
\[L_{stagel}=L_{cls}+\lambda_{1}L_{mmd}+\lambda_{2}\
orm{W}^{2} \tag{4}\]  
\[L_{mmd}=\frac{1}{B}\
orm{\sum_{i=1}^{B}\left[fca\left(A_{I}^{i}\right)-fca \left(A_{u}^{i}\right)\right]}^{2} \tag{5}\]  
\[L_{cls}=-\frac{1}{B}\sum_{i=1}^{B}\sum_{c=0}^{C-1}\delta\left(\gamma^{i}=c \right)\log p_{cls}^{c} \tag{6}\]  
\(\lambda_{1}\) and \(\lambda_{2}\) in Formula (4) are two balanced parameters. \(\
orm{W}^{2}\) is the regularization term to prevent overfitting. \(B\) is the size of the mini-batch during training. \(C\) is the number of categories defined for anomalies. \(fca(\cdot)\) represents the output of adaption layer. \(p_{cls}^{c}\) is the predicted probability of \(c^{\prime}\)th category. \(\delta\) is a 0-1 loss function. \(\gamma^{i}\) is the label of \(i^{\prime}\)th anomaly. In the first training stage, we use the classification layer to predict the unannotated anomalies simultaneously, in order to presumably infer the categories of anomalies. A soft label is defined as the category of the maximum probability predicted by the model in the whole process of transfer learning. The mathematical form is shown as follows:  
\[SL=\underset{c\in[0,C-1]}{\text{argmax}}\left[\frac{1}{T}\sum_{i=1}^{T}p_{ soft}^{c}\right] \tag{7}\]  
where \(T\) is the testing times of all the unannotated anomalies. \(p_{soft}\) is the predicted probability vector of the unannotated data. Figure 5: Transfer learning in two stages. (**a**) First stage transfer: fine-tuning with an adaptive fully connected layer. (**b**) Second stage transfer: knowledge transfer with soft labels for joint learning. In the first stage, the network is fine-tuned by the labeled image while predicting the soft labels for the unlabeled image. In the second stage, the network is trained by both labeled image and unlabeled image. A fully connected adaptive layer is adopted for feature transfer. In fact, the unlabeled data is predicted by the model every time the transfer training is performed. The output of the softmax function is a probability vector, the values of which reflects the possibility that it belongs to each category. Finally, categories corresponding to maximum values are considered as the soft labels according to the mean predictions throughout the training process. In the second training stage, both annotated data and unannotated data are used to train the similarity metric network. Except for the FC layer, the parameters of each layer are initialized with these trained in the first stage. When transferring the model from the first stage to the second stage, only the FC layer need to be completely trained while other layers just need to be fine-tuned. The outputs of the FC layer are two values, indicating their similarity and dissimilarity probabilities of the two inputs respectively. The softmax with cross-entropy loss is used to measure the similarity:  
\[L_{simi}=-\frac{1}{B}\sum_{i=1}^{B}\sum_{s=0}^{1}\delta\Big{(}S^{i}=s\Big{)} \log p_{sim}^{s}\Big{(}A1_{I}^{i},A2_{I}^{i}\Big{)}-\frac{\lambda}{B}\sum_{i=1} ^{B}\sum_{s=0}^{1}\delta\Big{(}S^{i}=s\Big{)}\log p_{sim}^{s}\Big{(}A1_{I}^{i},A2_{u}^{i}\Big{)} \tag{8}\]  
where \(A1\) and \(A2\) are the two inputs of two branches. The loss function contains two parts. If one of the inputs is the anomalies with the soft label, a coefficient \(\lambda\) is set to weaken the influence of this part. \(S^{i}\) is the target similarity. If two inputs and the embedded word vector belong to the same category, we set \(S^{i}=1\). In order to make the network converge quickly on the incomplete data-sets, we control the class of input samples of one branch to be the same as that of word vectors. In the same way, the categories of samples for comparison and the word vector are unified. In this way, the similarity measurement of anomalies based on the complementary text is realized.