# [PERSON] SOM Architecture for Intrusion Detection on In-Vehicle Communication Networks
## 6 Experimental Results and Discussion
The XYF networks were trained on the expanded grids initialing the neurons codebook vectors in the \(R_{X}\) and \(R_{Y}\) maps using the neurons codebook vectors returned in output by the previously trained XYF network. The XYF network trained using the \(4\times 4\) neurons map greatly improved the performance of the model in all the datasets (Figure 3e). Although the high number of periodic, sporadic and even aperiodic messages with different CAN IDs were sent at different times and with different frequencies, the network returned with an accuracy of 100% in identifying attack messages in the _DoS_, _spoofing gear_ and _RPM_ datasets (Table 5). Training the XYF network with the \(8\times 8\) neurons map, we can see a slight improvement in accuracy only in the _fuzzy_ dataset. Despite the presence of messages with 2017 different CAN IDs sent totally at random in the CAN bus, the network achieved an accuracy of 99.70%. In the _DoS_ and _spoofing gear_ datasets, the accuracy was the same, and in the _spoofing RPM_ and _mixed_ datasets, it was even worse than in the network trained using the \(4\times 4\) neurons map (Table 5). The XYF-K algorithm was also implemented on the expanded neurons maps. The XYF-K algorithm slightly improved the accuracy of the XYF network in the _spoofing RPM_ and _fuzzy_ datasets. In Figures 8-10, the distances between each input message data vector \(X_{i}\) to the corresponding winning unit returned by the XYF network were plotted for each map size. Table 6 also shows the related metrics. The boxplots highlight a clear separation in the distances for the normal and attack messages in all the datasets. This is less evident in the _fuzzy_ dataset, due to the completely random structure of the dataset, and, consequently, in the _mixed_ dataset. Variability in the distance is very low in the network trained with a \(2\times 2\) map both for normal and for attack messages with standard deviation <0.01 in all the datasets. An increase in variability is evident when training the network using the expanded maps. These results are consistent with the accuracy of the models and confirm how clearly the XYF Kohonen network highlights a distinction between the attack and normal message data. The higher the separation in distance between the two groups, the higher the accuracy of prediction is. Figure 8: Distances between the input vectors and the winning neurons for XYF network on the \(2\times 2\) neurons map. Figure 10: _Cont._  
Figure 9: Distances between the input vectors and the winning neurons for the XYF network on the \(4\times 4\) neurons map. For the XYF-K algorithm, we computed all the evaluation metrics as described in Section 5.4. They are shown in Table 7. Precision, recall and F1 assume values of 100% in the _DoS_, _spoofing gear_ and _RPM_ datasets for all map sizes. High levels of precision are also shown for the _fuzzy_ dataset, 99.77% using the \(8\times 8\) neurons map, and for the _mixed_ dataset, 99.19% with the \(4\times 4\) neurons map. A relevant result is given by the FNR that is 0% in the _DoS_, _spoofing gear_ and _RPM_ datasets, 0.90% in the _fuzzy_ dataset and 1.61% in the _mixed_ dataset, both with \(4\times 4\) and \(8\times 8\) neurons maps. Table 8 shows a comparison between the XYF-K algorithm and other IDS approaches proposed to detect attack messages sent on the CAN bus. We compared the proposed method in terms of the system's accuracy with all the studies that, at the state of the art, tested different IDS approaches on the same dataset (**RQ2**).