# V-SOC4 AS: A Vehicle-SOC for Improving Automotive Security
## 5 Experimental Results and Discussion
### Fuzzing Attack  
In this attack, an attacker sends a random ID and a CAN data frame. The following approach was considered to detect the fuzzing attack with SIEM: each ECU has its identifier determined at the design phase and consequently, the SOC analyst could create the database with ECU identifiers. For this reason, a reference set was created on QRadar by putting CAN IDs without attacks. Usually, a reference set contains all unique values that can also be used to store business data such as IP address, and username. The reference set contains the unique CAN IDs for the OpelAstra and RenaultClio dataset. These IDs have been taken from \"full_data_capture.log\" present in the OpelAstra and RenaultClio dataset that contains 2690069 packets captured with a total of 85 unique CAN IDs for the first dataset, and 386567 packets captured with 55 unique CAN IDs for the second dataset [41]. In Figure 14, the JSON messages regarding the Fuzzing attack were created in this way:  
* The first five messages are used for the OpelAstra vehicle. The last five for the RenaultClio vehicle. The UUID related to the OpelAstra will not be changed, while a new one is generated for the RenaultClio. * The UUID_ECU is the same for both vehicles. In this way, it is possible to simulate two vehicles that mount the same ECU and are attacked at the same time. Figure 15 shows the rule created to identify the fuzzing attacks. The first row works similar to the DoS rule. The second row is used to avoid being recognized as an event of DoS attack. In the third row, if the ID of the current payload sent to QRadar does not exist in the table, the offense will be raised. The actions chosen for the fuzzing attack are shown in Figure 16. The severity value was set to eight because in this case, this attack can impair the behavior of the vehicle by sending random identifiers. Credibility and relevance are the same as the DoS attack. Figure 14: The JSON messages created to test the Fuzzing rule. Figure 15: Fuzzing rule in QRadar. Figure 17a shows the generated offense when the five fuzzing messages are sent and analyzed by the SIEM. This offense corresponds to the OpelAstra (green rectangle corresponds to the UUID). Instead, Figure 17b shows the offense generated for the RenaultClio. Figure 16: The actions chosen for the fuzzing attack. Figure 17: Generated offenses when a fuzzing attack is detected. (**a**) Shows the offense regarding the vehicle model OpelAstra with five identified attacks (red rectangle). In (**b**) the figure refers to the RenaultClio with five identified attacks. The reference set contains the CAN_IDs detected by the SIEM as a fuzzing attack. In addition, the reference map of set contains the vehicleModel as a key and the UUID_ECU as a value. In this approach, using the reference map of set, if the attack occurs in multiple vehicles, the information of where the attack happened and whether multiple vehicles are involved are available. For example, with the UUID of the ECU, the SOC analyst could control if other vehicles mount the same ECU and proceed to implement security measures before the attack is carried out in these vehicles. The metrics accuracy, precision, recall, and F1-score are used to evaluate the SIEM's detection performance [22]. For the DoS attack, the number of injected messages analyzed was 40.015. As we see in Section 5.2 Figure 13, with the created rule, the SIEM successfully analyzes all the messages. In this case, the SIEM obtained 100% accuracy, precision, recall, and F1-score because all 40.015 messages were recognized correctly (see Section 5.1 for the dataset details). Considering the fuzzing attacks, Section 5.3 Figure 17a,b show that SIEM was able to detect all 10 CAN ID attacks for both vehicle datasets. In addition, IBM QRadar SIEM reaches 100% accuracy, precision, recall, and F1-score. Table 2 summarizes the results obtained with IBM QRadar SIEM. The results show that the SIEM, while using a rule-based system, can identify attacks that might occur on the CAN bus.