## 5 Performance Evaluation
### Performance of Detection and Mitigation Mechanism  
This subsection mainly tests the detection and mitigation mechanism based on the dynamic model and security detection. Considering the safety of the heavy-duty robot and the simplicity of the test, we established a test platform in the laboratory environment. As shown in Figure 9, we use a self-developed 7-DOF manipulator to simulate the heavy-duty robot. The application operating system and communication bus are the same as the heavy-duty robot. The data processing is completed in the cloud server, which communicates with the robot controller via the Ethernet. An IPC is used to simulate attackers, while receiving processed data from the cloud and randomly injecting some malicious control commands to replace data from the cloud server. The simulation attacker here adopts a PC software written in C# to send control commands containing malicious instructions to detect the effectiveness of the intrusion detection and mitigation mechanism. In this process, there is no change in the transmission of the communication protocol and the volume of data. Furthermore, in order to improve the detection efficiency of the system, only the dynamic model of the first three joints of heavy-duty robots was modelled. This is reasonable for that the first three joints contribute most to the robot's positions, while the other four joints mainly affecting the orientation of the manipulator [23]. The 10 frames of data are sent per second between the simulated attacker and robot controller. Of course, the frequency is adjustable. In order to test the performance of the detection and recovery mechanism, we did something based on the established test platform as follows:  
1. From the 5 ths of communication establishment, the injected commands randomly changed 1-2 continuous data with the information of joint torque, rotation angle, and trajectory. 2. From the 15 ths, the aforementioned information in 3-4 continuous data was randomly changed. 3. From the 25 ths, the aforementioned information in 3-5 continuous data was randomly changed. 4. The above steps were repeated in 15 sets of data. Meanwhile, the experimental phenomena and the number of alarms record were recorded by log records. Finally, we got the results of the test. Figure 9(a) gives part of the attack data in the red line, and one group of data detection results is shown in Figure 9(b). In the Figure 9(b), 1 indicates that an attack is detected and the mitigation mechanism was triggered. 0 means that no attack is detected, that is, the data are trusted. There is no security alarm on the left side of the red line, while a security alarm occurs per second on the right side of the red line except the 23 ths. Figure 9: Test platform. The statistics of the corresponding log records are shown in Table 3. We find that the log records are in accordance with the corresponding detection attack results. There is an undetected attack at 23.8 s, which means that the modified data (at least one of the data) are within the range of trustworthy thresholds. The analysis is consistent with the Table 3. In the 33.4-33.8 s, targeted attacks have been detected five times in a row, triggering a safe shutdown mechanism. During the test, there was no manipulator shaking or sudden braking occurred from the log records, which proved that the pre-estimation instructions of the dynamic model could eliminate transient and discontinuous attacks. When the attacks are continuous and can't be relieved by the mitigation mechanism, the mitigation mechanism will ensure a safe shutdown. Then, we expanded the number of trials to 100 sets. We got that the detection accuracy is 94.0% and the true positive rate is 97.8%, which proved the effectiveness of the proposed intrusion detection and mitigation mechanism. In summary, the detection and mitigation mechanism of heavy-duty robots based on IDBN and dynamic model can effectively mitigate targeted attacks, which come from cyber-domain and physical-domain.