# Cyber-Physical Attack Detection and Recovery Based on RNN in Automotive Brake Systems
## 4 Experimental Results
### Mean Squared Error  
MSE can be generally used to investigate how accurately fusion algorithms calculate vehicle speeds. When the experiment is performed with test datasets of additional attack, MSEs of RF, TV, and WA are 1.904, 1.9089, and 1.7786, respectively (see Table 11). WA shows the lowest MSE among the algorithms. Since there are differences between the raw wheel speed data and the vehicle speed data as mentioned in Section 2, there also exist differences between the speed data estimated based on the wheel speed data and vehicle speed data.  
The reference vehicle speed and the vehicle speed estimated by the fusion algorithms during acceleration are illustrated in Figure 8a. The reference vehicle speed is plotted as stairs since Hyundai Motor provides the vehicle speed as only integer number via CAN. Figure 8b displays the enlarged part of the purple box in Figure 8a. Before 3.3 seconds there is no difference between performances of the algorithms, but after 3.3 seconds WA estimates the vehicle speed more closely. The vehicle speed and the wheel speed input to our system to explain the cause in the same range of Figure 8b are presented in Figure 8c. Attacks occur on RR and RL before 3.3 seconds while attacks happen on only RL after 3.3 seconds. As all of the three algorithms calculate average except data of sensors on which attacks occur, they are the same before 3.3 seconds. In case of WA after 3.3 seconds, the average is calculated by assigning high weights of rear sensors. Therefore, WA speed is calculated close to RR data.  
Figure 9a shows the reference speed and the speed estimated by the fusion algorithms while the vehicle speed is kept close to constant speed. Figure 9b illustrates the enlarged part of the purple box in Figure 9a. It is difficult to find differences between performances of the algorithms as the vehicle maintains the vehicle speed. We can see that the reference speed goes back and forth between 58 and 59 km/h and is represented as the only integer. Hence, the integer 1 km/h seems to change if 0.01 km/h, which is a specific decimal, changes.  
The reference speed and the speed estimated by the fusion algorithms during braking are displayed in Figure 10a. Figure 10b presents an enlarged part of the purple box in Figure 10a. The speed estimated by RF and TV is similar, but WA shows a closer estimation of vehicle speed. Wheel speed data for result analysis are shown in Figure 10c. All wheel speed sensors are in normal operation, and the data of FR and FL are actually close to the vehicle speed data. MSE of WA is lower than those of RF and TV because WA estimates vehicle speed by much weighting the front sensors during braking. Thus, WA, which assigns different weights depending on a situation, is safer than other algorithms in terms of handling data integrity. WA was applied with data of a sedan car in this paper. Further research is needed to make a system applicable to vehicles to have different data patterns.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Algorithms** & **Recursive Filter** & **Threshold Voting** & **Weighted Average** \\ \hline MSE & 1.904 & 1.9089 & 1.7786 \\ \hline \hline \end{tabular}
\end{table}
Table 11: Mean squared error (MSE).  
Figure 8: Comparison of fusion algorithms during acceleration: (**a**) Acceleration; (**b**) Vehicle speed and estimated speed; (**c**) Vehicle speed and wheel speed to inject false data into.  
Figure 9: Comparison of fusion algorithms during constant speed: (**a**) Constant speed; (**b**) Vehicle speed and estimated speed.