# Evaluation of a Reputation Management Technique for Autonomous Vehicles
## 7 Results  
The results demonstrate the efficacy of the system. The three malicious vehicles all received significantly lower operating and reporting scores than the normal vehicles. These results are depicted in Figure 5, which shows the average operating and reporting scores that were generated from running the base model 1000 times for 5000 turns with 30 vehicles. Notably, the operating score for the malicious vehicles was much lower than the reporting score. This is likely due to the fact that the methods for calculating operating reputation are more punitive against non-credible transactions. With an average operating error of 0.175 and reporting error of 0.160, the data indicates that the simulation was able to effectively distinguish malicious vehicles from normal ones. Focus now turns to analyzing the impact of system parameters. First, the impact of velocity is assessed. Figure 6 shows that error remains relatively steady for velocities from 0.1 to 0.7, but operating error exhibits a noticeable increase once velocity goes beyond 0.7. For vehicles whose most recent transactions were rated poorly, a high velocity will adversely affect their reputation. The inverse applies as well, with high velocities having the potential to greatly increase a vehicle's reputation if its last transactions were rated positively. As a result, an increased velocity creates higher variability in the reputation of vehicles since it is more dependent on the most recent transactions, regardless of the vehicle's prior history. The reporting error was less affected by increased velocity because it is unlikely that the few malicious witnesses will outweigh the score of the other normal witnesses. This is due to the randomization and distribution of vehicles. The velocity parameter was effective at aggregating reputation across multiple transactions since the average operating and reporting error were both below 0.2 for a majority of velocity values. While the difference in error remains negligible for velocities below 0.7, this is likely due to the lack of variability  
Figure 5: Operating and reporting scores from a single simulation run. Figure 6: Average operating and reporting error from using various velocity values. in the simulation. Since vehicles did not exhibit any sporadic change in status (e.g., from normal to malicious), their most recent transactions were relatively representative of their behavior throughout the entire simulation. Future work may include the introduction of vehicle nodes that spontaneously (or over time) change their status by broadcasting more or fewer malicious transactions. This could reflect the real-world possibility of a vehicle having a change in ownership or being compromised by a cyber-criminal. Next, the impact of different levels of malicious vehicles is assessed. Figure 7 shows the results of the simulation with 10%, 50%, and 90% of the vehicles being malicious while the remaining 90%, 50%, and 10%, respectively, are normal. The proportion of malicious vehicles demonstrates a distinct positive correlation to the error of the simulation. At a propMalicious of 0.9, the network is dominated by malicious vehicles, normalizing this type of behavior. As a result, the error values are much closer to 1, as malicious vehicles are intentionally scoring transactions inaccurately. Observing the propMalicious at values of 0.5 and 0.9 shows that the effectiveness of the weighting mechanisms quickly deteriorates as the concentration of malicious vehicles in the network increases. These findings indicate that the model does not perform well in environments that are highly concentrated with malicious vehicles. This may indicate susceptibility to the system being vulnerable to mob-of-vehicle attacks. This result is due to the consensus-based approach to calculating reputation that heavily relies on a majority of witnesses being normal. While large-scale autonomous vehicle environments do not currently exist, the likelihood of a majority of vehicles being malicious is negligible, absent a wide-spread attack.