## 3 Black-Box Penetration Testing Framework for ICVs
### Penetration Testing Case Set in ICVTest
ICVs may face cybersecurity threats across 10 fields depicted in Figure 2. Within the vehicle, the on-board ECU consists of a hardware board with firmware or an operating system running on it. Different ECUs are interconnected through the vehicle bus to form an in-vehicle network. Outside the vehicle, sensors enable the automobile to perceive its surrounding environment and internal state. Sensor data is transmitted via the bus system to corresponding ECUs for analysis purposes. The radio communication module serves as a gateway between off-vehicle networks and the in-vehicle network. In addition to connecting vehicles with cloud service platforms, communication technology also facilitates connectivity with mobile terminal devices such as smartphones, introducing additional security risks for ICVs. Furthermore, the advent of ICVs brings forth significant concerns regarding data privacy while providing services. Based on the threats identified in Figure 2, ICVTest abstracts the vehicle cybersecurity test case set into 10 fields. The common cybersecurity threats that need to be verified by security tests are summarized in each test case set. 1. Hardware Security Test Case Set  
Hardware security threats encompass printed circuit board (PCB) vulnerabilities, processor chip weaknesses, memory chip susceptibilities, hardware debugging interface risks, and on-board bus vulnerabilities. In the case of PCB in ECU, there is potential for information leakage regarding integrated circuit (IC) chip models, interface details, and bus protocol. The processor chip may inadvertently disclose side channel information such as electromagnetic signals, timing data, and power consumption patterns [33] thereby  
Figure 2: Ten Security Fields of ICVTest. facilitating attackers to launch timing attacks [34], power consumption attacks [35], and electromagnetic attacks [36]. Additionally, fault injection techniques [37] pose significant security risks to processor chips. Attacks like voltage fault injection [38], electromagnetic fault injection [39], and laser fault injection [40] can alter the operational logic of the processor. Data storage chips are also susceptible to data residue threats. For instance, flash memory can be read by a programmer. Furthermore, hardware debugging interfaces like joint test action group (JTAG), serial wire debug (SWD), and USB provide attackers with opportunities to access internal storage data of the system on chip (SoC). Attackers may also monitor buses such as serial peripheral interface (SPI) or inter-integrated circuit (I2C), leading to potential data leakage risks. 2. \(\quad\)  
Firmware Security Test Case Set  
Private information, such as passwords, keys, network resource addresses, usernames, and email addresses, may be stored in plain text within the firmware. Malicious attackers can decipher the operational logic of the TOT through reverse engineering techniques. Furthermore, these attackers can even assess whether the target firmware possesses cyber-security vulnerabilities within real-world operating environments using dynamic analysis. 3. \(\quad\)  
In-Vehicle Bus Security Test Case Set  
ICVs employ various in-vehicle bus protocols, including CAN bus, FlexRay bus, local interconnection network (LIN) bus, media oriented system transport (MOST) bus, and in-vehicle ethernet bus. On one hand, certain in-vehicle bus protocols lack essential security mechanisms such as encryption, authentication, and integrity to meet the specific requirements of low-latency communication within vehicles. On the other hand, some application layer protocols for in-vehicle buses allow external individuals to access and control the vehicle's functionalities. Examples include unified diagnostic services (UDS) and scalable service-oriented middleware over IP (SOME/IP). The absence of adequate security measures renders these in-vehicle buses vulnerable to potential attackers who can gain control over them, resulting in severe consequences. 4.