###### Abstract  
Intelligent connected vehicles (ICVs) are equipped with extensive electronic control units which offer convenience but also pose significant cybersecurity risks. Penetration testing, recommended in ISO/SAE 21434 \"Road vehicles--Cybersecurity engineering\", is an effective approach to identify cybersecurity vulnerabilities in ICVs. However, there is limited research on vehicle penetration testing from a black-box perspective due to the complex architecture of ICVs. Additionally, no penetration testing framework has been proposed to guide security testers on conducting penetration testing for the whole vehicle. The lack of framework guidance results in the inexperienced security testers being uncertain about the processes to follow for conducting penetration testing. Moreover, the inexperienced security testers are unsure about which tests to perform in order to systematically evaluate the vehicle's cybersecurity. To enhance the penetration testing efficiency of ICVs, this paper presents a black-box penetration testing framework, ICVTest. ICVTest proposes a standardized penetration testing process to facilitate step-by-step completion of the penetration testing, thereby addressing the issue of inexperienced testers lacking guidance on how to initiate work when confronted with ICV. Also, ICVTest includes 10 sets of test cases covering hardware and software security tests. Testers can select appropriate test cases based on the specific cybersecurity threats faced by the target object, thereby reducing the complexity of penetration testing tasks. Furthermore, we have developed a vehicle cybersecurity testing platform for ICVTest that seamlessly integrates various testing tools. The platform enables even novice testers to conduct vehicle black-box penetration testing in accordance with the given guidance which addresses the current industry's challenge of an overwhelming number of testing tasks coupled with a shortage of skilled professionals. For the first time, we propose a comprehensive black-box penetration testing framework and implement the framework in the form of a cybersecurity testing platform. We apply ICVTest to evaluate an electric vehicle manufactured in 2021 for assessing the framework's availability. With the aid of ICVTest, even testers with limited experience in automotive penetration can effectively evaluate the security risks of ICVs. In our experiments, numerous cybersecurity vulnerabilities were identified involving in-vehicle sensors, remote vehicle control systems, and in-vehicle controller area network (CAN) bus. intelligent connected vehicles; penetration testing; black-box; cybersecurity; test case set 4.0.0.0000-0000-0000-0000-0000-00000-00000-00000-00000-00000-00000-00000-00000-000000-000000-0000000-  
security mechanisms create favourable conditions for attackers to control vehicles [7; 8; 9; 10]. Inadequate security protection regarding in-vehicle sensors pose significant cybersecurity challenges to autonomous driving [11; 12; 13]. Cybersecurity threats increase the possibility of malicious attacks, triggering numerous cybersecurity attacks, and endangering the safety of vehicles and people. To tackle cybersecurity challenges in the automotive industry, the ISO/SAE 21434 \"Road vehicles--Cybersecurity engineering\" [14] standard provides detailed specifications for cybersecurity activities throughout all stages of the vehicle life cycle to prevent potential cybersecurity risks. As a result, ISO/SAE 21434 has emerged as the industry's prevailing standard for managing cybersecurity. Both automobile manufacturers and suppliers systematically assess product's cybersecurity through penetration testing during development and validation phases to ensure compliance with vehicle type approval (VTA). Penetration testing is a crucial approach for identifying potential cybersecurity vulnerabilities in ICVs throughout their entire life cycle.