# A Survey of Vehicle to Everything (V2X) Testing
## 5 V2X Testing Methods
### Function Testing  
Application function testing can be used to determine whether an application can be triggered and take reasonable actions in different scenarios. It can guarantee the reliability and effectiveness of V2X applications. According to the environment, function testing can be divided into two types: laboratory testing and field testing. There is no doubt that the field testing is closest to reality. But if we want to do the V2X field testing, the testing scenarios will cost a lot of time and money. Besides, most of the scenarios are hard to build and the number of scenarios is huge. So a virtual function testing system in the lab is a perfect solution. A virtual function testing system can be built only by software simulators, which means that all the testing scenarios are virtual. [PERSON] and [PERSON] [62] presented a simulation framework for testing and evaluation, which was aimed to test platooning. They used VTI for driving simulation and Plexe for traffic and network simulations. All the simulations were virtual and the framework had advantage that it was cheap to build and easy to extend its function such safety warning. But it depended  
Figure 4: Test System Architecture Based on TTCN-3 [57]. severely on the models such as communication model and traffic model. If a model didn't accurately reflect the real situation, the result of testing could be wrong leading invalid testing. [PERSON] and [PERSON] [63] contrasted several simulators for VANETs which were old but some of them has been updated. They did the comparative study from the aspect of software characteristics, graphical user interface, accuracy of simulation, ease of use, popularity, input requirements and output visualization capabilities. Accuracy of simulation and ease of use were the most important for a simulator because they impacted directly on the testing results. [PERSON] and [PERSON] [64] had proposed a V2 XREF of V2X Runtime Emulation Framework that can be used to perform the evaluation and validation of vehicle safety applications and safety systems along the various safety scenarios. Their emulation framework was also based software simulation and could be used in function testing. [PERSON] and [PERSON] [65] had presented a new virtualization-based approach for emulating VANET by which application function testing could be performed. [PERSON] [66] designed the simulation environment for V2X protocol and application testing. The test environment is divided into three parts, traffic simulators using VISSIM, network simulators using NS-3, and application simulators using MATLAB. The test environment adopts software simulation, so the underlying physical communication process cannot be tested. And because the tested application is not real and complete application software, it can only be tested against the application core algorithm and performance requirements. [PERSON] [67] designed a test environment for testing V2X applications including three subsystems: input subsystem using NS-2, core subsystem to manage data and client framework subsystem providing a virtual test environment. [PERSON] and [PERSON] [68] built a general testing framework based on Veins for securing VANET applications. [PERSON] and [PERSON] [69] tested a Platooning Management Protocol with VSimRTI framework. The virtual simulation has its limitations which can't accurately reflect objective facts. Traditional Hardware-in-the-Loop (HIL) can be used in the V2X testing. [PERSON] [70; 71] proposed the Ego-Vehicle Interface (EVI) to integrate these very different types of simulators including HIL simulators and VANET simulators. [PERSON] and [PERSON] [72] designed a hardware-in-the-loop (HiL) V2X simulation framework to offer a cost-efficient and simple toolset. In the function testing with HIL, the device under test could be seen as a black-box and the testing environments were generated by the simulators. Its advantage is that we can test the V2X devices instead of the applications or algorithms.