# A Survey of Scenario Generation for Automated Vehicle Testing and Validation
## 3 Scenario Generation Methods
### Adaptive Test Scenario Generation Methods
#### 3.2.1 Reinforcement-Learning-Based Methods
The article pointed out that AVs can be flexibly tested through a large number of interactions between the AV and surrounding vehicles (SVs), and the different behaviors and strategies of the SVs play a crucial role in this process. The different driving behaviors of the SVs constitute hypothetical factors in the test scenarios, so a better description of the driving behaviors of SVs is needed. The article used a utility function to model and generate aggressive, conservative, and normal driving policies by adjusting the hyperparameters of the utility function, because a utility function-based method can combine multiple driving decisions to model driving behaviors, and therefore can be more easily extended to describe rich and complex behaviors. However, this method may have limitations in capturing the full range of interactions between an AV and surrounding vehicles. This may affect the comprehensiveness of the test scenario, as some subtle driving behaviors and interactions may not be fully represented. In [45], an interactive critical scenario generation method based on in-depth crash data was proposed. First, this method extracted the crash data that occurred at an intersection from the China In-depth Mobility Safety Study-Traffic Accident (CIMSS-TA) database to generate an initialized scenario state, including the relative position, speed, direction, driving intention, and acceleration of the ego vehicle (the AV under test) and the objective vehicle. A conditional tabular generative adversarial network (CTGAN) was then used to extrapolate the scenario, to increase its diversity. Then, the TD3 algorithm was used as the driving algorithm for the BV. This makes use of Baidu's \"Apollo\" platform, which is an open-sourced autonomous driving ecosystem that provides a suite of tools and algorithms for the development and deployment of AVs. It supports key functionalities such as perception, planning, control, and localization. In the study, Apollo was deployed within the SVL simulator to perform three distinct driving tasks: turning left, going straight, and turning right at intersections, reconstructed from crash data. These tasks allowed the system to engage in dynamic interactions with the BV controlled by the TD3 algorithm. Through these interactions, critical and realistic scenarios were created, challenging Apollo's decision-making and control algorithms. The authors also developed a simulator platform called RL-Scenario and used this platform to train an RL agent model. Crash rate, generalized-time-to-collision (GTTC), and post-encroachment time (PET) were the evaluation metrics used for analysis. The proposed method is based on real traffic accident data, especially intersection accident data, so the generated scenarios reflect high-risk situations that may occur on actual roads. In addition, this method uses a CTGAN, which can generate diverse test scenarios and expand the coverage of scenarios. The TD3 reinforcement learning algorithm is used to control the objective vehicle, so that it can dynamically adjust its behavior according to the actions of the ego vehicle. This interactive scenario generation can effectively simulate complex driving situations. However, since this method relies on historical accident data, and accidents themselves are low-probability events, although it can generate high-risk scenarios, it may ignore some potential risk scenarios that have not yet occurred, especially those based on natural driving data (NDD) scenarios. And although the scenario library was expanded through a CTGAN, some generated scenarios may be too extreme and may not fully comply with actual road conditions. In addition, this method was mainly verified on the Baidu Apollo, and different autonomous driving systems use different control algorithms, which means that this method may lack adaptability to other autonomous driving platforms and have poor versatility.