# A Survey of Scenario Generation for Automated Vehicle Testing and Validation
## 3 Scenario Generation Methods
### Adaptive Test Scenario Generation Methods
#### 3.2.1 Reinforcement-Learning-Based Methods
Finally, this method only considers the interaction between a single objective vehicle and an ego vehicle, and does not involve complex interaction scenarios between multiple vehicles. In reality, traffic conditions are often more complex, so it is necessary to consider generating scenarios involving multiple data sources and interactions between multiple vehicles. There have been some articles that clustered the generated scenarios after using reinforcement learning to train the driving strategy of BVs, because clustering can help better manage the generated corner cases and be better applied in the subsequent testing of autonomous vehicles. The study in [46] proposed a unified framework for generating corner cases for decision-making systems in autonomous vehicles. Since the real-world traffic environment is high-dimensional, the article proposed using a Markov decision process (MDP) to describe the traffic environment, and used a deep Q-network (DQN) [47] in DRL to learn the optimal behavior policy of a BV, thereby making the BV behave more aggressively when interacting with the CAV, and therefore systematically generating corner cases in complex driving environments. In order to better manage the generated corner cases and apply them in subsequent tests, the article used principal component analysis (PCA) [48] to reduce the dimensions of the corner cases and extract the principal features, and then used K-means [49] and DBSCAN [50] algorithms to cluster the corner cases to identify the most valuable corner cases. The proposed method could comprehensively evaluate the performance of an AV in various scenarios by generating and analyzing corner cases, which can help identify potential vulnerabilities of the AV system and improve the security of the system. Furthermore, the method can help improve the performance of AVs using feature extraction and clustering techniques to identify valuable corner cases, which can reveal specific challenges and critical situations. Despite that, the training of deep reinforcement learning models for generating corner cases can be very time-consuming and computationally resource-intensive, especially for complex scenarios. Second, the proposed method has limited applicability to different driving environments and scenarios. It is currently limited to highway driving environments and may not be directly applicable to other driving scenarios such as urban driving environments. Therefore, further adjustment and expansion are needed to achieve a wider range of applications. Finally, this method uses a DRL model to generate corner cases, and the DRL model may lack interpretability, making it difficult to understand the AV's decision-making process in various scenarios. Another study that used clustering techniques to classify generated risk scenarios was [51]. The article used deep reinforcement learning to generate adversarial environments for the evaluation of an ego vehicle. In order to better simulate the mixed cooperation and competition interaction between AVs and environment vehicles, the article designed a non-zero-sum reward function based on domain knowledge and traffic rules, and expressed the scenarios as an MDP. During training, ensemble reinforcement learning was used to collect various risk scenarios. After training, a non-parametric Bayesian approach was used to cluster the generated risk scenarios to increase the collision rate between the AVs and environment vehicles. The framework proposed with this method is adaptive, so it can generate a temporal adversarial environment based on the behavior of the vehicle being tested, thereby providing a dynamic and responsive evaluation method, so the generated scenarios are more realistic. In addition, by designing a non-zero-sum reward function, a mixed cooperative and competitive interaction relationship between the ego vehicle and the environment vehicle is better simulated, making the scenario more realistic.