# A Survey of Scenario Generation for Automated Vehicle Testing and Validation
## 3 Scenario Generation Methods
### Adaptive Test Scenario Generation Methods
#### 3.2.1 Reinforcement-Learning-Based Methods  
The reinforcement-learning-based approach is to use a BV to automatically generate challenging driving scenarios. The agent interacts with a virtual environment, assesses the current scenario, takes appropriate actions, and receives rewards based on the AV's performance. Through continuous optimization of its strategy, the agent generates scenarios that push the AV system's performance to its limits. This method allows for the exploration of complex, diverse, and extreme test conditions, enhancing the test coverage and improving system safety. In [14], a set of human-like social-driven models were built to generate evolving test scenarios. The study combined level-k game theory [41] and DRL frameworks based on previous work to train game-like driving policies. Level-k game theory refers to the hypothesis that participants in a game predict the behavior of others based on different levels of reasoning and formulate corresponding strategies, where each level of reasoning is based on the expectation of lower-level behavior [42]. In the study, level-k game theory was used to build the interaction logic between intelligent driver models and DRL was used to evolve the interaction between them, while the driving model was given game-like driving policies of competitive, mutual, and cooperative by shaping the reward function. Among these, a competitive driving policy means that the driving behavior of the car executing this driving policy is very aggressive. In a lane-changing scenario, it will constantly try to change lanes. In a car-following scenario, it will always try to follow the car at a shorter distance. This driving policy simulates drivers who behave aggressively in the real world. A mutual driving policy simulates the driving behavior of ordinary human drivers in the real world. Under this driving policy, if the vehicle finds that there are other vehicles in front trying to change lanes into its own lane, the vehicles with the mutual driving policy will slow down appropriately to allow the vehicle in front to change lanes more safely. If it is found that the vehicle in front is still driving slowly after changing lanes, the vehicles with the mutual driving policy will try to change to other lanes to restore a normal driving speed. The vehicle behavior of a cooperative driving policy is relatively more conservative. Vehicles executing this driving policy will generally not actively change lanes in lane changing scenarios, unless there are special circumstances. When other vehicles change lanes to their lane, they will obviously slow down to avoid the lane-changing vehicles. The TD3 algorithm [43] was then used to realize continuous control of the car's following behavior and human-like social driving policies for ego vehicle decision-making. Finally, these driving models were used in BVs to generate evolving scenarios. The generated test scenarios were highly efficient for AV testing and the generated driving environment was realistic and challenging. However, the paper only used three driver models, which does not fully represent the diversity of human driving behaviors, may limit the range of scenarios that can be generated, and may not fully capture the complexity of real-world driving scenarios. Additionally, the generated driving policies lacked interpretability. The paper used deep reinforcement learning to train the driver models, which can make policies difficult to interpret and understand. This can make it challenging to identify the specific factors that influenced the background vehicle behavior in a generated scenario. A sampling method of interpretable, operable corner cases was proposed in [44]. This was based on the idea of sampling a small number of challenging scenarios from all possible scenarios to test autonomous vehicles.