Making the right decision in traffic is a challenging task that is highly dependent on individual preferences as well as the surrounding environment. Therefore it is hard to model solely based on expert knowledge. In this work we use Deep Reinforcement Learning to learn maneuver decisions based on a compact semantic state representation. This ensures a consistent model of the environment across scenarios as well as a behavior adaptation function, enabling on-line changes of desired behaviors without re-training. The input for the neural network is a simulated object list similar to that of Radar or Lidar sensors, superimposed by a relational semantic scene description. The state as well as the reward are extended by a behavior adaptation function and a parameterization respectively. With little expert knowledge and a set of mid-level actions, it can be seen that the agent is capable to adhere to traffic rules and learns to drive safely in a variety of situations.
Due to the limitations of state machines, current research has expanded on the initial efforts by creating more complex and formal models: A mixture of POMDP, stochastic non-linear MPC and domain knowledge can be used to generate lane change decisions in a variety of traffic scenarios @cite_33 . Capturing the mutual dependency of maneuver decisions between different agents, planning can be conducted with foresight @cite_13 @cite_18 . While @cite_13 plans only the next maneuver focusing on the reduction of collision probabilities between all traffic participants, @cite_34 explicitly addresses longer planning horizons and the replanning capabilities of others.
@cite_18 @cite_34 @cite_13 @cite_33
2344985987 2134239466 1919962963
This paper presents a novel cooperative-driving prediction and planning framework for dynamic environments based on the methods of game theory. The proposed algorithm can be used for highly automated driving on highways or as a sophisticated prediction module for advanced driver-assistance systems with no need for intervehicle communication. The main contribution of this paper is a model-based interaction-aware motion prediction of all vehicles in a scene. In contrast to other state-of-the-art approaches, the system also models the replanning capabilities of all drivers. With that, the driving strategy is able to capture complex interactions between vehicles, thus planning maneuver sequences over longer time horizons. It also enables an accurate prediction of traffic for the next immediate time step. The prediction model is supported by an interpretation of what other drivers intend to do, how they interact with traffic, and the ongoing observation. As part of the prediction loop, the proposed planning strategy incorporates the expected reactions of all traffic participants, offering cooperative and robust driving decisions. By means of experimental results under simulated highway scenarios, the validity of the proposed concept and its real-time capability is demonstrated. The probabilistic prediction of road traffic scenarios is addressed. One result is a probabilistic occupancy of traffic participants, and the other result is the collision risk for autonomous vehicles when executing a planned maneuver. The probabilistic occupancy of surrounding traffic participants helps to plan the maneuver of an autonomous vehicle, whereas the computed collision risk helps to decide if a planned maneuver should be executed. Two methods for the probabilistic prediction are presented and compared: 1) Markov chain abstraction and 2) Monte Carlo simulation. The performance of both methods is evaluated with respect to the prediction of the probabilistic occupancy and the collision risk. For each comparison test, we use the same models that generate the probabilistic behavior of traffic participants, where the generation of these data is not compared with real-world data. However, the results independently show the behavior generation that Markov chains are preferred for the probabilistic occupancy, whereas Monte Carlo simulation is clearly preferred for determining the collision risk. Estimating and predicting traffic situations over time is an essential capability for sophisticated driver assistance systems and autonomous driving. When longer prediction horizons are needed, e.g., in decision making or motion planning, the uncertainty induced by incomplete environment perception and stochastic situation development over time cannot be neglected without sacrificing robustness and safety. Building consistent probabilistic models of drivers interactions with the environment, the road network and other traffic participants poses a complex problem. In this paper, we model the decision making process of drivers by building a hierarchical Dynamic Bayesian Model that describes physical relationships as well as the driver's behaviors and plans. This way, the uncertainties in the process on all abstraction levels can be handled in a mathematically consistent way. As drivers behaviors are difficult to model, we present an approach for learning continuous, non-linear, context-dependent models for the behavior of traffic participants. We propose an Expectation Maximization (EM) approach for learning the models integrated in the DBN from unlabeled observations. Experiments show a significant improvement in estimation and prediction accuracy over standard models which only consider vehicle dynamics. Finally, a novel approach to tactical decision making for autonomous driving is outlined. It is based on a continuous Partially Observable Markov Decision Process (POMDP) that uses the presented model for prediction. In order to determine a cooperative driving strategy, it is beneficial for an autonomous vehicle to incorporate the intended motion of surrounding vehicles within its own motion planning. However, as intentions cannot be measured directly and the motion of multiple vehicles often are highly interdependent, this incorporation has proven challenging. In this paper, the problem of maneuver estimation is addressed, focusing on situations with close interaction between traffic participants. Therefore, we define collective maneuvers based on trajectory homotopy, describing the relative motion of multiple vehicles in a scene. Representing maneuvers by sample trajectories, maneuver-dependent prediction models of the vehicle states can be defined. This allows for a Bayesian estimation of maneuver probabilities given observations of the real motion. The approach is evaluated by simulation in overtaking scenarios with oncoming traffic and merging scenarios at an intersection.