title: A Hybrid Approach to Automated Driving Unifying Prediction and Planning
Creators: name: Rosbach, Sascha
Publisher: name: UNSPECIFIED
Year: 2024

description: Fully automated driving is nearing a stage of large-scale deployment, where the vehicles will interact with many traffic participants within urban traffic environments. The success of the deployments requires reliable decision-making that generalizes over a variety of situations.
The conventional modular architecture, encompassing perception, prediction, planning, and control, has been pivotal for fully automated driving, allowing large teams to work simultaneously on the architecture. However, the generalization remains a challenge. 
This thesis proposes a hybrid approach to automated driving.
It draws upon the interpretability intrinsic to traditional modular design and combines this with the generalization capabilities of deep learning.

The first part of this thesis examines the real-world applicability of the direct perception paradigm. The paradigm directly ties perception to control, striving to streamline modular architectures by focusing on essential features to implement the desired driving behaviors rather than explicitly modeling and evaluating the complete environment. The approach employs multi-task learning to predict affordances for driving that directly supply the inputs for lateral and longitudinal controllers. However, the system's operational domain is confined due to rule-based behavior planning, making it infeasible to address unexpected situations. To overcome these limitations, the subsequent work in this thesis builds upon the modular architecture by integrating deep learning. An environmental model and model predictive planner are utilized, leveraging high-resolution action sampling to generate a diverse set of driving policies.
These policies have implicit behaviors ranging from lane-changing and emergency braking to merging into time gaps between vehicles, eliminating the need for explicit hierarchical behavior modeling.

The second part of this thesis is concerned with bringing the modular architecture into an offline training loop and aligning the behavior of the model predictive planner with the preferences of human drivers. The first proposed method automates the tedious reward function tuning process that domain experts usually perform manually. The sampled policies of the planner enable maximum entropy inverse reinforcement learning to be tractable within high-dimensional continuous action spaces, utilizing path integral features. The succeeding method uses deep learning to predict situation-dependent reward functions, enabling generalization across diverse driving situations. The network inputs all sampled driving policies to combine environment and vehicle dynamics features and predicts situation-dependent weights of the reward function.
Later work proposes policy and temporal attention mechanisms for the network designed to produce consistent driving behaviors while adapting the reward function for consecutive planning cycles.

The third part of this thesis again focuses on streamlining the modular architecture after tackling the problem of reward function generation. The proposed approach is designed to leverage deep learning-based situation understanding. 
It focuses on making the explicit future motion prediction of surrounding objects optional. This is achieved by learning from an exhaustively sampling model predictive planner driving in real-world situations. The method unifies prediction and planning by predicting pixel state value sequences of the planning algorithm that implicitly encode driving comfort, reachability, safety, and object interaction.

This thesis provides an important step towards the scalability of automated driving by learning what is difficult to model by hand while preserving interpretability and the interfaces to incorporate explicit reasoning. This hybrid approach allows joint optimization of prediction and planning essential to implement humanlike, assertive, and safe driving in interactive driving environments.