One Model to Drift Them All: Physics-Informed Conditional Diffusion Model for Driving at the Limits

Authors: franck  djeumou
thomas  lew
nan  ding
michael  thompson
makoto  suminaka
marcus  greiff
john  subosits
Venue: robot-learning_org_CoRL_2024_Conference
Published: 2024-06-04 18:15:00

Abstract: Enabling autonomous vehicles to reliably operate at the limits of handling— where tire forces are saturated — would improve their safety, particularly in scenarios like emergency obstacle avoidance or adverse weather conditions.
However, unlocking this capability is challenging due to the task's dynamic nature and the high sensitivity to uncertain multimodal properties of the road, vehicle, and their dynamic interactions.
Motivated by these challenges, we propose a framework to learn a conditional diffusion model for high-performance vehicle control using an unlabelled multimodal trajectory dataset.
We design the diffusion model to capture the distribution of parameters of a physics-informed data-driven dynamics model.
By conditioning the generation process on online measurements, we integrate the diffusion model into a real-time model predictive control framework for driving at the limits, and show that it can adapt on the fly to a given vehicle and environment.
Extensive experiments on a Toyota Supra and a Lexus LC 500 show that a single diffusion model enables reliable autonomous drifting on both vehicles when operating with different tires in varying road conditions.
The model matches the performance of task-specific expert models while outperforming them in generalization to unseen conditions, paving the way towards a general, reliable method for autonomous driving at the limits of handling.

Main paper: 
 introduction: 
 Existing autonomous vehicles are constrained to operate at a fraction of their full handling potential. Designing algorithms to reliably control vehicles beyond these engineered limits would unlock faster and more reliable responses to diverse safety-critical situations [1, 2] such as driving on ice and avoiding sudden obstacles, scenarios where the vehicle may use all of the available tire-road friction, causing it to slide across the road [3, 4, 5] . However, driving at the limits of handling is challenging due to the task's dynamic nature, the high sensitivity to model mismatch, and the uncertain properties of the road, the vehicle, and their dynamic interactions. In addition, the high cost of collecting a dataset for driving at the limits, the complex vehicle dynamics, and the safety considerations complicate the use of imitation learning and reinforcement learning strategies. These challenges motivate the development of a model capable of exploiting physics knowledge and capturing complex forms of uncertainties while being amenable to real-time autonomous vehicle control.Diffusion models [6, 7, 8, 9] have shown to be highly capable of representing complex, highdimensional, and multimodal distributions from data. However, their direct use for driving at the limits is not straightforward. The limitations of classical diffusion models include the question of how to leverage prior physics knowledge to improve data efficiency and interpretability, and the considerable model inference time, which can be a bottleneck for high-bandwidth control.Contribution. We propose a conditional diffusion vehicle model for control-oriented modeling of driving at the limits of handling under uncertainties. By predicting the parameters of a physicsinformed neural stochastic differential equation dynamics model, the model has four key properties.• By encoding prior physics knowledge as an inductive bias, the proposed diffusion-based vehicle model is interpretable and generalizes to new environments from small amounts of data.• The model can capture complex multimodal distributions over the vehicle model's parameters.• The model can adapt on the fly to various test-time vehicles and road conditions by conditioning on measurements of the vehicle's interaction with the world.• By predicting the parameters of a physics-based model, as opposed to directly predicting state trajectories, the model inference and control loops are decoupled. This hierarchical approach unlocks diffusion sampling at low rates and high-frequency predictive control.We integrate the diffusion model in a real-time nonlinear model predictive control framework for autonomous driving at the friction limits, and extensively validate it on a Toyota Supra and a Lexus LC500. Our results showcase that a single diffusion model can reliably control both vehicles on challenging drifting tasks involving different road conditions and vehicle properties, see Figure 1 . 
related work: 
 Several works have explored autonomous driving at the limits of handling, both in the context of racing [10, 11, 12, 13] and drifting [14, 5, 4, 15] . These methods identify the parameters of a physics-based vehicle dynamics model [16, 17, 4, 18, 19] or train a neural network model [15, 20] , and subsequently use it for model-based optimal control. Nonlinear model predictive control (MPC) is the go-to control strategy in such settings and has demonstrated high-performance tracking ability in challenging racing and drifting tasks [12, 11, 4, 15] . However, the performance of MPC is limited by the fidelity of the vehicle model, designed to capture a single vehicle with given tires operating in specific road conditions. In contrast, using an unlabeled trajectory dataset, we train a single generative vehicle model with online adaptation capabilities that enable autonomous driving at the limits of handling on different vehicles in varying road conditions.Diffusion models have emerged as a powerful tool for generating complex and multimodal distributions in continuous domains such as images [21, 22] , 3D contents [23, 24] , planning and control [25, 26, 27, 28, 29, 30, 31] , time series [32, 33] , and physics processes [34, 35, 36] . However, all these works learn to represent, in a black-box manner, distributions for which samples are directly available in the training data. In contrast, we train a diffusion model to generate samples from a latent space of vehicle dynamics parameters that are not in the training data. Our approach relates to research in latent diffusion models [23, 37] . But instead of learning encoder and decoder networks to map between the data and latent spaces, we impose a structure on the latent space through physics-informed neural stochastic differential equations.Neural ordinary differential equations [38, 39, 40, 41, 42] , Koopman operators [43, 44, 45, 46, 47, 48] , classical system identification [49, 50, 51, 52, 53] , Gaussian processes [54, 55, 56, 57, 58] , and neural stochastic differential equations (SDEs) [59, 60, 61, 62, 63] have been widely studied for modeling uncertain dynamical systems from data. These models are either deterministic or can capture only a single mode of the training dataset's distribution, along with the uncertainty around the mode. On the other hand, Bayesian inference on the parameters of some of these models can, in theory, capture the multimodal distribution of the model parameters. However, classical Monte Carlo-based methods [64, 65, 66, 67] do not scale well with large models and datasets, while variational inference-based methods [68, 69, 70] are limited by the choice of variational family used to approximate multimodal posteriors. To address these limitations, recent works [71, 72, 73, 36] have highlighted the scalability and expressivity of diffusion models when approximating the posterior distribution in Bayesian inference. We leverage such expressivity to capture a multimodal distribution over the parameters of a vehicle model expressed as a neural SDE: A model shown in [63] to improve long-term prediction accuracy and uncertainty estimate compared to deep Gaussian-based models [74, 75] .Our modeling approach is similar to meta-learning methods [76, 77, 78] since the diffusion model learns offline to predict the parameters of a neural SDE model while adapting online to different vehicles and environment conditions. However, in contrast to existing meta-learning approaches where the dataset has task-specific labels, we train our model from an unlabeled dataset of vehicle trajectories.In addition, while conditioning the offline-trained diffusion model enables online adaptation, we emphasize that no gradient updates or any sort of regression on the model parameters are performed online for adaptation, as is typically done in meta-learning or online learning [79, 80, 81, 82, 83] .3 Method We assume access to a dataset T = {τ 1 , . . . , τ |T | } of vehicle trajectories τ = {(x t0 , u t0 ), . . . , (x t |τ | , u t |τ | )}, where each (x t , u t ) denotes a state-control pair. We denote by x t:T f = [x t , . . . , x t+T f ] and u t:T f = [u t , . . . , u t+T f ] the future state and control sequence from time t to time t + T f , and by x Tp:t and u Tp:t the past sequence from t -T p to t. We also use τ t:T f , τ Tp:t , and τ Tp:t:T f to denote the future, past, and full sequence of state-control pairs at time t, respectively. We consider that the dataset's trajectories may be collected from vehicles with different physical and tire specifications, and operating on various road conditions while performing different tasks. Thus, the distribution over its trajectories is complex and challenging to model. The dataset is also characterized as unlabelled since no information (per trajectories) about the vehicles and environments is provided for model identification other than the state-control pairs.We present our approach to learning physics-constrained generative models for autonomous driving at the limits. The main idea is to integrate the structure and the existing physics knowledge of driving at the limits into the design of a neural SDE model parametrized by θ. Then, we train a state-control history conditioned diffusion model, parameterized by ψ, to output a distribution p ψ (θ|x Tp:t , u Tp:t ) over the neural SDE parameters θ, which is then used to generate the future state trajectories or models to use by an MPC controller. However, existing approaches to training diffusion models cannot be directly applied here as they would require unavailable access to a dataset of neural SDE parameters. The proposed approach (Section 3.1) generates such a dataset and consistently improves its parameters with respect to the trajectories in T while simultaneously training the diffusion model.Neural SDEs for modeling. Neural SDEs [84, 85, 59, 60, 63] offer a principled approach for modeling uncertain dynamical systems due to their ability to encode prior physics knowledge from first principles, their calibrated uncertainties, and their expressiveness from using neural networks:EQUATIONwhere x ∈ X ⊆ R nx is the state, u ∈ U ⊆ R nu is the control input, t is the time, W is the n x -th dimensional Wiener process, and f θ : X × U → R nx and Σ θ : X × U → R nx×nx + are the parametrized drift and diffusion terms, respectively, and the equation is interpreted as an SDE in the Itô sense. Under smoothness assumptions [86, 87] on f θ , Σ θ , we can efficiently sample a distribution of predicted trajectories via numerical integration of the neural SDE [88, 89] . Assuming approximate Gaussian transitions between discrete times of a numerical SDE integrator, the negative log-likelihood (NLL) loss of a state sequence x t:T f given the controls u t:T f can be estimated byEQUATIONwith z 2 A := z Az for z ∈ R nx , A ∈ R nx×nx , and where Σ θ s := Σ θ (x θ s , u s ) and xθ t:T f is a sample sequence obtained from the SDE integration for a fixed θ and the initial conditions x t and u t:T f . Optimize for θ t using (4) and θ loc .10:Append (τ Tp:t , θ t ) to D.11:Update ψ using (5) on batches from D. EQUATIONwhere J nll is the negative log-likelihood defined in (2), and τ ∼ T is a sampled trajectory from the dataset. The term λ traj controls the regularization term R(θ) that enforces available prior knowledge on the neural SDE parameters; see Appendix A.4 for details on the design choices in this section.Parameter dataset generation via local optimization. Next, we use the estimate θ loc to iteratively generate a dataset D = {(τ Tp:t , θ t ), . . . } of short state-control trajectories τ Tp:t and model parameters θ t that are consistent with (a) the data sequence τ Tp:t , (b) the immediate future sequence τ t:T f , and (c) additional sequences τ Tp:t k :T f in a neighborhood of the initial data sequence τ Tp:t:T f . The timesteps t k are sampled from a uniform distribution U of fixed width centered at t. The local optimal parameter θ t that is added to the parameters dataset D is given by θ t = argmin θ J loc (τ Tp:t , θ, θ loc ) withEQUATION)where the term in λ loc regularizes θ to be close to the estimate θ loc . The additional sequences τ Tp:t k :T f help refine the uncertainty estimates of the neural SDE model. Using short sequences τ Tp:t k :T f , with t k in a small time window W ∈ R + around t, helps ensure that they can be explained by a single parameter vector θ t , since properties of the system may otherwise vary over a trajectory τ Tp:t:T f if τ Tp:t:T f is too long. We optimize for θ t using gradient descent on (4) starting from an initial guess sampled from a Gaussian distribution centered at θ loc . We found that regularizing to the estimate θ loc in the objective (4) stabilizes the optimization process.Training the diffusion model. Given the generated neural SDE parameters in D, we update the parameters ψ of our conditional denoising diffusion model with gradient descent as in [7] , on a lossEQUATION)whereγ k := k i=1 (1 -β i ) with β i ∈ (0, 1)being a linear noise schedule to gradually distort the parameters dataset D, N (0, I) is the standard normal distribution, K is the number of diffusion steps, and ψ is the denoising neural network predicting the noise added during the noising process. Here, h is a function that maps the history τ Tp:t to a feature space for conditioning the diffusion model:EQUATIONand ∆t -1Tp:t = [(t Tp+1 -t Tp ) -1 , . . . , (t t -t t-1 ) -1 ].We use K = 1000 noising steps as in [7] . Iteratively refining the parameter dataset. As the diffusion model training progresses, its predictions of the local parameters θ t become more accurate while the initial estimate θ loc computed in (3) may become a suboptimal initial guess for θ t . Thus, after some number of diffusion training steps, we use the generative model to refine the dataset D by sampling a neural SDE parameter that would serve as an initial estimate θ loc and a regularizer for the local optimization problem (4).Online diffusion model inference. Algorithm 2 outlines a simple online strategy for sampling the neural SDE parameters conditioned on online measurements. The algorithm maintains the history of state-control pairs and uses it as a source for generating, validating, and scoring the neural SDE parameters. At each time step of the algorithm, a set T gen of state-action sequences is sampled from the history dataset T hist to generate a set of parameters θ t conditioned on each τ Tp:t ∈ T gen . This is done in K steps, by sampling θK ∼ N (0, I) and refining θk-1 ∼ N (µ ψ ( θk , k, h(τ Tp:t )), σ k I) withEQUATIONbefore letting θ t = θ0 . Finally, a set T val ⊆ T hist is sampled and used to validate the generated parameters and compute their scores, defined as the loss J traj (θ, T val ) in ( 3) plus a 2-norm regularization term that penalizes the distance to the n best = 5 best previously generated parameters {θ p } n best p=1 . We found that this last term helps ensure consistent updates during the online inference process. 
application to autonomous driving at the limits of handling: 
 Physics-constrained neural SDE model. We now introduce the uncertainty-aware and physicsconstrained neural SDE model for driving at the limits of handling. We employ the commonly used single-track model [90, 16, 91, 18, 17] as a foundation to describe the nonlinear dynamics of the vehicle. The vehicle position is expressed in a curvilinear coordinate system relative to a reference trajectory [14, 19, 12] . Specifically, the position coordinate is described by the distance s along the reference trajectory, the relative heading ∆φ with respect to a reference heading φ ref , and the lateral deviation e from the path. The proposed neural SDE model is given byEQUATIONwhere prior knowledge comes from the matrix M θ (•, •) that depends on vehicle parameters such as the mass m θ , yaw moment of inertia I θ z , rotational inertia of the drivetrain I θ w , tire radius R θ , and distances from the center of gravity to the front and rear axles a θ and b θ . The control input u = [δ, τ e ] is the steering angle and engine torque, respectively. The state x = [r, V, β, ω r , e, ∆φ, s] includes the yaw rate r, velocity V , sideslip angle β, rear wheelspeed ω r , lateral error e, and angular deviation ∆φ. Lastly,F θ = [F θ xf , F θ yf , F θ xr , F θ yr ]represents the tire forces between the vehicle and the road. These unknown tire forces F θ are learned as functions of the state and control inputs.Modeling the dynamic interaction between the tires and the uncertain road surface is crucial for accurately controlling a vehicle at the limits of handling. To do so, we incorporate into our neural SDE model a version of the neural-ExpTanh tire model [15] , a physics-informed neural tire model that captures the nonlinearities and saturation effects of tire forces, and that has shown to better predict tire forces than previous models used in the literature. We refer to Appendix A.1 for the derivations of M θ (•, •) and the neural tire force models used in the experiments.Model predictive control (MPC) for autonomous drifting. The MPC tracks a reference trajectory under actuator and actuator rate constraints by solving at each time t the optimization problemEQUATIONsubject to xk+1 = SDESolve(x k , ūk ; θ best ) ∀k = 0, ..., H, x0 = x t , ū0:H ∈ U, u0:H ∈ Ū (9b) where x k is the state at timestep k, x1:H+1 denotes the state trajectory over the prediction horizon of length H + 1, and ū0:H are the corresponding control signals. SDESolve is any differentiable SDE integration scheme, in our case a simple Euler-Maruyama method, parameterized using the best parameters θ best found by Algorithm 2. The states x1:H+1 are thus random variables, and the expectation in (9a) is evaluated using Monte Carlo. We use 2 particles in our experiments. The system is actuated by u t = ū 0 , where ū 0 is the optimal solution to (9) constrained with x0 = x t . 
results: 
 We validate the proposed framework on two vehicles in scenarios with different tires, operating gears, and road conditions. First, we verify the capabilities of the conditional diffusion model to capture the complexity of the unlabelled dataset and adapt its predictions online (Section 4.1). Second, we show that the model can adapt to different tires (Section 4.2). Third, we demonstrate the high performance of the method in diverse scenarios (Section 4.3). Finally, using a small amount of data collected on wet surfaces, we show that the framework enables drifting at the limits of handling in heavy rain (Section 4.4). We provide further details on the experiments in Appendix A.Experimental vehicles. We deploy the approach on a Toyota Supra and a Lexus LC500, as shown in Figure 1 . The Supra is modified with a more powerful engine and more responsive actuators, whereas the Lexus is kept with factory settings, making it a particularly challenging platform for autonomous drifting. The two vehicles' large differences in dynamics responses make them ideal platforms for evaluating the robustness and generalization capabilities of our approach. For both vehicles, we use onboard vehicle state estimation using a GPS and IMU, and we use the CPU of a ruggedized PC to run the diffusion model inference at 2Hz and compute control inputs using MPC at 200Hz. Training dataset. We train the diffusion model on a total of 84 manual and autonomous driving and drifting trajectories from the two vehicles. The duration of each trajectory is between 10 and 90 seconds. It consists of 5 manual driving trajectories pushing the car to the limits of handling, whereas the remaining trajectories are autonomous drifting experiments comprising of failed and successful attempts. The dataset includes driving data in different gears (affecting the effectiveness of the throttle input) and using tires with different physical properties (affecting the vehicle's dynamics and steering input effectiveness). The composition of the dataset is summarized in Figure 3 .Baselines. We compare with neural SDE dynamics models (referred to as BaseSDE or Expert depending on context) trained on specific vehicle-tire-gear subsets of the dataset in Figure 3 . Each baseline is trained with the loss function in (3) and a regularization term R(θ) encoding prior knowledge about the parameters (m, a, b, I z , I w , R). The resulting models are Expert since they are optimized for specific scenarios, but they may perform poorly when deployed in different conditions. We condition the model on two trajectories from the Lexus vehicle dataset (see Figure 3 ), where the vehicle is either accelerating in a straight line (gripping) or drifting (sliding). In Figure 4 , we report two predicted parameters (I θ w , c θ 1 ) corresponding to the rear wheel inertia and the maximum total force that can be generated by the rear tires; see Appendix A.1. By conditioning on the straight-line (grip) trajectory, the model returns a multimodal parameter distribution due to the lack of information about tire friction properties, as tire forces are not saturated. Interestingly, by conditioning on a trajectory where the vehicle is sliding and tire forces are saturated, the model predicts a tight unimodal distribution of the parameters. The left mode of the two parameters from the gripping phase has collapsed due to sufficient information to infer vehicle properties. This example shows that the proposed conditional diffusion model captures multimodal parameter distributions and adapts its predictions based on the information contained in the trajectory. We further highlight the generalization capability of the proposed method by studying the closed-loop tracking performance of the Lexus when operating with various tires. We report tracking performance on two reference trajectories in Table 1 . The controller using the Expert model accurately tracks the reference trajectory. In contrast, using the baseline trained only on type 3 tire data is insufficient to track the reference, and leads to spinning out. This baseline fails to initiate the drift, due to differences in the cornering stiffness between the two tires leading to a poor prediction of how fast the vehicle will saturate the rear tires. The BaseSDE model trained on data with type 2 and type 3 tires can drift with a higher tracking error. On the donut trajectory, this baseline drifts on a circle with a larger radius due to a tire force modeling mismatch. Finally, the diffusion model quickly infers the tire properties, which results in adaptive tracking performance that matches Expert performance.We evaluate the framework on a range of scenarios, including reference trajectories that are not in the dataset, and compare its performance with an Expert trained on the relevant subset of the dataset. Drifting results on the Toyota Supra.We report performance in tracking different reference trajectories in Figure 5 and Table 2 . The diffusion model enables drifting maneuvers with tracking performance that is comparable to expert models, while simultaneously having the advantage of being trained on an unstructured dataset. This demonstrates the model's ability to adapt at test time to the specific vehicle setting and road condition based on the online observation. Interestingly, the framework succeeds in accurately tracking the Slalom trajectory (first column of Figure 5 ), which is not part of any maneuvers in the training dataset. Tracking this trajectory requires the vehicle to operate outside of the training data distribution, given the rapid changes between circles of different radii while accelerating during the last transition of the trajectory. We speculate that this ability to generalize results from the prior physics knowledge encoded in the neural SDE vehicle model. Drifting results on the Lexus LC500.Tracking results in Figure 6 and Table 3 show that using the diffusion model enables accurate tracking with performance comparable to expert models. Again, the proposed method is capable of performing a donut trajectory in second gear, although no second-gear trajectory from the Lexus is in the dataset. Moreover, only unsuccessful Figure-8 trajectories on the Lexus are in the dataset, yet both methods using the expert and diffusion models succeed in tracking this trajectory, thanks to the physics structure encoded in the neural SDE model. Lastly, we augment the dataset with 3 manual drifting trajectories collected in the rain and 4 autonomous drifting donut trajectories on a wet surface with failed and successful attempts on the Lexus vehicle. Then, we retrain the diffusion model and report the performance of the resulting controller deployed in heavy rain in Figure 7 . The proposed approach is capable of drifting with only 1.47 m as the lateral RMS error (e) and 4.79 deg as the RMS slip angle (β) error. Drifting in such rainy conditions is particularly challenging. Indeed, accurate friction modeling is critical to successfully initiating the drift without spinning out, especially using a commercial vehicle such as the Lexus. Stabilizing the vehicle in rainy conditions is particularly difficult due to the increased effect of small friction variations on the handling characteristics of the vehicle, and because friction parameters vary over space due to the terrain drying unevenly over time. These results indicate that a single generative model, trained on a majority of data collected on high-friction surfaces, has the potential to enable reliable autonomous driving at the limits in both high and low-friction conditions. 
conclusions: 
 We propose a physics-informed generative vehicle model for autonomous driving at the limits of handling. By decoupling model inference and control, this hierarchical approach combines the expressiveness of a diffusion model with the high-rate replanning and reliability of model predictive control. Through extensive autonomous drifting experiments on a Toyota Supra and Lexus LC500, we demonstrate that a single conditional diffusion model, trained on unlabelled trajectories from both vehicles operating in various conditions, can enable adaptive, robust, and real-time autonomous driving at the limits of handling.Limitations and future work. Although the diffusion model predicts a multimodal distribution of parameters of the neural SDE vehicle dynamics model, the current model predictive controller only uses one predicted parameter set for control at a time. Fully reasoning over predicted distribution using risk-sensitive algorithms would potentially lead to additional robustness and inform data collection via active exploration to best reduce uncertainties online. Finally, while only validated on drifting tasks, the generality of the proposed hierarchical method and the experiments indicate that the approach could potentially be used in other autonomous driving and robotics applications. 
 
 Appendix: This section provides additional details on the neural SDE vehicle model, the experimental vehicles and training dataset, the expert models used as baselines, the diffusion model training, and the model predictive control formulation.We implement all the numerical experiments (training the models and the gradient-based model predictive control solver) using the Python library JAX [92] to take advantage of its automatic differentiation and just-in-time compilation features. We use Python 3.8.5 for the experiments and train all our models on a laptop computer with an Intel R Xeon(R) W-11855M CPU (base frequency 3.30GHz), 12 cores, 32 GB of RAM, and a GeForce RTX 2060, TU10. We employ the commonly used single-track model [90, 16, 91, 18, 17] as a foundation to describe the nonlinear dynamics of the vehicle. The vehicle position is expressed in a curvilinear coordinate system relative to a reference trajectory [14, 19, 12] , as shown in Figure 8 . Specifically, the position coordinate is described by the distance s along the path, the relative heading ∆φ with respect to a planned course φ ref , and the lateral deviation e from the path. For simplicity, we assume only the steering and throttle are used for autonomous drifting and do not include brakes in the control inputs and dynamics. The proposed neural SDE model is given byEQUATIONwhere vehicle-specific parameters such as the mass m θ , yaw moment of inertia I θ z , rotational inertia of the drivetrain I θ w , the tire radius R θ , and the distances from the center of gravity to the front and rear axles a θ and b θ are included in the neural SDE parameters θ to learn. The control input u = [δ, τ e ] is the steering angle and engine torque, respectively. E θ (τ e ) is a parameterized polynomial function that maps the engine torque to the wheel torque through the gear ratio G. The state x = [r, V, β, ω r , e, ∆φ, s] includes the yaw rate r, velocity V , sideslip angle β, rear wheelspeed ω r , lateral error e, and angular deviation ∆φ. The evolution of the path-dependent variables e, ∆φ, and s are well described by basic kinematicsEQUATIONEQUATIONEQUATIONwhere κ ref (s) is the curvature of the reference path, and we use β and ṡ as an abuse of notation for the drift terms of β and s, respectively. Lastly, the unknown lateral and longitudinal tire forces F θ xf , F θ yf , F θ xr , and F θ yr are parametrized and learned as functions of the state and control inputs. brake-by-wire and throttle-by-wire capabilities. These modifications make the platform extremely suitable for (autonomous) Formula drift, as they allow for precise and fast control of the vehicle's dynamics. The Lexus, on the other hand, has lower-performance actuators and is not designed to be used as a drifting vehicle. Ultimately, these two platforms have completely different dynamics, which makes them ideal for evaluating the robustness and generalization capabilities of our approach.We use an Oxford Technical Systems (OxTS) RT4003 v2 RTK-GPS/IMU system for localization and vehicle state estimation for both vehicles. The MPC controller is implemented on an Intel Xeon E-2278GE (base frequency 3.30GHz) CPU Linux computer mounted on board the vehicles. The computer communicates with a low-level PID controller implemented on a dSpace MicroAutoBoxII (DS1401) to track desired steering angle δ and engine torque τ e . The MicroAutoBoxII receives commands from the MPC controller via UDP and sends actuator commands to the original equipment manufacturer's steering and engine electronic control units. All data are synchronized and recorded at a frequency of 62.5Hz on the Lexus and 100Hz on the Supra.Training dataset. We build a dataset of manual and autonomous driving and drifting trajectories on a closed circuit from both vehicles. The dataset contains a total of 84 trajectories, each trajectory with a duration between 10 and 90 seconds. It has 5 trajectories collected from manual driving with the intent of pushing the car to the limits of handling without any specific path-tracking maneuvers planned. The remaining trajectories are from autonomous drifting experiments with 28 from the Supra and the rest from the Lexus. The supra dataset contains failed and successful attempts at performing donut maneuvers in first gear and Figure-8 maneuvers in second gear. The Lexus dataset contains attempts at performing donut maneuvers in first gear and very limited (all failed) attempts at performing "Figure -8 " maneuvers in first gear. No second-gear drifting trajectories were provided in the training dataset for the Lexus. Additionally, 7% of the trajectories from the Lexus were collected with Tire 2, as opposed to Tire 3 used in the rest of the dataset. A notable difference between the two sets of tires is their cornering stiffness, which makes drift initiation strategies and tire dynamics different between the two sets of tires.Training dataset for drifting on heavy rain. We augmented the above training dataset with 7 additional drifting trajectories collected on a wet track with a vehicle identical to the Lexus LC500.3 of the trajectories were manually collected, while the remaining 4 were collected autonomously. The autonomous trajectories were collected using second-gear drifting maneuvers only on a donut trajectory. Despite the limited number of trajectories and restriction to second-gear drifting, we show in Section 4.4 that the diffusion model trained on this dataset generalizes to drifting in heavy rain on a first-gear donut trajectory. We recall that the expert models in the experiments are neural SDE models trained on specific vehicle-tire subsets of the dataset. Given a subset T exp of the training dataset T , we train the expert models by minimizing the negative log-likelihood J traj (θ, T exp ) defined in (3) with respect to the neural SDE parameters θ defined in Appendix A.1. The regularization term R(θ) encodes a Gaussian prior on the vehicle parameters m θ , I θ z , I θ w , R θ , a θ , and b θ . For each parameter, we use a Gaussian prior with a mean equal to the known (estimated) parameter value for the specific vehicle and a standard deviation of 1. The resulting R(θ) is given byEQUATION)where the parameters values m, I z , I w , R, a, and b for each vehicle are provided in Table 4 . The expert models are trained using Adam optimizer [93] with a learning rate of 10 -3 and a batch size of 64. We use λ traj = 10 -4 for the regularization term in the loss function (3) . During training, we discretize the sum in the expression of J nll (see (2) for the expression and (3) for the loss function) into a sum of N f = 20 discrete time steps t 0 , . . . , t N f , where t 0 = t, T f = t Nt 0 , and t i = t i-1 + ∆t i for i = 1, . . . , N f . We set ∆t i = U (1, 6) ∆t where ∆t is typically 0.01 for trajectories on the Supra and 0.016 for trajectories on the Lexus. We randomize the time steps ∆t i during training to improve the model's generalization when evaluated with an integration scheme that uses stepsizes other than the training stepsizes. This is typically the case when using the model in a model predictive controller; see Appendix A.5. We note that the model learns to predict state-action sequences with varying lengths T f between 0.4 seconds and 1.96 seconds. Lastly, we use 5 particles of the neural SDE to compute the expectation in the expression of J nll .Initial estimate of the neural SDE parameters. We train the initial estimate θ loc of the neural SDE parameters in a similar manner to the expert models in Appendix A.3. The main difference is that we use the full training dataset T instead of the vehicle-specific subset T exp , and thus we do not enforce any prior knowledge on the vehicle parameters as in (15) . Specifically, the regularization term R(θ) is now enforcing an uninformative Gaussian prior with mean 0 and standard deviation 1 on all the neural SDE parameters θ as followsEQUATION)where N θ is the number of parameters in θ, and θ in defined as in Appendix A.3.Parameter dataset generation via local optimization. The hyperparameters that define this step of the diffusion model training are the regularization parameter λ loc , the time window W , the history length T p , the future trajectory length T f , and the number of sequences τ Tp:t k :T f used to compute the expectation in (4). We set λ loc = 10 -3 , W = 10 seconds, and the number of sequences τ Tp:t k :T f to 5.In a similar manner to how we chose the time steps in the training of the expert models (see Appendix A.3), we discretize τ Tp:t and τ Tp:t k into N p = 10 discrete time steps with the same randomization of the step size ∆t i as in Appendix A.3. Thus, the maximum length of the history sequence is T p = 0.96 seconds. Besides, we keep the future trajectory length T f to be the same as in the training of the expert models, i.e., with N f = 20 discrete time steps and a maximum length of 1.96 seconds. We use 5 particles of the neural SDE to compute the expectation in the expression of J nll .We optimize the neural SDE parameters θ of the loss function (4) using gradient descent with Nesterov acceleration and an adaptive learning rate through Armijo line search. We set the maximum number of iterations to 1000 and the initial guess for the learning rate and neural SDE parameters to be respectively 0.01 and θ loc .Diffusion model training. We follow the procedure described in [7] to train all our diffusion models. The model ψ defining the generative process is represented as a standard feedforward neural network with three hidden layers of 256 neurons each. We use a sinusoidal positional encoding of the diffusion step k (see (5) ) as an input to the neural network defining ψ , instead of using k directly as the input. The encoding is done by scaling the diffusion step and concatenating its sine and cosine to the input of a feedforward neural network with two hidden layers of 32 and 16 neurons each and swish as the activation function. We use K = 1000 denoising steps, and a linear noise schedule β i ∈ (0, 1), where β i = 0.0001 + 0.02i/K for i = 0, . . . , K. We use Adam optimizer with a learning rate of 10 -4 and a batch size of 32 to train the diffusion model. Additionally, we perform 50 gradient updates of ψ for each step of Algorithm 1.Iteratively refining the parameter dataset. Given the history sequence τ Tp:t , we use the diffusion generation process defined in (7) to obtain a set of parameters {θ p t } 100 p=0 conditioned on τ Tp:t . Then, we use the future sequence τ t:T f to select the best parameter in terms of the negative log-likelihood J nll (θ p t , τ Tp:t:T f ) with 5 particles of the neural SDE to compute the expectation in its expression. The obtained best parameter is then used to update θ loc for better initialization and regularization of the local optimization problem (4). In our experiments, we refine θ loc at every step of Algorithm 1 only after the initial 20000 steps of the main training loop.Online diffusion model inference. In Algorithm 2, we use a sliding window to deal with the growing size of the dataset T hist and to account for changing vehicle-road properties or environment conditions. The maximum size of the sliding window T hist is typically set to 30 seconds worth of driving data. During online sampling for model predictive control, we randomly sample the set T gen to contain 5 history sequences τ Tp:tj and the set T val ⊆ T hist to contain 30 sequences τ t l :T f of the current history dataset T hist . Specifically, by indexing each state-action pair x t k , u t k with the corresponding discrete time, we can define a discrete distribution to sample sequences τ Tp:t k , by sampling time indexes t k and using the corresponding state-action as the endpoint of the sequence of length T p . In our experiments, we use an exponential distribution with a higher mass on the latest time indexes in T hist to generate T gen . Then, to generate T val , we pick the future sequences τ tj :T f corresponding to the sequences of T gen and sample the remaining validation sequences τ t l :T f in the same manner as T gen . The sequences in the validation dataset are selected to be as close as possible to the latest time in the history dataset. We use the diffusion model to generate a total of 100 parameters {θ t k } k conditioned on the sequences in T gen , and select the best parameter according to Algorithm 2.We use a custom proximal gradient-based solver with Nesterov acceleration and Armijo line search, inspired by the approach in [94] , to optimize the MPC problem 9a-9b. The state constraints, if any, and control rate constraints are enforced using slack variables, and the proximal operator for projecting the slack variables onto the feasible set. We use a first-order approximation to compute the control rate as in uk = (ū k+1ūk )/∆t k . On the other hand, the box constraints on the control input are simply enforced by projection onto the set at each iteration of the proximal gradient-based solver.Reference trajectories. The reference trajectories for the maps are generated offline via the quasiequilibrium strategy proposed in [19] . We start with a few waypoints in terms of the desired curvature κ ref and sideslip angle β ref as a function of path distance s. Then, for each point on the path, an equilibrium point is computed using the single-track bicycle model and the conditions κ = κ ref , β = β ref , φ = κ ref V , and V = ṙ = 0, yielding the fine-grained reference vehicle state x ref . We emphasize that the model used to generate the reference trajectories differs significantly from the neural SDE model used in the MPC controller. We reduce the over-reliance on possibly infeasible reference trajectory by using a cost function that penalizes the deviation from only in the sideslip angle β and wheel speed ω r , see (9a)-(9b).Lexus LC500. We use Q β = 120, Q e = 2.0, Q φ = 60.0, Q δ = 5, and Q τ = 10 -6 . The control set is given by U = [-0.52, 0.52] × [-1, 400] while the control rate set is given by Ū = [-0.9, 0.9] × [-3000, 400]. The problem is optimized over a horizon H = 30 with 25 short time steps of 0.05s and 5 long time steps of 0.15s. Thus, the total lookahead horizon amounts to 2s. Toyota Supra. We use Q β = 70, Q e = 3.0, Q φ = 30.0, Q δ = 1, and Q τ = 10 -7 . The control set is given by U = [-0.75, 0.75] × [-50, 350] while the control rate set is given by Ū = [-2, 2] × [-3000, 2000 ]. The problem is optimized over a horizon H = 30 with 25 short time steps of 0.05s and 5 long time steps of 0.15s. Thus, the total lookahead horizon amounts to 2s. Figure 18 : Lexus drifting on a first-gear donut trajectory on heavy rain.8th Conference on Robot Learning (CoRL 2024), Munich, Germany. 


Decision: The paper presents a method to map recent trajectory observations into parameter values of a neural SDE model via a diffusion model, which is, in turn, used in MPC to control a car at its limits.

Strengths
- The paper presents an interesting approach that enables driving at the limit without access to labeled training data.
- The presented experiments on two different real-world cars are compelling.

Limitations
- The paper needs to provide further clarification on some of the technical details to fully understand its contributions, such as the availability of the control inputs during training, the latency of online adaptation of the neural SDE, etc.
- It would also be helpful to comment on how the method would handle interactive driving scenarios.

**Comments post rebuttal**

The revised paper has satisfactorily addressed the outstanding reviewers' concerns/questions.

Reviews: review: title: title: A compelling piece of research with intriguing novelty likely underappreciated by the authors themselves
paper_summary: paper_summary: The authors designed and trained a diffusion model to map recent trajectory observations into parameter values of a neural SDE (stochastic differential equation) model, which is in turn used in MPC to control a car at the limits of handling (specifically tire dynamics). The trained diffusion model is shown to generalize across vehicle types, weather conditions, and reference trajectory geometry. The authors claim that their main contribution is a "reliable method" for autonmous driving "at the limits of handling".
main_review: main_review: The paper is generally well written with numerous places needing correction or improvement (see Questions for Rebuttal). The technical material is complex, the handling is adequate, and the real-world robotics integration is compelling. These in themselves do qualify the paper for acceptance.

However, while the authors framed their work as having to do with a "reliable method" for autonmous driving "at the limits of handling", it seems to this reviewer that (the understanding of) the research could be deepened. Specifically, the work could be understood as about a general architecture for autonomous driving or even robotics and with what has been done so far (about "drifting") as a mere proof of concept -- but a successful one at it. Even more specifically, while the authors called their method "meta learning", it could likely be more appropriately understood in terms of a hierarchical control and decision making architecture. Thus, what's said on Lines 46-47 "the model inference and control loops are decoupled, which enables diffusion sampling at low rates and high-frequency predictive control" could be more easily made sense of. Similarly, the complexity of Algorithm 1 could also likely be understood in terms of co-learning across a hierarchy (concerned with at least two levels). Furthermore, conceptually, we may understand the $\theta$ parameters at the interface between the diffusion model and the neural SDE for MPC as representing "affordances" or "regimes" of affordances, rather than merely as parameter values to the neural SDE. This way, the method is about merging a more semnatically rich representation (from general observation of the environment history, possibly including very complex interaction dynamics -- both tire with ground surface and car with other cars and pedestrians) with a structurally more delineated and thus interpretable control scheme. If under such a general light, the choice of diffusion model and the specific method developed (mostly in Algorithm 1) still make sense, then the contribution of the paper could be truly significant.
questions: questions: 1. Equation 1: "$dB$" should be "$dW$" to be consistent with Figure 1?

2. Line 125 the variable $z$ seems to have never been introduced.

3. Algorithm 1, Line 6: the condition "if Time to refine" seems to have never been explained.

4. Algorithm 1, Line 10: "Update D with ..." should be something like "Append ... to D" instead? "Update" is too vague.

5. Line 138: "$2W$" -- the variable $W$ was never introduced.

6. Lines 145-146: more explanation?

7. Lines 163-165: this seems quite standard in dealing with diffusion models. Should we just defer to the literature? Otherwise, if there's something new and special, explain!

8. Line 248: "This conclusion ..." -- it's not advicable to do anaphoric reference to the previous section at the beginning of a new section.

9. This reviewer disagrees with the "meta-learning" characterization (Line 86; see Review above for discussion). The authors are encouraged to consider their approach as hierarchical.

10. The authors are also encouraged to discuss Algorithm 1, especially it's complexity from a hierarchical perspective.

11. In terms of limitations, please also consider the method's relevance to handling general complexity of an interactive driving environment (with other road users). Now, in real world handling of driving in snow and ice, it's simply not true that the environment has only the ego car itself. This comment/request is of course also along the recommendation above of considering the method's general relevance.
limitations: limitations: 2
review_summary: review_summary: The recommendation is based on the current state of the paper: it introduced a method with novelty to address a meaningful enough challenge (automated car control in static environment at the limits of handling) and demonstrated the effectiveness of the method. The real-world demonstration, especially that of the generalization across car type, weather condition, and trajectory shape, is compelling. The way in which a diffusion model is combined with neural SDE, and through which with MPC, is interesting. That this reviewer thinks the potential of the general architecture goes beyond the authors' own realization certainly adds to the reviewer's own confierence in their recommendation but does not change what's recommended, which again is based on the strength of the current manuscript itself.
title: title: Well written paper on drifting real-world cars with strong and convincing experiments.
paper_summary: paper_summary: This paper deals with the challenge of operating cars on their limits, i.e. in drifting situations, which is critical for safe autonomous driving in critical situations. The authors propose to use a physics-informed SDE which parameters are learned by a diffusion model. The article shows very strong results in the experimental section where the proposed method is applied to two real-world cars.
main_review: main_review: The paper is well written and addresses a critical point in autonomous driving which is the operation of cars on their limit which is required in extreme situations. However, that is very challenging as we have typically limited data on these situations and the outcome is sensitive to parameters such as friction, ground, tires, etc. The authors overcome this issue in an original way by using an physics informed model where the parameters come from a diffusion model. In this way, the multimodal nature of the problem can be addressed. 

The paper presents a strong experimental verification of the proposed method on two real world cars drifting and following desired trajectories.

However, there are some issues that I’d like to see addressed.

-	It’s not clear to me why the limitation of \Sigma in (1) to be a diagonal matrix improves the model’s interpretability. In contrast, understanding the correlations between the states would enhance the understanding.

-	The authors claim the proposed methods can adapt to different tires, drift situations, etc. by observing the past trajectory and then using the diffusion model to give the “best” parameters to the MPC controller as stated in Algorithm 2. However, I don’t understand the relationship between the historic dataset and the T_gen and T_val. Is always the entire dataset T_hist used? If so, how do you deal with the growing size of this set? Would it make sense, to use some kind of sliding window to be able to adapt to suddenly changing situations such as the transition from a paved road to ice?

-	It seems that \Theta^1 in Alg. 2 is the same as \Theta_Best on (9b). If that’s the case, please be consistent with the name.
questions: questions: Please address the issues mentioned in the “review” section
limitations: limitations: 3
review_summary: review_summary: Paper with strong experimental verification on operating cars in drifting situation which should be accepted at Corl.
title: title: One Model to Drift Them All: Physics-Informed Conditional Diffusion Model for Driving at the Limits
paper_summary: paper_summary: The paper proposes a generative model that can handle driving at the limits. A physics informed neural SDE model is introduced for this. Experiments show that the model can perform drifting maneuvers on 2 different vehicles
main_review: main_review: The work proposed in the paper is quite interesting in that it proposed a method for tackling the task of driving at the limits without access to labeled training data. Experiments demonstrating drifting capabilities were done on 2 different vehicles (Toyota Supra and Lexus LC500).. These vehicles had large differences in their dynamic responses, thereby showing the  extent of their generalization and robustness capabilities.  Experiments were additionally done to test on differing tire, gear and road conditions.

However, at the same time some things are not clear and require further clarification:
The abstract/paper claims that the capability of operating at the limits of driving would improve safety in scenarios like emergency obstacle avoidance. I do not understand how demonstrating drifting/gripping capabilities would imply obstacle avoidance. This was not shown but only claimed in the paper. I reckon this would require additional sensors to first perceive the environment which may then be used to take evasive measure.

The term "multi-modal" has been used in multiple places in the paper in different contexts such as data, uncertainties, trajectories, properties. It is not clear what it precisely means in each context. I only found the explanation for multi-modal trajectory in L101-102.
questions: questions: In addition to the questions raised in the previous review section, there are some additional points that require explanation:

1) What are the control inputs (L117) and where do they come from? If they are already available, would this not be considered labeled data?  Also, are the controls not the final output for eventually controlling the vehicle?  

2) How fast is the online adaptation of the neural-SDE (L87)?
limitations: limitations: 3
review_summary: review_summary: The paper is an interesting read with relevant results for the robotics community. However, there are aspects which are not very clear and require further clarficiation.