Original: CCIL: C ONTEXT - CONDITIONED IMITATION LEARNING FOR URBAN DRIVING
Revised: CCIL: C ONTEXT - CONDITIONED IMITATION LEARNING FOR URBAN DRIVING

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Imitation learning is a promising solution to the challenging autonomous urban driving task as experienced human drivers can effortlessly tackle highly complex driving scenarios.
Revised: Imitation learning is a promising solution to the challenging autonomous urban driving task as experienced human drivers can effortlessly tackle highly complex driving scenarios.

Original: Behavior cloning is the most widely applied imitation learning approach in autonomous driving due to its exemption from potentially risky online interactions, but it suffers from the covariate shift issue.
Revised: Behavior cloning is the most widely applied imitation learning approach in autonomous driving due to its exemption from potentially risky online interactions, but it suffers from the covariate shift issue.

Original: To mitigate this problem, we propose a context-conditioned imitation learning approach that learns a policy to map the context state into the ego vehicle’s state instead of the typical formulation from both ego and context state to the ego action.
Revised: To mitigate this problem, we propose a context-conditioned imitation learning approach that learns a policy to map the context state into the ego vehicle’s state instead of the typical formulation from both ego and context state to the ego action.

Original: Besides, to make full use of the spatial and temporal relations in the context to infer the ego future states, we design a novel policy network based on the Transformer, whose attention mechanism has demonstrated excellent performance in capturing relations.
Revised: Besides, to make full use of the spatial and temporal relations in the context to infer the ego future states, we design a novel policy network based on the Transformer, whose attention mechanism has demonstrated excellent performance in capturing relations.

Original: Finally, during evaluation, a linear quadratic controller is employed to produce smooth planning based on the predicted states from the policy network.
Revised: Finally, during evaluation, a linear quadratic controller is employed to produce smooth planning based on the predicted states from the policy network.

Original: Experiments on the real-world large-scale Lyft and nuPlan datasets demonstrate that our method can surpass the state-of-the-art method significantly.
Revised: Experiments on the real-world large-scale Lyft and nuPlan datasets demonstrate that our method can surpass the state-of-the-art method significantly.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Planning a safe, comfortable, and efficient trajectory in a complex urban environment for a selfdriving vehicle (SDV) is an important and challenging task in autonomous driving (Yurtsever et al., 2020).
Revised: Planning a safe, comfortable, and efficient trajectory in a complex urban environment for a selfdriving vehicle (SDV) is an important and challenging task in autonomous driving (Yurtsever et al., 2020).

Original: Unlike highway driving (Henaff et al., 2019), urban driving requires handling more varied road geometry such as roundabouts and intersections while interacting with traffic lights, pedestrians, and other vehicles.
Revised: Unlike highway driving (Henaff et al., 2019), urban driving requires handling more varied road geometry such as roundabouts and intersections while interacting with traffic lights, pedestrians, and other vehicles.

Original: Classic manually-designed rule-based approaches (Fan et al., 2018) have achieved some success in industry but demand tedious human engineering to struggle with diverse real-world cases.
Revised: Classic manually-designed rule-based approaches (Fan et al., 2018) have achieved some success in industry but demand tedious human engineering to struggle with diverse real-world cases.

Original: Meanwhile, the rapid development of deep learning techniques motivates researchers (Bojarski et al., 2016; Pan et al., 2020) to employ a deep neural network to model the complicated driving policy.
Revised: Meanwhile, the rapid development of deep learning techniques motivates researchers (Bojarski et al., 2016; Pan et al., 2020) to employ a deep neural network to model the complicated driving policy.

Original: To learn such a policy, imitation learning (IL) from human drivers’ demonstrations is a promising solution as experienced drivers can tackle even extremely challenging situations, and their driving data can be collected at scale.
Revised: To learn such a policy, imitation learning (IL) from human drivers’ demonstrations is a promising solution as experienced drivers can tackle even extremely challenging situations, and their driving data can be collected at scale.

Original: The simplest IL algorithm is the behavior cloning (BC) method, which has wide applications in autonomous driving (Pomerleau, 1988; Bojarski et al., 2016; Codevilla et al., 2018b) due to its exemption from potentially dangerous online interactions.
Revised: The simplest IL algorithm is the behavior cloning (BC) method, which has wide applications in autonomous driving (Pomerleau, 1988; Bojarski et al., 2016; Codevilla et al., 2018b) due to its exemption from potentially dangerous online interactions.

Original: It learns a policy in a supervised fashion by minimizing the difference between the learner’s action and the expert’s action in the expert state distribution.
Revised: It learns a policy in a supervised fashion by minimizing the difference between the learner’s action and the expert’s action in the expert state distribution.

Original: However, the BC method suffers from the covariate shift issue (Ross et al., 2011), i.e . the state induced by the learner’s policy cumulatively deviates from the expert’s distribution.
Revised: However, the BC method suffers from the covariate shift issue (Ross et al., 2011), i.e . the state induced by the learner’s policy cumulatively deviates from the expert’s distribution.

Original: To overcome the covariate shift obstacle, existing methods such as DAgger (Ross et al., 2011) and DART (Laskey et al., 2017) query supervisor corrections at the learner’s states or perturbed expert’s states.
Revised: To overcome the covariate shift obstacle, existing methods such as DAgger (Ross et al., 2011) and DART (Laskey et al., 2017) query supervisor corrections at the learner’s states or perturbed expert’s states.

Original: Since human supervision is hard to collect, recent works like GAIL (Ho & Ermon, 2016) seek to provide feedback from a neural network-based discriminator to recover from out-ofdistribution states generated by the learner’s policy.
Revised: Since human supervision is hard to collect, recent works like GAIL (Ho & Ermon, 2016) seek to provide feedback from a neural network-based discriminator to recover from out-ofdistribution states generated by the learner’s policy.

Original: However, these data augmentation methods need either expert supervision or rolling out their policy in the real world or a realistic simulator, which are impractical in autonomous driving.
Revised: However, these data augmentation methods need either expert supervision or rolling out their policy in the real world or a realistic simulator, which are impractical in autonomous driving.

Original: Instead, some researchers attempt to constrain the learned policy formulation to ensure its robustness to the policy error by incorporating control theoretic prior knowledge.
Revised: Instead, some researchers attempt to constrain the learned policy formulation to ensure its robustness to the policy error by incorporating control theoretic prior knowledge.

Original: For example, Palan et al.
Revised: For example, Palan et al.

Original: ; Havens & Hu (2021) pose Kalman or linear matrix inequality constraints on the learned linear policy to guarantee its closed-loop stability in a linear time-invariant (LTI) system.
Revised: ; Havens & Hu (2021) pose Kalman or linear matrix inequality constraints on the learned linear policy to guarantee its closed-loop stability in a linear time-invariant (LTI) system.

Original: Yin et al.
Revised: Yin et al.

Original: (2021) relaxes the linear policy formulation into a simple feed-forward neural network representation and East (2022) extends the method in Havens & Hu (2021) to polynomial policy and dynamical system.
Revised: (2021) relaxes the linear policy formulation into a simple feed-forward neural network representation and East (2022) extends the method in Havens & Hu (2021) to polynomial policy and dynamical system.

Original: However, the urban driving task is too complex to be handled by these naive policy formulations.
Revised: However, the urban driving task is too complex to be handled by these naive policy formulations.

Original: To learn a stable and general urban driving policy by imitating only the offline human demonstrations, we propose a context-conditioned imitation learning (CCIL) method, where a policy network is learned to predict the SDV’s future states using only its observed context, different from classic policy taking both ego state and context as input to generate its action (Codevilla et al., 2019; Bansal et al., 2019).
Revised: To learn a stable and general urban driving policy by imitating only the offline human demonstrations, we propose a context-conditioned imitation learning (CCIL) method, where a policy network is learned to predict the SDV’s future states using only its observed context, different from classic policy taking both ego state and context as input to generate its action (Codevilla et al., 2019; Bansal et al., 2019).

Original: The motivation is that the ego state can be easily influenced by the policy’s error, thus leading to the catastrophic distribution shift.
Revised: The motivation is that the ego state can be easily influenced by the policy’s error, thus leading to the catastrophic distribution shift.

Original: On the contrary, the static context elements such as lanes or crosswalks will not be influenced by the SDV and the dynamic context element like human drivers will try to recover from its perturbation.
Revised: On the contrary, the static context elements such as lanes or crosswalks will not be influenced by the SDV and the dynamic context element like human drivers will try to recover from its perturbation.

Original: Thus, based on the stability assumption of the traffic system, we can prove theoretically that our policy formulation can achieve closed-loop stability, thus addressing the distribution shift issue.
Revised: Thus, based on the stability assumption of the traffic system, we can prove theoretically that our policy formulation can achieve closed-loop stability, thus addressing the distribution shift issue.

Original: In practice, as it becomes challenging to accurately plan a trajectory for a SDV without its historical trajectory, we construct our policy network based on
Revised: In practice, as it becomes challenging to accurately plan a trajectory for a SDV without its historical trajectory, we construct our policy network based on

Original: Transformer (Vaswani et al., 2017) to exploit spatial and temporal relation information in highly interactive and constrained urban driving scenarios.
Revised: Transformer (Vaswani et al., 2017) to exploit spatial and temporal relation information in highly interactive and constrained urban driving scenarios.

Original: Furthermore, during evaluation, we employ a linear-quadratic regulator (LQR) ( ˚Astr ¨ om & Murray, 2021) to yield a smooth action based on the
Revised: Furthermore, during evaluation, we employ a linear-quadratic regulator (LQR) ( ˚Astr ¨ om & Murray, 2021) to yield a smooth action based on the

Original: SDV’s current and predicted states.
Revised: SDV’s current and predicted states.

Original: The main contributions of this paper can be summarized as follows:
Revised: The main contributions of this paper can be summarized as follows:

Original: To address the covariate shift issue in offline imitation learning, we propose a novel contextconditioned imitation learning method, where a policy is learned to output the ego state using only its context as input.
Revised: To address the covariate shift issue in offline imitation learning, we propose a novel contextconditioned imitation learning method, where a policy is learned to output the ego state using only its context as input.

Original: A robustness assurance is provided for our policy formulation based on an assumption of the context’s stability.
Revised: A robustness assurance is provided for our policy formulation based on an assumption of the context’s stability.

Original: To apply our method to urban driving, we remove the explicit ego state information input and propose a new ego-perturbed goal-oriented coordinate system to reduce the implicit ego information in the coordinate system.
Revised: To apply our method to urban driving, we remove the explicit ego state information input and propose a new ego-perturbed goal-oriented coordinate system to reduce the implicit ego information in the coordinate system.

Original: Besides, we design a Transformer-based planning network to make full use of the spatial and temporal information in the context.
Revised: Besides, we design a Transformer-based planning network to make full use of the spatial and temporal information in the context.

Original: To verify the effectiveness of our approach, we benchmark the real-world large-scale urban driving Lyft (Houston et al., 2020) and nuPlan (Caesar et al., 2021) datasets with state-of-the-art performance.
Revised: To verify the effectiveness of our approach, we benchmark the real-world large-scale urban driving Lyft (Houston et al., 2020) and nuPlan (Caesar et al., 2021) datasets with state-of-the-art performance.

Original: The video and code can be found at https://sites.google.com/view/ contextconditionedil .
Revised: The video and code can be found at https://sites.google.com/view/ contextconditionedil .

Original:  R ELATED W ORK 2. I MITATION LEARNING FOR AUTONOMOUS DRIVING
Revised:  R ELATED W ORK 2. I MITATION LEARNING FOR AUTONOMOUS DRIVING

Original: The objective of applying IL in autonomous driving is to learn driving behavior mimicking human drivers.
Revised: The objective of applying IL in autonomous driving is to learn driving behavior mimicking human drivers.

Original: The most straightforward solution is BC which minimizes the difference between the learner’s and the expert’s action on the expert state without demanding extra manually labeled data and online interaction.
Revised: The most straightforward solution is BC which minimizes the difference between the learner’s and the expert’s action on the expert state without demanding extra manually labeled data and online interaction.

Original: Early BC applications in autonomous driving such as ALVINN (Pomerleau, 1988) and PilotNet (Bojarski et al., 2016) learn an end-to-end policy that directly maps sensor inputs to vehicle control commands using a large amount of human driving experience.
Revised: Early BC applications in autonomous driving such as ALVINN (Pomerleau, 1988) and PilotNet (Bojarski et al., 2016) learn an end-to-end policy that directly maps sensor inputs to vehicle control commands using a large amount of human driving experience.

Original: Recently, ChauffeurNet (Bansal et al., 2019) provides intermediate planning using perception results to improve generalization and transparency.
Revised: Recently, ChauffeurNet (Bansal et al., 2019) provides intermediate planning using perception results to improve generalization and transparency.

Original: However, the BC approach typically leads to the covariate shift between the training distribution and deploying distribution, as minor errors in the policy can lead to deviating from the expert state and then larger errors.
Revised: However, the BC approach typically leads to the covariate shift between the training distribution and deploying distribution, as minor errors in the policy can lead to deviating from the expert state and then larger errors.

Original: IL methods to address the distribution shift challenge can be categorized into online methods and offline model-free and model-based methods.
Revised: IL methods to address the distribution shift challenge can be categorized into online methods and offline model-free and model-based methods.

Original: Online methods try to directly match the expert stateaction distribution instead of matching the expert state conditioned action distribution like BC.
Revised: Online methods try to directly match the expert stateaction distribution instead of matching the expert state conditioned action distribution like BC.

Original: For instance, the method in Zhang & Cho (2017) based on DAgger (Ross et al., 2011) queries supervisor actions at the state the learner visits and then adds the new data into the dataset, thus adjusting the expert state distribution to match the learner’s state distribution.
Revised: For instance, the method in Zhang & Cho (2017) based on DAgger (Ross et al., 2011) queries supervisor actions at the state the learner visits and then adds the new data into the dataset, thus adjusting the expert state distribution to match the learner’s state distribution.

Original: To get rid of the requirement of interactive expert, several methods like Wang et al.
Revised: To get rid of the requirement of interactive expert, several methods like Wang et al.

Original: based on adversarial imitation learning (Ho & Ermon, 2016) utilize a discriminator to measure the difference between learner and expert’s state- action distribution and then compute the reward in reinforcement learning.
Revised: based on adversarial imitation learning (Ho & Ermon, 2016) utilize a discriminator to measure the difference between learner and expert’s state- action distribution and then compute the reward in reinforcement learning.

Original: By raising the policy’s accumulated reward, the policy’s state distribution will get close to the expert distribution.
Revised: By raising the policy’s accumulated reward, the policy’s state distribution will get close to the expert distribution.

Original: However, the reinforcement loop needs online interaction with the environment to generate the policy’s state distribution, which is hard to deploy in safe-critical tasks like autonomous driving.
Revised: However, the reinforcement loop needs online interaction with the environment to generate the policy’s state distribution, which is hard to deploy in safe-critical tasks like autonomous driving.

Original: On the contrary, our method matches ego state distribution conditioned on context, which can be learned by supervised learning without interaction with the environment or access to expert supervision.
Revised: On the contrary, our method matches ego state distribution conditioned on context, which can be learned by supervised learning without interaction with the environment or access to expert supervision.

Original: The most popular model-free IL methods are based on DART (Laskey et al., 2017) which avoids the compounding error by providing synthetic examples of how to recover from the deviated state.
Revised: The most popular model-free IL methods are based on DART (Laskey et al., 2017) which avoids the compounding error by providing synthetic examples of how to recover from the deviated state.

Original: In Codevilla et al.
Revised: In Codevilla et al.

Original: (2018b), they inject temporally correlated noise into the trajectory to simulate gradual drift away from the desired trajectory.
Revised: (2018b), they inject temporally correlated noise into the trajectory to simulate gradual drift away from the desired trajectory.

Original: Alternatively, ChauffeurNet (Bansal et al., 2019) adds a uniform perturbation to the current SDV state and fits a new smooth trajectory that brings the SDV back to the original target location.
Revised: Alternatively, ChauffeurNet (Bansal et al., 2019) adds a uniform perturbation to the current SDV state and fits a new smooth trajectory that brings the SDV back to the original target location.

Original: However, these rule-based trajectory augmentation methods are hard to cover the real motion distribution induced by the learner’s policy and the policy is very likely to learn a propensity for perturbed driving.
Revised: However, these rule-based trajectory augmentation methods are hard to cover the real motion distribution induced by the learner’s policy and the policy is very likely to learn a propensity for perturbed driving.

Original: Our method also applies perturbation to the SDV’s current position but its role is to blur the ego position information instead of data augmentation.
Revised: Our method also applies perturbation to the SDV’s current position but its role is to blur the ego position information instead of data augmentation.

Original: Therefore, our method does not have the trajectory smoothing process during training.
Revised: Therefore, our method does not have the trajectory smoothing process during training.

Original: Our method is also model-free but we seek to endow the policy with robustness properties by constraining its formulation without resorting to recovery examples.
Revised: Our method is also model-free but we seek to endow the policy with robustness properties by constraining its formulation without resorting to recovery examples.

Original: Model-based IL methods address the distribution shift by minimizing the difference between a trajectory rolling out in a differentiable learned or data-driven model with the expert trajectory.
Revised: Model-based IL methods address the distribution shift by minimizing the difference between a trajectory rolling out in a differentiable learned or data-driven model with the expert trajectory.

Original: PPUU (Amos et al., 2018) first learn a dynamics model based on variational autoencoder (Kingma & Welling, 2013) from data and then train the policy network to output actions that lead to a similar trajectory as the expert trajectory.
Revised: PPUU (Amos et al., 2018) first learn a dynamics model based on variational autoencoder (Kingma & Welling, 2013) from data and then train the policy network to output actions that lead to a similar trajectory as the expert trajectory.

Original: Since the dynamics model is differentiable, the actions can receive gradients from multiple time steps ahead which can penalize actions that will lead to large diverges in the future even if the instantaneous divergence is small.
Revised: Since the dynamics model is differentiable, the actions can receive gradients from multiple time steps ahead which can penalize actions that will lead to large diverges in the future even if the instantaneous divergence is small.

Original: Instead of learning a model,
Revised: Instead of learning a model,

Original: UrbanDriver (Scheel et al., 2022) constructs a differentiable data-driven model using recorded perception data and High Definition (HD) maps, where new observations are calculated by a coordinate transformation based on the pose of controlled SDV and collected data.
Revised: UrbanDriver (Scheel et al., 2022) constructs a differentiable data-driven model using recorded perception data and High Definition (HD) maps, where new observations are calculated by a coordinate transformation based on the pose of controlled SDV and collected data.

Original: However, the performance of the model-based approach is limited by the model’s accuracy.
Revised: However, the performance of the model-based approach is limited by the model’s accuracy.

Original: 2. I MITATION LEARNING WITH ROBUSTNESS
Revised: 2. I MITATION LEARNING WITH ROBUSTNESS

Original: IL is different from supervised learning by deploying the policy under dynamics, whose robustness is considered to be the learned policy’s ability to recover from policy errors.
Revised: IL is different from supervised learning by deploying the policy under dynamics, whose robustness is considered to be the learned policy’s ability to recover from policy errors.

Original: Some researchers have attempted to learn a policy with a stability guarantee using control-theoretic methods by constraining policy and system dynamics.
Revised: Some researchers have attempted to learn a policy with a stability guarantee using control-theoretic methods by constraining policy and system dynamics.

Original: Taylor Series IL (Pfrommer et al., 2022) proves that the trajectory induced by the learner and expert will be close if their derivative difference at expert states is small but computing the high-order derivatives of the expert policy are difficult without sufficient data.
Revised: Taylor Series IL (Pfrommer et al., 2022) proves that the trajectory induced by the learner and expert will be close if their derivative difference at expert states is small but computing the high-order derivatives of the expert policy are difficult without sufficient data.

Original: Others learn a robust linear (Palan et al., 2020; Havens & Hu, 2021) or simple feed-forward neural network (Yin et al., 2021) control policy for a linear dynamical system by posing constraints on the policy.
Revised: Others learn a robust linear (Palan et al., 2020; Havens & Hu, 2021) or simple feed-forward neural network (Yin et al., 2021) control policy for a linear dynamical system by posing constraints on the policy.

Original: Even though the authors in East (2022) extend the robustness guarantee to the polynomial system and policy, obtaining guarantees on close-loop stability for the nonlinear autonomous driving system remains a challenge.
Revised: Even though the authors in East (2022) extend the robustness guarantee to the polynomial system and policy, obtaining guarantees on close-loop stability for the nonlinear autonomous driving system remains a challenge.

Original: Recently, the CMILe method (Tu et al., 2022) is capable of training nonlinear policies with the same safety guarantees as the expert but requires online expert access as
Revised: Recently, the CMILe method (Tu et al., 2022) is capable of training nonlinear policies with the same safety guarantees as the expert but requires online expert access as

Original: DAgger.
Revised: DAgger.

Original: However, our method only constrains our policy in the formulation of receiving context state and producing ego state, whose stability is guaranteed under a mild input-to-state stability assumption on the environment dynamics.
Revised: However, our method only constrains our policy in the formulation of receiving context state and producing ego state, whose stability is guaranteed under a mild input-to-state stability assumption on the environment dynamics.

Original:  T HEORETICAL A NALYSIS
Revised:  T HEORETICAL A NALYSIS

Original: We first introduce the notations and definitions used in this paper.
Revised: We first introduce the notations and definitions used in this paper.

Original: For any vector x ∈ R n , ∥ x ∥ p stands for its L p norm and ∥ x ∥ for L 2 norm.
Revised: For any vector x ∈ R n , ∥ x ∥ p stands for its L p norm and ∥ x ∥ for L 2 norm.

Original: For any square matrix A ∈ R n × n , ∥ A ∥ denotes its induced L 2 norm and ρ ( A ) is its spectral radius (the maximum of the absolute values of its eigenvalues).
Revised: For any square matrix A ∈ R n × n , ∥ A ∥ denotes its induced L 2 norm and ρ ( A ) is its spectral radius (the maximum of the absolute values of its eigenvalues).

Original: For every induced matrix norm,
Revised: For every induced matrix norm,

Original: Definition 1 (Comparison Functions)
Revised: Definition 1 (Comparison Functions)

Original: A function γ : R ⩾ 

Revised: A function γ : R ⩾

Original: → R ⩾ 0 is class K if it is continuous, strictly increasing and satisfies γ (0)
Revised: → R ⩾ 0 is class K if it is continuous, strictly increasing and satisfies γ (0)

Original: = 0 .
Revised: = 0 .

Original: A function β ( x, t ) : R ⩾ 

Revised: A function β ( x, t ) : R ⩾

Original: × R ⩾ 

Revised: × R ⩾

Original: → R ⩾ 0 is class KL if it is continuous, β ( · , t ) is class K for each t and β ( x, · ) is decreasing for each x .
Revised: → R ⩾ 0 is class KL if it is continuous, β ( · , t ) is class K for each t and β ( x, · ) is decreasing for each x .

Original: Definition 2 (input-to-state stability (ISS) (Sontag, 2008))
Revised: Definition 2 (input-to-state stability (ISS) (Sontag, 2008))

Original: A discrete-time system x t + = f
Revised: A discrete-time system x t + = f

Original: ( x t , u t ) , x 0 = ξ is input-to-state stable if there exists a class KL function β and a class K function γ such that, for each bounded input u and initial condition ξ and t ∈ Z + , it holds that:
Revised: ( x t , u t ) , x 0 = ξ is input-to-state stable if there exists a class KL function β and a class K function γ such that, for each bounded input u and initial condition ξ and t ∈ Z + , it holds that:

Original: [Equation]
Revised: [Equation]

Original: where ∥ u ∥ ∞ = sup t ∈ Z + ( u t ) is the sup norm of the input.
Revised: where ∥ u ∥ ∞ = sup t ∈ Z + ( u t ) is the sup norm of the input.

Original: If a system satisfies the ISS property, the distance between any two trajectories must eventually be bounded by its input signal and independent of initial conditions.
Revised: If a system satisfies the ISS property, the distance between any two trajectories must eventually be bounded by its input signal and independent of initial conditions.

Original: We consider performing imitation learning in a nonlinear discrete-time system:
Revised: We consider performing imitation learning in a nonlinear discrete-time system:

Original: [Equation]
Revised: [Equation]

Original: where x t ∈ R n is the system state at time t ∈ N and u t ∈ R u is the control input.
Revised: where x t ∈ R n is the system state at time t ∈ N and u t ∈ R u is the control input.

Original: For an autonomous driving system, its state can be separated into two parts x t =
Revised: For an autonomous driving system, its state can be separated into two parts x t =

Original: ( s t , c t ) : an SDV state s t ∈ R m controlled by the control input u and context state c t ∈ R n − m which is influenced by the
Revised: ( s t , c t ) : an SDV state s t ∈ R m controlled by the control input u and context state c t ∈ R n − m which is influenced by the

Original: SDV state.
Revised: SDV state.

Original: By viewing the policy error as a disturbance to the system, we apply linearization to the nonlinear system at the expert’s state x ∗ to obtain the following linear subsystem:
Revised: By viewing the policy error as a disturbance to the system, we apply linearization to the nonlinear system at the expert’s state x ∗ to obtain the following linear subsystem:

Original: [Equation]
Revised: [Equation]

Original: with solution:
Revised: with solution:

Original: [Equation]
Revised: [Equation]

Original: Here with abuse of notation, we use the same symbol c t , s t for the context deviation c t − c ∗ t and
Revised: Here with abuse of notation, we use the same symbol c t , s t for the context deviation c t − c ∗ t and

Original: SDV state deviation s t − s ∗ t .
Revised: SDV state deviation s t − s ∗ t .

Original: Because the real-world traffic system without the SDV state deviation c t +1 =
Revised: Because the real-world traffic system without the SDV state deviation c t +1 =

Original: Ac t is stable, we have ρ ( A ) < 1 .
Revised: Ac t is stable, we have ρ ( A ) < 1 .

Original: For such a Schur matrix, there are constants c > 0 and 0 ⩽ σ < 1 such that ∥ A t ∥ ⩽ cσ t (Jiang & Wang, 2001).
Revised: For such a Schur matrix, there are constants c > 0 and 0 ⩽ σ < 1 such that ∥ A t ∥ ⩽ cσ t (Jiang & Wang, 2001).

Original: Besides, the context in traffic systems generally includes static map elements that are not influenced by SDV vehicles and intelligent human drivers who can quickly recover from the other vehicle’s perturbation.
Revised: Besides, the context in traffic systems generally includes static map elements that are not influenced by SDV vehicles and intelligent human drivers who can quickly recover from the other vehicle’s perturbation.

Original: It is natural to assume the subsystem in equationto be input-to-state stable (ISS) with β ( r, t ) <
Revised: It is natural to assume the subsystem in equationto be input-to-state stable (ISS) with β ( r, t ) <

Original: σ t − 1 r and γ ( r ) ⩽ ϵr , where ϵ ⩾ 0 .
Revised: σ t − 1 r and γ ( r ) ⩽ ϵr , where ϵ ⩾ 0 .

Original: Combining equation 1 and equation 4, we have:
Revised: Combining equation 1 and equation 4, we have:

Original: [Equation]
Revised: [Equation]

Original: which implies that c < 1 σ , and
Revised: which implies that c < 1 σ , and

Original: ∥ B ∥ ⩽ ϵ (1 − σ ) c .
Revised: ∥ B ∥ ⩽ ϵ (1 − σ ) c .

Original: Next, we study how the policy in our formulation can be stable under this ISS system.
Revised: Next, we study how the policy in our formulation can be stable under this ISS system.

Original: We learn a context state feedback control s t = u t = f θ ( c t ) which directly maps the context state to the
Revised: We learn a context state feedback control s t = u t = f θ ( c t ) which directly maps the context state to the

Original: SDV state.
Revised: SDV state.

Original: For simplicity, we consider a linear policy u t =
Revised: For simplicity, we consider a linear policy u t =

Original: Kc t .
Revised: Kc t .

Original: Then, the linear subsystem in equation 3 can be simplified as c t +

Revised: Then, the linear subsystem in equation 3 can be simplified as c t +

Original: =
Revised: =

Original: ( A + BK ) c t whose stable condition is ρ ( A + BK ) < 1 .
Revised: ( A + BK ) c t whose stable condition is ρ ( A + BK ) < 1 .

Original: Thus, we have if ∥ K ∥ <
Revised: Thus, we have if ∥ K ∥ <

Original: c
Revised: c

Original: (1 − cσ )
Revised: (1 − cσ )

Original: ϵ (1 − σ ) , then
Revised: ϵ (1 − σ ) , then

Original: [Equation]
Revised: [Equation]

Original: This result shows that the stability of closed-loop autonomous driving can be guaranteed as long as the context system is stable enough and the norm of our policy is sufficiently small.
Revised: This result shows that the stability of closed-loop autonomous driving can be guaranteed as long as the context system is stable enough and the norm of our policy is sufficiently small.

Original: In practice, we will add a L 2 norm regularization into the training loss of our neural network policy to internalize this stability prior.
Revised: In practice, we will add a L 2 norm regularization into the training loss of our neural network policy to internalize this stability prior.

Original: Finally, we show how the naive BC policy formulation which takes both the SDV state and context state and generates the displacement from the current state can fail with even a fixed context c t = 0 .
Revised: Finally, we show how the naive BC policy formulation which takes both the SDV state and context state and generates the displacement from the current state can fail with even a fixed context c t = 0 .

Original: We consider a linear policy: u t = K bc s t and update the state s t
Revised: We consider a linear policy: u t = K bc s t and update the state s t

Original: +1 =
Revised: +1 =

Original: s t + u t .
Revised: s t + u t .

Original: Then, the dynamics is s t +

Revised: Then, the dynamics is s t +

Original: = ( I + K bc ) s t .
Revised: = ( I + K bc ) s t .

Original: Because ∥ I ∥ = 1 , we cannot obtain the stable condition ρ ( I + K bc ) <by limiting the ∥ K bc ∥ as our policy formulation.
Revised: Because ∥ I ∥ = 1 , we cannot obtain the stable condition ρ ( I + K bc ) <by limiting the ∥ K bc ∥ as our policy formulation.

Original: Thus, the system’s stability cannot be guaranteed.
Revised: Thus, the system’s stability cannot be guaranteed.

Original: This simple analysis supports our choice to overlook the SDV history and current state in the learning to avoid causal confounding and distribution shift.
Revised: This simple analysis supports our choice to overlook the SDV history and current state in the learning to avoid causal confounding and distribution shift.

Original: Its advantage will also be validated in our experiments showing that this can significantly decrease the off-road rate.
Revised: Its advantage will also be validated in our experiments showing that this can significantly decrease the off-road rate.

Original:  M ETHOD
Revised:  M ETHOD

Original: As shown in Figure 1, we apply our context-conditioned imitation learning method to learn a policy for urban driving.
Revised: As shown in Figure 1, we apply our context-conditioned imitation learning method to learn a policy for urban driving.

Original: The policy network consists of a spatial encoder and a temporal encoder.
Revised: The policy network consists of a spatial encoder and a temporal encoder.

Original: The policy network is trained to minimize the L 1 distance between its predictions and the ground truth trajectories.
Revised: The policy network is trained to minimize the L 1 distance between its predictions and the ground truth trajectories.

Original: During the evaluation, we employ a LQR controller to generate smooth planning based on the predicted trajectory from the policy network.
Revised: During the evaluation, we employ a LQR controller to generate smooth planning based on the predicted trajectory from the policy network.

Original: 4. I NPUT REPRESENTATION
Revised: 4. I NPUT REPRESENTATION

Original: Following Bansal et al.
Revised: Following Bansal et al.

Original: (2019), the intermediate representations such as oriented bounding boxes from a perception system and road networks from HD maps rather than raw sensor data (such as camera images or lidar points clouds) are harnessed to improve generalization and interpretability.
Revised: (2019), the intermediate representations such as oriented bounding boxes from a perception system and road networks from HD maps rather than raw sensor data (such as camera images or lidar points clouds) are harnessed to improve generalization and interpretability.

Original: In detail, there are two types of inputs to the policy network: map context and traffic participants.
Revised: In detail, there are two types of inputs to the policy network: map context and traffic participants.

Original: For the map context, we consider two types of map elements, which are crosswalks and lanes.
Revised: For the map context, we consider two types of map elements, which are crosswalks and lanes.

Original: Each map element is presented in a polyline which is a sequence of vectors with different associated features as Gao et al.
Revised: Each map element is presented in a polyline which is a sequence of vectors with different associated features as Gao et al.

Original: The features of crosswalk vectors are their initial and terminal point’s position and sequence orders, and lane vectors have additional features such as their traffic light state and lane width.
Revised: The features of crosswalk vectors are their initial and terminal point’s position and sequence orders, and lane vectors have additional features such as their traffic light state and lane width.

Original: For the traffic participants, we divide them into ego vehicle and other agents.
Revised: For the traffic participants, we divide them into ego vehicle and other agents.

Original: For the ego vehicle, as discussed above, we should overlook the features which can be influenced by the learned policy to overcome the distribution shift obstacle.
Revised: For the ego vehicle, as discussed above, we should overlook the features which can be influenced by the learned policy to overcome the distribution shift obstacle.

Original: Therefore, we only input its mission goal position while removing its history.
Revised: Therefore, we only input its mission goal position while removing its history.

Original: For each other agent, its features are composed of its type (e.g. car, bicycle, and cyclist), sizes, centroids, and orientations of bounding boxes at several past time steps.
Revised: For each other agent, its features are composed of its type (e.g. car, bicycle, and cyclist), sizes, centroids, and orientations of bounding boxes at several past time steps.

Original: Even though there is no explicit SDV state in the network input, the observations are typically translated to the SDV coordinate system using its rear axle’s midpoint as origin and orientation as x-axis direction in previous works (Bojarski et al., 2016; Scheel et al., 2022).
Revised: Even though there is no explicit SDV state in the network input, the observations are typically translated to the SDV coordinate system using its rear axle’s midpoint as origin and orientation as x-axis direction in previous works (Bojarski et al., 2016; Scheel et al., 2022).

Original: This implicit incorporation of the SDV state can still cause a distribution shift of inputs.
Revised: This implicit incorporation of the SDV state can still cause a distribution shift of inputs.

Original: To reduce the implicit influence and ensure local observability, we adopt an ego-perturbed goal-oriented coordinate system whose origin is the SDV position adding a zero mean Gaussian perturbation, and x -axis direction is toward its goal position.
Revised: To reduce the implicit influence and ensure local observability, we adopt an ego-perturbed goal-oriented coordinate system whose origin is the SDV position adding a zero mean Gaussian perturbation, and x -axis direction is toward its goal position.

Original: Note that the perturbation is only required during training and the goal position can be replaced with any point that is not influenced by the driving policy.
Revised: Note that the perturbation is only required during training and the goal position can be replaced with any point that is not influenced by the driving policy.

Original: 4. P OLICY N ETWORK
Revised: 4. P OLICY N ETWORK

Original: The policy network is designed to directly plan a trajectory for the SDV using only its context.
Revised: The policy network is designed to directly plan a trajectory for the SDV using only its context.

Original: Since it is more challenging to plan without the historical trajectory, we need to make full use of the spatial-temporal interactions between the SDV and context to infer its future states.
Revised: Since it is more challenging to plan without the historical trajectory, we need to make full use of the spatial-temporal interactions between the SDV and context to infer its future states.

Original: Thus, we first embed the observation at each time step into an observation feature by a spatial encoder, and then the observation features in current and previous time steps are embedded to generate the predictions by a temporal encoder.
Revised: Thus, we first embed the observation at each time step into an observation feature by a spatial encoder, and then the observation features in current and previous time steps are embedded to generate the predictions by a temporal encoder.

Original: We construct our policy network based on the Transformer whose multi-head attention mechanism has demonstrated excellent performance in capturing relations.
Revised: We construct our policy network based on the Transformer whose multi-head attention mechanism has demonstrated excellent performance in capturing relations.

Original: Spatial Encoder: To capture the spatial relations between vectors of the same polyline such as a lane or crosswalk, we employ a local Transformer encoder as the vector-vector interaction encoder.
Revised: Spatial Encoder: To capture the spatial relations between vectors of the same polyline such as a lane or crosswalk, we employ a local Transformer encoder as the vector-vector interaction encoder.

Original: Subsequently, we aggregate the features of vectors belonging to the same polyline by max-pooling to obtain polyline-level features.
Revised: Subsequently, we aggregate the features of vectors belonging to the same polyline by max-pooling to obtain polyline-level features.

Original: To obtain the agent and goal features, two multi-layer perceptions (MLP) are applied.
Revised: To obtain the agent and goal features, two multi-layer perceptions (MLP) are applied.

Original: To model the high-order interactions between the goal, agents, and map elements, we utilize a global Transformer encoder whose goal embedding is output as observation features.
Revised: To model the high-order interactions between the goal, agents, and map elements, we utilize a global Transformer encoder whose goal embedding is output as observation features.

Original: Temporal Encoder:
Revised: Temporal Encoder:

Original: To take the temporal information and interaction into account, a Transformer encoder with a causal self-attention mask is employed to embed H historical observation features with a step interval I .
Revised: To take the temporal information and interaction into account, a Transformer encoder with a causal self-attention mask is employed to embed H historical observation features with a step interval I .

Original: Then, each hidden state of the Transformer is decoded by a full-connected linear layer to generate a prediction of the SDV future state at future T time steps.
Revised: Then, each hidden state of the Transformer is decoded by a full-connected linear layer to generate a prediction of the SDV future state at future T time steps.

Original: For simplicity, we represent the SDV state with its rear axle’s midpoint coordinate and orientation.
Revised: For simplicity, we represent the SDV state with its rear axle’s midpoint coordinate and orientation.

Original: 4. T RAINING P ROCESS
Revised: 4. T RAINING P ROCESS

Original: We leverage supervised learning like BC to train the policy network by minimizing the L 1 distance between the predictions and ground truth trajectories, as L 1 is more correlated to driving performance compared with the mean square error (Codevilla et al., 2018a).
Revised: We leverage supervised learning like BC to train the policy network by minimizing the L 1 distance between the predictions and ground truth trajectories, as L 1 is more correlated to driving performance compared with the mean square error (Codevilla et al., 2018a).

Original: To help the network converge and generalize, we additionally introduce an auxiliary task of minimizing the L 1 error at all previous time steps and apply a squared L 2 norm regularization to the network parameters θ inspired by equation 6.
Revised: To help the network converge and generalize, we additionally introduce an auxiliary task of minimizing the L 1 error at all previous time steps and apply a squared L 2 norm regularization to the network parameters θ inspired by equation 6.

Original: The final loss is:
Revised: The final loss is:

Original: [Equation]
Revised: [Equation]

Original: where s t is the ground-truth SDV state at time step t and ˆ s thI is the state prediction at the historical time step hI for its future t time step.
Revised: where s t is the ground-truth SDV state at time step t and ˆ s thI is the state prediction at the historical time step hI for its future t time step.

Original: µ is an auxiliary hyperparameter and λ is a regularization factor.
Revised: µ is an auxiliary hyperparameter and λ is a regularization factor.

Original: 4. E VALUATION P ROCESS
Revised: 4. E VALUATION P ROCESS

Original: Even though the prediction without SDV state input is more stable, it has difficulty ensuring the smoothness of the predicted trajectory from the current state.
Revised: Even though the prediction without SDV state input is more stable, it has difficulty ensuring the smoothness of the predicted trajectory from the current state.

Original: Prior works such as Vitelli et al.
Revised: Prior works such as Vitelli et al.

Original: ; Amos et al.
Revised: ; Amos et al.

Original: (2018) usually add a differentiable kinematic layer into the policy network to generate physically feasible planning.
Revised: (2018) usually add a differentiable kinematic layer into the policy network to generate physically feasible planning.

Original: However, the differentiable kinematic layer will bring in the current SDV state which is undesirable for learning a policy.
Revised: However, the differentiable kinematic layer will bring in the current SDV state which is undesirable for learning a policy.

Original: Therefore, we choose to obtain a smooth trajectory during evaluation by a linear-quadratic regulator (LQR) ( ˚Astr ¨ om & Murray, 2021) which can efficiently minimize the total commutative quadratic cost of a linear dynamic system.
Revised: Therefore, we choose to obtain a smooth trajectory during evaluation by a linear-quadratic regulator (LQR) ( ˚Astr ¨ om & Murray, 2021) which can efficiently minimize the total commutative quadratic cost of a linear dynamic system.

Original: We consider a finite-horizon, discrete-time linear system whose dynamics is described by:
Revised: We consider a finite-horizon, discrete-time linear system whose dynamics is described by:

Original: [Equation]
Revised: [Equation]

Original: 
Revised: 

Original: I
Revised: I

Original: D D
Revised: D D

Original: I
Revised: I

Original: D
Revised: D

Original: I   (cid:34) p t ˙ p t ¨ p t (cid:35) +  
Revised: I   (cid:34) p t ˙ p t ¨ p t (cid:35) +  

Original: D
Revised: D

Original: D
Revised: D

Original: I   u t , (8) where D is a diagonal matrix with the time interval of each step as diagonal entries, ˙ p
Revised: I   u t , (8) where D is a diagonal matrix with the time interval of each step as diagonal entries, ˙ p

Original: = ( v t , w t ) , ¨ p t = ( a t , α t ) , u t = ( u pt , u αt ) denote the positional and angular velocity, acceleration, and control respectively, subject to a quadratic cost function:
Revised: = ( v t , w t ) , ¨ p t = ( a t , α t ) , u t = ( u pt , u αt ) denote the positional and angular velocity, acceleration, and control respectively, subject to a quadratic cost function:

Original: [Equation]
Revised: [Equation]

Original: where p t is regarded as the planning trajectory and the predicted states ˆ s 0 ,t at the current step from the policy network are regarded as target poses; w w , w a , w α and w u are the weights to balance the positional accuracy and the smoothness of the planned trajectory.
Revised: where p t is regarded as the planning trajectory and the predicted states ˆ s 0 ,t at the current step from the policy network are regarded as target poses; w w , w a , w α and w u are the weights to balance the positional accuracy and the smoothness of the planned trajectory.

Original: At the end of the optimization, the
Revised: At the end of the optimization, the

Original: LQR will output a smooth trajectory for the SDV to follow.
Revised: LQR will output a smooth trajectory for the SDV to follow.

Original:  E XPERIMENTS 5. D ATASET
Revised:  E XPERIMENTS 5. D ATASET

Original: To benchmark our method’s performance, we conduct experiments on two real-world large-scale datasets: Lyft Level 5 Prediction Dataset (Houston et al., 2020): contains about 1000h urban driving demonstrations in Palo Alto, which have been separated into independent scenes of nearly 25s at 10Hz.
Revised: To benchmark our method’s performance, we conduct experiments on two real-world large-scale datasets: Lyft Level 5 Prediction Dataset (Houston et al., 2020): contains about 1000h urban driving demonstrations in Palo Alto, which have been separated into independent scenes of nearly 25s at 10Hz.

Original: We train our network on the provided 100h (16265 scenes) subset as Scheel et al.
Revised: We train our network on the provided 100h (16265 scenes) subset as Scheel et al.

Original: (2022) and test with all 16220 validation scenes.
Revised: (2022) and test with all 16220 validation scenes.

Original: nuPlan Dataset (Caesar et al., 2021) provides an urban driving dataset with 1312h of human driving data from 4 cities (Boston, Pittsburgh, Las
Revised: nuPlan Dataset (Caesar et al., 2021) provides an urban driving dataset with 1312h of human driving data from 4 cities (Boston, Pittsburgh, Las

Original: Vegas, and Singapore).
Revised: Vegas, and Singapore).

Original: Due to the huge difference in traffic rules and patterns in different cities, we extract driving data in Las Vegas as Phan-Minh et al.
Revised: Due to the huge difference in traffic rules and patterns in different cities, we extract driving data in Las Vegas as Phan-Minh et al.

Original: Then, we separate the data into independent scenes of 25s at 10Hz like Lyft and filter out the scene without a mission goal.
Revised: Then, we separate the data into independent scenes of 25s at 10Hz like Lyft and filter out the scene without a mission goal.

Original: After filtering, we obtain 63181 training, 4774 validation, and 6386 testing scenes.
Revised: After filtering, we obtain 63181 training, 4774 validation, and 6386 testing scenes.

Original: More details about the data prepossessing and corresponding model details are presented in the appendix.
Revised: More details about the data prepossessing and corresponding model details are presented in the appendix.

Original: 5. C LOSED -L OOP E VALUATION
Revised: 5. C LOSED -L OOP E VALUATION

Original: To evaluate the closed-loop performance of our method, we use a log-replay simulator as prior work (Scheel et al., 2022; Huang et al., 2022).
Revised: To evaluate the closed-loop performance of our method, we use a log-replay simulator as prior work (Scheel et al., 2022; Huang et al., 2022).

Original: At each step in the log-replay simulator, the SDV vehicle updates its state according to the planned trajectory, while the other agents are assumed to follow their recorded trajectories in the dataset.
Revised: At each step in the log-replay simulator, the SDV vehicle updates its state according to the planned trajectory, while the other agents are assumed to follow their recorded trajectories in the dataset.

Original: For both datasets, we evaluate our method for 25s at 10Hz with the following metrics:
Revised: For both datasets, we evaluate our method for 25s at 10Hz with the following metrics:

Original: Collision Rate : If the SDV collides with the other agents at any time step in a scene, the scene is deemed as a collision scene.
Revised: Collision Rate : If the SDV collides with the other agents at any time step in a scene, the scene is deemed as a collision scene.

Original: The collision rate is the ratio of the collision scenes in all scenes.
Revised: The collision rate is the ratio of the collision scenes in all scenes.

Original: Off-road Rate :
Revised: Off-road Rate :

Original: In nuPlan, we use the official drivable area compliance metric, i.e . the distance of a corner of SDV’s bounding box from the drivable area is more than 0.3m.
Revised: In nuPlan, we use the official drivable area compliance metric, i.e . the distance of a corner of SDV’s bounding box from the drivable area is more than 0.3m.

Original: But in Lyft, without access to the drivable area, if the SDV deviates laterally from the human driver’s ground truth more than 2m like in a scene, the scene will be deemed off-road as Scheel et al.
Revised: But in Lyft, without access to the drivable area, if the SDV deviates laterally from the human driver’s ground truth more than 2m like in a scene, the scene will be deemed off-road as Scheel et al.

Original: Discomfort : we count the rate of the absolute value of the acceleration more than 3 m / s 2 over time steps to quantify the comfort and feasibility of the planned trajectory.
Revised: Discomfort : we count the rate of the absolute value of the acceleration more than 3 m / s 2 over time steps to quantify the comfort and feasibility of the planned trajectory.

Original: L2 : we use the average L2 position errors between the roll-out trajectory and the human driver’s ground truth to quantify the human driving similarity.
Revised: L2 : we use the average L2 position errors between the roll-out trajectory and the human driver’s ground truth to quantify the human driving similarity.

Original: 5. P ERFORMANCE E VALUATION
Revised: 5. P ERFORMANCE E VALUATION

Original: On the Lyft dataset, we compare our methods against three state-of-the-art methods to demonstrate the advantage of the proposed framework:
Revised: On the Lyft dataset, we compare our methods against three state-of-the-art methods to demonstrate the advantage of the proposed framework:

Original: Raster-perturb : an
Revised: Raster-perturb : an

Original: official BC planning
Revised: official BC planning

Original: baseline
Revised: baseline

Original: (Houston et al., 2020), which based on ResNet receives a Bird-Eye-View (BEV) representation of the scene surrounding the SDV and produces a trajectory of position and yaw displacements.
Revised: (Houston et al., 2020), which based on ResNet receives a Bird-Eye-View (BEV) representation of the scene surrounding the SDV and produces a trajectory of position and yaw displacements.

Original: To augment data, a perturbation is applied to the current SDV position, and then a new kinematically feasible trajectory to reach the original endpoint is generated as ChauffeurNet.
Revised: To augment data, a perturbation is applied to the current SDV position, and then a new kinematically feasible trajectory to reach the original endpoint is generated as ChauffeurNet.

Original: [Table]
Revised: [Table]

Original: BC-
Revised: BC-

Original: perturb : a BC model provided by Scheel et al.
Revised: perturb : a BC model provided by Scheel et al.

Original: with
Revised: with

Original: the
Revised: the

Original: same
Revised: same

Original: trajectory perturbation and output
Revised: trajectory perturbation and output

Original: as Raster-perturb
Revised: as Raster-perturb

Original: but
Revised: but

Original: its
Revised: its

Original: inputs are
Revised: inputs are

Original: presented in
Revised: presented in

Original: vector formulation,
Revised: vector formulation,

Original: which are
Revised: which are

Original: processed
Revised: processed

Original: by a VectorNet (Gao et al., 2020) and Transformer
Revised: by a VectorNet (Gao et al., 2020) and Transformer

Original: UrbanDriver :
Revised: UrbanDriver :

Original: an offline policy gradient method proposed by Scheel et al.
Revised: an offline policy gradient method proposed by Scheel et al.

Original: to imitate the ex
Revised: to imitate the ex

Original: - pert’s
Revised: - pert’s

Original: policy exploiting a differentiable data-driven simulator with the same data and
Revised: policy exploiting a differentiable data-driven simulator with the same data and

Original: model structure
Revised: model structure

Original: as BC-perturb
Revised: as BC-perturb

Original: On the nuPlan dataset, we compare our methods against two official baselines learned by BC due to a lack of other prior works:
Revised: On the nuPlan dataset, we compare our methods against two official baselines learned by BC due to a lack of other prior works:

Original: Raster : a raster-based model that uses the ResNet-50 backbone to encode SDV, agent, and map information as raster layers to plan the SDV’s trajectory.
Revised: Raster : a raster-based model that uses the ResNet-50 backbone to encode SDV, agent, and map information as raster layers to plan the SDV’s trajectory.

Original: LaneGCN-perturb : a vector-based model that uses a series of MLPs to encode SDV and agent signals and one LaneGCN (Liang et al., 2020) to encode vector-map elements and a fusion network to capture lane and agent intra/inter-interactions through attention layers.
Revised: LaneGCN-perturb : a vector-based model that uses a series of MLPs to encode SDV and agent signals and one LaneGCN (Liang et al., 2020) to encode vector-map elements and a fusion network to capture lane and agent intra/inter-interactions through attention layers.

Original: To augment data, the SDV trajectories are perturbed and other agents are randomly dropped out.
Revised: To augment data, the SDV trajectories are perturbed and other agents are randomly dropped out.

Original: In addition to these models provided by prior works, we also learn our model by a representative method in behavior cloning and offline reinforcement learning on both datasets:
Revised: In addition to these models provided by prior works, we also learn our model by a representative method in behavior cloning and offline reinforcement learning on both datasets:

Original: TD3+BC (Fujimoto & Gu, 2021): it adds behavior cloning term to the policy updating of Twin
Revised: TD3+BC (Fujimoto & Gu, 2021): it adds behavior cloning term to the policy updating of Twin

Original: Delayed Deep Deterministic Policy Gradient (TD3) (Fujimoto et al., 2018) for implicit policy constraint.
Revised: Delayed Deep Deterministic Policy Gradient (TD3) (Fujimoto et al., 2018) for implicit policy constraint.

Original: We construct the offline RL dataset by applying the trajectory perturbation augmentation and consider the collision and comfort for the reward design.
Revised: We construct the offline RL dataset by applying the trajectory perturbation augmentation and consider the collision and comfort for the reward design.

Original: Vector-Chauffeur : our model learned using the same data augmentation method as ChauffeurNet including trajectory perturbation and ego past motion dropout.
Revised: Vector-Chauffeur : our model learned using the same data augmentation method as ChauffeurNet including trajectory perturbation and ego past motion dropout.

Original: And we represent the data in the same coordinate system which uses the ego location as the origin and its heading perturbed by a uniform noise as the orientation.
Revised: And we represent the data in the same coordinate system which uses the ego location as the origin and its heading perturbed by a uniform noise as the orientation.

Original: The performance is shown in Table 1.
Revised: The performance is shown in Table 1.

Original: It demonstrates that our method can outperform previous work significantly on both datasets.
Revised: It demonstrates that our method can outperform previous work significantly on both datasets.

Original: The collision rate on the nuPlan dataset is higher than the Lyft dataset because nuPlan’s scenarios are more complex with more road agents.
Revised: The collision rate on the nuPlan dataset is higher than the Lyft dataset because nuPlan’s scenarios are more complex with more road agents.

Original: 5. A BLATION S TUDY
Revised: 5. A BLATION S TUDY

Original: The following ablation experiments on the Lyft dataset are used to expose the significance of different components of our model, whose results are shown in Table 2:
Revised: The following ablation experiments on the Lyft dataset are used to expose the significance of different components of our model, whose results are shown in Table 2:

Original: [Table]
Revised: [Table]

Original: Network input : We first study the importance of removing explicit ego information from the network input.
Revised: Network input : We first study the importance of removing explicit ego information from the network input.

Original: We consider two ways to bring the explicit ego input back.
Revised: We consider two ways to bring the explicit ego input back.

Original: The first one is to directly input ego past positions and then process it similarly to the goal position.
Revised: The first one is to directly input ego past positions and then process it similarly to the goal position.

Original: The other one is to additionally introduce a dropout of 50% at the ego input during training as Bansal et al.
Revised: The other one is to additionally introduce a dropout of 50% at the ego input during training as Bansal et al.

Original: We can observe that in both explicit ego information input ways, there is a steep drop in the collision and off-road rate and L2 distance due to the covariate shift issue.
Revised: We can observe that in both explicit ego information input ways, there is a steep drop in the collision and off-road rate and L2 distance due to the covariate shift issue.

Original: Coordinate system : To analyze the impact of the implicit ego information in the coordinate system, we first consider replacing our ego-perturbed goal-oriented coordinate system with the ego-centric coordinate system in ChauffeurNet using orientation uniformly within the heading.
Revised: Coordinate system : To analyze the impact of the implicit ego information in the coordinate system, we first consider replacing our ego-perturbed goal-oriented coordinate system with the ego-centric coordinate system in ChauffeurNet using orientation uniformly around the heading.

Original: We observe that the ChauffeurNet coordinate system leads to inferior closed-loop performance.
Revised: We observe that the ChauffeurNet coordinate system leads to inferior closed-loop performance.

Original: In our coordinate system, we add a Gaussian noise with zero mean to the ego current position to obtain the origin.
Revised: In our coordinate system, we add a Gaussian noise with zero mean to the ego current position to obtain the origin.

Original: Thus, we can increase the Gaussian noise’s standard deviation to diminish the implicit ego information.
Revised: Thus, we can increase the Gaussian noise’s standard deviation to diminish the implicit ego information.

Original: We can observe that with the increasing of the standard deviation, the discomfort increases but the off-road rate and closed-loop L2 decreases, which implies that the implicit ego information may improve the instantaneous prediction accuracy but deteriorate the closed-loop performance.
Revised: We can observe that with the increasing of the standard deviation, the discomfort increases but the off-road rate and closed-loop L2 decreases, which implies that the implicit ego information may improve the instantaneous prediction accuracy but deteriorate the closed-loop performance.

Original: Architecture : To demonstrate the importance of the temporal information and the effectiveness of the causal Transformer in capturing the temporal interactions, we replace it with a MLP.
Revised: Architecture : To demonstrate the importance of the temporal information and the effectiveness of the causal Transformer in capturing the temporal interactions, we replace it with a MLP.

Original: We find that the causal Transformer can achieve better performance.
Revised: We find that the causal Transformer can achieve better performance.

Original: Besides, to show the importance of the
Revised: Besides, to show the importance of the

Original: LQR module in enhancing comfort, we ablate it.
Revised: LQR module in enhancing comfort, we ablate it.

Original: Removing the LQR mechanism leads to a huge drop in comfort.
Revised: Removing the LQR mechanism leads to a huge drop in comfort.

Original: Loss : To demonstrate the effectiveness of the additional term, we remove the auxiliary and regularization loss term separately.
Revised: Loss : To demonstrate the effectiveness of the additional term, we remove the auxiliary and regularization loss term separately.

Original: The result in Table 2 shows that they are both beneficial to improving performance.
Revised: The result in Table 2 shows that they are both beneficial to improving performance.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We have proposed a new offline imitation learning method to mitigate the distribution shift of behavior cloning, where we learn a neural network to predict the SDV’s future positions and orientations without SDV information by making full use of its constraints from and interaction with the context.
Revised: We have proposed a new offline imitation learning method to mitigate the distribution shift of behavior cloning, where we learn a neural network to predict the SDV’s future positions and orientations without SDV information by making full use of its constraints from and interaction with the context.

Original: Firstly, we analyze theoretically the stability of our policy formulation.
Revised: Firstly, we analyze theoretically the stability of our policy formulation.

Original: In addition to removing the explicit SDV information input to the network, we present a new SDV-perturbed goal-oriented coordinate system for representing represent the observation input to remove the implicit SDV information and ensure local observability.
Revised: In addition to removing the explicit SDV information input to the network, we present a new SDV-perturbed goal-oriented coordinate system for representing represent the observation input to remove the implicit SDV information and ensure local observability.

Original: We design a Transformer-based network to make full use of the history information to handle the more challenging learning task.
Revised: We design a Transformer-based network to make full use of the history information to handle the more challenging learning task.

Original: Finally, we demonstrate the effectiveness of our approach in two real-world large-scale datasets with state-of-the-art performance
Revised: Finally, we demonstrate the effectiveness of our approach in two real-world large-scale datasets with state-of-the-art performance.

Original: A M AP PREPOSSESSING
Revised: A M AP PREPOSSESSING

Original: In Lyft, there are only two types of map elements: lane and crosswalk.
Revised: In Lyft, there are only two types of map elements: lane and crosswalk.

Original: In nuPlan, the map elements consist of a lane, lane connector, intersection, stop line, crosswalk, walkway, and car park.
Revised: In nuPlan, the map elements consist of a lane, lane connector, intersection, stop line, crosswalk, walkway, and car park.

Original: We categorize them into polyline elements including lanes and lane connectors, and polygon elements including stop lines, crosswalks, intersections, walkways, and car parks.
Revised: We categorize them into polyline elements including lanes and lane connectors, and polygon elements including stop lines, crosswalks, intersections, walkways, and car parks.

Original: For each polyline element, it is approximated by a sequence of vectors with an interval of 3 meters.
Revised: For each polyline element, it is approximated by a sequence of vectors with an interval of 3 meters.

Original: For both datasets, we prepossess the polyline map into a graph to take advantage of the topological information.
Revised: For both datasets, we prepossess the polyline map into a graph to take advantage of the topological information.

Original: We connect each vector with its nearest left, right, and next vector if exists.
Revised: We connect each vector with its nearest left, right, and next vector if exists.

Original: The nearest left or right vector means the nearest vector of its reachable left or right polyline.
Revised: The nearest left or right vector means the nearest vector of its reachable left or right polyline.

Original: And the distance between vectors is represented by the Euclidean distance between their starting points.
Revised: And the distance between vectors is represented by the Euclidean distance between their starting points.

Original: After building the vector graph, we can compute the travel distance between any two vectors using
Revised: After building the vector graph, we can compute the travel distance between any two vectors using

Original: Dijkstra’s algorithm.
Revised: Dijkstra’s algorithm.

Original: For both datasets, the polyline vector has shared features including coordinates of starting and ending point, distance to left and right vectors, distance to the mission goal, traffic light states (red or green), and the sequence order.
Revised: For both datasets, the polyline vector has shared features including coordinates of starting and ending point, distance to left and right vectors, distance to the mission goal, traffic light states (red or green), and the sequence order.

Original: However, lane vectors in Lyft have a lane width feature because the vectors approximate computed middle lines of lanes, while the lane or lane connector vectors in nuPlan have a lane left and width feature because the vectors approximate an annotated baseline.
Revised: However, lane vectors in Lyft have a lane width feature because the vectors approximate computed middle lines of lanes, while the lane or lane connector vectors in nuPlan have a lane left and width feature because the vectors approximate an annotated baseline.

Original: In addition, the polyline vectors in nuPlan have an additional speed limit and type features.
Revised: In addition, the polyline vectors in nuPlan have an additional speed limit and type features.

Original: For each polygon element, we approximate it also with a sequence of vectors.
Revised: For each polygon element, we approximate it also with a sequence of vectors.

Original: In the Lyft dataset, the crosswalk vectors are directly constructed by connecting the original sequential annotation coordinates.
Revised: In the Lyft dataset, the crosswalk vectors are directly constructed by connecting the original sequential annotation coordinates.

Original: But we approximate each polygon with fixed 20 vectors because the annotation point number is too large for elements like intersection.
Revised: But we approximate each polygon with fixed 20 vectors because the annotation point number is too large for elements like intersection.

Original: For both datasets, each polygon vector has features including coordinates of starting and ending points, and its order sequence, while vectors in nuPlan have additional type features.
Revised: For both datasets, each polygon vector has features including coordinates of starting and ending points, and its order sequence, while vectors in nuPlan have additional type features.

Original: The inputs to our neural network are composed of two types of map elements: polyline and polygon, and two types of agent elements: other agents and ego goal.
Revised: The inputs to our neural network are composed of two types of map elements: polyline and polygon, and two types of agent elements: other agents and ego goal.

Original: The missing inputs are padded with zeros and masked out when calculating the attention.
Revised: The missing inputs are padded with zeros and masked out when calculating the attention.

Original: The origin is the perturbed SDV position.
Revised: The origin is the perturbed SDV position.

Original: Polyline : 30 topologically nearest polylines with vectors whose starting points are within 35m from the origin.
Revised: Polyline : 30 topologically nearest polylines with vectors whose starting points are within 35m from the origin.

Original: The topological distance between the origin and a polyline is the minimum of topological distances between the origin and its vectors.
Revised: The topological distance between the origin and a polyline is the minimum of topological distances between the origin and its vectors.

Original: Polygons : 20 polygons whose boundaries are within 35m from the origin.
Revised: Polygons : 20 polygons whose boundaries are within 35m from the origin.

Original: If there are more than 20 polygons, they are selected according to the importance of their types: stop line, crosswalk, intersection, walkway, and car park.
Revised: If there are more than 20 polygons, they are selected according to the importance of their types: stop line, crosswalk, intersection, walkway, and car park.

Original: Agents : the nearest 30 agents whose oriented boxes’ centroids are within 50m from the origin.
Revised: Agents : the nearest 30 agents whose oriented boxes’ centroids are within 50m from the origin.

Original: The agent features in Lyft include its centroid coordinates, yaws, shapes, types, and relative times in the past 2 and current steps.
Revised: The agent features in Lyft include its centroid coordinates, yaws, shapes, types, and relative times in the past 2 and current steps.

Original: The nuPlan agents additionally have velocity features as they are provided.
Revised: The nuPlan agents additionally have velocity features as they are provided.

Original: Goal : the x, y coordinates of the SDV’s mission goal of a scene.
Revised: Goal : the x, y coordinates of the SDV’s mission goal of a scene.

Original: In Lyft, the mission goal is not provided, so we regard the ending point of the lane where the SDV locates at the last time step of the scene as the mission goal.
Revised: In Lyft, the mission goal is not provided, so we regard the ending point of the lane where the SDV locates at the last time step of the scene as the mission goal.

Original: In nuPlan, we directly take the provided mission goal at the last time step of the scene as the scene mission goal.
Revised: In nuPlan, we directly take the provided mission goal at the last time step of the scene as the scene mission goal.

Original: B M ODEL
Revised: B M ODEL

Original: For both datasets.
Revised: For both datasets.

Original: the same model architecture is used, whose hyper-parameters are listed in Table 3.
Revised: the same model architecture is used, whose hyper-parameters are listed in Table 3.

Original: C T RAINING
Revised: C T RAINING

Original: Our model is trained using the Adam optimizer with a learning rate of 0.0005, 10000 steps linear warm-up, β = (0 . 9 , 0 . 999) , and batch size 128.
Revised: Our model is trained using the Adam optimizer with a learning rate of 0.0005, 10000 steps linear warm-up, β = (0 . 9 , 0 . 999) , and batch size 128.

Original: We stop training after 30 epochs and select the model with smallest validation collision rate for evaluation.
Revised: We stop training after 30 epochs and select the model with smallest validation collision rate for evaluation.

Original: We train all models except the pretrained model in Lyft dataset independently for 3 times and then report the mean and std of their performances.
Revised: We train all models except the pretrained model in Lyft dataset independently for 3 times and then report the mean and std of their performances.

Original: [Table]
Revised: [Table]

Original: D E VALUATION
Revised: D E VALUATION

Original: For Lyft baselines, we directly evaluate the pretrained model provided by the Lyft dataset and UrbanDriver.
Revised: For Lyft baselines, we directly evaluate the pretrained model provided by the Lyft dataset and UrbanDriver.

Original: The Raster-perturb are model trained on train.zarr forepochs at https://github.com/woven-planet/l5kit/blob/master/examples/ planning/train.ipynb .
Revised: The Raster-perturb are model trained on train.zarr forepochs at https://github.com/woven-planet/l5kit/blob/master/examples/ planning/train.ipynb .

Original: The BC-perturb and UrbanDriver are from Open Loop and Urban
Revised: The BC-perturb and UrbanDriver are from Open Loop and Urban

Original: Driver in https://github.com/woven-planet/l5kit/blob/master/examples/ urban_driver/train.ipynb .
Revised: Driver in https://github.com/woven-planet/l5kit/blob/master/examples/ urban_driver/train.ipynb .

Original: We evaluate all Lyft models from the second time step to compute current velocity using position information as input to the LQR with zero assumed initial acceleration.
Revised: We evaluate all Lyft models from the second time step to compute current velocity using position information as input to the LQR with zero assumed initial acceleration.

Original: For nuPlan baselines, we train the official models by ourselves using provided hyper-parameters and evaluate from the first time step because the velocity and acceleration information is provided.
Revised: For nuPlan baselines, we train the official models by ourselves using provided hyper-parameters and evaluate from the first time step because the velocity and acceleration information is provided.

Original: E R UNTIME
Revised: E R UNTIME

Original: We conduct runtime experiments using a single Nvidia GeForce GTX 1080 GPU and an Intel i78700@3.2GHz CPU on Lyft dataset.
Revised: We conduct runtime experiments using a single Nvidia GeForce GTX 1080 GPU and an Intel i78700@3.2GHz CPU on Lyft dataset.

Original: We measure the runtime of each method its mean and std over all time-steps in an evaluation.
Revised: We measure the runtime of each method its mean and std over all time-steps in an evaluation.

Original: The runtime results shown in Table 4 consider all components in each model including data-prepossessing, model inference and control.
Revised: The runtime results shown in Table 4 consider all components in each model including data-prepossessing, model inference and control.

Original: We observe that our architecture can achieve higher data processing efficiency and medium model inference efficiency compared with other methods.
Revised: We observe that our architecture can achieve higher data processing efficiency and medium model inference efficiency compared with other methods.

Original: Our method takes longer total execution time due to the extra LQR control module which does not exist in prior works because they only focus on optimizing positional accuracy but not comfort.
Revised: Our method takes longer total execution time due to the extra LQR control module which does not exist in prior works because they only focus on optimizing positional accuracy but not comfort.

Original: However, our approach can still be executed in real-time on this hardware.
Revised: However, our approach can still be executed in real-time on this hardware.

Original: [Table]
Revised: [Table]

Original: F T OY E XPERIMENT
Revised: F T OY E XPERIMENT

Original: We design a toy experiment to vividly show our method’s ability to reduce compounding error.
Revised: We design a toy experiment to vividly show our method’s ability to reduce compounding error.

Original: In the experiment, we use synthetic data from a very ideal and simplified scenario where a SDV moves under ring road network with fixed 1 m/s at 1Hz, as shown in Figure 2.
Revised: In the experiment, we use synthetic data from a very ideal and simplified scenario where a SDV moves under ring road network with fixed 1 m/s at 1Hz, as shown in Figure 2.

Original: During training, the radius of the ring is a variable with a range from 10m to 100m and the circular lane are represented as a series of fixed lane points with same interval of nearly 1m.
Revised: During training, the radius of the ring is a variable with a range from 10m to 100m and the circular lane are represented as a series of fixed lane points with same interval of nearly 1m.

Original: We compare our CCIL methods with several baseline methods introduced above including BC, BCperturb, UrbanDriver.
Revised: We compare our CCIL methods with several baseline methods introduced above including BC, BCperturb, UrbanDriver.

Original: The inputs of these baseline methods are composed of two parts: SDV state (its position and orientation in the past 10 time step) and context (nearest 10 lane points) in the ego coordinate system, while our methods only takes the context in the past 10 time step as inputs in the ego-perturbed center-oriented coordinate system.
Revised: The inputs of these baseline methods are composed of two parts: SDV state (its position and orientation in the past 10 time step) and context (nearest 10 lane points) in the ego coordinate system, while our methods only takes the context in the past 10 time step as inputs in the ego-perturbed center-oriented coordinate system.

Original: The ego-perturbed origin-oriented coordinate system means using the SDV position added a zero-mean one-std Gaussian perturbation as the origin and orientation to the center of the circular road as the x-axis direction.
Revised: The ego-perturbed origin-oriented coordinate system means using the SDV position added a zero-mean one-std Gaussian perturbation as the origin and orientation to the center of the circular road as the x-axis direction.

Original: The trajectory perturbation is applied to augment the data in the BC-perturb method.
Revised: The trajectory perturbation is applied to augment the data in the BC-perturb method.

Original: For outputs, the BC and CCIL method generate the relative position and yaw at the next time step and the BC-perturb method produces the next 10 time steps.
Revised: For outputs, the BC and CCIL method generate the relative position and yaw at the next time step and the BC-perturb method produces the next 10 time steps.

Original: In the UrbanDriver method, we unroll the policy for 32 time steps.
Revised: In the UrbanDriver method, we unroll the policy for 32 time steps.

Original: In each methods, we employ a two-layer MLP with hidden size of 128 as a policy network.
Revised: In each methods, we employ a two-layer MLP with hidden size of 128 as a policy network.

Original: We train the neural networks using Adam optimizer of with a learning rate of 0.0001 with random initial weights for 100 times.
Revised: We train the neural networks using Adam optimizer of with a learning rate of 0.0001 with random initial weights for 100 times.

Original: We stop training after 10000 steps and then unroll the policy from one random starting point on the circle of radius 50m for 100 time steps.
Revised: We stop training after 10000 steps and then unroll the policy from one random starting point on the circle of radius 50m for 100 time steps.

Original: The 100 closed-loop trajectories for each method are depicted in Figure 3.
Revised: The 100 closed-loop trajectories for each method are depicted in Figure 3.

Original: We can observe that some trajectories in the baseline methods deviate from the road due to the covariate shift issue but the trajectories in our method keep following the route.
Revised: We can observe that some trajectories in the baseline methods deviate from the road due to the covariate shift issue but the trajectories in our method keep following the route.