DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Models
Authors: Licheng Wen
Daocheng Fu
Xin Li
Xinyu Cai
Tao MA
Pinlong Cai
Min Dou
Botian Shi
Liang He
Yu Qiao
Venue: ICLR 2024 poster (2024)
Keywords: Autonmous Driving
Large Language Model
Embodied AI
Knowledge-driven
Primary area: applications to robotics, autonomy, planning
TL;DR: We propose a framework that can instill human knowledge into autonomous driving systems with the help of large language models.

Abstract: Recent advancements in autonomous driving have relied on data-driven approaches, which are widely adopted but face challenges including dataset bias, overfitting, and uninterpretability. 
Drawing inspiration from the knowledge-driven nature of human driving, we explore the question of how to instill similar capabilities into autonomous driving systems and summarize a paradigm that integrates an interactive environment, a driver agent, as well as a memory component to address this question. 
Leveraging large language models (LLMs) with emergent abilities, we propose the DiLu framework, which combines a Reasoning and a Reflection module to enable the system to perform decision-making based on common-sense knowledge and evolve continuously. 
Extensive experiments prove DiLu's capability to accumulate experience and demonstrate a significant advantage in generalization ability over reinforcement learning-based methods.
Moreover, DiLu is able to directly acquire experiences from real-world datasets which highlights its potential to be deployed on practical autonomous driving systems.
To the best of our knowledge, we are the first to leverage knowledge-driven capability in decision-making for autonomous vehicles. Through the proposed DiLu framework, LLM is strengthened to apply knowledge and to reason causally in the autonomous driving domain.

Project page: https://pjlab-adg.github.io/DiLu/

Published as a conference paper at ICLR 2024
DILU : A KNOWLEDGE-DRIVEN APPROACH TO AUTONOMOUS DRIVING WITH LARGE LANGUAGE MODELS
Licheng Wen 1** Daocheng Ful, Xin Li? Xinyu Cai', Tao Ma
Pinlong Cai', Min Dou', Botian Shi',,, Liang Hee, Yu Qiao
1 Shanghai Artificial Intelligence Laboratory
2 East China Normal University, The Chinese University of Hong Kong
Equal contributions; Corresponding author shibotian@pjlab.oo org. cn
ABSTRACT
Recent advancements in aulonomous driving have relied on data-driven ap- proaches, which are widely adopted but face challenges including dataset. bias, overfitting, and uninterpretability. Drawing inspiration from the knowledge-driven nature of human driving, explore the question of how to instill similar capabilities into autonomous driving systems and sum- marize paradigm that integrates interactive environment, driver agent, as well as a memory component to address this qucstion. Loveraging large language models (LLMs) with emergent abilities, we propose the DiLu framework, which combines a Reasoning and a Reflection module to enable the system perform decision-makinn based on common-sens, knowledge and evolve continuously. Extensive experiments prove DiLu's capability to accumulate experience and demonstrate a significant advantage general- ization ability over reinforcement teasning-based methods. Moreover, DiL.u is able directly acquire experiences from real-world datasets which high- lights its potential to be deployed on practical autonomous driving systems. To the best of our knowledge, we are the first to leverage knowledge-driven capability in rocision-making for autonomous vehicles. Through the pro- posed DiLu framework, LLM is strengthened to apply knowledge and to reason causally in the autonomous driving domain.
Project page: https://pjlabb: wit.ggttt..... 1o/DiLu/
INTRODUCTION
Autonomous driving has witnessed remarkable advancements in recent years, propelled by the data-driven manner (Bogdoll al., 2021 Chen al. 2023a b). These data-driven algorithms strive to capture and model the underlying distributions of the accumulated data (Bolte al.) 2019 Zhou Beyerer 2023), but they always encounter challenges such as datasel. bias, overlitting, and uninterpretability (Codevilla, et al. 2019 Jin et al. 2023 Exploring methods mitigate these challenges could lead to deeper understanding of driving scenarios and more rational decision decision-making potentially enhancing the performance of autonomous driving systems.
Drawing inspiration from the profound question posed by LeCun 2022: "Why con adolescent learn to drive a car in about 20 hours of practice and know how to act in many situations he/she has never encountered before?". explore the core principles that underlie human driving skills and raise a pivotal distinction: human driving is fundamentally driiie,,e- opposed to data-driven. For example, when faced with situation

---

Published as a conference paper at ICLR 2024
Observe
Update
Recall
Act
Reflect
Reason
query
Environment
Driver Agent
Memory
Continuous Evolution
Figure 1: The knowledge-driven paradigm for autonomous driving system, including an interactive environment, a driver agent with recall, reasoning and reflection abilities, along with an independent memory module. Driver agent continuously evolves to observe the environment, query, update experiences from the memory module, and make decisions to control the ego vehicle.
where the truck ahead is danger of losing its cargo, humans can rely on common sense and explainable reasoning to ensure safe distance is maintained between vehicles. Conversely, data-driven methods rely on a large quantity of similar data to fit this scenario which lacks environment. comprehension and limits generalization ability (Heidecker
al. 2021 Chen
et. all 2022 Wen et al. 2023). Furthermore, this task requires significant human la aboo and financial resources to collect and annotate driving data handle varied real-world scenarios. This observation catalyzes fundamental question: How can we instill such knowledge- driven capabilitics of human drivers into an autonomous driving system?
Recent advancements in large language models (LLMs) with emergent abilitics offer an ideal embodiment of human knowledge, providing valuable insights toward addressing this ques- Lion. LLMs possess exceptional human-level abilities and show strong abilities in robotics manipulation (Driess et al. 2023a Huang et al. 2023a b). multi-modal understanding (Gao
et. al_ 2023 and lifelong skill learning Wang et al. 2023 Zhu et al.) 2023b). However, just as humans may need 20 hours of practice to learn to drive, LLMs cannot successfully perform the driving task without any experience or guidance. Through these analyses, summarize the knowledge-driven paradigm for autonomous driving systems, as illustrated in Figure including three components: (1) an environment with which an agent interact; (2) driver agent with recall, reasoning, and reflection abilities; (3) memory component to persist experiences. In continuous evolution, the driver agent observes the environment, queries, and updates experiences from memory and performs decision-making.
Following the paradigm above, we design a novel framework named DiLu illustrated in F Specifically, the driver agent utilizes the Reasoning Module to query experiences from the Memory Module and leverage the common-sense knowledge of the LLM to generate decisions based on current scenarios. It then employs the Reflection Module to identify safe and unsafe decisions produced by the Reasoning Module, subsequently refining them into correct decisions using the knowledge embedded in the LLM. These safe or revised decisions are then updated into the Memory Module.
Extensive experiments demonstrate that the proposed framework DiLu can leverage L.M to make proper decisions for the autonomous driving system. We design a closed-loop driving environment and prove that DiLu perform better and better with the experience accu- mulated in the memory module. Remarkably, with only 40 memory items, DiLu achieves comparable performance to the reinforcement learning (RL) based methods that have ex- tensively trained over 600,000 episodes, but with a much stronger generalization ability to diverse scenarios. Moreover, DiLu's ability to directly acquire experiences from real-world datasets highlights its potential to be deployed on practical autonomous driving systems.
The contributions of our work are summarized follows:
To the best of our knowledge, we are the first to leverage knowledge-driven capability in decision-making for autonomous vehicles from the perspective of how humans drive. We summarize the knowledge-driven paradigm that involves an interactive environment, driver agent, well as memory component.
2

---

Published as a conference paper at ICLR 2024
We propose novel framework called DiLu which implements the above paradigm to address the closed-loop driving tasks. The framework incorporates a Memory Module to record the experiences, and leverages LLM to facilitate reasoning and reflection processes.
Extensive experimental results highlight DiLu's capability to contimuously accu- mulate experience by interacting with the environment. Moreover, DiLa exhibits stronger generalization ability than RI-based methods and demonstrates the po- tential to be applied in practical autonomous driving systems.
RELATED WORKS
2.1 ADVANCEMENTS IN LARGE LANGUACE MODELS
Large language models (LLMs) are the category of Transformer-basee language models that are characterized by having an enormous number of parameters, typically numbering the hundreds of billions or even more. These models are trained on massive Lext dalasets, en- abling them to understand natural language and performn a wide range of complex lasks, primarily through text generation and comprehension (Zhao el al.] 2023). Some well-known
examples LLMs include GPT-3 (Brown et al. 2020) PaLM Chowdhery et al. 2022) and
LLaMA (Touvron et al. 2023) GPT-4 (Achiam et al. 2023 The emergent abilities of LLMs are one of the most significant characteristies that distinguish them from smaller language models. Specifically, in-context learning (ICL) (Brown et al.] 2020 instruction follow- ing (Ouyang et al. 2022 Wei et al. (2221) and reasoning with chain-ol-thought (CoT) (Wei el. a 2022) are three typical emergent abilities for LLMs. OpenAl's pursuit of LLMs has led to The achievernntt of two remarkable milestones: ChatGPT (OpenAl) 2023) and GPT- (Achiam et al. 2023). These two milestones signify significant advancements in LLMs capabilities, particularly in natural language understanding and generation. Notably, recent developments in large LLMs have showcased human-like intelligence and hold the potential to propel us closer to the realm of Artificial General Intelligence (AGT) (Zhao et al. 2023 Zhu et al. 2023a
2.2 ADVANCED TASKS BASED ON LARGE LANGUAGE MODEL
Owing to the superior capability of common-sense knowledge embedded in LLMs, they are widely adopted for diverse tasks (Sammani et al. 2022 Bubeck et al.) 2023 Schick et al.
2023). Furthermore, there has emerged a flourishing research area that leverages LLMs to
create autonomous agents endowed with human -ikan-like capabilities (Chowdhery et al. 2022 Yao et al. 2022: Park et al.) 2023 Fu et al.] 2024 Zhu et al.) 2023b Li ct al., 2023). ).
In particular, LLMs are shown to possess a wealth of actionable knowledge that can be extracted for robot manipulation in the form of reasoning and planning- For instance, DDiess et al. 2023bb proposed embodied language models that directly integrate real-world continuous sensor data into language models, establishing a direct link between words and perceptual information. Voyager (Wang et al.) 2023) introduces lifelong learning through the incorporation of prompting mechanisms, a skill library, and self-verification. These three modules are all grounded by LLM and empower the agent to learn more sophisticated et al., 2023b) leveraged LLMs to generate robot
behaviors. Similarly, Voxposer(Huann trajectories for a wide range of manipulation tasks, guided by open-ended instructions and objects. Simultaneously, motivated by insights from AGI and the principles of embodied AI (Pfeifer & lida [2004 Duan et al. (022). the field of autonomous driving is also undergoing a profound transformation. An open-loop driving commentator LINGO-1 is proposed by Wayve (023)) which combines vision, language and action enhance how to interpret and train the driving models. LLMs have demonstrated remarkable human-level capabilities across various domains, but we observe that they lack the inherent ability to engage with and comprehend the complex driving environment as humans. In contrast, autonomous vehicles depend on systems that can actively interact with and understand the driving environment. To bridge this gap, propose novel knowledge-driven autonomous driving paradigm and DiLu framework that enables LLMs to comprehend driving environments and drive by incorporating human knowledge.
3

---

Published as a conference paper at ICLR 2024
Reasoning Module
Memory Module
Few-chet Experiences
Obsave
Scenario Descriptor
Recall
S
Decison Decoder
Prompts Generator
Environment
Experience Sequences
Large Language Models
Summarization
Correction
Opdste
Reflection Module
Revised experiences
Different types
experiences
The framework of DiLu. It consists of four modules: Reflection, and Memory. In DiLu, the Reasoning module can observe the environment, gen- crate prompts by combining scenario descriptions and experiences the Memory module, and decode responses from the LLM to finish decision-making. Concurrently, the Refiec- tion module evaluates these decisions, identifies the unsafe decision to the experiences, and finally updates the revised experiences into the Memory module.
Figure 2:
Environment, Reasoning,
METHODOLOGY
3.1
OVERVIEW
Based on the knovledge-drieen paradigm for aulonomous driving systems introduced pre- viously, we propose a practical framework called DiLu, as illustrated in Figure 2 DiLu consists of four core modules: Environment, Reasoning, Reflection, and Memory. In partic- ular, the Reasoning module begins by observing the environment and obtaining descriptions of the current scenario. Concurrently, prompt generator is employed to combine this nario description with the fow-shot experiences of similar situations, which retricved from the Memory module. These prompts then fed into an out-of-the-box Large Language Model (LLM), and the decision decoder make an action by decoding LLM's response.
This process iterated within the Reasoning module, resulting in time-series decision quences. Subsequently, we employ the Reflection module to assess past decision sequences, categorizing them either safe or unsafe. The unsafe decisions are revised and these refined decisions are finally updated back into the Memory module. Detailed implementations of the Memory, Reasoning, and Reflection modules will be claborated the following soctions.
3.2 MEMORY MODULE
Without few-shot experiences, the out-of-the-box LLMs fail to perform precise reasoning when tackling the complex closed-loop driving tasks. Therefore, we employ a Memory mod- ule to store the experiences from past driving scenarios, which include the decision prompts, reasoning processes, and other valuable information. The memory stored in the memory module consists of two parts: scene descriptions and corresponding reasoning processes. The scene description provides a detailed account of the situation, serving the key to the memory module for retrieving similar memories. The reasoning process, on the other hand, records the appropriate method for handling the situation. This is the value of the memory, guiding the agent towards the correct driving logic. The memory module is constructed in three stages: initialization, memory recall, and memory storage.
Initialization:
The Initialization of memory module akin to a human attending driving school before hitting the road. We select few scenarios and manually outline the correct reasoning and decision- decision-making processes for these situations to form the initial memory. These memories instruct the agent on the correct decision-making process for driving.
4

---

Published as a conference paper at ICLR 2024
Observe
Vectorize
Recall
Scenario Descriptor
Memory Module
Prompt Generator
System Prompts
Textuol Description
Few-shot Experiences
Large Language Model with
Decision Decoder
Chain- Thoughts
Action
Current Frome
Next Frome
Figure 3: Reasoning module. We leverage the LLM's common-senss knowledge and query the experiences from Momory modulc to make decisions based on the scenario observation.
Memory recall: At each decision frame, the agent reccives a textual description of the driving scenario. Before making a decision, the current driving scenario is embedded into vector, which serves as the memory key. This key is then clustered and searched to find the closest scenarios (Johnson et al. 2019) in the memory module and their corresponding reasoning processes, or memories. These recalled memories are provided to the agent in few-shot format to assist making accurate reasoning and decisions for the current scenario.
Memory storage: As the agent makes correct reasoning and decisions, reflects on the correct reasoning process, gains driving experience. We embed the scene description into key, pair it with the reasoning process to form and store memory in the memory module.
3.3 REASONING MODULE
In the Reasoning module, we utilize the experiences derived from the Memory module and the common-sense knowledge of the LLM to perform decision-making for the current traffic scenario. Specifically, the reasoning procedure is illustrated Figure including the following procedures: (1) encode the scenario by descriptor; (2) recall several experience from the Memory module; (3) generate the prompt; (4) feed the prompt into the LLM; (5) decode the action from the LLM's response. The detailed prompt design be found in Appendix A.2
Encode the scenario by descriptor: To facilitate DiLu's understanding of the cur- rent traffic conditions, the scenario descriptor transcribes the present scenario data into descriptive text. The scenario descriptor follows standard sentence structure and utilizes natural language lo offer a comprehensive depiction of the ongoing driving scenario. This description contains static road details well dynamic information regarding the ego vehicle and surrounding vehicles within the scenario. These generated descriptions are then used as input for the prompt generator and keys acquire relevant few-shot experiences from the Memory module.
During the reasoning process, the description of the current driving scenario is also embedded into a vector, as illustrated in Figure 3 This vector is then used to initiate a similarity query within the Memory module, searching for the top k similar siluations. The resulting paired scene descriptions and reasoning procedures assemble as few-shot experiences, which are then integrated into the prompt generator.
Recall several experience from the Memory module:
Generate the prompt: As depicted the blue dashed box Figure the prompts for each frame consist of three key components: system prompts, textual descriptions from scenario descriptor, and the few-shot experience. Within the system prompts, we offer a concise overview of the closed-loop driving task, which includes an introduction the con- tent and format of task inputs and outputs, along with constraints governing the reasoning process. In each decision frame, construct tailored prompts based the current driving

---

Published as a conference paper at ICLR 2024
Decision Sequences
Memory Module
Summarization
#0
Key Frome Sampling
#t0
#$2
Update
#1
Safe
#22
Mistake Correction using
Experience Revise
#3
Updote
System Prompts
Mistoke frome
#4
Unsofe
#4
Correction
Initial experience
success experience
unsafe experience
revised experience
Figure 4: Reflection module. The Reflection module takes recorded decisions from closed- loop driving tasks as input, it utilizes a summarization and correction module to identify safe and unsafe decisions, subsequently revising them into correct decisions through the human knowledge embedded in LLM. Finally, these safe or revised decisions are updated into Memory module.
scenario. These prompts subsequently employed by LLM to engage in reasoning and determine the appropriate action for the current frame.
Feed the prompt into the LLM: Since the closcd-loop driving task requires com- plex reasoning process to make proper decisions, we employ the Chain-of-Thought (CoT) prompting techniques introduced by Wei al. (2022). These techniques require LLM to generate a sequence of sentences that describe the step-by-step reasoning logic, ultimately leading to the final decision. This approach is adopted due to the inherent complexity and variability of driving scenarios, which may lead to hallucinations if the language model di- rectly produces decision results. Also, the generated reasoning process facilitates further refinement and modification of incorrect decisions by the Reflection module in Section 3.4
Decode the action from the LLM's responsc: After feoding the prompt into the LLM, we decode the final decision in action decoder. The action decoder translates LLM's decision outcomes into actions for the ego vehicle and provides leedback to the environment. By repeating the procedures above, establish closed-loop decision decision-making system.
3.4 REFLECTION MODULE
In the Reasoning module, we use the LLM to undertake closed-loop driving tasks with the support of the proposed Memory module. As a next step we hope to accumulate valuable experiences and enrich the Memory module upon the conclusion of a driving session. To achieve this goal, we propose the Reflection module in DiLu, which continuously learns from past driving experiences. DiLu can progressively improve its performance through the Reflection module, similar to the progression of novice becoming an experienced driver.
The Reflection module is illustrated in Figure 4 During the closed-loop driving task, we record the prompts used as input based on the driving scenario and the corresponding decisions generated by the LLM for each decision frame. Once driving session concludes, we obtain decision sequence, e.g., 5 decision frames from to in Figure When the session ends without any collisions DiLu proceeds to sample several key decision frames from the sequence. These frames then directly become part of the historical driving experience and enrich the Memory module.
hazardous incidents, indicating a successful session,
On the contrary, if the current session is terminated due to hazardous situations such as collisions with other vehicles, this indicates that the driver agent has made inaccurate de- cisions. It is crucial for the system to rectify the unsafe decision made by the Reasoning module. Thanks to the interpretable chain-of-thouughn responses, causes of dangerous situations. Certainly, we can ask a human expert to complete such an error correction process. However, our goal is to make the autonomous driving system learn
can easily find the

---

Published as a conference paper at ICLR 2024
from mistakes on its own. We discover that LLM can effectively act mistake rectifier. Our approach is to use the driving scenarios which incorrect decisions occurred, together with the original reasoning output, prompts for LLM. We instruct LLM to pinpoint the reasons behind the incorrect decision and provide the correct one. We also ask LLM to pro- pose strategies in order to avoid similar errors in the future. Finally, the correct reasoning process and the revised decision learned from the mistakes are retained in Memory module.
4 EXPERIMENTS
In our experimental setup, we utilize the well-established Highway-env as our simulation environment, which is widely used form in the fields of autonomous driving and tactical Docision-making [Leurent 2018). This environment provides several driving models and offers realistic multi-vehicle interaction environment. In addition, the density of vehicles and the number of lanes in the environment can be freely adjusted. The detailed sctup of DiLu framework and Highway-em are described in Appendix A.1 The demonstration video of DiLu can be found in the project page: htttp::/pplabbe adg. terg.iithub... io/DiLu/
4.1 THE VALIDATION OF THE DILU FRAMEWORK
In this section, we primarily focus validating the effectiveness of the DiLu framework, in particular the reasoning and reflection process with or without the Memory module. The DiLu framework without the Memory module is referred to the 0-shot baseline, and conduct comparative experiments with 1-shot, 3-shots, and 5-shots experiences to demon- strate the necessity of experience accumulation. The initial Memory module contains human-crafted experiences. These experiences are then used in different few-shot settings for reasoning and reflection, allowing for continuous accumulation and updating of experi- ences. We conduct comparative experiments when the Memory module has 5, 20, and 40 experiences, respectively. Each setting is repeated 10 times with different seeds. The results are shown as a box plot in Figure 5 (a). Success Steps (SS) is the number of consecutive frames without collision, an SS of 30 means that the ego car has completed the driving task. We found that as the number of experiences in the Memory module increases, the perfor- mance of the DiLu framework improves in all few-shot settings. Notably, the 5-shots setting successfully completes the closed-loop driving task in the majority of cases in scenarios with 20 experiences. Furthermore, when the Memory module is cquipped with 40 experiences, all trials achieve median SS of over 25. In comparison, when the framework lacks the Memory module and runs 0-shot experiment, no tasks are successfully conducted and the median SS is below 5. This indicates that LLM cannol directly perform the closed-loop driving tasks without any adaptation.
In addition, we observed that for fixed number of memory items, the performance of the framework improved as the number of few-shot experiences increased. Specifically, with 40 memory items, the 5-shot framework successfully passed almost all tests, while the median SS for the 3-shots and 1-shot frameworks were 27 and 25, respectively. This can be attributod to the fact that a higher number of few-shots includes a diverse range of experiences in similar driving scenarios. When fed into the LLM as prompts, these allow the LLM to draw on a wider range of information and decision strategies, thereby facilitating more rational decisions. Several detailed case studies can be found in Appendix A.3
4.2 COMPARISON WITH REINFORCEMENT LEARNING METHOD
We conducted comparative experiments between DiLu and the SOTA reinforcement learning (RL) method in Highway-eny, GRAD (Graph Representation for Autonomous Driving) XXi Sukthankar, 2022). GRAD an RL-based approach specifically designed for autonomous driving scenarios, it generates global scene representation that includes estimated future trajectories of other vehicles. We trained GRAD under the setting of lane-4-density- which means the 4-lane motorway scenarios with vehicle density of 2.0. The training details are given in Appendix A.4 and A.55 As a comparison, DiLu used 40 experiences purely obtained from lane-4-density- setting. We defined the success rate (SR) driving
7

---

Published as a conference paper at ICLR 2024
25
Success steps (55
D-Khot
1-hhot
3-thots
S-shots
lone 5 densiyy
20
3-shots
5-chots
Memory items number
Few-shots type
(b)
Figure 5: (a) Quantitative experiments with different experiences in Memory module and dilferent few-shot numbers. Notably, the 5-shots setting achieve maxinum(30) simulation steps with both 20 and 40 memory items. (b) Generalizability experiment on environment with different traffic density using 20 memory items.
without any collision for 30 decision frames and then conducted experiments under three environment settings: fane-4-densit-2, Lane-5-density-2.55 and lane-6-density-3 Each setup includes 10 test scenarios with different seeds.
The results are illustrated in Figure 6 (a). Firstly, in the lane-4-density-2 setting, DiLu uses only 40 experience in the Memory module to achieve 70% SR while the GRAD converges 69% SR after 600,000 training episodes. We found that many failures from GRAD are due to the inability to brake in time, resulting in collisions with the vehicle ahead. This bocause reinforcement learning methods tend fit the environment and fail to take human-like driving knowledge into consideration. Secondly, GRAD optimized on 1ame-4-density-2 to -ade---denstt-2.55 and 1ame-6-density-3 settings. We observe that both methods suffer varying degrees of performance degradation in the migrated environments, as the number lanes changes and traffic density increases. environment, DiLa still maintains 35%
migrale DiLu and
However, in the most complex lane-5-densitys SR without extra optimization, while the GRAD has an 85% performance drop. DiLu accumulated experience in one environment can be generalized into another one, while RL method tends to overfit the training environment.
4.3 EXPERIMENTS ON GENERALIZATION AND TRANSFORMATION
Data-driven approaches often overfit the training environment, while human knowledge should be domain-aguostic and can be generalized to different environments. We perform several experiments to evaluate DiLu's gencralization and transformation abilitics.
Generalization ability in different environments. We verify whether the experiences obtained DiLu's Memory module have generalization capabilities. More formally, we used 20 experiences obtained from the 1ane-4-density-2 environment, and conducted experiments in the fane-5-density- setting, testing the closed-loop performance of 3- shot and 5-shot respectively. The experimental results are shown in Figure 5 (b). As we can see, the 3-shot version of DiLu achieves 13 median SS on the setting of Lame-4-density- while decreasing
5 media SS under the lane-6-density- setting. But in the 5-shot version, we achieve 30-23 median SS degradation in the same situation. This suggests that the ability generalize is better with more few-shot experience fed into the LLM.
Transformation ability using real-world dataset. Since DiLu's Memory module stores experiences in the form of natural language text, contains environment-agnosiic knowl- edge that can be readily transferred to different environments. To illustrate this capability, we created two Memory modules, each containing 20 experiences extracted from two dis- tinct sources: (1) Highway-env and (2) CitySim, a dataset comprising real-workd vehicle trajectory data (Zheng et al. 2023 We subsequently evaluated these modules on the lane Lane-4-density- and lane- - ------ddn------- scenarios within the Highway-env environment. The experimental results are presented Figure (b). In comparison to a system without any prior experiences (depicted by the gray dash-dot line), CitySim-contributeed knowledge
8

---

Published as a conference paper at ICLR 2024
80%
70%
DiLu
30
65%
All evaliated on Highway env
0016
Success steps (5S)
15
40%
35%
10
20%6
10%
0%
lane
2
laneSidensity-3
Highnaysen
CitySim
Environment type
Source Experience
(b)
Figure 6: (a) Performance comparison with GRAD in different types of motorway environ- ments. Both two methods are only optimized on lane-4 4ane-4-density-2 set teltings and evaluate on Lane-t-dessity-2 --n----enssty-2.. 5 and 1ane-5 -ane-5-denstt-- respectively. (b) Ex- periments with experience from different domains.
enhanced DiLu's performance. Also, when transferring experiences to a more congested environment (from lane-4-density-2 lane-5-density-3) the memory derived from real-world data exhibited superior robustness compared to the memory accumulated solely within the simulation domain.
4.4 EPFECTIVENESS OF TWO MEMORY TYPES IN THE REFLECTION MODULE
In this section, we explore the signifi- cance of incorporating successful expe- riences and revised unsafe experiences in the Refleetion module through an ablation study. We adopt the Mem- ory module containing 20 initial periences and observe the performance change during the new memory accu- mulation. The results are shown in Ta- ble D The baseline indicates the sys- tem using the Memory module with 20 initial experiences. Then we add 12 success memories and correction memories into the baseline successively. In the experiments, the median success step of the baseline is only 10. However, after updating with new experiences, all two methods achieve median suc- cess step of over 20, and the method with both types of experiences shows higher success steps on all statistical measures. Therefore, it is both reasonable and effective to add two different. types of experiences during reflection.
Effectiveness of two memory types in the Reflection module Success Steps. MIN, Q1, Median, Q3, MAX means the quartile statistical evaluation.
Table 1:
Methods
MIN
Median
Q3
MAX
baseline
2.0
22.0
30.0
manccens_nemory
3.0
29.3
30.0
Etorrectiio_nenoryy
4.0
30.0
30.0
5.0
30.0
30.0
CONCLUSION
In this paper, we explore the realm of instilling human-level knowledge into autonomous driving systems. We summarize knowledge-driven paradigın and propose the DiLu frame- work, which includes a memory module for recording past driving experiences and an agent equipped with Reasoning and Reflection modules. Extensive experimental results showcase DiLu's capability to continuously accumulate experience and exhibit its strong generaliza- tion ability compared to the SOTA RI-based method. Moreover, DiLa''s ability to directly acquire experiences from real-world datasets highlights its potential to be deployed tical autonomous driving systems. The DiLu framework, while effective, is not without limitations. Presently, it experiences decision-making latency of 5-10 seconds, encompass- ing LLM inference and API response times. Additionally, it does not completely eradicate hallucinations generated by LLMs. Future improvements could capitalize on recent advances in LLM compression and optimization, aiming to enhance both efficiency and effectiveness.
prac-

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
The research was supported by Shanghai Artificial Intelligence Laboratory, the National Key R&D Program of China (Grant No. 20222D0160104) and the Science and Technology Commission of Shanghai Municipality (Grant Nos. 22DZ1100102 and 23YF1462900).
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. orXiv preprint arXi::20030077 2023.
Daniel Bogdoll, Jasmin Breitenstein, Florian Heidecker, Maarlen Bieshaar, Bernhard Sick, Tim Fingscheidt, and Marius Zöllner. Description of corner cases in automated driving: Goals and challenges. In Proceedings of the IEEE/CVF International Conference Computer Vision Workshop, pp. 1023-1028, 2021.
Jan-Aike Bolte, Andreas Bar, Daniel Lipinski, and Tim Fingscheidt. Towards corner detection for autonomous driving. In 2019 IEEE Intelligent vehicles symposium (IV), pp. 438-445. IEEE, 2019.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Lan- guage models are few-shot learners. Advances in neural information processing systems, 33:1877 1901, 2020.
Sébastien Bubeck, Varun Chandrasckaran, Ronen Eldan, Johannes Gchrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint Xrxiv:2303. 121122203.11,,, 2023.
Long Chen, Yuchen Li, Chao Huang, Bai Li, Yang Xing, Daxin Tian, Li Li, Zhongxu Hu, Xiaoxiang Na, Zixuan Li, et al. Milestones in autonomous driving and intelligent vehicles: Survey of surveys. IEEE Transactions on Intelligent Vehicles, (2D::0046-0555 2022.
Long Chen, Yuchen Li, Chao Huang, Yang Xing, Daxin Tian, Li Li, Zhongxu Hu, Siyu Teng, Chen Lv, Jinjun Wang, et al. Milestones in autonomous driving and intelligent vehicles- -arttaattt 1: Control, computing system design, commuicattin,, hd map, testing, and human behaviors. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2023a.
Long Chen, Siyu Teng, Bai Li, Xiaoxiang Na, Yuchen Li, Zixuan Li, Jinjun Wang, Dongpu Cao, Nanning Zheng, and Fei- Yue Wang. Milestones in autonomous driving and intelligent vehicles part ii: Perception and planning. IEEE Transactions on Systems, Man, and Cyberneties: Systems, 2023b.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. arXiv preprint ar arXiv:22440223 2022.
Felipe Codevilla, Eder Santana, Antonio M López, and Adrien Gaidon. Exploring the limitations of behavior cloning for autonomous driving. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. 9329 9338, 2019.
Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanct, Daniel Duckworth, Sergey Levine, Vincent Van- houcke, Karol Hausman, Mare Toussaint, Klaus Greff, Andy Zeng, Igor Mordatch, and Pete Florence. Palm-e: An embodied multimodal language model. In orXiv preprint arXio:2303.0333 03377, 2023a.
10

---

Published as a conference paper at ICLR 2024
Danny Driess, Fei Xia, Mehdi SM Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, et al. Palm-e: An embodied multimodal language model. arXiv preprint arXiv:2033.033 2023b.
Jiafei Duan, Samson Yu, Hui Li Tan, Hongyuan Zhu, and Cheston Tan. A survey of embodied ai: From simulators to research tasks. IEEE Transactions Emerging Topics in Computational Intelligence, 6(2):230-24) 244,, 2022.
Daocheng Fu, Xin Li, Licheng Wen, Min Dou, Pinlong Cai, Botian Shi, and Yu Qiao. Drive like a human: Rethinking autonomous driving with large language models. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision: Workshop, pp. 910991199999 2024.
Peng Gao, Jiaming Han, Renrui Zhang, Ziyi Lin, Shijie Geng, Aojun Zhou, Wei Zhang, Pan Lu, Conghui He, Xiangyu Yue, Hongsheng Li, and Yu Qiao. Tlama-adapter v2: Parameter-efficent visual instruction model. arXiv preprint 1550: 2023.
Florian Heidecker, Jasmin Breitenstein, Kevin Rösch, Jonas Löhdefink, Maarten Bieshaar, Christoph Stiller, Tim Fingscheidt, and Bernhard Sick. An application-driven concep- tualization corner cases for perception highly automated driving. In 2021 IEEE Intelligent Vehicles Symposium (IV), pp. 644 651. IEEE, 2021.
Siyuan Huang, Zhengkai Jiang, Hao Dong, Yu Qiao, Peng Gao, and Hongsheng Li. In- struct2act: Mapping multi-modality instructions robotic actions with large language model. orXiv preprint ar arxiv:2305. 11176, 2023a.
Wenlong Huang, Chen Wang, Ruohan Zhang, Yunzhu Li, Jiajun Wu, and Li Fei-Fei. Vox- poser: Composable 3d value maps for robotic manipulation with language models. arXiv preprint arXiv:2307.08 059973, 2023b.
Ye Jin, Xiaoxi Shen, Huiling Peng, Xiaoan Liu, Jingli Qin, Jiayang Li, Jintao Xic, Peizhong Gao, Guyue Zhou, and Jiangtao Gong. Surrealdriver: Designing generative driver agent sinulation framework in urban contexts based on large language model. arXiv preprint arXiv:2309, 18193, 2023.
Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with gpus. IEEE Transactions Big Dala, 7(3):535 547, 2019.
Yann LeCun. A path towards autonomous machine intelligence version 0.9. 2, 2022-06-27. Open Review, 62, 2022.
Edouard Leurent. An environment for autonomous driving decision-making. https://
github.com/elle wtt..///eeeeueeeeiiiaaaaaaeeeeee 2018.
Xin Li, Yeqi Bai, Pinlong Cai, Licheng Wen, Daocheng Fu, Bo Zhang, Xuemong Yang, Xinyu Cai, Tao Ma, Jianfci Guo, et al. Towards knowledge-driven autonomous driving. 2023.
Xix preprind
Thco X Olausson, Joevana Priya Inala, Chenglong Wang, Jianfeng Gao, and Armando Solar-Lezama. Is self-repair a silver bullet for code generation? arXiv preprint ar arXio:2306. 09896, 2023.
OpenAL. Introducing chatgpt. https://openai.. http::///ppaaa..cooo/oog//agagg 2023.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730-22 -77744,, 2022.
Joon Sung Park, Joseph C O'Brien, Carrie J Cai, Meredith Ringel Morris, Percy Liang, and Michael Bernstein. Generative agents: Interactive simulacra of human behavior. arXiv preprint arXim:2304 2023.
11

---

Published as a conference paper at ICLR 2024
Rolf Pfeifer and Fumiya lida. Embodied artificial intelligence: Trends and challenges. In Embodied Artificial Intelligence: International Seminar, Dagstuhl Castle, Germany, July 7-11, 2003. Revised Papers, pp. 1-26. Springer, 2004.
Fawaz Sammani, Tanmoy Mukherjee, and Nikos Deligiannis. Nix-gpt: A model for nat- ural language explanations in vision and vision-language tasks. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 8322-8332, 2022.
Timo Schick, Jane Dwivedi-Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. arXiv preprint 0rr:22300 76711, 2023.
Hugo
Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinel, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.139 13971, 2023.
Guanzhi Wang, Yuqi Xie, Yunfan Jiang. Ajay Mandlekar, Chaowci Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. arXiv preprint 0rXiv:2305.16201 2023.
Wayve. Lingo-1: Exploring natural language for autonomous driving. https://wayve: ai/ Bhinking/Lingo-aa natural-uuaaaaaa -aaguugge ceattooooous-drivvia
2023.
Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. Finetuned language models are zero-shot
learners. Xix preprint arXi:22109.014 096616 2021.
Jason Wei, Xuezhi Wang, Dalc Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in Newral Information Processing Systems, 35:24824 24837, 2022.
Licheng Wen, Pinlong Cai, Daocheng Fu, Song Mao, and Yikang Li. Bringing diversity to autonomous vehicles: An interpretable multi-vehicle decision-making and planning framework. In Proceedings of the 2023 International Conference Autonomous Agents and Multiagent Systems, AAMAS '23, pp. 2571-2573, Richland, SC, 2023.
Zerong Xi and Gita Sukthankar. A graph representation for autonomous driving. In The 36th Conference on Newral Information Processing Systems Workshop, 2022.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. arXiv preprint arXio:2210.0366 2022.
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. A survey of large language models. Xix preprint arXiv:3303. 18223, 2023.
Ou Zheng, Mohamed Abdel-Aty, Lishengsa Yue, Amr Abdelraouf, Zijin Wang, and Nada Mahnoud. Cilysim: A drone-based vehicle trajectory dataset for safety-oriented research and digital twins. Trunsportation Research Record, 2023.
Jingxing Zhou and Jürgen Beyerer. Corner cases in data-driven automated driving: Defini- tions, properties and solutions. In 2023 IEEE Intelligent Vehicles Symposium (IV), pp. 8. IEEE, 2023.
Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:234,.1059. 2023a.
Xizhon Zhu, Yuntao Chen, Hao Tian, Chenxin Tao, Weijie Su, Chenyu Yang, Gao Huang, Bin Li, Lewei Lu, Xiaogang Wang, et al. Ghost in the minecraft: Generally capable agents for open-world enviroments via large language models with based knowledge and memory. Xiv preprint or arXi::230.. 1.....777 17144, 2023b.
12

---

Published as a conference paper at ICLR 2024
A APPENDIX
A.1 DETAILED SETUP OF THE EXPERIMENT
Within our DiLu framework, the large language model we adopt is the GPT family developed by OpenAL. GPT-3.5 (OpenAl, 2023) is used in the Reasoning module of the framework, which is responsible for making reasonable decisions for the ego vehicle. GPT-4 is used in the Reflection module since it has demonstrated significantly improved self-repair and fact-checking capabilities compared to GPT-3.5 (Bubeck al. 2023 Olausson et al. 2023) ). To serve as the Memory module in the DiLu framework, we adopt Chroma an open-source embedding vector database. The scenario descriptions are transformed into vectors using model of OpenAT.
the text-embedding-an
In terms of the setup for highway-env, we directly obtain vehicle information from the underlying simulation and imput it into the scenario descriptor. This information only includes each vehicle's position, speed, and acceleration data in the current frame, without any decision intent or potential risk information, as shown in Figure 7 Meta-actions are used as the decision output in our experiments, which include five discrete actions to control the ego vehicle: acceleration, maintaining speed, deceleration, and lane changes to the left and right. For each closed-loop driving lask, we define a successful completion time of 30 seconds, with decision-making frequency of 1Hz. This means that. if the ego vehicle can navigate through traffic at a reasonable speed and without collisions for 30 seconds, we consider the task to be successfully completed. Unless otherwise stated, our experimental environment is four-lane motorway with a vehicle density of 2.0, representing sccnarios with relatively high traffic density and complexity. All other scttings follow the simulator's default configurations.
A.2 PROMPTS EXAMPLE
In this section, we detail the specific prompts design in the reasoning and reflection modules.
Reasoning prompts As mentioned in the article, the prompts for the reasoning module primarily consist. of three parts: system prompts, scenario description, and few-shot expe- rience. Specifically, shown in Figure 7 the system prompts section is entirely fixed and mainly includes an introduction to the closed-loop driving task, instructions for input and output, and formatting requirements for LLM responses. Most of the scenario description is fixed, but three parts are directly related to the scenario and are dynamically generated based on the current decision frame. The driving scenario description contains information about the positions, speeds, and accelerations of the ego vehicle and surrounding key vehi- cles. It's important to note that we only embed the text of the driving scenario description into vectors and use it as query input to the memory module. Available action includes all meta-actions. The driving intention can be input by humans to modify the vehicle's behavior, with the default intention being: "Your driving intention is to drive safely and avoid collisions."
As for the few-shot experience, it is entirely obtained from the memory module. Each experience consists of human-LLM dialogue pair, where the human question includes the scenario description at that decision frame, and the LLM response represents the correct (or correct after revised) reasoning and decision made by the driver agent. The extracted experiences directly utilized with few-shot prompting technique to input into the large language model, enabling in-context learning. Figure demonstrates the results of a 3-shot experience query, which includes two "keep speed" decisions and one "decelerate" decision. It's important to note that consistency in decisions not a requirement within the few-shot. experience.
13

---

Published as a conference paper at ICLR 2024
Highway-env
ego
992
368
568
128
System Prompts
You are ChatGPT, large language model trained by OpenAI. Now act mature driving assistant, driver complex driving You will he given detailed description of the driving scenario of curzent frame along with your history of previous decisions. will slse given the available actions allowed to take. these elementa delimised by Your response should use the following formal:
advice
<ressoning?
<ressoning>
<repest until have decision> Respense
conly output one action name or explanation. The output decision must be unique and not ambiguous, for example iI decida decalearate, then output "4'> sure
you
any
#ade to aeparate every atep.
Pew-shot experiences
#1 Experiance:
#2 Experience:
#k Experience: Bunan questiens:
Bunan Questions:
Hunan Questions:
response:
response:
reaponse:
Scenario Description Above messages are sone examples you decision successfully in similar scenarios current scenario. You should refer examples make a decision for the current scenario.
You are driving road with lanes, and are currently driving
Your position 1(408.13, 0.c0)*, speed is 24.03 $n/st, acceleration is -0.05 $n/s*28, and lane position is 408.13 snt.
There are other vehicles driving around you, and below is their basic information:
your right of you. The position of it is 74222.2, 4.00)", speed is 17.25 #n/s#, scceleration is #n/s*28, lane position 422.72 ent.
driving
Venicie '712" is driving the same lane as you and is abead oz (344222,, 0.00)' speed
Here is the current
you. position acceleration is -1.49 tm/s*24, and lane position is 442.23 ent
#0с* Driving scenario description: (scenario description)
#0е# Driving Intensions:
driving intension drive safely and avoid collisions
(driving
*ее* Available actions:
(available, actions)
available actions renain in the current lane with current. apeed Action_ids Turn-right change lane to the right of current lane Action_idi id: Acceleration
You can stop reasoning action to take.
vaiid
vehiole Action_id; beceleration decelerate the vehicla Action_
Figure 7: The prompts template for reasoning module. The prompts in the grey box are fixed, while the prompts the coloured box are different dopending on the current scenario. The few-shot experiences are detailed demonstrated in Figure
14

---

Published as a conference paper at ICLR 2024
Ranan Question:
Sescription
stop ressaning you
#1 Experiene seperience
response:
have acttonn to choose fror. Now, would 1ike know which action in posetol a. can accelerate, then idle, finelly decelerate. can alxo try to change lanex but
sbould tirst chack
csation and nos Loo Eroquently.
vant to know if can accelerace, so need observe the car in front of ne on the current lane, which oar discance between and car
464.26 432.11 32.11 x, and cifference speed 20.90 and apeed is 1.42 n/s slower than nise.
228"" sraveling 12.11 shead distanee ls pafe ay apeed already highez the front on cureest lane. to should accelerase. carnot acsalarate, want know can mwintain ny current speed. need to observe bue above caloulation. distance beiveen larger than my preferced following my speed.
-3.02 a/a.
cur tront "196"
ne on current lase, speed
speed higher
Responze
Homan Question:
Bcre la the 22 sesnario:
4444 briving scenario description: Deseription the scene, onlited here... eses briving Incensions: bescrip:ion of the intension, axitted here... eee* Available actions: Description of the available actions, enitted here... can stop reweoning once you have valid action take.
LIM response:
#2 Experience
have actionn onn to choose froe. Now, 1 uould Tike to know which actioon in passible a.
should riret chack ir can acostarate, then idle, finally decelerate. can also try to change lanex but with
cxation and not ton reaquantly.
vant to know it can accelerace, so I seed to observe the car in tront of me on the cursent lase, shich is car "p52". The distence between zu and car "352" is 422.15 390.19 22.26 n, and the citterence in speed is 14,00 24.03 -9.95 a/a. cae "252" is traveling 22.25 in abead of me and its speed is 9:35 n/a slower than nise. this distance is safe and ay apeed is olready highez than the front cas on the cureest lane. 5o should sot acceleza:e.
Since connos aeselerate, I wans to know if con idle. 1 seed to observe the car behind se on the cuzzens lane. which ls caz "996". i The distance between ae and car "976" is 199, 89 411, 65 -11.79 E. and the difference in speed is 24.03 29.90 0:15 n/s, car "976" is traveling 31.19 a behind ne and its speed is 0.13 a/s slower than rine. This distance is too close and speed is already higher than the rear our on the current lane, If idte, the distance betieen se and car "926" xill further enal Ter, so whould nor idle. order to keep safe dixtance
the can, can only decelerate. mecalaration is faaxible action. Great, zake ay docision now. Decipion: Decelesation Respotae to userieeee
Question:
Btene 23 acenario:
evew Driving acenarto deccription: Desoription the eome, onitted here... briving Insensious: beseripsion the intension, wxitted here... Available actioon: Description the availabie actione, enitted here... stop zeasoning eace you have valid action take.
#3 Exper Lence
reeponse:
have actions to chocse fror. 1 would know action possible. check accelorato, idie. finally deoclerase. con also tsy so change lascs coution nos froquently. to
between accelerace,
ne lase, u, speed
ear
speed
speed higher my speed. somslarate,
my presucred
Reeponze to seee
Figure 8: The 3-shot prompts example for the scenario shown in Figure 7 DiLu recall top three most similar experience from the memory module which contains 2 idle and 1 deceleration decision. The few-shot experience is entered into GPT-3.5 as a "human-L.LM" dialogue.
15

---

Published as a conference paper at ICLR 2024
System Prompts
You are Chatdr?, a large language nosel trained by OpenAI. Now you act as a mature driving assistant, who can give accurate and correct advice for hunan driver in comples urban driving scenarios maariiaa... You will be given a datailed description of the driving scanario of current. frame along with the available actions allowed to take.
Nake sure to include Name to separate every step.
Reflection Prompts
Here is the current acenaria:
... Message
Driving acenaric description: (acenario, dedderiptpiion ettt Driving Intensions:
(honan message)
ChatGPT Response
(lin_response)
(driving seriving_iiteer intennsinnn)
Now, you know this action ChataPt output cause colliaon after taking this action, which meane there are some mistake in ChatGPT resoning and cause the wrong action.
нчни Available actions:
actions))
You can atop reasoning once you have valid action take.
Flease carefully check every reasoning in Chatop? response and find out the mistake in the reasoning process ChateFT, and also output your corrected vereion of ChatePT respense.
Your ansuer should use the following format:
to choose fron.
44н* Analysis of the nistake:
Now, I would like to know which action is possible.
<Your ana.ysia of the mistake in Chatert reasoning process>
**** What should Chatger do to avoid such errors in the
future:
creasoning process>
<Your answer>
**** Corrected version of ChatePT reeponse:
<Your corrected version of ChatCPT response>
Response user:#200
Figure 9: The prompts template for reflection module. The prompts in the grey box are fixed, while the prompts in the coloured box are different depending the current scenario.
Reflection prompts Figure 9 presents the template for the reflection module's prompts, primarily comprising two sections: system prompts and reflection prompts. The system prompts section is entirely fixed and mainly consists of an introduction to the reflection task, along with instructions and formatting requirements for LLM responses. On the other hand, the reflection prompts section includes the scenario description of the erroneous decision and the faulty reasoning process made by the LLM. We require the reflection module to produce three components: an error analysis, corrected reasoning and decision-making, and suggestions on how to avoid making the same mistake in the future.
A.3 CASE STUDY
In this section, let's demonstrate several case studies.
First, we present the results of the reasoning module for three cases, as shown in Figure 10 In Case 1, the green ego car is closely following the car 368 in front, and their speeds are similar. Initially, the driver agent explores whether it can accelerate lane. The agent determines that, since the ego car's speed is similar to the car in front, there's no need to accelerate. Subsequently, the driver agent explores the possibility of maintaining the current speed. Through reasoning, the agent concludes that although the distance between the vehicles is slightly shorter than the idcal following distance, it's safe to maintain the speed because the ego car's speed is slightly lower than that of the car in front. Lastly, the agent verifies the feasibility of changing lanes to the right and calculates that the ego car's speed lower than the car in the right lane by 5.88 m/s, with a safe following distance. Therefore, changing lanes to the right. is also safe. In the end, the agent decides change lanes to the right. In Case 2, the scenario The agent observes a long distance with the car in front and that the ego car's speed is lower than that car. Consequently, not perform further calculations regarding maintaining speed or changing lanos, aligning with the typical thought process and reasoning human drivers. In Case 3, we changed the driving intention in the prompts from the default to "I need to change to the rightmost lone." " The driver agent recognizes this intent. Although determines that it can maintain speed in the current lane, it continues to evaluate whether changing lanes to the right in the current lane is feasible. Ultimately, considering the intention, the agent chooses to change lanes to the right maintaining speed.
the current.
relatively simple.
decides to accelerate. In this case, the agent does
16

---

Published as a conference paper at ICLR 2024
37%
ego
13.6
1568
Reasoning Decision
1 have actions to choose froa, Bow, I would like to knsw which acties is possible. I should First check if ner accelerate, then idln, finally decel erate. can also try to change lanes but wi paution and not too freguently.
current speed difference which should ancelerate. can maintain ay cursent speed. need to ebssrve car front of me on the current lane, which car "368". Due to above caloslation, the distance between and car 3665" is 25.55 s, and the difference in speed is 0.27 n/s, the distance 29 29.55 as lazgez than my preferzed caz following diatance 20 but my apeed is bit lower than the front car on the current lane. So can naintain current speed.
accelerate, need to
La the
my speed Since cannot accelezate, want
distance
ear
- I can to change lanes to the right, but need to check if ia safe. I need to observe the par on the lane ny right, which is "s6a". the distance between and "560" 541.40 528.13 12.07 and difference
distance 5.88 my speed
"568"
of its opeed right
change creat, make decislor. now. Decision: Turo-right
Case
ego
928
BBQ
Reasoning and Decision like
possible. secciorate, then idio, finally deceierate.
should
1xy change lancs
cantion and not too freqiently.
want to know caz "696". the distance between zs and car "695" is $12.67 373.04 44.63 n, and the difference in speed is 17.15 16.00 1.15 a/x. car "696" is traveling 44.63 ahead of me and its speed in 1.15 n/s faster than nine. this distance is sañe and speed is already lover than the front on the cuzzent lane. 8o can accelerate. azeat, can make decisios. now. Decision: Acceleration
can accelerate,
need to observe the oar in front of re on the current lane, which is
ego
280
Case
Driving intention: need change to the rightssst lane to get off highway.
Reasoning and Decision
driving intentions, previoux successful exarplen to make dectatio.
Based on curzent acenario
considur available
zefer
want to lnsw if I can accelerate, so I need to observe the car in front of ze on the current lone, which is oar "eze" the distance between za and car "aze" is 646.37 393.03 53.34 i, and the difference in speed is 21.28 21 a -3.93 n/s, car "424" in traveling 53.34 abead of and its speed is 3.93 n/e slover than zine. this distance in sače and speed is already higher than the front not acte larate.
the cureent lase. 1 should
socelerate,
the prefeszed following
caleulation, distance
speed
distance 53.56 higher
front Now, need to consider my driving intention to change the right lane. should cbeck
30 it,
apead highar
cursent lane.
and speed of the care the right late. The our on the right laze ahead of ne is car "280" The dietance between and eaz "180" in $29.23 393.99 35.20 x, and the difference in speed is 19.59 25.21 -9.62 n/s. cas "259" ix travel kng 36.20 n wheed of me and itx speed is 7.62 n/s alower then mine. This dixtance ix safe and speed is higher than the front oaz on the right lase. tharefore, is aafe to change the right lane.
Considering the above analysia and the driving intention, Deciision: Turn-right
make decision
Figure 10: Case study for reasoning module
17

---

Published as a conference paper at ICLR 2024
265
1504
707
128i
eg
408
Original Decision
given acenario,
acalyze situation
driving
sirst, let's 's conaider accelerating. The vhhicle "128" is ahoad of ne in ny curzent lane. It is position 531.70 which quite far from current position of 480.91 the difference in speed between 18.19 19.34 -1.15 n/s.
speed than mine, socelerating would only increase the speed difference and potentially lead to collision. Therefore, accelerating current apeed. The
cuitable
Next, let's consider maintaining current lane, and its speed is slightly slower than nine. If naintain current speed, will gradually speed is not a fessible option.
ny
which may result collision. Therefore, msintaining
New, let's coasider changing lanes. The vehicle en the right lane, '408", is shead of me. It is 480.91
which is relatively difference in speed between as 17.82 19.34 -1.52 n/s. The distance between us appropriate, and its speed is slower than nine. Therefore, changing lanes to the right seens like collision.
uy
feasible
Based on the analysis, the best driving decision in this scenario to change lanes to the right. Decision: Tura-rigat
Crash Analyze
The mistake in ChatoPT's reasoning lies in its foasibility changing the right. It chose change lanes to the right
Lesson Learned
ChatcP? should consider not only the speed
of vehicles also their relative pesitioning when naking deciaions about lane changes. important
based on the slower speed of the vehicle ('409") in the vehicles. However, it did not consider the zelative position and speed of the vehicle in the right lane, which
enough space and tine to safely complete the lane change without causing collition.
changing. The vehicle in the right lane is only elightly abead and noving slower, which means there's high risk of colliston change attergted.
might involve complex calculations of relative speed, distance, and tine.
Revised Decision
Based the given scenario, let's analyza the aituation make best driving First, let's le consider sccelersting. The vehicle "228" is shead of ne in ny current lane. It is position 531.70 which
from
position of
difference in its apeed
between la 18.19 19.34 -1.15 n/a. Since than mine, sccelerating would only increase the speed difference and patentially lead to colliaion.
suitable option.
Next, let's consider maintaining our current speed. The vehiole "12e" ahead of me in ny current lane, and its speed is slightly slower than nine. If maintain current speed, will may
gradually
up
collision.
maintaining ny
speed
is not fessible option.
Now, lat's consider changing lanes. The vehiele the right lane, "408", is ahead of ne. It at position 481.25 which relstively in speed between us 17.82 19.34 = -1,52 n/s. The distance between un is 481.25 480.91 0.34 Given the relative speed, the time collision would be 0.34 1.52 0.22 seconds, which eafely change lanes. Siven these considerations, the safest course of action would be to decelerate, which would increase the distance between and the vehicle shead the lane, reducing the risk of collisica.
ny
position
changing
right
plan.
Decision: Decelerate
Figure 11: Case study for reflection module
18

---

Published as a conference paper at ICLR 2024
Next, in Figure 11 present result of the reflection module. In this case, the ego traveling in the second right lane and made the incorrect decision to change lanes to the right, resulting in collision with 408. Consequently, the reflection module intervenes to correct the mistake. First, it conducts an analysis of the cause of the collision. Upon reviewing the original reasoning process, GPT-4 astutely identifies the source of the error. It realizes that the imitial decision did not take into account the relative distance between the vehicles in the right lane. In fact, car 408 was merely "slightly ahead" of the ego car, contrary to what the initial decision process described as an "appropriate" distance. Subsoquently, in the revised decision process, the driver agent supplements and correctly calculates the relative position of the ego car and the car in the right lane. It also includes the calculation of "time to collision" (which was completely absent the original decision). Based on this calculation, it determines that "time to collision is too short to safely change lanes." As a result, chooses to decelerate, avoiding the collision. Finally, the reflection module summarizes the lesson learned from this error, emphasizing the importance of ensuring that there is enough space and time to safcly complete a lane change without causing a collision: "It's important to ensure that there's 's enough space and time to safely complete the lane change without causing a collision."
A.4 TRAINING SETTINGS OF GRAD
35
30
episode_reee reward
25
20
15
10
100000
200000
300000
400000
500000
600000
triiiin_stteps
Rewards for each episode evaluated during the training process. Each data point represents the average of independent runs and has been smoothed using a window size of 9. The shadows correspond to the standard deviations of these runs.
Figure 12:
Here we introduce the training settings of the RL method GRAD for the comparing with the DiLu. The rewards for each episode of GRAD are evaluated during the training process as shown in Figure [2 During the training, We the Observations consisting of the kinematic status of the agent and the surrounding vehicles, specifically their coordinates, velocity, and heading. The maximum number of surrounding vehicles that can be observed 32, and their speeds range from 15 to 25 units. As for the Action, it is discrete and comprises the following options: OneLaneLeft, Idle, OneLaneRight, SpeedLevelUp, and SpeedLevelDown. The available speed levels are 10, 15, 20, 25, and 32 units. The reward system includes two components, one is that a reward of 0.2 is granted for each time step survived by the agent, and the other one is that the reward is linearly mapped from the driving speed (10, 32), reward value between 0 and 0.8. Moreover, Each model trained for 600,000 action steps and is subsequently evaluated over 24 episodes using deterministic policies.
19

---

Published as a conference paper at ICLR 2024
A.5 DIFFERENT DENSITY LEVEL
In order to give the reader an intuitive understanding of the different vehicle densities under Highway-env, we show in Figure 13 screenshols of number of scenarios for the experimental part of this paper for three different trallic densities. It can be seen that with increasing vehicle density, there more vehicles in the scenario, and the vehicle spacing and vehicle speed are reduced.
4-lane 2.0 density
5-lane 2.5 density
5-lane 3. 0 density
Figure 13: Different vehicle density in highway-env
20


Reviews:
[{"review_id": "YPqtN1ZD3L", "reviewer": "Reviewer_LJSU", "summary": "This paper presents a novel framework for autonomous driving systems based on LLM and tailored components. Contributions of this paper are several folds:\n\n- Knowledge-Driven Paradigm: The paper introduces a knowledge-driven paradigm for autonomous driving, differentiating it from existing data-driven approaches. This paradigm is inspired by human driving, which relies more on knowledge and understanding rather than mere data accumulation.\n\n- DiLu Framework: The authors propose the DiLu framework, integrating large language models (LLMs) with autonomous driving systems. Several modules are proposed based on recent advances of AI agent: A Reasoning Module that utilizes LLMs for decision-making based on common-sense knowledge; A Reflection Module that assesses decisions and updates them based on safety and correctness, using the knowledge from LLMs.\n\n- Experimentation and Results: Extensive experiments demonstrate the framework's capability to make proper decisions, its strong generalization ability, and the potential for real-world application. The paper compares DiLu with reinforcement learning methods, showing its superior performance in generalization and adaptability.", "questions": "While LLM-based agent systems have shown success in various embodied systems, the adaptation of it in the AV tasks is still unclear to the reviewer. AI agent system has shown prominent success in task planning for open world robotic tasks, but AV has a different setting (with different challenges). The motivation and advantages of using AI agent system for AV needs to be elaborate more.\nOn the other hand, the authors didn't evaluate the proposed framework thoroughly enough (with only simple metrics and simple baselines). This further raises questions of the reviewer regarding how promising or what are the key advantages of using AI agent system in AV setting.\nFinally, the proposed AI agent follows a typical setup compared to the other existing works in robotics tasks. The authors should highlight more on the unique challenges and design choices tailored for the AV task.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Innovative Approach: The integration of LLMs into autonomous driving systems represents a significant shift from traditional data-driven methods, potentially offering more adaptable and human-like decision-making.\n\n- Generalization Ability: DiLu shows a strong ability to generalize from one environment to another, a crucial aspect for real-world applicability.\n\n- Continuous Learning: The framework's ability to continuously evolve and improve through its memory and reflection modules is a key strength.", "weaknesses": "- Complexity and Scalability: The integration of LLMs and the need for continuous updating and reflection may introduce complexity, potentially impacting the scalability of the system.\n\n- Real-World Application: While the framework shows promise, the transition from controlled experiments to real-world application can be challenging, given the unpredictable nature of real-world environments.\n\n- Dependence on LLMs: The framework's reliance on LLMs means that its performance is heavily dependent on the capabilities and limitations of these models.\n\n- Evaluation thoroughness: The authors only evaluate the proposed methods with oversimplied metrics (collisions) and compared to a simple baseline (RL). The limitation of the evaluation poses a question mark on how such system actually performs in the real driving scenarios, compared to sota autonomous driving systems.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QR5UQkVxgV", "reviewer": "Reviewer_RmuT", "summary": "This work proposes a novel and interesting approach leveraging LLMs in autonomous driving to perform knowledge-based reasoning about making high level driving decisions. The approach is motivated by how humans learn to drive. There are three straightforward pieces of the method: reasoning, recall, and reflection. The method is evaluated in simulating driving scenarios and positively compared against a SOTA RL method and ablations of the approach.", "questions": "It is fairly odd in the experiments that two different GPT versions are used. Why did the authors not just use GPT-4?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The proposed method is well-motivated by human behavior and generally clearly explained. The experiments justify each portion of the method for achieving the goal task of autonomous driving.  The method is novel, simple, and has potential to be used in the real world. Overall, an interesting perspective on the self-driving car problem.", "weaknesses": "The memory module requires more description in Section 3.2. The process of storing experiences is somewhat unclear. The writing could be interpreted to mean that every scenario is stored separately or that the similarity between the keys is used to map similar experiences to the same memory store (which seems to be what the authors are actually doing). Either a new figure or updates to the existing figures would also add to clarity and precision.\n\nThis paper never discusses limitations. I strongly recommend making room to discuss the relationship between this approach and approaches which focus on safety. In fact, the “reflection” module is being presented as a safety mechanism. However, the trustworthiness of the results from the LLM is never discussed. Diving into reliability and limitations is important in a method which claims to address safety for transparency in a safety critical task where results are currently deployed in the real world.\n\nI thought the following claim in the abstract was slightly misleading given LINGO-1 (which the authors do cite). “To the best of our knowledge, we are the first to instill knowledge-driven capability into autonomous driving systems from the perspective of how humans drive.” I think that the correct way to phrase what the authors mean is specifically saying that they are the first to “use human-like knowledge-based reasoning to make autonomous driving decisions” or something similar since leveraging it in decision making is the distinction with prior work. “Instill” is a vague term which could also describe what LINGO-1 is doing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xN8k5wi0lW", "reviewer": "Reviewer_48LR", "summary": "The paper presents a framework for utilizing the few shot and self-correction capabilities of LLMs for the task of AV planning, where the following abilities of the framework are highlighted:\n- Store successful experiences in memory and leverage them to improve future rollouts through similarity retrieval and usage in few-shot prompting.\n- Ability to learn from unsuccessful experiences (ones with collisions) by applying LLM self-correction and storing the modified experience among the successful experiences in the memory\n\nThe above components, dubbed as reasoning and reflection modules respectively, are integrated along with memory in a closed loop setting without any back-propagation objective. \n\nA number of prompting techniques including chain-of-thought and few-shot prompting are used to get better reasoning. The environment used for experiments (Highway-Env) only requires four discrete decisions, hence the LLM is prompted to select one amongst these four decisions for each frame after going through CoT reasoning.\n\nThe experiments are used to demonstrate the following key claims:\n- The memory module combined with few-shot prompting provides much better results than using no memory module (zero-shot) or using lesser shots. \n- The more the number of experiences in the memory, the better.\n- The ability to generalize is better with more few-shot experiences fed into the LLM\n- Adding successful and corrected experiences both help in improving performance\n- Better generalization capability compared to RL method GRAD.", "questions": "- What kind of diverse interactions do we get from the Highway-env simulator? Would it be possible to evaluate the framework under more interactive / challenging conditions, especially wrt agent interactions? It would be interesting to see the generalization to intersections, interactions with peds, aggressive agents etc.\n\n- The correction experiences intuitively should provide a strong boost to performance since they are akin to hard example mining and injecting reasoning about the negative outcomes. However the corresponding results in Table 1 do not show strong improvements. Is it possible understudied and warrants more extensive experimentations?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The motivating idea of human knowledge distillation for AV planning is sound, interesting, and under-explored.\n\n- The overall framework formulation towards leveraging LLMs via appropriate prompting, retrieval, and self-correction is interesting and well set up. It would have been exciting to see formulations for LLMs assisting planning stacks (instead of directly doing discrete action decision making) - which could be much more valuable to existing systems.  \n\n- The flywheel effect created from storing both successful and unsuccessful + corrected experiences in memory is an important contribution.\n\n- The paper provides a good foundation for other exciting work to build upon, especially with the promise to open source upon acceptance.\n\n- The experiments are fairly extensive towards investigating all the different components of the proposed framework.", "weaknesses": "-  One of the main proposed advantages is better generalization through instilling human knowledge-driven capabilities instead of a data-driven only approach. However, the experimental settings derived from HighwayEnv are too restrictive to help extrapolate how such LLM based reasoning would perform on diverse new scenes using retrieval + few shot prompting. While it is perfectly fine to work with restricted settings and smaller datasets for new research work, the bridge to answer the most interesting questions is too long.\n\n- As mentioned in strengths section, providing directions and initial experiments on assisting planning stacks (instead of directly doing discrete action decision making) could provide a lot of value.\n\n- The experiment settings used to demonstrate generalization are not too convincing. The number of lanes and traffic density is changed, but this is still an extremely similar environment where the retrieved few-shot scenarios could be nearly directly applicable.\n\n- Under the above setting, it is possible that with a large enough memory module the task reduces to simply copying the answer from one of the retrieved experiences. It would be good to see a baseline where the decision from one of the retrieved experiences is used as is (voting with mixture of experts or winner takes all)\n\n- The metric movement with CitySim in Figure 7b and Table 1 correction row do not seem significant to make the corresponding claims?\n\n- Nit: The key frame sampling for successful experiences seems like an important detail that has not been explained.\n\n- Minor nit: The claim for this being the first work addressing AV planning via leveraging LLMs might need to be revised with recent papers like GPT driver (depending on chronology).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "noWUVYaBif", "reviewer": "Reviewer_Cxkm", "summary": "The paper includes a LLM into a framework which controls the decisions of a driving agent in a simulator. The authors define a concept, they call knowledge-based driving, and argue how their framework implements this and performs better than data-based methods. They test against one reinforcement learning based baseline in the Highway-env simulator.", "questions": "- What are the more precise concepts of knowledge-driven human driving that inspire this?\n- Instill knowledge-driven capabilities sounds very vague. Methods that acquire experiences from real-world dataset covers all learning-based methods depending on what you mean with acquire. The abstract could be more concrete, it's hard to take away anything apart from that a LLM seems to do decision making while following a continuous learning scheme. \n\nThe language is hard to follow and the citations do not seem to support the claims well. In the Introduction, the sentence \"This phenomenon inevitably leads to the marginal performance of data-driven methods.\" is one example for a broad claim without enough support in citations. There are autonomous cars driving in cities today with vision algorithms which are data-driven. They do not show marginal performance. The citations for this claim are one work describing a methodology to categorize corner cases in three common sensor modalities, so not very related, and the second citation \"Chen et al. 2022\" seems to be a catch all \"survey of surveys\" which is a large list of autonomous driving surveys with some added, partially trivial, thoughts. \n\nOther examples where statements are too broad and hard to understand are: \"Furthermore, this task is particularly formidable and expensive for autonomous driving systems due to the complex challenge of iterating diverse and unpredictable driving scenarios.\" What is this sentence supposed to say? The authors should heavily simplify their language to deliver their points clearer. Formidable and expensive can be understood in many different ways and distracts from the core argument the authors want to make.\n\nClaims that the knowledge-based system is how a human drives can not be supported by the current state of research and not by the citations in this paper. I think the paper would benefit from not making the claim that they imitate a system in humans but limit themselves to saying, they designed a framework to include an LLM in a continuous learning setting where it outperforms certain other approaches. \n\nPlease add enough details from Johnson et al. 2019 to understand the vector similarity on an idea level. Make the paper more self-complete.\n\nWhat is the data the LLM is trained on? If the training data contains driving situations from several countries, how to make sure it is following the appropriate traffic rules?\n\nWhat are the 5 human crafted experiences and why are they needed?\n\nFigure 7 a) could be easily replaced by a table to save space. \n\nIt is a bit unsatisfactory to have only a comparison against one baseline which was re-trained on this particular data. Is there no standard scenario on Highway-Env or another RL-based approach that was already applied to Highway-env to compare against? I could not find one myself so I don't see this as a downside in my rating but I think it would make the paper stronger if the authors could find a way to compare against more than one baseline.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "The idea to use an LLM for scenario understanding and decision making in driving is very interesting. The authors have proposed a decent suggestion for integration and practically showed that it works.\n\nThe figures help getting a good top-level overview of the modules. Some parts are missing like how does the correction module work which is only an arrow in Figure 5?", "weaknesses": "The language could be clearer, less vague and heavily simplified to make the arguments easier to understand. \n\nThe evaluation against a single self-trained RL baseline makes it hard to estimate the performance. Since it seems there is no limit to the perception of the own agent, another fair comparison would be a simpler approach where a statistical or rule-based approach would have all information of all cars and drive. Without having a state-of-the-art RL method that is already optimized on highway-env it is hard to see if the performance gain comes from the proposed method or from the failure to adopt the RL method on the task.\n\nConceptually it is hard to imagine right now how this is supposed to drive in real time. Is the video sped up or slowed down? It is a challenge of the last decade how to get convolutional networks fast enough to be usable in a car. The computation challenges are not discussed at all. What is the reaction time of this and is execution speed a bottleneck?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)