Paper: DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model (ICLR-2024)

Reviewer comment (Novelty):
The model relies on behavior cloning for end-to-end driving. This is the first end-to-end method tried out. The limitation is very well-known, e.g. can not handle distribution drift. For more information, please see

End-to-end Autonomous Driving: Challenges and Frontiers
Li Chen, Penghao Wu, Kashyap Chitta, Bernhard Jaeger, Andreas Geiger, Hongyang Li

Author response:
Thanks for the information. We have citated this paper in our revised manuscript. At this stage, DriveGPT4 is for open-loop vehicle control. The behavior cloning issue mentioned in this question should be carefully studied in the close-loop control, which is the research direction in the next stage.

Reviewer comment (Evaluation):
The paper relies on ChatGPT to evaluate on vehicle action description, action justification, question and answering. ChatGPT can exhibit well-known bias such as position bias, style bias. Given the data is generated by ChatGPT and the baseline ADAPT does not use ChatGPT, ChatGPT evaluation could be very well biased in favor of DriveGPT4 simply because it prefers its own style.

Author response:
The fixed QAs are not generated by ChatGPT, and they are directly from the BDD-X dataset. For fixed QAs, ChatGPT is used to generate more patterns of the question, but the answer stays fixed and not altered. During evaluation, the generated answers are not in ChatGPT style, but in BDD-X style. Thus ChatGPT evaluation does not show biased favor for this, and the results can be reliable. 

In addition, compared with conventional evaluation metrics, the ChatGPT metric is much more powerful since it can measure complicated semantic meanings/logic/etc. Evaluation with ChatGPT/GPT4 is commonly conducted in many recent papers about multimodal LLM [1,2]. Therefore, leveraging ChatGPT for evaluation should be a good way for comparison.

Reviewer comment (Experiments):
The action prediction task should be evaluated with strong end-to-end autonomous driving baselines.

Author response:
DriveGPT4 is proposed for interpretable end-to-end autonomous driving, thus it should be able to predict the text explanations as well as vehicle actions at the same time. ADAPT is the previous SOTA for this problem, thus DriveGPT4 is mainly compared with ADAPT, instead of other baselines that only focus on either interpretability or end-to-end vehicle control.

Reviewer comment (Experiments):
For architectural choices, why not just fine-tune Video-LLaMA with the visual instruction tuning dataset based on the BDD-X dataset? Do you employ position encoding for video tokens?

Author response:
Yes, positional encoding is used in DriveGPT4 for video frames. DriveGPT4 is basaed on a video understanding framework Valley, which is similar to Video-LLaMA but presents better video understanding ability. Thus, DriveGPT4 follows Valley for model design and tuning.

Reviewer comment (Experiments):
In ablation study, it will be great to see the contribution of global features by removing F0^G ⊕ F1^G ⊕ ... ⊕ FN^G.

Author response:
Thanks for the suggestion. The results have been added into the ablation study.  Please refer to Table. 6 in the revised manuscript.

Reviewer comment (Experiments):
The paper uses Yolo-8 for object detection, I wonder if the authors can comment on whether open-set detectors like grounding DINO could be better.

Author response:
We tried the mentioned open-set detectors for object detection during data generation. But since we only care about limited object classes, e.g., vehicle, pedestrians, traffic light, we did not obvious notable difference between YOLO and open-set detectors. YOLO is already effective and efficient enough for our specific applications.

Reviewer comment (Presentation):
The training stages, methods, datasets, and tasks are scattered. It would help if they can be summarized in a table. Furthermore, the architecture figure should have information on the training stages, which part is frozen or trainable.

Author response:
Thanks for the suggestion. We have modified the manuscript accordingly. Please refer to the revised manuscript for details.

Reviewer comment (Experiments):
DriveGPT4 dataset is based on BDD-X dataset. DriveLLM2023 (https://github.com/OpenDriveLab/DriveLM) dataset augments NuScene with QA + Scene Description from the perspective of perception, prediction and planning with Logic. Can the model be evaluated on this dataset as well? Specifically how well DriveGPT4 performs on high level planning decisions compared with DriveLLM?

Author response:
DriveLLM does not release any code or code at this stage yet. As long as DriveLM releases their data, we will adapt our model on it and conduct additional evaluations.

Reviewer comment (Experiments):
As the model is an end-to-end autonomous driving model, the ability to make driving plans or control predictions is critical. However, from the model design and the ablation experiments, it seems that the model might just simply extrapolate the input control signals. The authors should also include more possible baselines for this task (for example, a simple transformer with the same input as the proposed model which is directly trained on the prediction task). Besides, only predicting the speed and steer angles might not be enough to claim it as an end-to-end autonomous driving model.

Author response:
Thanks for this good question. In the revised manuscript, history information is removed from the input. Now DriveGPT4 makes prediction only based on video frames and input human questions. Please refer to Table.3 in the revised manuscript for the new results. From the results, we can see that DriveGPT4 still outperforms the previous SOTA ADAPT (8-frames for fair comparison).

Reviewer comment (Experiments):
The paper claims that the proposed model still has the general multi-modal conversation ability. Then the author should evaluate it on the general multi-modal benchmarks to validate this.

Author response:
Sorry for the overclaim. In the original paper, we meant that DriveGPT4 is more flexible than ADAPT to answer questions in more patterns raised by humans. Thus, this ability is the additional question-answering ability, which is evaluated in Table 4 of the manuscript. DriveGPT4 is specially designed for autonomous driving scenarios, and it is not trained for other general multimodal problems, e.g., image understanding, video understanding, etc. Overclaims have been removed from the revised manuscript.

Reviewer comment (Evaluation):
For the experiments about interpretability, the proposed method shows consistently inferior performance in terms of standard metrics. It is true that the classic metrics might have certain problems, but so does the ChatGPT-based evaluation. As the training data is partially generated by ChatGPT, ChatGPT might prefer the responses similar to its own generated ones during evaluation.

Author response:
The answers to fixed QA pairs are directly from BDD-X dataset, and the model is trained to answer in BDD-X label style instead of ChatGPT style. Besides, no matter what style of the generated response, ChatGPT compares it with the BDD-X label, so evaluation bias should not happen here. 

Compared with conventional evaluation metrics, the ChatGPT metric is much more powerful since it can measure complicated semantic meanings/logic/etc. Evaluation with ChatGPT/GPT4 is commonly conducted in many recent papers about multimodal LLM [1,2]. Therefore, leveraging ChatGPT for evaluation should be a good way for comparison. 

**References**

[1] Liu, Haotian, et al. "Visual instruction tuning." NIPS, 2023.

[2] Luo, Ruipu, et al. "Valley: Video Assistant with Large Language model Enhanced abilitY." arXiv:2306.07207, 2023.

Reviewer comment (Experiments):
For the control prediction task, DriveGPT4 takes historical control signals as input while other models do not. From the ablation study, the results of 'No control signals' are close to ADAPT-32, suggesting that the contribution comes from the history information rather than from the LLM, which is considered unfair.

Author response:
**4. For the control prediction task, DriveGPT4 takes historical control signals as input while others do not. It can be observed from the ablation study, that the results of 'No control signals' are close to ADAPT-32. The contribution comes from the history information, rather than LLM, which is unfair.** 

Thanks for this good question. In the revised manuscript, history information is removed from the input. Now DriveGPT4 makes prediction only based on video frames and input human questions. Please refer to Table.3 in the revised manuscript for the new results. From the results, we can see that DriveGPT4 still outperforms the previous SOTA ADAPT (8-frames for fair comparison).

Reviewer comment (Evaluation):
It remains an open question whether flexible answers can help real-world applications of autonomous driving.

Author response:
**5. It is an open question if flexible answers can help real-world applications of autonomous driving.** 

Yes. Flexible questions can be used for better human-user interaction, by providing flexible interface for users. So that users can better understand the vehicle action, and this improves the vehicle transparency and trustworthiness. Besides, flexible QAs can train the model to better understand the driving scenarios, which is helpful for the fixed-QAs as well.

Reviewer comment (Novelty):
This is not the first work on interpretable end-to-end autonomous driving, as several prior works are already listed in the related works. Even the claim of language-based interpretability is not precise since there are multiple types of interpretability. The reviewer believes it is fair to say this is LLM-based.

Author response:
**6. It is not the first work on interpretable end-to-end autonomous driving** 

Sorry for the typo. This problem has been fixed in the revised manuscript.

Reviewer comment (Novelty):
From the start, it is questionable whether interpretability is indeed the unsolved problem hindering the commercialization and development of autonomous driving. As stated in the first paragraph of the introduction, the issue with popular modular-based methods is not interpretability.

Author response:
**7. From the very beginning, is interpretability indeed the unsolved problem that hinders the commercialization and development of autonomous driving** 

Yes. One critical problem that prevents commercialized autonomous driving is that autonomous vehicles are black boxes. The decisions and actions of autonomous vehicles cannot be understood by humans, which raises severe ethical and law issues. Once the autonomous vehicle leverages deep learning modules, the transparency and trustworthiness of it is always an inevitable problem in the way of commercialized autonomous driving.  LLM could be a potential tool to open this black box by training the LLM to explain for itself when it makes a decision to control the vehicle.

Reviewer comment (Novelty):
The technical contribution is limited. While there are contributions from instruction tuning for dataset generation, the video tokenization part is based on Valley, and the action part is very similar to existing works such as RT-2.

Author response:
- Thanks for the suggestion. We will add it into the comparison in a later stage considering limited time and resources. Valley is proved to be better than Video-LLaMA for general video understanding, thus in the past we directly adapt Valley for our task.

Reviewer comment (Experiments):
Why not use ground truth from the BDD dataset? The detection results from YOLOv8 could be inaccurate, and the reviewer wonders if there is a truck in the provided figure (Table 1).

Author response:
**8. Why not use ground truth from the BDD dataset? The detection results from YOLOv8 could be inaccurate. I am wondering if there is a truck in the provided figure (Table 1).** 

The original BDD-K dataset only has ground truth object bounding boxes for keyframes, which is not enough for our data processing. Thus we need to detect objects in each frame by ourselves. Thanks for pointing out the inaccurately detected truck. We further tuned the settings of YOLO8 and fixed the problem. Inaccurate detection happens sometimes and cannot be completely avoided. However these incorrect detected objects do not seriously affect the QA pairs generated by ChatGPT. The overall quality of the additional QA pairs generated by ChatGPT is assured.

Reviewer comment (Reproducibility):
How are bounding box coordinates normalized, and in which coordinates are they normalized?

Author response:
**9. How bounding box coordinates are normalized? In which coordinates are they normalized?** 

The bounding boxes (x_min,x_max,x_max,y_max) are normalized based on the image size. This follows LLaVA [1] for instruction tuning data generation.

Reviewer comment (Experiments):
If the model can generalize to video games, it would be worth trying to generalize to CARLA and evaluating it in a closed-loop manner.

Author response:
**10. If the model can generalize to video games, it is worth trying generalizing to CARLA and evaluating it in a closed-loop manner.** 

At this stage, DriveGPT4 does not work on close-loop control. The problems of applying LLM for close-loop control, such as drifting, consistent prediction, efficiency, are not well studied  yet. This is a promising future research topic, and we will investigate it in the next step.

Reviewer comment (Presentation):
The short name 'DriveGPT4' may not be very appropriate, as the core method is finetuned from LLaMA 2 and does not use GPT4.

Author response:
**11. The short name 'DriveGPT4' is not very appropriate, as the core method is finetuned from LLaMA 2 and does not use GPT4.** 

Thanks for the comment. The name “DriveGPT4” does not derive from its utilized LLM. “GPT” here indicates we want to make the autonomous vehicle able to chat with human users in a similar way to ChatGPT/GPT4. Besides, ChatGPT is used for data engineering in our project. Therefore, the model is named DriveGPT4.

Reviewer comment (Experiments):
A notable limitation of the paper is its focus on a dataset comprised solely of salient images related to autonomous driving. This does not accurately represent real-world conditions, where sensors capture a multitude of irrelevant or non-critical images. This selective approach raises questions about the model's susceptibility to overfitting and its reliance on human oversight for attention guidance. Future work could benefit from evaluating the fine-tuned LLM on a more diverse set of images to assess the model's generalization capabilities beyond the curated dataset.

Author response:
At this stage, this paper is an early research to study the possibility of using LLM for autonomous driving, thus real-world conditions with complicated scenarios are not the focus of this paper. We try to make this model work on the specific dataset first before it can be adapted to more generalized autonomous driving situations.

Reviewer comment (Experiments):
The paper may be perceived as over-ambitious in its claims, particularly given the title "DriveGPT4," which implies a comprehensive solution for Level 4 autonomous vehicles. However, the proposed system focuses narrowly on generating control signals, neglecting other essential aspects like perception, planning, and behavior. This narrow focus limits the model's utility in real-world applications. Additionally, the paper does not sufficiently demonstrate the model's ability to address long-tail or zero-shot cases, further constraining its practical relevance. While DriveGPT4 shows promise for specific tasks like auto-labeling, its current form falls short of making it a broadly applicable solution in the autonomous driving ecosystem.

Author response:
Thanks for the comment. In the original manuscript, there do exist some overclaims, and they have been fixed in the revised manuscript. The name “DriveGPT4” is inspired by Mini-GPT4, where “4” represents multimodality, and it does not indicate Level-4 autonomous driving ability. We have highlighted this delaration in the revised manuscript.