Paper: SwapTransformer: Highway Overtaking Tactical Planner Model via Imitation Learning on OSHA Dataset (ICLR-2024)

Reviewer comment (Experiments):
The baselines used in this paper are all variants of the author's own methods. There are no comparisons against other state-of-the-art methods.

Author response:
We appreciate the reviewer for this concern. We would like to mention that most of the SOTA methods are focused on motion forecasting using end-to-end methods. Also, our main contribution to this work is to propose an approach for imitation learning especially behavioral cloning rather than focusing on motion forecasting. Moreover, we are focusing on modular autonomous driving stacks where high-level decision-making is the interface between the model and the low-level controller. One of the benefits of using a modular architecture is that we can apply it to real vehicles equipped with the Travel Assist controller. These make our approach and dataset unique compared to other works.

Reviewer comment (Novelty):
The proposed method can only do as well as the rule-based method. What are the advantages of the proposed method over the rule-based method?

Author response:
The rule-based method serves as a surrogate for a human driver in our problem context to help us prove the Swap Transformer concept. However, the rule-based driver, even in the simplest scenarios, requires a  significant effort to develop and can't be extended to more intricate states (city traffic, entering and exiting highways, etc.). During simulation, the rule-base played an instrumental part in the data collection process, thus enabling our model to undergo training. In real-world scenarios, the rule-based driver is replaced by an expert driver.

Reviewer comment (Evaluation):
There is no variance or confidence interval on the main results in Table 2 over repeated trials or different network seeds. Since some of the improvements are quite small compared to baselines (1-2%), it would be useful to know if the improvement is repeatable / consistent.

Author response:
we thank you for this concern you brought up. We agree with you in this regard. With this respect, we added average and standard deviation to Table 2 to explicitly address that. As shown in the updated Table 2, the standard deviation is reported for speed difference and time-to-finish. Those confidence intervals for the SwapTransformer (ours) for both metrics are relatively in range compared to other evaluated models.

Reviewer comment (Experiments):
I have some concerns on the training procedure: Was any form of validation set or early stopping used to help prevent overfitting and tune hyperparameters? Were multiple seeds used? The manuscript solely states that all models were trained for 300 epochs.

Author response:
We appreciate the reviewer for this concern. We are trying to address each of the points separately. At the beginning of this research, we did offline validation and early stopping. However, offline validation does not necessarily show the performance of the model in all possible situations at the inference time. We observed that those models that have very good metrics in validation, do not perform well in a live interaction in simulation. Also, comparing an ongoing trained model with a validation set needs to match the prediction with the ground truth exactly at the same time-slot, this comparison with validation does not seem reasonable for planning approaches. However, for research on vehicle motion forecasting, validation sets are pretty useful for early stopping or having a good understanding of the training phase. We added this information to the paper to Section A.6. In fact, we tried different versions of training before going into evaluation and inference. We did hyperparameter tuning on different aspects. For instance, on the naive transformer model, we tried different versions of ResNet such as 18, 50, and 101 and we realized that there are not that many differences between different versions. We also tried a customized vision encoder instead of Resnet but the performance was not as good as Resnet 18. On WandB, we tried different sweeping mechanisms on different hyperparameters such as the learning rate of the optimizer and scheduler. In addition to that, we tried a multi-head approach vs a single-head approach. In a multi-head approach, the optimizer, loss function, and schedulers were considered independent. Based on the training metrics (loss values), the result was better compared to a single-head approach. However, the generalization on inference was better using a single-head approach. Moreover, we tried different numbers of layers and neurons for each single head of the model to find the right number of layers to have a good performance and also to keep the model light enough to be able to run the model in the vehicle. All of these hyperparameters tuning were applied on the naive transformer before applying the auxiliary tasks and swap feature. This information was not mentioned in the paper due to space limitations. We appreciate your concern, and we added this information to Section 4.2 (possibly appendix Section A.6). Regarding the multiple seed concern (if I understand it correctly), the seed was mentioned for the evaluation. The seed was used for the simulation evaluation and it was fixed between different models. This guarantees to have the same pattern of randomness when we compare different models together. If the seed is mentioned as a concern to make sure that different models are initialized with same random values at the beginning, that is correct.

Reviewer comment (Evaluation):
I assume the Ours method in Table 2 is Transformer + Aux + Swap? Quite often it seems that the addition of the auxiliary loss or swap operation actually decreases performance but for some reason when combined together (ours approach) performance improves beyond the original transformer architecture. Can some intuition be given for this behavior?

Author response:
That is correct, Our method means Transformer + Aux + Swap, we fixed that in the paper to explicitly mention it.

Reviewer comment (Evaluation):
There does not appear to be any intuition as to why the swap operation would improve performance. This is further confusing by the fact that the transformer + swap architecture seems to harm performance versus the transformer architecture alone (Table 2).

Author response:
We are trying to address both 3 and 4 here:

The combination of auxiliary tasks and dimension-swapping exhibits synergistic effects, enhancing the model's performance by leveraging both strategies concurrently. However, when individually trained—either with Transformer+Aux or Transformer+Swap—both models exhibit a tendency to underfit within the 300-epoch computational budget. In the swap transformer, we 'swap' dimensions between the time horizon and features. If trained longer, Transformer+Aux or Transformer+Swap by themselves would outpace the Transformer in evaluation. Hence, the Swap mechanism and auxiliary tasks are complementary in terms of faster convergence during training because of the additional signals they provide for each other. We mentioned these details in Section~4.3.

Reviewer comment (Evaluation):
I feel that the evaluation metrics (speed difference, time to finish, overtake ratio) do not holistically evaluate the method. Although the proposed method finishes faster and overtakes more often, is it possible that some of these were risky maneuvers (i.e. network predicted overtake when it shouldn’t have)? I think it would have been useful to show the model’s test prediction accuracy versus the expert to alleviate these concerns (i.e. is it switching lanes when it should be).

Author response:
The concern raised regarding the evaluation metrics (speed difference, time to finish, overtake ratio) is valid. While our proposed method demonstrates faster completion and more frequent overtakes, there is a legitimate question about the potential inclusion of risky maneuvers, particularly if the network predicted overtakes when it shouldn't have. In our simulations with the Travel Assist controller, safety measures inherently prevent such risky actions, such as abrupt lane changes. Addressing your suggestion about comparing our model's predictions to a rule-based approach, it's important to note that the rule-based method introduces a level of randomness in behavior, making it challenging to precisely evaluate against our approach. The rule-based driver incorporates diverse behaviors, including maintaining safe distances within a lane, during lane changes. While we recognize the value of comparing our model's predictions to a rule-based system, the inherent variability in rule-based behavior complicates a straightforward assessment. We opted for the chosen evaluation metrics due to this complexity.

Reviewer comment (Experiments):
Were the 50 test/evaluation episodes consistent across all models for fairness? I assume each was randomized but I would hope that the random conditions were repeated for all models.

Author response:
We appreciate your valid concern. The same seed is used for all evaluation between different models. As you mentioned, we tried to put evaluation fairly between models. The randomness that we could apply at the inference/eval time accounted for traffic behavior like the density of other cars, placement of other cars, and velocity behavior. All of these randomness were defined based on some average and standard deviation (same mean and std for all of them).