Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: Instead of building a module-based ADS, this work aims to achieve off-road self-driving end-to-end. Concretely, a driving policy trained in a simulator can be directly transferred into the real world. The core technique enabling zero-shot transfer is a module built on RCAN, which can bridge the gap between the images captured in the real world and those in the simulator. Therefore, this method can avoid the dangerous data collection process of trial and error. Also, this method can be trained efficiently. With this method, the policy trained in 48 hours is comparable to a rule-based policy, which requires a huge human budget.
strengths_and_weaknesses: **Strength**:
1) This work builds a complete pipeline to show that training in a simulator and transferring the learned policy to the real world is feasible
2) The idea that mapping RGB images from different domains (simulation/reality) to the segmentation domain is interesting. This also enables the zero-shot transfer.
3) It is exciting to see the real vehicle experiment since it is difficult to do hardware experiments and a lot of researchers only validate their idea in simulators.

**Weakness**
1) The originality of the method is limited. It seems that this work transfers RCAN to the driving task with a small modification.
2) The standard deviation is missing in all tables. If repeated experiments are conducted, please report std. 
3) The real-world demonstrative video is not striking enough. It shows unsmooth trajectories, and the setting is simple. Actually, learning to drive via RL is achieved several years ago: https://wayve.ai/blog/learning-to-drive-in-a-day-with-reinforcement-learning/.  This video is more attractive than the demo in the supplementary material. In my opinion, the core contribution of this work is to build such a hardware system and demonstrate that zero-shot sim-to-real autonomous driving can be completed. Therefore, the quality of trajectories in demo video is the most important part. If smooth and long driving trajectories or more complex scenarios can be shown, I will consider increasing my score.


typo:
line 156-157: two two approaches
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: The method is an incremental work built on RCAN, and thus originality is limited. Although hardware experiments are conducted, the driving trajectories are not of high quality. 
issues: N/A
review_title: null

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper addresses the problem of developing an
end-to-end controller for autonomous driving. In
particular, the authors focus on training a
controller in simulation (where no real data is
required) and then bridging the sim-to-real gap
via a segmentation network (also trained on
simulation data). Evaluation is provided in both
an online and offline setting.
strengths_and_weaknesses: Overall, the paper is well written and the problem
is well motivated. Bridging the sim-to-real gap
would enable the development of controllers that
do not require real data and yet are exhaustively
tested in simulation. I also appreciate the
comparison between the proposed system and the
classical software stack approach. However, I do
have a few comments that can be used to improve
the paper's quality.

My first comment has to do with the comparison
with the classical approach. The authors make a
very strong claim that "our trained RL agent can
perform as well as a sophisticated, model-based 
autonomous driving stack". However, the comparison
is performed in a very controlled setting, with
short tasks, where the goal is known, and in 
environments that are likely too similar to the
simulation environment and hence not exhibiting 
significant distribution shift. In order to
justify this claim, I think the authors need to
illustrate cases where the proposed method fails,
especially in comparison with the classical
method. Of course, experiments where the classical
method fails would be equally interesting.

Furthermore, the contribution over the RCAN method
seems unclear to me. It seems that the proposed
method is almost identical to RCAN in terms of
the conversion to a "canonical" representation of
each observation, the main difference being that
the controller is trained using TD3. In my opinion,
this is not a major contribution. If the authors
focus more on the off-road application, perhaps
that would separate it more from the RCAN work.

Related to the above, it is not clear to me that
the GAN training adjustment (i.e., the image
convolution) is needed. In particular, the original
GAN training algorithm is known to be unstable,
and multiple improvements have been proposed, e.g.,
Wasserstein GANs. The authors are encouraged to
explore this issue further.

Minor: 

I do not understand why the past trajectory is in
R^{10,3}. If I understand correctly, the trajectory
consists of 10 time steps, but I do not get where
the 3 comes from -- it seems that state is in R^2.

I also do not see why the action space consists
of 5 action tuples. Are these 5 tuples predictions?
If so, the authors are encouraged to comment on this
design choice, as well as on the significance on
the number 5.

A lower-level comment I have is that multiple
symbols in the paper are not properly explained.
For example, L_eq equation on page 4 has several
unintroduced terms, e.g., G_x, G_d, etc.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are not well addressed
summary_of_recommendation: I am recommending a weak reject because I don't think the contributions are significant enough. In particular, it seems that the improvements over RCAN are minor and that the GAN training improvements are not justified through experiments and comparisons.
issues: 1. I would like to see more justification for the proposed image convolution approach to training the GAN. In particular, Wassertein GANs are known to overcome many of the GAN training stability issues, so that would be one alternative approach to compare with. For example, the authors can look at this paper and code within: https://proceedings.neurips.cc/paper/2017/file/892c3b1c6dccd52936e27cbd0ff683d6-Paper.pdf

2. The authors are encourage to discuss in greater detail the differences with RCAN and/or provide an experimental comparison.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: 
The paper proposes sim2seg, which aims to surpass the sim-to-real gap for off-road autonomous driving without using real-world data. The proposed method converts simulated RGB images of terrain to segmentation and depth masks and subsequently enables converting real-world images. The segmented images are then used to train a navigation policy using reinforcement learning. The proposed approach is proven efficient in reaching goals and performs obstacle avoidance during zero-shot transfer. The paper also investigates the various factors necessary for sim2seg’s performance.
strengths_and_weaknesses: Strengths:

+ The paper provides a clean and straightforward method for sim-to-real learning for off-road autonomous driving. 
+ The paper is well written and easy to follow for a broad robotics audience. 
+ The proposed method is evaluated on a physical robot platform. It appears to cross the sim-to-real visual gap successfully.
+ The authors have comprehensively detailed the approach's limitations, providing insights into the future direction of this research.


Weakness:

+ It will be helpful to include more details of the approach in the Introduction section. 
+ It might also be helpful to understand the approach from a high level by including a diagram describing the various modules of the proposed approach.

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: The paper presents a method for solving the challenging sim-to-real transfer problem without using any real-data. The paper is well written and easy to follow. The experimental validation on the physical hardware platform makes the paper a good contribution.
issues: + I would request the authors provide more details of their approach in the Introduction section.
+ Please check the indentation in line 207.
+ Please include details on the method's runtime, how the approach can adapt when sensor data is obtained at lesser frame rates, and if the approach is robust to the various lighting conditions.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper presents an autonomous end-to-end off-road driving approach which can be trained using only simulation data. To enable good sim2real transfer, segmentation maps are proposed as an intermediate representation and trained in an adversarial fashion inspired by RCAN. The segmentation maps can be easily generated in simulation and are shown to provide a suitable level of abstraction. The authors show experimental evidence that the method performs at least as good as a classical autonomous driving stack and that it can be deployed closed-loop on a real-world autonomous offroad vehicle.
strengths_and_weaknesses: The main strengths of the paper are its real-world experiments and evaluations that demonstrate the applicability of the proposed method. Using a true-to-scale demonstrator is a great contribution, in contrast to using only small RC cars or simulation only experiments. 

On a more detailed level, the insights into what a suitable trajectory parametrization is for learning are interesting as they are not only specific to offroad driving but most likely a general finding. It would be interesting to see (as possible future work) if it is possible to design a neural controller that is directly capable of controlling the vehicle. Such a controller might inherently have advantages because it has access to all the information in the network and it thus might learn ‘careful’ driving when there is high uncertainty, or similar human-like behaviour. 

The detailed description of the method in Sec. 3 is of great value to the reader and the community. It is well written and informs the reader about the key details of the proposed Sim2Seg approach.

However, since the experiments are a key part of this work, some clarifications might be required to further strengthen the results:
- It seems that the real-world rollouts collected for evaluation purposes are collected by having a human operator drive the vehicle around. It is stated that those generated trajectories are an “efficient” way to reach some specified goal. First, what does efficient mean in this context? Energy efficient? Time optimal? Second, since the reference for an “efficient” performance is generated by a human, there is no guarantee that there might not be an even more efficient solution, w.r.t to the efficiency-metric. That also implies that comparing the (possibly suboptimal) human trajectory with the results of the autonomous driving method might lead to a less accurate evaluation because the autonomous vehicle might be able to find a better trajectory.
- What are the ground truth trajectories $\tau$ and how are they recorded? Is a trajectory simply a list of positions associated with a time-stamp? Is it recorded via GPS only, RTK-GPS, GPS+IMU, ...
- In the evaluation (sec. 5.1) two metrics are used: L2 and GT, whereas the L2 metric only measures the difference between the trajectory endpoint and the specified goal. It is unclear what exactly the GT value characterizes. Why is cosine-similarity between two trajectories (assumed to be positions and 2D or 3D space) a good measure? Why is no classical measure like average trajectory error used for the evaluation? It would be very convincing to also provide the trajectory errors as well.

In the introduction and related work it is mentioned that a problem of RL methods is that it is very difficult to cover all possible environments, lighting cases, etc and that there are often “visual discrepancies”. The presented method learns the segmentation maps from directly from images. It would be very interesting to add details as to why the intermediate representation “segmentation map” is helpful to improve the robustness when unseen imaging conditions are observed. 

Minor points:
- What low-level controller has been used to track the rollout? 
- in line 300, there is a typo: “ultiple” instead of “multiple”
- Fig 3 is presumably done using wrapfig and the next page looks messed up.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: The paper presents an approach that is based on RCAN work, but adapts the methodology to the new segmentation map intermediate representation. The work is well executed, although some details of the experimental evaluation need to be clarified, especially with respect to the choice of the optimal trajectory that the method is compared to. The inclusion of real-world experiments is really nicely done, especially considering the fact that the autonomous vehicles used in this work is true-to-scale. 
At this point, I recommend "weak reject" because some parts of the experimental evaluation are not perfectly convincing. If the paper is improved a bit more, it will be a valuable contribution to the community.
issues: The authors should address the following issues to further strengthen their paper:
- Explain what a trajectory is and how the “GT” metric is computed.
- Motivate why the “GT” metric is a good choice compared to more classical metrics such as average trajectory error.
- Add details what the “efficient” trajectories executed by the human operator are and in what sense they are efficient. 
- Explain why the difference to such an “efficient” trajectory is a good metric to evaluate the approach. Is it not possible, that the trajectory found by Sim2Seg is better?
- Discuss why adding the segmentation map intermediate representation helps overcome the issue that during training not all possible scenarios can be observed. Overall, the segmentation is also machine-learned and thus might suffer from the same problems as a direct end-to-end approach.


**

** AFTER REBUTTAL PERIOD **

**

The authors have addressed my comments in a satisfactory way: Specifically:
- _Explain what a trajectory is and how the “GT” metric is computed_: The explanations in the experimental section address this point fully.
- _Motivate why the “GT” metric is a good choice compared to more classical metrics such as average trajectory error_: As the authors have included the ATE as well in the revised version, this point is void. The ATE very clearly show that the proposed sim2seg method outperforms the baselines.
 - _Add details what the “efficient” trajectories executed by the human operator are and in what sense they are efficient.  Explain why the difference to such an “efficient” trajectory is a good metric to evaluate the approach. Is it not possible, that the trajectory found by Sim2Seg is better?_: The definition what efficient is (close to human driver), is still questionable but for the goal of this work, it makes sense.
- _Discuss why adding the segmentation map intermediate representation helps overcome the issue that during training not all possible scenarios can be observed. Overall, the segmentation is also machine-learned and thus might suffer from the same problems as a direct end-to-end approach._ The argument made by the authors that the segmentation map has much smaller dimension than the original image space is a good point. Furthermore, the authors have added ablations in section 5.4 which strengthen this argument.
review_title: null

Meta-review: metareview: The paper is well written and the addressed problem is well motivated. The approach of using the segmentation map as an intermediate representation to solve the sim2real problem is interesting and the evaluation on a real, true-to-scale demonstrator vehicle was seen as a strong point and valuable contribution by all reviewers. In contrast, there was much more of a divided assessment regarding the originality of the approach and the significance of the contributions. In particular, the authors should address the concerns of reviewers McEn and Vqbb that the presented method seems to be a rather straightforward transfer of RCAN to the autonomous driving domain, and that progress over the original RCAN method remains unclear.  

Other issues to address:
* No standard deviation is given in any of the results tables
* An illustration of failure cases of both the proposed method the classical method would be helpful to strengthen claims
* It is unclear that the proposed GAN training adjustment is really justified, especially in light of other GAN modifications that address training stability such as Wasserstein GANs
* More details on the method's runtime, robustness to slower sensor sampling frequencies to various lighting conditions would be helpful
* More details on trajectory definition and metrics used for evaluation would be appreciated
* Please discuss why adding the segmentation map intermediate representation helps to overcome the issue that not all possible scenarios can be observed during training, given that the map is itself learned from data

=== 

Post rebuttal/discussion update:

Thank you for providing a detailed rebuttal. In the internal discussion, reviewers agreed that their concerns were addressed properly and expressed the feeling that the paper adds value to the community.
confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Poster)