Sim-to-Real via Sim-to-Seg: End-to-end Off-road Autonomous Driving Without Real Data
Authors: John So
Amber Xie
Sunggoo Jung
Jeffrey Edlund
Rohan Thakker
Ali-akbar Agha-mohammadi
Pieter Abbeel
Stephen James
Venue: CoRL 2022 Poster (2022)
Keywords: Sim-to-Real
Reinforcement Learning
Autonomous Driving

Abstract: Autonomous driving is complex, requiring sophisticated 3D scene understanding, localization, mapping, and control. Rather than explicitly modelling and fusing each of these components, we instead consider an end-to-end approach via reinforcement learning (RL). However, collecting exploration driving data in the real world is impractical and dangerous. While training in simulation and deploying visual sim-to-real techniques has worked well for robot manipulation, deploying beyond controlled workspace viewpoints remains a challenge. In this paper, we address this challenge by presenting Sim2Seg, a re-imagining of RCAN that crosses the visual reality gap for off-road autonomous driving, without using any real-world data. This is done by learning to translate randomized simulation images into simulated segmentation and depth maps, subsequently enabling real-world images to also be translated. This allows us to train an end-to-end RL policy in simulation, and directly deploy in the real-world. Our approach, which can be trained in 48 hours on 1 GPU, can perform equally as well as a classical perception and control stack that took thousands of engineering hours over several months to build. We hope this work motivates future end-to-end autonomous driving research.

Sim-to-Real via Sim-to-Seg: End-to-end Off-road Autonomous Driving Without Real Data
John So*t Amber Xie*t
Sunggoo Jung'
Jeffrey Edlund
Rohan Thakkeri
Ali Agha-mohaamadii Pieter Abbeel Stephen James'
UC Berkeley
+NASA Jet Propulsion Laboratory "Equal Contribution
Abstract: Autonomous driving is complex, requiring sophisticated 3D scene un- derstanding, localization, mapping, and control. Rather than explicitly modelling and fusing each of these components, we instead consider an end-to-end ap- proach via reinforcement learning (RL). However, collecting exploration driving data in the real world is impractical and dangerous. While training in simulation and deploying visual sim-to-real techniques has worked well for robot manipu- lation, deploying beyond controlled workspace viewpoints remains a challenge. In this paper, we address this challenge by presenting Sim2Seg, re-imagining of RCAN [1] that crosses the visual reality gap for off-road autonomous driving, without using any real-world data. This is done by learning to translate random- ized simulation images into simulated segmentation and depth maps, subsequently enabling real-world images to also be translated. This allows us to train an end-to- end RL policy in simulation, and directly deploy in the real-world. Our approach, which can be trained in 48 hours on GPU, can perform equally as well as a classical perception and control stack that took thousands of engineering hours over several months to build. We hope this work motivates future end-to-end au- tonomous driving research. Code and videos available our project page.
Keywords: Sim-to-Real, Reinforcement Learning, Autonomous Driving
1 Introduction
Sim
Real
Simplifying large autonomous driving software stacks, which are usually composed of 3D scene understanding, localization, mapping, and control, is promising goal. While these stacks can indeed perform well in range of scenarios, they do suffer from error propagation through each of the modules, and tend to require large engineer- ing overhead. However, attempting to solve autonomous driving problem in purely end-to-end manner, where observations are mapped directly to actions, also has its downfalls. For one, these methods usually data in- tensive, and particular for reinforcement learning (RL), collecting exploration driving data in the real world is im- practical and dangerous.
Sim2Seg
π
To overcome the data burden, large-scale simulations can be employed to collect experience from a large num- ber of parallel agents. However, then have to con- sider the visual and dynamic discrepancies between the simulation and reality. Sim-to-real transfer approaches, such as domain adaptation [2, 1] and domain randomiza- tion [3, 4, 5, 6] exist for this reason. These however, have mostly been applied in tasks with a fixed camera view- point, such manipulation tasks where the camera usually points down towards a bin [2, 1] or table [3], or faces wall [4, 5J.
Figure 1: Sim2Seg learns a mapping be- tween randomized images and segmen- tation maps for zero-shot transfer.
6th Conference on Robot Learning (CoRI. 2022), Auckland, New Zealand.

---

Unlike these robot manipulation setups, autonomous navigation with full-scale vehicles in off-road settings are far less controlled, often featuring aspects such as drastically different lighting, glare, dust, long visual horizons, and complex backgrounds. In addition, unlike in standard autonomous street driving, the vehicle must traverse off-road terrains, which have less uniform dynamics and may include natural obstacles such as bushes, rocks, bumps, and more, which may not be observed during training.
In this paper, we investigate how similar sim-to-real techniques used for smaller scale robotic do- mains, such as robot manipulation, can be scaled to the significantly more visually challenging do- main of off-road autonomous driving of a large vehicle. To this end, we propose Sim2Seg, sim-to- real method designed with the challenges of off-terrain autonomous navigation in mind. Combined with a deep RL policy, Sim2Seg is the first work to effectively employ primarily visual sim-to-real transfer for rof- autonomous driving-
Contributions
We highlight our contributions below:
We improve the discriminator component within RCAN [1] by convolving the output segmen- tation maps to produce feature maps that are then fed to a discriminator to evaluate.
We explore suitable action modes that encourage safe trajectory proposals without diminishing the model's capability.
We show that our trained RL policy can perform well as sophisticated, model-based au- tonomous driving stack.
We show the first primarily visual sim-to-real transfer method for end-to-end RL autonomous driving in complex off-road terrains that requires no real world data.
2
Related Work
Model-based Autonomous Driving Most off-road autonomous driving approaches center around scene understanding approaches. Due the challenges of natural obstacles such bushes, trees, and rocks; uneven terrain; and various static and dynamic uncertainties [7], the robot must con- stantly assess the traversability of the terrain. Geometry-base based methods involve constructing ter- rain map [8, 9] from depth measurements from sensors such as LiDAR, stereo cameras, etc. This terrain map is used to generate a traversability cost by performing stability analysis, using features like surface normals, maximum or minimum height of the terrain, etc., which can be used by motion planning and control algorithms to plan vehicle's actions [10, 11, 12, 13]. Papadakis [14] provides a survey of several other off-roat driving algorithms. In lieu of these methods, we focus end-to-end autonomous driving directly from pixels, avoiding these engincering layers.
End-to-End Autonomous Driving End-to-end autonomous driving at large remains an unsolved problem: most approaches have narrowed their scope to urban environments [15, 16), as it is most relevant for everyday human transportation. In addition, most methods simplify the problem of general navigation by assuming static environments [17] and real-world datasets [15], which is limiting factor for more difficult terrains and navigation tasks like off-road autonomous driving.
Since urban environments produce unique challenges of multi-agent interactions and following traf- fie rules, many end-to-end approaches greatly simplify the environment to focus on navigation on roads. For instance, Chu et al. [18] and Nair al. [19] both reduce the environment static, toy car racing environments. While Kendall et al. [20] trains a visual RL policy end-to-end in simula- tion. they focuses explicitly on lane-following in a static environment and requires real-world policy rollouts for few-shot policy transfer. Offline real-world data has also been crucial to many methods. Ram [21] trains end-to-end in CARLA [22] an urban driving simulator, but bridge the visual to--ll gap. requires real-world data in order to enhance simulator images. VISTA [15] focuses on street navigation and relies upon a data-driven simulator, which synthesizes new viewpoints of scene based on offline data, making it difficult to quickly simulate new scenes. Osinski et al. [17] uses real-world images and ground-truth semantic segmentation maps to learn segmentations. We instead focus on goal-conditioned policy for the explicit task of off-road navigation and obstacle avoidance, and zero-shot policy transfer to the real world via learning a shared representation from a diverse set of high-fidelity simulations.
2

---

Sim-to-Real in the Visual Domain Perception forms the basis of many tasks, from smaller-scat robotics tasks large-scale vehicles. To ensure consistent perception across simulators and real- world images, popular technique used is domain randomization [23, 4, 16], in which input obser- vations are randomized to prevent overfitting to simulator images, ensure adaptability to a variety of conditions, and encourage extraction of meaningful features such object shapes and locations.
In the case that real-world data easily accessible, domain adaptation popular method used to extract consistencies across the two domains. Pixel-level domain adaptation, for instance, improves pixel-level consistencies by restylizing simulator images [24, 25]. For robotics tasks involving ob- jects that are critical to the scene, additional losses are penalized: RetinaGAN ensures object con- sistencies using pretrained object detector [26]. and RL-CycleGAN [27] penalizes differences in Q-values. Feature-level domain adaptation learns shared features across both domains [28, 29, 30]. notable that many of these environments are based around grasping and other robotics tasks, which have fixed camera viewpoints, controlled environments, and relatively easier data collection
processes in comparison with off vehicles.
The most direct analog of our approach is RCAN[]], which approaches visual sim-to-real translation in robotic grasping via learning a shared RGB canonical space using Pix2Pix model [31]. While directly inspired by RCAN, our method has distinct differences, which we highlight: (1) RCAN converts visual inputs canonicalized RGB images. Sim2Seg converts visual imputs to one-hot segmentation maps, which are different, relatively lower dimensional, modality. (2) RCAN dis- criminators take paired RGB images. To improve discriminator stability, our discriminator takes in learned features of input images and segmentation maps. (3) RCAN approaches robotic manip- ulation tasks. Sim2Seg approaches autonomous driving in the much more difficult environment of offroad environments, which presents additional challenges as described in Section 1.
3 Method
In this section, we detail our method, Sim2Seg, which is summarised in Figure It consists of two primary components: (1) inspired by RCAN [1], we train Sim2Seg model that translate ran- domized RGB images from simulation into shared canonical form, which we define semantic segmentation map. (2) We then train a goal-reaching policy within this canonical representation to navigate in off-terrain environments. During inference, we use frozen pretrained Sim2Seg model to perform zero-shot transfer on real-world images.
3.1 Simulation
We create several simulation environments using the Unity engine. Unity fulfills several key desider- ata, notably high-fidelity visual observations and dynamics, open-source vehicle components and scenes [32], and the ability to apply custom domain randomization techniques. Unity also integrates well with RL training with the Unity ML-Agents toolkit, which provides Gym interface for train- ing [33]. For our purposes, we further modify ML-Agents to support instance-level parallelism. allowing us train multiple agents per built executable.
To maximally train our policy to a variety of off-road environments and generate a diverse dataset of simulator data, select 3 differentt simulated scenes - dubbed Meadow [34], Landscapes [35], and Canyon [36]- with semantically diverse visual environments. During training, the policy trains simultaneously all 3 environments, leveraging the most out of the simulation training phase and ensuring adaptability to a variety of scenes.
3.2 Sim2Real via Sim2Seg
To bridge the visual gaps between simulators and real-world data, we use Sim2Seg model con- vert randomized image domains into our chosen canonical form, a segmentation map consisting of six classes: trees/bushes, ground, sky, rocks, road, and logs. Visualization colors can be found in Appendix B. We believe this is useful for oof-road vehicles because it simplifies the unnecessary de- tails, textures, and colors of images and identifies different types of obstacles, which gives important information on obstacles and areas to avoid (see Figure 2).
3

---

Randomized Simulation
Zero-shot Real
To bridge the visual sim-to-real gap, apply combinations of texture randomization, camera position and intrinsic randomization, and lighting color and direction randomization to vary- ing scenes in Unity, and learn mappings to ground truth segmentation maps.
Figure 2:
Domain Randomization For data collection, we collect 200k pairs of paired RGB images, seg- mentation maps, and depth data per environment, using a random policy. We apply domain ran- domization to textures of all objects, lighting color and direction, camera field-of-view and camera position. For the best map to the canonical state, we train a Sim2Seg model per each environment and separate Sim2Seg model for the combined dataset. During training, we use each environment's Sim2Seg model, which avoids distribution shifts between the Unity environments, At real-world test time, we also consider a Sim2Seg model trained all environments achieve optimal performance.
3.3 Sim2Seg Training
Following RCAN [1]. we use conditional GANs (cGAN) with the U-Net Architecture [37, 31] to translate pairs of randomized simulation images into their canonical segmentation map representa- tion. During train time, we use paired simulation data (I., Te, Ma, mo)i where I. is a random- ized RGB simulation image, Te is the canonicalized segmentation map, and m. is the canonicalized depth map, and m. the obstacle mask.
Unlike RCAN [1], we modify our objective to adapt to the task of autonomous driving. While RCAN predicts the canonical image, segmentation map, and depth map, simplify problem to predicting the segmentation map, which is simultaneously our canonical image; and the depth map, which is used as auxiliary.
Thus, to encourage visual similarity, we optimize the following objective:
(I) where leg, is the cross-entropy loss, leg, the LI loss, my my is the element-wise product, and Xe, Aa the weighting of the losses. We denote G (x.) to be the segmentation output and Ga(x.) to be the depth output of randomized simulation image Zg.-
Leg (G) Ezy. [Aoteg_ (G, (2.), x.)) + Xaleas (Ga(Ia); ma)]
Adversarial Objective We employ a discriminator D(x) which outputs the probability that the RGB image and segmentation map is a pair from the simulation dataset. Because our canonical output consists of probabilities for segmentation classes, we experimented with few approaches to use to featurize the combination of RGB and segmentation map.
LGAN D) Er, ... (log ((f(f, xe)] Ez, [log(1 Df f(G((x)))
(2)
Because our Sim2Seg model outputs segmentation class logits while the ground truth segmentation maps are one-hot encodings, discriminating directly on image and segmentation pairs immediately trivial. In addition, because segmentation classes are discrete, class and pass the gradient back to the generator. To remedy this issue, we initially attempted two two approaches: (1) sampling with gumbel-softmax [38] and (2) approximately the arg-max with soft arg-max [39], both of which are differentiable. Despite this, GAN training remained unstable, discriminating between real and fake pairs easy, likely due noisy gumbel-softmax samples and soft-argmax values in areas of uncertainty.
cannot sample segmentation
Thus, to circumvent these issues, we choose instead to separately convolve the simulation image and segmentation maps to an equal number of channels to use as paired features for the discriminator to evaluate [40]. This allows for gradient flow without leading to unrealistic segmentation maps.

---

We find this to be effective for Sim2Seg, as it circumvents the problems of different modalities and instead allows for stabler discrimination in feature space, and we present ablations in Appendix E.
3.4 End-to-end Driving with RL
Using Sim2Seg, we train short-horizon navigation policy using RL; specifically, we consider goal-conditioned policy conditioned on visual and odometry data. Sim2Seg is inherently compat- ible with any visual learner, but we choose to TD3 [41] with image augmentation [42] backbone RL algorithm. TD3 an off- algorithm, and so enables goul-conditionin and rela- belling in training pipeline. We detail specific details about our architecture and hyperparameters in Appendix B.
We adjust the TD3 backbone policy with our augmented visual observations and state information for our domain. Before inference, Sim2Seg translates the input RGB image Or E (332256,266) into a one-hot, C-class segmentation map Ct E (R,22,66,266) Additionally, we condition on egocentric past trajectory Ty E 12,0,3 the current 2D state Sa IR2 and the goal state $y R2. The encoded segmentation map and state information are flattened and concatenated to achieve a final representation.
Observation Space
Action Space We parameterize the policy's actor as an LSTM which outputs series of 5 action tuples, each consisting of steering angle 0 E - and acceleration E [0, 1J). The policy then performs a temporal rollout of the actions to form a trajectory, which is consumed by lower- level controller for vehicle commands. This parameterization comes with several benefits; proposing rollouts instead of vehicle torque commands mitigates the dynamics sim-to-real gap. Furthermore, proposing multiple continuous actions allows the policy to propose waypoints and consider more coherent short-term plans, while giving a notion of safety when executing in the real world. We perform an analysis of different action parameterizations in Appendix 5.3.
We design a simple reward function to best achieve goals while producing safe behaviors, notably obstacle avoidance:
Reward Function
(5g: Sa. a) >g
( (Sg. s.) + AuTu($gs 8a, a) + Agra(a) + Aere(s,a,s))
where r is goad-condi sparse reward, ru is upright reward intended to incentivize smooth terrains (i.e. less rocky and flatter terrains), re is steer penalty intended to discourage bang-bang control, and Te is collision penalty:
100
iflls, - 8 ll2 <
ro(s,a,s') = {-1
if collision otherwise
To (5g) 8a)
[-1
otherwise
0
Tu Tu Sa, a) = 180
r- (ap,a) = -111112
where 0 is the max of the roll and pitch angles between the vehicle body and world frames, and collisions are detected via Unity. To supplement training, we additionally leverage Hindsight Expe- rience Replay, [43]: by relabeling sampled trajectories (T, 8a, 8g) with goal achieved later in the same trajectory (T, Sa, 8a,8% sax) during training, we obtain more signal from the sparse reward rg.
RGRD
Al
3.5 Real-world Vehicle Integration
For real-world evaluation, we use Polaris $4 1000 Turbo RZR equipped with a variety of perception sensors, in- cluding an inertial measurement unit (IMU), stereo cam- era pairs, and LiDARs (see Figure 3). Note that we only use monocular RGB camera when deploying. The Po- laris also includes computing resources and a "drive by wire" system to autonomously control the vehicle (ac- celerate, change gears, brake, steer). NeBula [44] has been integrated with the vehicle and utilizes a ROS stack with planners for varying goal horizons. To integrate our
Figure 3: Polaris hardware with sensor suite. a) raw image, (b) cropped image input Sim2Seg (c) segmented result using on-board computer.
5

---

model with the Nebula autonomy system, we further test in the ARL simulator, a photorealistic simulator of the Meadow environment already integrated with the ROS stack.
During real-world deployment, we plan in an iterative closed loop at 10 Hz, executing actions until the policy receives sufficient information. The policy stores buffer of received messages, and only executes when timestamps are synchronized within 100ms. Output trajectories are consumed by PID controller, which translates trajectories into low level vehicle commands.
4 Experimental Setup
Helendale Desert
b) Arroyo Seco Trails
Preliminary: Classical Baseline We con- struct the classical baseline by leveraging the the Nebula software stack [44]. Our spe- cific implementation uses localization estimates from a LiDAR's inertial odometry to fuse depth scans temporally which are used estimate traversability cost using a settling-based geo- metric analysis [45]. This traversability map is used by kinodynamic motion planner [46] to generate collision-free trajectories that are fol- lowed by lower level PID-based tracking controller.
Figure 4: Real-world off-road evaluation data gathered by our platform at a) Helendale, Mojave Dessert and b) Arroyo Seco Trails, Altadena
Offiin Riall Transfer Evaluation To evaluate our policy's ability to perform visual sim- to-real transfer, we evaluate our policy using offline rosbag datasets of real-world, manual rollouts with the Polaris vehicle. Given rosbags of vehicle interactions, {or Sg. Tp:T"). where o is the observation at t = 0. sy is the real achieved goal at some t > 3 the vehicle's real-world trajectory from
first construct dataset D
seconds, Tp is the past trajectory up t = O. To calculate T" and Tp. we transform the recorded odometry into the vehicle's 's egocentric frame. We are then able to perform the model's 's full inference pipeline using o, &g. and Tp to produce predicted trajectory т. Following this method, we construct two different real-world datasets from environments semantically and visually different from the set of training environments, which features the vehicle navigating winding trails, rocks, and vegetation (see Figure 4).
O, and
Online Sim-to-Real Transfer Evaluation We also perform qualitative evaluation of closed-loop control with our policy fully in the real world using the system described in Section 3.5 in the Arroyo environment.
5
Experimental Results
We seek to answer two primary questions: (1) Is Sim2Seg able to efficiently reach goals and perform obstacle avoidance during zero-shot transfer? (2) What factors are most necessary for Sim2Seg's performance?
5.1 Goal Reaching and Obstacle Avoidance
We evaluate the policy's is capacity for obstacle nav- igation through comparison against real world roll- out r", under the assumption that r* is efficient trajectory to reach sy while avoiding obstacles. Fol- lowing the offline evaluation setup described in Sec- tion 4, we define efficient as (1) reaching 3, a timely manner (sy is the offset in f = 3 seconds), and (2) avoiding obstacles (the human driver pur- posefully avoids collisions).
Figure 5: Timelapse of experimental demon-
stration of zero-shot transfer of goal follow- ing while avoiding obstacles.
We compare both the normalized distance (denoted as L2) between the requested goal and the trajectory endpoint, and the angle difference between 10 evenly sampled points from r* and (denoted as GT) We also include absolute trajectory error (denoted as ATE), a classical measure of trajectory alignment. Additionally, to account for the possibility that reaches the goal more

---

directly than r", we also introduce GTG. which is the GT metric described above between the shortest distance sy and т. lower L2 and GTg indicates that the policy outputs trajectories that reach the goal (goal reaching); lower GT and ATE indicates that the policy aligns well with the human trajectory (obstacle avoidance).
To understand the gains provided by our method, we consider Sim2Seg's performance against sev- eral baselines detailed below. The results are listed in Table 1.
Random: An untrained policy, i.e., random actions.
Domain Randomization: We train an identical policy without the shared representation space learned through Sim2Seg, instead leveraging only our domain randomization methods.
Classical:
We consider existing classical autonomous stack described in Section 3.5.
Data
Method
GT
ATE +
GT |
L2
Random
0.354
0.002
1.693
0.411
0.343
0.005
0.691
1 0.003
Arroyo
DR
0.292 0.026
0,945
= 0.209
0.280
0.022
0,452 + 0.008
Classical
0.070
0.812
0.087
0.653
Sim2Seg (ours)
0. 14147 + 0.027
0.471
= 0.012
0.163
+ 0.019
0.287
+ 0.024
Random
0.318 0.003
2.128
0.029
0.350
0.751
d: 0.005
Helendale
DR
0.305 0.039
1.546
0.374
0.238
0.042
0,501
4: 0.028
Classical*
0.106
2.498
0.200
0.868
Sim2Seg (ours)
0.158 1: 0.007
0.747
0.381
0.210
1: 0.013
0.383 1: 0.026
Table I: Offline evaluation metrics of our method against broad set of baselines, including do- main randomization (DR) and random actions (Random). Standard deviation shown across 5 seeds. *Note, Classical is not an RL solution, and so there are no seeds. Classical takes in LiDAR, whereas our other methods only takes in image and odometry inputs.
Furthermore, we demonstrate zero-shot transfer by deploying the algorithm a real-world off-road environment passenger-size vehicle shown in Figure 5. In the included example, the policy identifies the rock as an obstacle, and creates a trajectory to navigate around it. Further videos can be found on the project webpage.
5.2 Ablation: Simulator and Real-World Data
We additionally compare the performance of our model against the sources of training data. Under 'I Sim Env', we train only on one environment (Meadow), instead of all three simulator environ- ments. Under 'Real- World', we additionally introduce a limited offline dataset of transitions, in- spired by [1]. To construct this dataset, we invert the temporal rollout described in Section 3.4 on rosbags consisting of trajectories in a separate area of the Arroyo environment to obtain transitions fo, Sy, Tp J. and add these to the replay buffer. During training, we sample 50% of our batch from the real-world transitions. Notably, this setup amenable to using large amounts of offline, non-segmented data; RGB images are still segmented zero-shot according to a frozen pretrained Sim2Seg model. We experiment with different amounts of real-world data, including O, 400, and 1200 transitions. Reducing the number of simulation environments doesn't seem to significantly harm model performance; we hypothesize that this could be large part due to Meadow and Ar- royo's 's similarities relatively flat landscapes. However, introducing even little 400 transitions produces improved results and allows for even smoother transfer to the real world, shown in Table 2.
Ablation
GT J
ATE 1
GTG
L2 |
Sim2Seg Baseline
0.147 1. 0.027
0.471 1: 0.012
0.163 1 0.019
0.287
0.024
Sim Env
0.189 + 0.020
0.445 + 0.077
0.174 + 0.021
0.270
0.068
400 Real World Transitions
0.149 + 0.030
0.489
0.057
0.142
+ 0.026
0.266
0.153
1200 Real World Transitions
0.146 + 0.040
0.448
+: 0.156
0.141
+: 0.041
0.432
0.060
Action Step
0.309 1: 0.135
0.910
+: 0.213
0.234
1: 0.060
0.388
0.076
Action = 10 Steps
0.158 J: 0.045
0.489
0.216
0.180 1: 0.046
0.314 0.058
Table 2: Incorporating worl- data is a scalable method to improve policy performance. Standard deviation shown across 5 seeds.
7

---

5.3 Ablation: Trajectory Parameterization
We additionally experiment with the number of actions A our policy autoregressively outputs. means the policy only takes one action at time; A = 5 means the policy, parameterized by an LSTM, generates 5 actions and takes 5 environment steps at time. We evaluate the performance of short- horizon policy (A 1), medium-horizon policy (A = 5), and long-horizon policy (4 = 10). Our short-horizon policy performs noticeably worse, whereas our medium and short-horizon policies perform similarly well. This seems to suggest that learning longer-range behaviors can be important.
Figure 6: Qualitative zero-shot transfer results of our final Sim2Seg model. Our simulator envi- ronments range from a grassy meadow to a rocky canyon environment, yet with sufficient domain- randomization, we can achieve strong performance in the unseen and quite different real-world environments. Row consists of observations we consider particularly challenging. Sim2Seg is able to generalize harsh lighting and extreme shadows.
6 Conclusion and Limitations
In this paper, we have investigated transferring end-to-end off-road autonomous driving policies from simulation to reality. To accomplish this, we have presented Sim2Seg, which converts ran- domized simulated RGB images into segmentation masks, and subsequently enables real-world im- ages to also be converted. Given that our driving policy is trained in these canonical segmentation environments, possible run policies trained simulation directly in the real world. When evaluating on real-world data, we are able to perform equally well as classical perception and control stack that took thousands of engineering hours over several months to build.
We differ from RCAN in that the discriminator on the visual features of the generated segmentation mask (rather than the canonical RGB image in RCAN). Given that our driving policy is trained in these canonical segmentation environments, it is possible to run policies trained in simulation directly in the real world. We show that only is this superior training directly on domain randomization, but also that we are able to perform equally as well as classical perception and control stack that took thousands engineering hours over several months to build. To the best of our knowledge, this is the most ambitious visual sim sim-to-real transfer accomplished to date. We hope this inspires future sim-to-real work on even more ambitious domains.
Limitations Improving the quality of the Sim2Seg model is a key priority; qualitative analysis of segmentation maps shows difficulty with shadows (which may be perceived new objects) and low- contrast scenes (such as the small shrubs depicted in Figure 6). We hope that such shortcomings can be addressed by introducing stronger shadow randomization techniques, and generally increasing the number of environments trained on.
Currently, Sim2Seg's 's policy is trained on horizons of 20 meters in length, and thus is only effective at this range. An effective horizon on the range of 50 meters or more would likely be much more practical, especially when navigating higher speeds. Our policy currently is capable of avoiding obstacles short range as demonstrated needing to traverse around a forest to reach goal, has not been tested. Training with temporally and spatially longer horizons inherently introduces more complexity to RL; this is an active area of research for
Section 5.1; however, long horizon reasoning, such as
One particular area of interest is trajectory proposals terms of waypoints. Waypoint proposals would allow to explore other methods such as Bezier curve parameterizations and discrete trajec- tory libraries [47], and would provide further proof that Sim2Seg can generalize different policy architectures.

---

Acknowledgments
This work was supported by DARPA RACER and Hong Kong Centre for Logisties Robotics. We would like to thank Valentin Ibars for his help running real-world experiments.
References
[1] S. James, P. Wohlhart, M. Kalakrishnan, D. Kalashnikov, A. Irpan, J. Ibarz, S. Levine, R. Hadsell, and K. Bousmalis. Sim-to-real via sim-to-sim: Data-efficient robotic grasping via
adaptation networks. In IEEE Conference on Computer Vision and Pattern Recognition, pages 12627-12637, 2019.
[2] K. Bousmalis, A. Irpan, P. Wohlhart, Y. Bai, M. Kelcey, M. Kalakrishnan, L. Downs, J. Ibarz, Pastor, K. Konolige, S. Levine, and V. Vanhoucke. Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping. IEEE Intl. Conference Robotics and Automation, 2018.
[3] Tobin, Fong, A. Ray, J. Schneider, W. Zaremba, and Abbeel. Domain randomization for transferring deep neural networks from simulation to the real world. In IEEE Intl. Conference on Intelligent Robots and Systems, 2017.
[4] S. James, A. J. Davison, and E. Johns. Transferring end-to-end visuomotor control from sim- ulation real world for multi-stage task. Conference on Robot Learning. 2017.
[5] J. Matas, S. James, and A. J. Davison. Sim-to-real reinforcement learning for deformable object manipulation. Conference on Robot Learning, 2018.
[6] S. James, M. Bloesch, and A. J. Davison. Task-embedded control networks for few-shot imi- tation learning. In Conference on Robot Learning, pages 783-795. PMLR, 2018.
[7] U. Muller, Ben, E. Cosatto, B. Flepp, and Cun. Off-road obstacle avoidance through end-to-end learning, Advances in neural information processing systems, 18, 2005.
[8] Fankhauser, M. Bloesch, C. Gehring, M. Hutter, and R. Siegwart. Robot-centric elevation mapping with uncertainty estimates. In International Conference Climbing and Wulking Robots (CLAWAR), 2014.
[9] A. Horuung K. M. Wurm, M. Bennewitz, Stachniss, and W. Burgard. Octomap: An efficient probabilistic 3d mapping framework based on octrees. Autonomous robots, 2013.
[10] K. Otsu, G. Matheron, S. Ghosh, O. Toupet, and M. Ono. Fast approximate clearance evalua- tion for rovers with articulated suspension systems. Journal of Field Robotics, 77857776 2019. ISSN 15564967. doi: 0oi:1001102//e 10.1002/rob.21822
[11] K. Konolige, M. Agrawal, M. R. Blas, Bolles, B. Gerkey, J. Sola, and A. Sundaresan. Mapping, navigation, and learning for off-road traversal. Journal of Field Robotics, 26(1): 88-113, 2009.
[12] D. D. Fan, K. Otsu, Y. Kubo, A. Dixit, J. Burdick, and A.-A. Agha-Mohammadi Step: Stochastic traversability evaluation and planning for risk-aware off-road navigation. arXiv preprint NiXi:22033.02828. 2021.
[13] P. Krüsi, P. Furgale, M. Bosse, and R. Siegwart. Driving on point clouds: Motion planning, trajectory optimization, and terrain assessment generic nonplanar environments. Journal of Field Robotics, 34(5):940-95) 2017.
[14] Papadakis. Terrain traversability analysis methods for unmanned ground vehicles: survey. Engineering Applications of Artificial Intelligence, 26(4):1373-1383 2013. ISSN 09521976. doi: 101 10166/..1gaa/pppee www..engapai.2213..0m 2013. 006.
URL. http: doi oro/ 1016/ engappai.

---

[15] A. Amini, I. Gilitschenski. J. Phillips. J. Moseyko, R. Banerjee, S. Karaman, and D. Rus. Learning robust control policies for end-to-end autonomous driving from data-driven simula- tion. IEFE Robotics and Automation Letters, 5(2): 143:11143-1150, 2020. doi:10.1 109/.RA.2020. 2966414.
[16] Y.J. R. Chu, T.-H. Wei, J.-B. Huang, Y.-H. Chen, and I.-C. Wu. Sim-to-real transfer for miniature autonomous car racing, 2020. URL https://arxis: arg/abs/20111 01661006..
[17] B. Osinski, A. Jakubowski, P. Milos, P. Ziecina, C. Galias, S. Homoceanu, and H. Michalewski. Simulation-based reinforcement learning for real-world autonomous driving, 2019. URL https://arxdv..rg/// httpg//ab///91111 12905.
[18] Y.R. Chu, T. Wei, J. Huang, Y. Chen, and I. Wu. Sim-to-real transfer for miniature autonomous car racing. CoRR, abs/201 105617, 2020. URL http:s::///xvvvvr///// atrg/abs/20111 0561777711........
[19] U. R. Nair, S. Sharma, U. S. Parihar, M. S. Menon, and S. Vidapanakal. Bridging sim2real gap using image gradients for the task of end-to-end autonomous driving, 2022. URI https: // araai.i..v. arxiv.org/ abs 2248107481......
[20] A. Kendall, J. Hawke, D. Janz, P. Mazur, D. Reda, J.-M. Allen, V.-D Lam, A. Bewley, and
A. Shah. Learning drive in day, 2018. URL UuR.https://arzivv.rr///// 1001/00117000.....
[21] Ram, Jan 2022. URL https: www.llaac....o http:sss//ppf/022122222-2-2
pdf.
[22] A. Dosovitskiy, G. Ros, F. Codevilla, A. M. López, and V. Koltun. CARLA: an open ur- ban driving simulator. CoRR, 10./171,,,3938, 2017. URL httpt://axxii..oo wwt.aarwww.....agboooo/ / 03938.
[23] Sadeghi and S. Levine. CAD2RL: Real single-image flight without single real image. In Robotics: Science and Systems, 2017.
[24] K. Bousmalis, N. Silberman, D. Dohan, D. Erhan, and D. Krishnan. Unsupervised pixel-level domain adaptation with generative adversarial networks, 2017.
[25] Z. Yi, H. Zhang, Tan, and M. Gong. Dualgan: Unsupervised dual learning for image-to- image translation, 2018.
[26] D. Ho, K. Rao, Z. Xu, E. Jang, M. Khansari, and Y. Bai. Retinagan: An object-aware approach to sim-to-real transfer, 2020. URL https://arxiv org/ ab//a/as//11111 03148.
[27] K. Rao, C. Harris, A. Irpan, S. Levine, J. Ibarz. and M. Khansari. Rl-cyclegan: Reinforcement learning aware simulation-to-ret CoRR, hbs/2000.0900,, 2020. URL htttstss///axxi abs/2006. 09001.
[28] Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, and V. Lempitsky. Domain-adversarial training of neural networks. 2015. doi: 10040/..55///.. 1505.07818. URL https://arxiv www..org/aaaaa.aa abg//55/1505 /105. 07818.
[29] B. Sun, J. Feng, and K. Saenko. Return of frustratingly easy domain adaptation, 2015.
[30] N. Sünderhauf, O. Brock, W. Scheirer, R. Hadsell, D. Fox, J. Leitner, B. Upcroft, P. Abbeel, W. Burgard, M. Milford, and Corke. The limits and potentials of deep learning for robotics, 2018.
[31] O. Ronneberger, P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmentation. CoRR, abs/1505.04597, 2015. URL http::///axxw.or//aassee .4tps 04597.
[32] Realistic buggy kit. realistic- brrissic--bugy-kiii -ig------ Accessed: 2022-02-13.
https://assetstore unity. commppackage/ttoos//pyy.....
[33] A. Juliani, V.P. Berges, E. Teng, A. Cohen, J. Harper, C. Elion, C. Goy, Y. Gao, H. Henry,
M. Mattar, and D. Lange. Unity: general platform for intelligent agents, 2020.
[34] Meadow environment dynamic nature. https: httpss//asssetstoree unity. wwt.ccoo/pccceeecc pomm/paaaages3a// wwwpraaddddeeeiioeeeeeeee wwtprtttddynnniinnnnnn 132195. Accessed: 2022-03-02.
10

---

[35] Real landscapes valley forest. https://assetsttrr. www.tuntntttttteeeeoee com/packages//3//
Accessed: 2022-04-18.
[36] Castle valley collection 1. attp:://aaseesttrre.niiii unity. wtw.:uuttttooo//accagg/////////
hwr.:aanttt/aaaaaeee////earrrrrr 2022-04-18.
Accessed:
[37] Isola, J.-Y. Zhu, Zhou, and A. A. Efros. Image-to-image translation with conditional
adversarial networks, 2016. URL puttps://arxee http://wxw......bbb////aaaa wrwg//////....////000000 07004.
[38] E. Jang, S. Gu, and B. Poole. Categorical reparameteriization with gumbel-softmax, 2017.
[39] C. Finn, X. Y. Tan, Y. Duan, T. Darrell, S. Levine, and Abbeel. Learning visual feature spaces for robotic manipulation with deep spatial autoencoders. CoRR, abs/1509.061 2015. URI. http::tt/// / /ttp//xxvvv.//////// wwwpeiiivorrr/aeeee wtwg//a//////ooo 06113.
[40] A. Abdollahi, B. Pradhan, Gite, and A. Alamri. Building footprint extraction from high resolution aerial images using generative adversarial network (gan) architecture. IEEE Access, 8, 11 2020. doi: 100../1/000AACCSS.2000000 Www.:CCEEESSS2...33..00 3038223
[41] S. Fujimoto, H. Van Hoof, and D. Meger. Addressing function approximation error in actor- critic methods. Inil. Conference on Machine Learning, 2018.
[42] D. Yarats, Fergus, Lazaric, and L. Pinto. Mastering visual continuous control: Im- proved data-augmented reinforcement learning, 2021. URL attps://ai http///axxvvv.. org/abs/ 2107. 09645.
[43] M. Andrychowicz, Wolski, A. Ray, J. Schneider, Fong. P. Welinder, McGrew, Tobin, P. Abbeel, and W. Zaremba. Hindsight experience replay, 2017. htt.tttss////ii...ot org/
abs/1707 01495.
[44] A. Agha, K. Otsu, B. Morrell, D. D. Fan, R. Thakker, A. Santamaria- S.-K. Kim, A. Bouman, X. Lei, J. Edlund, et al. Nebula: Quest for robotic autonomy in challenging envi- ronments; team costar at the darpa subterranean challenge. arXiv preprint arXiv:2 1470, 2021.
[45] R. Thakker, N. Alatur, D. D. Fan, Tordesillas, M. Paton, K. Otsu, O. Toupet, and A.-a. Aghammohammadi. Autonomous off-road navigation over extreme terrains with perceptually- challenging conditions. In International Symposium on Experimental Robotics, pages 161- 173. Springer, 2020.
[46] G. Williams, A. Aldrich, and E. A. Theodorou. Model predictive path integral control: From theory to parallel computation. Journal of Guidance, Control, and Dynamics, 40(2):344-377, 2017.
[47] N. Dashora. D. Shin, D. Shah, H. Leopold, D. Fan, A. Mahha-Mohammdii.. N. Rhinehart, and Levine. Hybrid imitative planning with geometric and predictive costs in off-road environ- ments, 2021. URL https: :ttps://arraa //arxiva/vvv////////a org/ aaaaaaas//111 /2111 10948.
[48] I. Gulrajani, Ahmed, M. Arjovsky, V. Dumoulin, and A. Courville. Improved training of wasserstein gans, 2017. URL httth::t////aaee htttpp://aaxiv.orgee.. wwg.abs//////eoooooooooooeeeeeet tabs//104. 00028.


Reviews:
[{"review_id": "q4KPxZJGf0t", "reviewer": "Reviewer_McEn", "summary": "This paper addresses the problem of developing an\nend-to-end controller for autonomous driving. In\nparticular, the authors focus on training a\ncontroller in simulation (where no real data is\nrequired) and then bridging the sim-to-real gap\nvia a segmentation network (also trained on\nsimulation data). Evaluation is provided in both\nan online and offline setting.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Overall, the paper is well written and the problem\nis well motivated. Bridging the sim-to-real gap\nwould enable the development of controllers that\ndo not require real data and yet are exhaustively\ntested in simulation. I also appreciate the\ncomparison between the proposed system and the\nclassical software stack approach. However, I do\nhave a few comments that can be used to improve\nthe paper's quality.\n\nMy first comment has to do with the comparison\nwith the classical approach. The authors make a\nvery strong claim that \"our trained RL agent can\nperform as well as a sophisticated, model-based \nautonomous driving stack\". However, the comparison\nis performed in a very controlled setting, with\nshort tasks, where the goal is known, and in \nenvironments that are likely too similar to the\nsimulation environment and hence not exhibiting \nsignificant distribution shift. In order to\njustify this claim, I think the authors need to\nillustrate cases where the proposed method fails,\nespecially in comparison with the classical\nmethod. Of course, experiments where the classical\nmethod fails would be equally interesting.\n\nFurthermore, the contribution over the RCAN method\nseems unclear to me. It seems that the proposed\nmethod is almost identical to RCAN in terms of\nthe conversion to a \"canonical\" representation of\neach observation, the main difference being that\nthe controller is trained using TD3. In my opinion,\nthis is not a major contribution. If the authors\nfocus more on the off-road application, perhaps\nthat would separate it more from the RCAN work.\n\nRelated to the above, it is not clear to me that\nthe GAN training adjustment (i.e., the image\nconvolution) is needed. In particular, the original\nGAN training algorithm is known to be unstable,\nand multiple improvements have been proposed, e.g.,\nWasserstein GANs. The authors are encouraged to\nexplore this issue further.\n\nMinor: \n\nI do not understand why the past trajectory is in\nR^{10,3}. If I understand correctly, the trajectory\nconsists of 10 time steps, but I do not get where\nthe 3 comes from -- it seems that state is in R^2.\n\nI also do not see why the action space consists\nof 5 action tuples. Are these 5 tuples predictions?\nIf so, the authors are encouraged to comment on this\ndesign choice, as well as on the significance on\nthe number 5.\n\nA lower-level comment I have is that multiple\nsymbols in the paper are not properly explained.\nFor example, L_eq equation on page 4 has several\nunintroduced terms, e.g., G_x, G_d, etc.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "I am recommending a weak reject because I don't think the contributions are significant enough. In particular, it seems that the improvements over RCAN are minor and that the GAN training improvements are not justified through experiments and comparisons.", "technical_quality": "Fair: Significant additional evidence required to support the claims.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are not well addressed", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "2: Unsure, difficult to assess.", "reviewer_expertise": "4: The reviewer is confident but not absolutely certain that the evaluation is correct"}, "extra_text": {"issues": "1. I would like to see more justification for the proposed image convolution approach to training the GAN. In particular, Wassertein GANs are known to overcome many of the GAN training stability issues, so that would be one alternative approach to compare with. For example, the authors can look at this paper and code within: https://proceedings.neurips.cc/paper/2017/file/892c3b1c6dccd52936e27cbd0ff683d6-Paper.pdf\n\n2. The authors are encourage to discuss in greater detail the differences with RCAN and/or provide an experimental comparison."}}, {"review_id": "dYsbaVEO5S", "reviewer": "Reviewer_aEoJ", "summary": "\nThe paper proposes sim2seg, which aims to surpass the sim-to-real gap for off-road autonomous driving without using real-world data. The proposed method converts simulated RGB images of terrain to segmentation and depth masks and subsequently enables converting real-world images. The segmented images are then used to train a navigation policy using reinforcement learning. The proposed approach is proven efficient in reaching goals and performs obstacle avoidance during zero-shot transfer. The paper also investigates the various factors necessary for sim2seg’s performance.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n+ The paper provides a clean and straightforward method for sim-to-real learning for off-road autonomous driving. \n+ The paper is well written and easy to follow for a broad robotics audience. \n+ The proposed method is evaluated on a physical robot platform. It appears to cross the sim-to-real visual gap successfully.\n+ The authors have comprehensively detailed the approach's limitations, providing insights into the future direction of this research.\n\n\nWeakness:\n\n+ It will be helpful to include more details of the approach in the Introduction section. \n+ It might also be helpful to understand the approach from a high level by including a diagram describing the various modules of the proposed approach.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The paper presents a method for solving the challenging sim-to-real transfer problem without using any real-data. The paper is well written and easy to follow. The experimental validation on the physical hardware platform makes the paper a good contribution.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Excellent: Enjoyable to read.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "reviewer_expertise": "5: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature"}, "extra_text": {"issues": "+ I would request the authors provide more details of their approach in the Introduction section.\n+ Please check the indentation in line 207.\n+ Please include details on the method's runtime, how the approach can adapt when sensor data is obtained at lesser frame rates, and if the approach is robust to the various lighting conditions.\n\n"}}, {"review_id": "NWRxSaTnA0", "reviewer": "Reviewer_ACNP", "summary": "The paper presents an autonomous end-to-end off-road driving approach which can be trained using only simulation data. To enable good sim2real transfer, segmentation maps are proposed as an intermediate representation and trained in an adversarial fashion inspired by RCAN. The segmentation maps can be easily generated in simulation and are shown to provide a suitable level of abstraction. The authors show experimental evidence that the method performs at least as good as a classical autonomous driving stack and that it can be deployed closed-loop on a real-world autonomous offroad vehicle.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The main strengths of the paper are its real-world experiments and evaluations that demonstrate the applicability of the proposed method. Using a true-to-scale demonstrator is a great contribution, in contrast to using only small RC cars or simulation only experiments. \n\nOn a more detailed level, the insights into what a suitable trajectory parametrization is for learning are interesting as they are not only specific to offroad driving but most likely a general finding. It would be interesting to see (as possible future work) if it is possible to design a neural controller that is directly capable of controlling the vehicle. Such a controller might inherently have advantages because it has access to all the information in the network and it thus might learn ‘careful’ driving when there is high uncertainty, or similar human-like behaviour. \n\nThe detailed description of the method in Sec. 3 is of great value to the reader and the community. It is well written and informs the reader about the key details of the proposed Sim2Seg approach.\n\nHowever, since the experiments are a key part of this work, some clarifications might be required to further strengthen the results:\n- It seems that the real-world rollouts collected for evaluation purposes are collected by having a human operator drive the vehicle around. It is stated that those generated trajectories are an “efficient” way to reach some specified goal. First, what does efficient mean in this context? Energy efficient? Time optimal? Second, since the reference for an “efficient” performance is generated by a human, there is no guarantee that there might not be an even more efficient solution, w.r.t to the efficiency-metric. That also implies that comparing the (possibly suboptimal) human trajectory with the results of the autonomous driving method might lead to a less accurate evaluation because the autonomous vehicle might be able to find a better trajectory.\n- What are the ground truth trajectories $\\tau$ and how are they recorded? Is a trajectory simply a list of positions associated with a time-stamp? Is it recorded via GPS only, RTK-GPS, GPS+IMU, ...\n- In the evaluation (sec. 5.1) two metrics are used: L2 and GT, whereas the L2 metric only measures the difference between the trajectory endpoint and the specified goal. It is unclear what exactly the GT value characterizes. Why is cosine-similarity between two trajectories (assumed to be positions and 2D or 3D space) a good measure? Why is no classical measure like average trajectory error used for the evaluation? It would be very convincing to also provide the trajectory errors as well.\n\nIn the introduction and related work it is mentioned that a problem of RL methods is that it is very difficult to cover all possible environments, lighting cases, etc and that there are often “visual discrepancies”. The presented method learns the segmentation maps from directly from images. It would be very interesting to add details as to why the intermediate representation “segmentation map” is helpful to improve the robustness when unseen imaging conditions are observed. \n\nMinor points:\n- What low-level controller has been used to track the rollout? \n- in line 300, there is a typo: “ultiple” instead of “multiple”\n- Fig 3 is presumably done using wrapfig and the next page looks messed up.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The paper presents an approach that is based on RCAN work, but adapts the methodology to the new segmentation map intermediate representation. The work is well executed, although some details of the experimental evaluation need to be clarified, especially with respect to the choice of the optimal trajectory that the method is compared to. The inclusion of real-world experiments is really nicely done, especially considering the fact that the autonomous vehicles used in this work is true-to-scale. \nAt this point, I recommend \"weak reject\" because some parts of the experimental evaluation are not perfectly convincing. If the paper is improved a bit more, it will be a valuable contribution to the community.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "3: The reviewer is fairly confident that the evaluation is correct"}, "extra_text": {"issues": "The authors should address the following issues to further strengthen their paper:\n- Explain what a trajectory is and how the “GT” metric is computed.\n- Motivate why the “GT” metric is a good choice compared to more classical metrics such as average trajectory error.\n- Add details what the “efficient” trajectories executed by the human operator are and in what sense they are efficient. \n- Explain why the difference to such an “efficient” trajectory is a good metric to evaluate the approach. Is it not possible, that the trajectory found by Sim2Seg is better?\n- Discuss why adding the segmentation map intermediate representation helps overcome the issue that during training not all possible scenarios can be observed. Overall, the segmentation is also machine-learned and thus might suffer from the same problems as a direct end-to-end approach.\n\n\n**\n\n** AFTER REBUTTAL PERIOD **\n\n**\n\nThe authors have addressed my comments in a satisfactory way: Specifically:\n- _Explain what a trajectory is and how the “GT” metric is computed_: The explanations in the experimental section address this point fully.\n- _Motivate why the “GT” metric is a good choice compared to more classical metrics such as average trajectory error_: As the authors have included the ATE as well in the revised version, this point is void. The ATE very clearly show that the proposed sim2seg method outperforms the baselines.\n - _Add details what the “efficient” trajectories executed by the human operator are and in what sense they are efficient.  Explain why the difference to such an “efficient” trajectory is a good metric to evaluate the approach. Is it not possible, that the trajectory found by Sim2Seg is better?_: The definition what efficient is (close to human driver), is still questionable but for the goal of this work, it makes sense.\n- _Discuss why adding the segmentation map intermediate representation helps overcome the issue that during training not all possible scenarios can be observed. Overall, the segmentation is also machine-learned and thus might suffer from the same problems as a direct end-to-end approach._ The argument made by the authors that the segmentation map has much smaller dimension than the original image space is a good point. Furthermore, the authors have added ablations in section 5.4 which strengthen this argument.\n\n"}}, {"review_id": "7C9uwvLuuSK", "reviewer": "Reviewer_Vqbb", "summary": "Instead of building a module-based ADS, this work aims to achieve off-road self-driving end-to-end. Concretely, a driving policy trained in a simulator can be directly transferred into the real world. The core technique enabling zero-shot transfer is a module built on RCAN, which can bridge the gap between the images captured in the real world and those in the simulator. Therefore, this method can avoid the dangerous data collection process of trial and error. Also, this method can be trained efficiently. With this method, the policy trained in 48 hours is comparable to a rule-based policy, which requires a huge human budget.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strength**:\n1) This work builds a complete pipeline to show that training in a simulator and transferring the learned policy to the real world is feasible\n2) The idea that mapping RGB images from different domains (simulation/reality) to the segmentation domain is interesting. This also enables the zero-shot transfer.\n3) It is exciting to see the real vehicle experiment since it is difficult to do hardware experiments and a lot of researchers only validate their idea in simulators.\n\n**Weakness**\n1) The originality of the method is limited. It seems that this work transfers RCAN to the driving task with a small modification.\n2) The standard deviation is missing in all tables. If repeated experiments are conducted, please report std. \n3) The real-world demonstrative video is not striking enough. It shows unsmooth trajectories, and the setting is simple. Actually, learning to drive via RL is achieved several years ago: https://wayve.ai/blog/learning-to-drive-in-a-day-with-reinforcement-learning/.  This video is more attractive than the demo in the supplementary material. In my opinion, the core contribution of this work is to build such a hardware system and demonstrate that zero-shot sim-to-real autonomous driving can be completed. Therefore, the quality of trajectories in demo video is the most important part. If smooth and long driving trajectories or more complex scenarios can be shown, I will consider increasing my score.\n\n\ntypo:\nline 156-157: two two approaches", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The method is an incremental work built on RCAN, and thus originality is limited. Although hardware experiments are conducted, the driving trajectories are not of high quality. ", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "issues": "N/A", "reviewer_expertise": "4: The reviewer is confident but not absolutely certain that the evaluation is correct"}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)