Venue: CoRL 2021 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper presents a method for navigation with ground robots in unstructured outdoors environments. The method in this paper is based on learning from demonstrations, where the goal is to reproduce recorded navigation behaviors conditioned on multi-modal sensor data. To do so, the paper describes an algorithm maps 3d tensors of features extracted from sensor streams, representing a time window of past sensor readings,  to control signals through a linear operator (mode-3 tensor product). Control signals to be executed by the robot are thus computed as a linear combination of  sensor features, with weights learned from demonstrations.  The approach introduces an extra term to deal with divergence of executed robot behaviours from demonstrated behaviours, and regularization terms that ensure that the executed behaviours are temporally consistent. The paper describes an iterative optimization algorithm to solve for the weight tensors, with theoretical analysis for its convergence and an experimental comparison against other learning from demonstration methods applied to the task of navigation over various terrains with a real ground robot.
strengths_and_weaknesses: The paper provides a  clean and simple method for learning to navigate unstructured environments from demonstrations. The paper is well written, with thorough theoretical analysis and experimental validation. 

The paper could do a better job in describing the implementation details of the algorithm. While some of this information is available in the appendix, it doesn't explain very well how the training and execution phases are carried out. The video shows a failure case of the approach, but it would be better if some discussion of the limitation of the approach was added to the paper. There is no  discussion about the computational requirements and the final performance of the algorithm as a function of dataset and temporal window size, which would make this paper a stronger contribution.
summary_of_recommendation: This paper  presents an approach for solving the difficult task of unstructured navigation. The theoretical analysis and experimental validation make this a good contribution for this conference.
issues: The paper is not clear in a couple implementation details. For instance, is algorithm 1 executed periodically at test time to include new data points for the actual robot behaviours? or does this happen in an offline data collection phase to construct A with the same size as Y? In terms of computational cost, how does the algorithm's performance scale with varying temporal window size c, and number of demonstrated trajectories n.

Line 36 therefor
Line 162 y(t) -y(t)
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper considers the problem of terrain adaptation for ground robots to traverse unstructured off-road terrains. The authors motivate the work by observing that in realistic scenarios, robots often fail to execute behaviors due to unexpected and difficult to model aspects such as wheel slip, reduction in tyre pressure etc. Hence, this paper presents a data-driven approach that learns to predict control inputs from observed features such that the robot is able to correct for unexpected observations and still consistently perform the behavior. The authors frame the problem as learning a weight tensor that associates a particular terrain feature captured at a time step with a particular behavior type. The optimization objective implicitly encodes an adaptation for correcting the behaviors online. Experiments are carried out in realistic outdoor settings on a data set captured in a certain types of unstructured terrains with features derived from multiple modalities. 
strengths_and_weaknesses: Strengths
- The authors tackle and important problem of robust execution of navigation behaviors with a data-driven approach. 
- The proposed model is demonstrated on a realistic and a challenging data set. 
- The authors have provided the code and a video demonstrating the results on a ground robot. 

Weaknesses
- The authors rely on a certain predefined feature sets such as HOG, Locally Binary Patters etc. Is it possible to ameliorate the need for feature extraction by using a neural-style approach that can learn feature  representations. 

summary_of_recommendation: Overall, the paper addresses a practical problem and provides strong results. However, the technical exposition can provide further details on how the paper advances the state of the art. Such an explanation is necessary for wider adoption of this approach. Otherwise the results are convincing. 
issues: Issues
- I would request the authors to provide details on how the work advances the state of the art and highlight the novelty of the approach.  
- The authors are also requested to explicitly state the assumptions made in the problem formulation. This would help understand the generality of the method. 
- Does the approach assume that all sensor measurements are available at all times? Can the approach work when one of the sensors fails or when the sensor data arrives at different periodicities.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: The core contribution of this work focuses on enabling ground robots to make consistent navigation behaviors in unstructured terrains including grass, sand, gravel, and various rocks. By taking in terrain features from the environment, their method learns to monitor the difference between the expected and actual navigation behaviors to compute an offset that reduces the difference.
strengths_and_weaknesses: The paper is easy to read for a broad robotics audience. While I am not as familiar with the problem addressed in this work, the problem definition, related works, and results are discussed in a manner that brings someone from outside the community up to speed quickly and shows convincing improvements relative to prior methods. Additionally, the notation provided in the approach is helpful to interpret this work and build upon it.

It might be helpful to some readers to understand your approach at a high-level before getting into the details in section 3.2A by providing a diagram describing all the interacting tensors, inputs, outputs, critical variables and transformations. For something like object detection, that usually translates into a network architecture. However, that might be different in your case. While the paper is tight on space, the first figure is communicating a familiar to most roboticists (i.e. actual behavior deviating from expected due to unaccounted variables). Perhaps combining that figure with a high-level diagram would be beneficial.

summary_of_recommendation: The paper is well organized, mentioning relevant related work. The paper clearly defines all notation, the problem definition, and approach. The paper presents a set of unstructured terrain traversal experiments including various terrain types and complexities. Additionally, the paper shows how the proposed method outperforms prior baselines from related work, considering multiple metrics including success, consistency, and others. I think the paper would be an excellent addition to CoRL and enjoyable to readers.
issues: Incorporating a high-level diagram as mentioned in strengths/weaknesses along with text describing it might be helpful to some readers.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper investigates off-road ground robot navigation, particularly aiming to learn a control policy that provides offsets to provided vehicle commands such that the vehicle does not get stuck as often. The technical contribution is an algorithm for updating the weights of the control policy using self-supervision (i.e. learning from windows of (state, expected trajectory, actual trajectory) tuples). The authors numerically compare the proposed method with three baseline algorithms on a husky robot in various off-road terrains.
strengths_and_weaknesses: Strengths:
- Navigating off-road is a challenging and important problem, and the authors motivate and explain the problem nicely. The paper is written well and it was enjoyable to read.
- The ability to learn in a self-supervised manner (without requiring annotation) is a great attribute of the approach
- The hardware experiments are indeed extensive: several runs of several baseline algorithms are compared on various terrains on a real hardware platform in several real off-road environments. That is a lot of work!

Weaknesses:
- It isn't obvious to me how novel the optimization algorithm is. It looks like a fairly standard regularized loss, and the authors propose to iteratively update the two weight matrices by taking the derivative and setting it to zero. If the convergence guarantee is novel then it would make sense to have it in the main text or at least a sketch of how the proof is done.
- The improvement over TRAL is rather small, and for n=10, are the differences statistically significant? "The clear performance improvement obtained by our approach is because of the closed-loop capability that considers robot states along with terrain characteristics when adapting to terrains." -- there's no ablation, so how to justify this claim?
- The quality/quantity/diversity of the training data is arguably the most important design "parameter" for the algorithm, so it would be great if the authors could provide some discussion on insights or impacts of these factors on the results.
summary_of_recommendation: The paper is solving an important and relevant problem, seems technically sound, and provides extensive hardware evaluation which can provide insights to future research in this direction. However, the technical novelty is not as clear to me, in terms of how unique the self-supervised formulation is, how unique the optimization algorithm is, and to what extent this is better than approaches such as actually modeling things like wheel slip/payload weight, off-road learning methods from Kahn et al, learning other types of offset terms (eg from Schoellig et al).
issues: Minor issues:
- The introduction repeats itself a few times
- The related work begins "Related research on robot terrain adaptation..." but this seems incomplete. The related work should better align with the paper's claimed contributions, such as a self-supervision methods, learning offset behaviors (e.g. Schoellig's work with UAVs), optimization methods with guarantees, state-of-the-art in off-road experimental validation, etc. What about the work of Kahn at Berkeley?
- " Due to momentum, we know the robot’s behaviors at the present time are dependent on previous time steps" -- ? depends on what is in the state vector
- " learning of non-linear tasks can be lifted to linear space" -- provide support for this claim?
- " for short periods of time (i.e., c = [1, 30]), the dynamics of the robot do not dramatically change and thus, the non-linearity in the robot is not severe" -- questionable... 
- "behavior" should be defined more clearly as it is used throughout the paper. Is it a trajectory? or something more?
- Is this really "adaptation" as the title suggests? It's more of learning an offset policy. Adaptation suggests an online component.
review_title: null

Meta-review: This paper presents a method for navigation with ground robots in unstructured outdoor environments. All the reviewers agreed on the good quality of the contribution. The reviewers particularly appreciated extensive robot experiments. 
- Several reviewers have pointed out the need for clearly highlighting the novel contributions of the proposed approach and how it advances the state-of-the-art. 
- Providing more details about the algorithm, training procedure, and the training/test datasets can significantly strengthen the paper. 
Besides addressing the major issues mentioned above, the authors should also revise the manuscript according to the other clarifications
requested and the suggestions provided. 

===== Post rebuttal ===== 
The authors have sufficiently addressed reviewers' concerns. All reviewers agree that the paper is well written and a good contribution. I recommend an acceptance.
Decision: Accept (Oral)