Venue: CoRL 2021 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This work aims at providing a systematic way to debug controllers and analyze behavior saliency, in order to avoid dangerous or unexpected situations. There are various approaches to do so, as direct testing of extreme and edge cases or providing a mathematical analysis of the controller. In most cases, this is either cumbersome or extremely difficult. The focus of the paper is to establish a framework to evaluate controllers and summarize robot performance with the specific purpose of informing accurate human models. One way of testing for controller failure is to roll it out on many different scenarios, but this is a notably difficult problem, considering that it involves sampling from a posterior which is predominated by zero-density regions (salient behaviors/edge cases). They present RoCUS, a method that enables understanding of robot controllers via posterior sampling of representative behaviors.
strengths_and_weaknesses: The paper is well written apart from some typos. They provide a solution to the problem of posterior sampling by relaxing the posterior and reformulate it such that one only has to tune an intuitive hyperparameter (alpha) which represents the volume of the samples. Like this, one can use methods like Metropolis-Hastings, as they do, to sample from the posterior. They also propose different alternative metrics for behavior understanding, that do not purely focus on task completion. 
One of the weaknesses of the paper was a lack of a baseline comparison (with any of the methods cited in the related work for example). It could be nice to see how it improves over a less sophisticated form of sampling.
summary_of_recommendation: This paper provides a complementary approach to direct testing of extreme cases, which is very useful when the controller is deployed in situations that could hurt humans. The two presented use cases (navigation and manipulation) provided interesting insights which could be further showcased and assessed in other more complicated environments, or better, more complicated tasks, in the lack of real robotic experiments.
issues: - Line 153: what do you mean when b tends to + or - infinity?
- Line 312: what do you mean with b=0?
- How does RoCUS compare to other task completion-based metrics?
- While describing figure 2 you mention that for a deterministic system and controller, the probability of a given trajectory is a delta. It is not clear to me if you are modeling also the initial condition of the dynamical system, together with the trajectory. 
- Would be interesting to see more use cases of matching vs maximal mode.
- The premises of the paper are very alluring and it would be interesting to see RoCUS applied to more environments/tasks.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: In this article, the authors propose a Bayesian framework that aims to find situation in robot planning tasks that lead to trajectories which exhibit specified behaviors. For this purpose, the task distribution is defined as the prior, and the behavior saliency is the likelihood. Then, the samples of the posterior are tasks that balance these two objectives. Two simulations demonstrate the benefits of the proposed approach. 
strengths_and_weaknesses: Strengths:
-	Interesting and time-relevant topic as it allows to study the behavior of “black-box” controller in robotics
-	Smart idea using the Bayesian framework
- Simulation with complex examples
-	Paper is well written and structured

Weakness
-	Some minor mistakes and missing definitions
-  The baseline of the simulation seems to be too simple

summary_of_recommendation: One of the most interesting papers I have reviewed this year. The proposed approach is highly relevant as more and more "black-box" controller are used in robotics. However, the results should be compared to other more complex metrics.

Update: Based on the authors' response, I updated my recommendation to "strong accept".
issues: - (2) What is No? I guess it's the normal distribution but the notation is uncommon and never introduced.
- Alg.1 : "numner" -> "number"
- Alg.1 : How does the choice of $N_B$ and $N_t$ affect the result?
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: Complex robotic controllers typically demonstrate both intended and emergent behavior. Whilst intended behavior is optimized for and measured (e.g. navigate to x without collisions, grasp object y), often complex controllers exhibit emergent behaviors (e.g. jerkiness, object clearance, path length) that are also important to understand and measure. This work presents RoCUS, a method for understanding emergent controller behavior using a Bayesian sampling-based method. Unlike other controller testing techniques, RoCUS focuses on sampling representative trajectories instead of extreme or edge case trajectories that exhibit the emergent behavior of interest. 

strengths_and_weaknesses: **Strengths**

1. The proposed method is interesting. It appears reasonably general (provided that a function measuring the behavior of interest can be defined, and the environment scenarios of interest parameterized) and potentially a very useful way to enhance understanding of robot controllers beyond measuring task completion.
    * Of particular value is the focus on representative trajectories.
2. The experimental evidence provided is interesting (taking into account the appendix), especially Table 1 which highlights the differences between controllers. This type of controller characterization provides a rich and valuable way of understanding controllers that is complementary to task completion metrics. 
3. The problem of characterizing and understanding emergent robot controller behavior is highly relevant to robot learning, especially if learning robot controllers are to be deployed in the real world. The problem setting is well motivated and explained in the paper. 
4. The related work is thorough.
5. The paper is well written, the clarity of presentation is high throughout
6. The video is a valuable addition, especially the chart presented starting at 45s
7. The behavior taxonomy is valuable, well described, and appear comprehensive
    * Potentially consider looking into obstacle clearance velocity (i.e. average speed at closest distance to obstacle) in future work.
8. Sufficient detail appears to be provided to reproduce the work taking into account the appendix. The code is also released.

**Weaknesses**

No baseline
* The main alternative to RoCUS presented in the paper is a simple top-k baseline however no results using this approach are presented. This makes it difficult to assess the benefits of RoCUS over the (much simpler) baseline.

Scope
* Whilst this paper is relevant to robot learning in that it proposes a method for better understanding complex and potentially high dimensional learned robot controllers, there is no evidence provided or discussion of how the results in simulation are relevant to physical robot controllers. Potential questions of interest include:
    1. Can this method be applied to a physical robot controller? If not, why not?
    2. If this method is only applicable in simulation, can it be used to understand physical robot controllers? If not, why not and how might this limitation be overcome in the future?

The results are a little challenging to interpret.
* Figure 5 and Table 1(appendix) indicates that there is significant variation between the different controllers’ emergent behavior. This is interesting, but it is challenging to interpret and compare controllers because the likelihood of the posterior samples is not clear. Is the likelihood of the posterior samples equal across all three controllers? It would be even more useful if the likelihood could be provided for each controller and behavior for all posterior means.
* To give an example, consider Table 1(appendix), Behavior: Length. The IL and DS controllers have a similar posterior mean length (161, 166 respectively), but quite different Prior Means (161, 201 respectively). It is not clear how to compare IL and DS on trajectory length for representative samples. Are trajectories with sample length of 161 and 166 equally likely for IL and DS?

A number of the most interesting results are located in the appendix, and the main paper’s experimental evidence is not as compelling without it. 
* Suggest moving Table 1 and Table 2 to the main paper.
* Suggest moving the discussion of the stochastic controller to the appendix to make space. Similarly the explanation of the RRT controller could be shortened since controller understanding instead of design is the main focus of the paper

Minor
* It is not clear why IL is used in one experimental setting (2D nav) and RL in another (Reaching)
* The sampling formulation (see eq 2) appears to be very similar to BAYES-TREX. It would be nice to see a brief discussion in the related work describing how RoCUS differs (e.g. trajectories vs classification, MCMC vs MH sampling).
* Notation: The use of No() for the Normal distribution is confusing since it is non-standard. The notation in [12] is clearer.
* Typo: 172 I think this should read (Line 7) instead of (Line 6).
* It is not clear why all the emergent behaviors are not evaluated for the 7DoF reaching task (Table 2) whilst they were for 2D nav (Table 1). It would be valuable to see Table 2 extended to include all of the behaviors from Table 1.
summary_of_recommendation: Thank you for this submission. Overall this is an interesting method that offers insight into an important problem in robot learning (understanding emergent controller behavior). The method is well explained and the results in simulation are interesting, especially when the appendix is taken into account. It is also a pleasure to read. However there are a few issues, described above, some of which are significant.
issues: * Please consider the four main issues of baseline, scope, results interpretation, and structure
* Please also address the minor points listed above.
review_title: null

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper introduces a Bayesian model for policy evaluation. They introduce in a principled way an approach to sample tasks and trajectories in which a certain desired metric, b, is maximized. To do so, the paper proposes to apply Metropolis Hasting over an approximated posterior distribution.

The paper proposes a set of behavioural metrics (trajectory length, time derivatives, straight line deviation, obstacle clearance and motion legibility) and evaluate the performance for a set of policies (Imitation Learning, Motion Planning, Reinforcement learning and Dynamic Systems) to evaluate the proposed policy evaluation method. 
strengths_and_weaknesses: Strengths:
1. Formulating policy evaluation for robust interaction with he environment as a bayesian inference problem is interesting as it provides well-formulated problem that allow us to get deeper insights and let us compare the different methods for policy evaluation.

Weakness:
1. The main weakness of the paper is the lack of qualitative metrics. The paper proposes an evaluation method, but only evaluates the performance of a set of policies with quantitative measures. 
2. RRT algorithm is well known and it would be enough to re-direct the reader to the original paper, rather than putting the algorithm itself in the paper
3. When applied the Dynamics-based approach, they first generate a task space trajectory and then map the trajectory to the configuration space by Inverse Kinematics. Instead, the performance of the DS based method would improve if it considers a multi-objective reactive motion generator in which obstacle avoidance elements can integrated [1,2,3] with the goto movements.
4. The paper lacks any information of the required amount of samples (from both trajectories and tasks) for proper evaluation.


[1] Khatib, Oussama, "Real-Time Obstacle Avoidance for Manipulators and Mobile Robots", 1986

[2] Ratliff, Nathan, et al., "Riemannian Motion Policies", 2018

[3] Urain, Julen et al., "Composable Energy Policies for Reactive Motion Generation and Reinforcement Learning", 2021
summary_of_recommendation: I suggest a weak rejection. The paper proposes a policy evaluation framework but it did not exploit it enough. The paper requires more metrics, more comparison, more policies and qualititative metrics for a better methods comparison.
issues: The authours might consider adding adding qualititative metrics to improve the policy evaluation.
review_title: null

Meta-review: The paper proposes an approach to analyze the "emergent" behaviors of a controller, assuming that the behavior metric can be defined. All reviewers agree that the paper addresses an important problem in robotics. 

However, to fully appreciate the contributions of the proposed method, the authors should qualitatively and quantitatively compare it against other baselines. The authors should also clearly discuss the challenges, if any, in deploying the proposed approach on real robots. 

Besides addressing the major issues mentioned above, the authors should also revise the manuscript according to the other clarifications requested and the suggestions provided.

===== Post rebuttal ===== 

The authors have sufficiently addressed reviewers' concerns; specifically, the comparison with additional baselines was particularly insightful. I recommend an acceptance. 
Decision: Accept (Poster)