Venue: CoRL 2021 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper tackles the problem of socially compliant navigation. Specifically, it clusters the pedestrians into groups, finds the convex hull of each group, and then included in the cost of an MPC controller for the robot. The method is validated to achieve better safety and somewhat better "comfort" on simulated scenarios, with injected noise.  
strengths_and_weaknesses: Overall, the propoed method is intuitive and sound. The method and its motivation is fairly intuitive: we want the robot to take into account the group formation it is in, for more socially compliant navigation. 

A main weakness is the simplicity of the robot model that is compatible. Lines 165-169 indicate that the method requires a fairly smalll number of discrete actions to be made. This suggests that in practice the proposed method is too slow, i.e. we cannot solve for continuous steering angle/ velocity. This begs the question, is it worthwhile to consider group dynamics? Would it be better to instead have a MPC controller running at high frequencies and adapt to simpler predictive models. It would be good if this could be investigated in the experiments -- it is unclear whether the MPC controllers used as comparison are also restricted to discrete actions, and are running at the same frequency.

Additionally, there are other methods that merit mentioning. Particularly, POMDP-based solutions DESPOT and its variants (HyP-DESPOT) (Adhiraj Somani et al., 2013, neurips; Panpan Cai et al., 2021 IJRR); and other control-based social navigation approaches (Weiming Zhi et al., 2021, ICRA).

Despite the concern about the simplicity of the robot model, the proposed method is fairly novel and solid. The paper is also accompanied by sufficient empirical evaluation. 
summary_of_recommendation: See above
issues: Need clarification on the simplicity of the robot dynamics. Discussion or ideally empirical evaluations on how much considering social groups add, instead of just having a controller that can be evaluated fast.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: The authors seek to demonstrate the benefits of modeling group structures in social navigation settings.

Their results clearly demonstrate the benefits of modeling groups over individuals for social navigation problems. They also show that their method is also robust to sensor noise.
strengths_and_weaknesses: The paper is clear and well written. It is easy to understand the contributions of the authors, and they clearly demonstrate their contributions. 

I appreciated the way the authors clearly laid out their hypothesis (H1-H4) in the evaluation section and then clearly laid out their analysis with respect to those same hypothesis(H1-H4). That organization style made it very easy to understand the questions they were asking, and immediately put the results into context.

Another hypothesis to explore that would be of interest would be comparing group-nopred and group-auto. This would directly highlight the impact of modeling future group motion using the auto encoder. I would hypothesize that group-auto would be “better across the board”: e.g. higher success rate, maintain a higher min-dist-to-pedestrian, have fewer trials with group intrusion, shorter path length. The charts seem to mostly suggest that this is the case, but there are some cases where group-nopred is “better” (e.g. Online, UNIV-F, Success rate). It would be nice to get an in depth explanation of some of these cases and what is going on there.

It would have been nice to see a sim2real deployment. It would have been nice to see how this system behaves in a couple real human scenarios where the environment is much less controlled than this simulation. It would also be nice to understand what are the main limitations of transferring this system to a real robot (i.e. compute bottlenecks, sensor failure modes, etc.)
summary_of_recommendation: I think the contribution in this paper is clearly demonstrated and with a with a bit more analysis (see above), it could be strengthened.
issues: See above
1. Experiments on a real robot
2. More analysis of group-nopred vs group-auto
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: The authors approach the problem of planning a robot trajectory through a dense pedestrian crowd. They cluster pedestrians into groups, and consider the combined personal space of such groups. Using past group-space image masks, future group-space masks are predicted by a 3D Convolutional Neural Network (CNN). A receding-horizon Model Predictive Control (MPC) approach is used to navigate the robot through the crowd.
strengths_and_weaknesses: Strengths

- Considering pedestrian crowds as larger groups than individuals is good insight, this research should encourage others in the field to consider "macroscopic" models.
- The evaluation of method work is significant, testing on a variety of public datasets. They also propose a method of testing via an interactive "Online" approach using ORCA; such a testing environment is very important when modelling interactive environments. In 4 cases, they prove their results are statistically significant.

Weaknesses

- From eq. (3), the learned model operates on each group individually. As such, there is no way for the model to capture inter-group interactions. This addition could be an interesting extension to this work.


summary_of_recommendation: The authors take an unconventional "macroscopic" approach to crowd prediction, and prove it is superior to state-of-the "microscopic" methods for online navigation. The authors clearly explain their approach, and back it up with an extensive evaluation section.

issues: - The group space prediction oracle is repeatedly referred to as an "autoencoder". As the model predicts the *future* from the *past*, the input is not the output: therefore the network is not strictly an *auto-* encoder. Consider changing the language, for example to "encoder-decoder framework".
- Footnote 1 mentions a homographic transform between "cartesian" coordinates and "image-space" coordinates. The purpose and details of this transform could be elaborated in the supplementary details.
- (optional, stylistic) There are no less than 5 variables associated with the letter 'g' in various fonts, 4 of them capital 'G'. It might aid understanding if the distinction between group *membership* and group *space* was clearer.
- (note) This reviewer knows of only one other work with this macroscopic space-time-image representation of pedestrian crowds that the authors should consider as part of the related work: “Dynamic Crowd Prediction for Social Navigation”, S. H. Kiss, K. Katuwandeniya, A. Alempijevic, and T. Vidal-Calleja, published to ICRA 2021.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper investigates motion planning among pedestrians, particularly focusing on the fact that pedestrians can travel in groups that have different dynamics/interactions than individual pedestrians. The technical contributions include a method for grouping/representing groups of pedestrians, an autoencoder to learn the group dynamics from a dataset, and an MPC formulation to plan collision-free paths with respect to the groups. The results include comparisons of various algorithms on fixed datasets and with simulations where agents follow ORCA policies.
strengths_and_weaknesses: Strengths:
- Groups are an important component of social robot navigation but most of the recent works on this topic focus on modeling individual pedestrians
- The paper is well organized, enjoyable to read, and the problem is well-motivated and explained.
- The approach of learning a group dynamics model and folding that into an MPC is interesting and makes sense, as MPC comes with collision guarantees, and it would be difficult to specify a group dynamics model without learning from data

Weaknesses:
- For groups, hardware experiments seem even more important than other social navigation problems, since the group dynamics are difficult to model and thus simulate. However, the authors designed insightful experiments in simulation and the challenges of running physical experiments (particularly with pedestrian groups) are totally understood here. Are there other ways to simulate group behaviors, other than ORCA (which is inherently single-agent) and playing back pre-recorded motions?
- The figures could be improved. I don't really understand Fig 5. Figs 3 and 4 are hard to see, especially since the tiny differences between algorithms are scrunched (e.g. for success rate, would it help to show failure rate instead?).
summary_of_recommendation: The paper lays out a principled way for group dynamics to be adopted into social navigation algorithms. Looking ahead, real-world testing of the idea will be important to determine the long-term impact of this type of modeling, though it seems like a promising direction.
issues: n/a
review_title: null

Meta-review: Summary: This paper proposes a method for modeling pedestrian agents in groups as opposed to as individuals. The proposed technique designates the groups using a neural network, and the groups’ motion is used within an MPC framework. The proposed method is supported by simulation results. 

Originality: The paper presents an interesting and original idea. Drawing inspiration from psychology, the paper proposes a macroscopic view of the world and show that this reduced fidelity can have benefits to improved navigation behavior. Reviewers commented positively on the appeal behind this idea. 

Clarity: The paper is very clear in presenting the problem, the intuition and technical approach to the solution, and organizing the analysis of the experimental results. Reviewers seemed generally convinced and interest in the idea of using group-based reasoning to induce socially-aware navigation. 

Quality: Reviewers commented on the thoughtful set-up of the experiments to approximate real-world conditions through using public datasets. However, a number of reviewers gave suggestions to further improve the experimental analysis. 

Significance: Reviewers generally saw the potential for impact for this work, but were mixed on the applicability of the proposed technique to real-world scenarios. (See “opportunities for improvement” for more details.) 

Pros: 
This work presents a compelling and novel framing of the navigation problem, a clearly described solution, and thoughtful analysis of experiments. 

Main areas of improvement: 

Reviewers broadly responded very positively to this paper. The major source of mixed feedback was whether or not the paper did enough to show that the proposed approach is practical. In particular, Reviewers rGtm, qTAW, and 6KBt suggest more discussion or simulation experiments to give insight into what it would take to deploy this technique on a real robot. I would suggest real-work applicability as an area of focus in the revision and rebuttal. 

Thank you for considering our feedback, and we look forward to seeing the updated paper. 

================
Final Decision

This paper seems like a clear accept - all reviewers recommended accept, even prior to revisions. Where I am not 100% certain is the boundary between oral v poster. I ultimately recommended oral because two reviewers recommended strong accept, all reviewers gave very good or above scores for technical quality, 3/4 gave very good scores for originality, and other scores were positive by consensus. 
Decision: Accept (Oral)