Venue: CoRL 2023 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: This paper describes a framework for solving long horizon Acquisition Policies for Robot-Assisted Feeding. The proposed method Visual Action Planning OveR Sequences (VAPORS) uses a hierarchal structure where the high level selects discrete manipulation strategies and the low level executes vision-parametrized policies. Also, this paper shows results on two real food manipulation tasks: robotic noodle acquisition and bimanual scooping with high efficiency and plate clearance.
review: Strengths:

The paper addresses interesting long-horizon robot-assisted feeding tasks -- robotic noodle acquisition and bimanual scooping. And the proposed method decouples the policy into separate high and low-level sub-policies.  The high-level part takes advantage of simulation for modeling high-level plate dynamics to leverage visual planning, and at the low level, the parameterized visual policy is executed for precise real manipulation. In general, the presentation for the method is clear and straightforward. Figures clearly express the algorithm step by step. There are sufficient quantitative results, baseline comparison and real-world execution videos look promising.

Weaknesses and suggestions: 

1) It seems to me the two low-level primitives for the noodle task are all used during the execution which may be easy for the high-level dynamics model/planner to plan. Could you add an ablation study in sim by adding some redundant primitives here to test whether the high-level planner could ignore those redundant primitives and be robust to those "noises"
2) To further show the generalization power of the proposed method, it needs to have an example beyond clearing plates. For example, the planner should be able to generate a plan for making a certain pattern for noodles (e.g make the noodle into a circle).
3) One low-bound baseline is needed: treat random select low-level primitives as a high level policy and test the results
4) Another interesting baseline comparison: use the data trained for the dynamics model to train a primitive selector and treat the selector as a high-level policy and test the results
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: The paper shows an interesting solution for long horizon Acquisition Policies.  Though more details, experiments, and some modifications are needed to be added to show the validity of the method, I am leaning toward accepting this work.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary_of_paper: The paper presents a hierarchical algorithm to learn how to sequence food acquisition primitives associated with robot-assisted feeding. Specifically, a high-level planning policy is learned in simulation (to avoid sim-to-real gap) and a low-level visual policy is learned offline on real-world data to execute parameterized primitives.
review: **Strengths**
- The paper is expertly-written and was easy to follow. Numerous design choices are well-justified.
- The paper addresses an important problem and does a great job of motivating it and discussing the associated literature.
- Although not emphasized in the paper, I wonder if the modified simulator can be a valuable contribution to the research community focused on robot-assisted feeding.
- The experimental design is comprehensive. I particularly appreciate the effort to quantify and categorize complex food acquisition tasks and settings.
- The paper also includes a user study to demonstrate that users tend to prefer the proposed method over baselines.
- The authors have made their code and the dataset open-source.

**Weaknesses**
- While I appreciate the sound arguments in the paper pertaining to comparisons with existing methods, the paper lacks empirical comparisons with existing algorithms to substantiate the qualitative arguments.
- As the authors acknowledge, the set of primitives explored is rather limited. Adding new primitives will likely involve expert-design of parameterization and integration into the pipeline.
- It is not clear why the baselines are omitted from the generalization experiment.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: The paper is well-written and addresses an impactful problem. The results conclusively demonstrate the effectiveness of the proposed method, including a user study. The paper lacks empirical comparisons with prior methods, but makes good qualitative arguments for how it improves upon them.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: This paper presents a system for long-horizon planning for bite acquisition for a food manipulating robot intended to eventually be used in assistive feeding.
review: Strengths:
- The proposed system tackles a challenging robotics problem (manipulation of food) and shows that including long-horizon planning for when to group food and when to acquire a bite can improve effectiveness at that task
- The evaluation includes real-world foods (restaurant meals) as well as simpler food (jellybeans).

Weaknesses:
- The preference survey, conducted with non-disabled participants, does not tell the reader anything about the appropriateness of the method for assisted feeding for people with disabilities.  It is not appropriate to evaluate assistive technologies by asking nondisabled people to imagine being disabled. (see Cynthia L. Bennett and Daniela K. Rosner. 2019. The Promise of Empathy: Design, Disability, and Knowing the "Other". In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). Association for Computing Machinery, New York, NY, USA, Paper 298, 1–13. https://doi.org/10.1145/3290605.3300528).  It would be better to leave this out or focus the study on how a co-diner would perceive the system.  If the perceptions of co-diners is not the point that the authors want to make, then they should conduct their study with disabled participants.  Overall, it is disappointing that the work has such high ecological validity in terms of the foods (ordered from real restaurants) and low ecological validity in terms of users, especially since the use of non-disabled proxies is not even listed as a limitation.
- Please clarify whether correction was used for the large number of statistical tests.


Minor points:
- In the first sentence, the "toll of meal preparation" is cited as a reason that people can't eat independently.  This is definitely not true (and the cited papers do not support this claim).

- I'm not sure that referring to sauce as adversarial is appropriate, given that sauce on noodles is a very normal configuration
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are not well addressed
summary_of_recommendation: Overall, this paper has some exciting technical components but I am very concerned about the user study; it is not appropriate to evaluate assistive technologies with nondisabled users without careful attention to the interpretation of results and some discussion of how the results might change with disabled users.  I currently lean towards rejection, but this is highly dependent on whether the authors can provide a convincing plan in the rebuttal for providing this care and nuance.

Post rebuttal comment:
I am trusting that the authors will update their paper appropriately, but they have made an effort to address my main concerns (although having made an effort to contact users is not the same as having validated with those users).  I will raise my score to a weak accept.
review_title: null

Meta-review: The reviewers commend the paper for being well-written and organized. The proposed framework for long horizon food acquisition policies for robot-assisted feeding was well received by reviewers. I encourage the authors to further address any remaining points raised in the reviews for the final version of the paper.
Decision: Accept (Poster)