Learning Sequential Acquisition Policies for Robot-Assisted Feeding
Authors: Priya Sundaresan
Jiajun Wu
Dorsa Sadigh
Venue: CoRL 2023 Poster (2023)
Keywords: Deformable Manipulation
Dexterous Manipulation

Abstract: A robot providing mealtime assistance must perform specialized maneuvers with various utensils in order to pick up and feed a range of food items. Beyond these dexterous low-level skills, an assistive robot must also plan these strategies in sequence over a long horizon to clear a plate and complete a meal. Previous methods in robot-assisted feeding introduce highly specialized primitives for food handling without a means to compose them together. Meanwhile, existing approaches to long-horizon manipulation lack the flexibility to embed highly specialized primitives into their frameworks. We propose Visual Action Planning OveR Sequences (VAPORS), a framework for long-horizon food acquisition. VAPORS learns a policy for high-level action selection by leveraging learned latent plate dynamics in simulation. To carry out sequential plans in the real world, VAPORS delegates action execution to visually parameterized primitives. We validate our approach on complex real-world acquisition trials involving noodle acquisition and bimanual scooping of jelly beans. Across 38 plates, VAPORS acquires much more efficiently than baselines, generalizes across realistic plate variations such as toppings and sauces, and qualitatively appeals to user feeding preferences in a survey conducted across 49 individuals. Code, datasets, videos, and supplementary materials can be found on our website: https://sites.google.com/view/vaporsbot.

Learning Sequential Acquisition Policies for Robot-Assisted Feeding
Priya Sundaresan, Jiajun Wu, Dorsa Sadigh Stanford University
United States
stanford. Srreaaafossttfoord priyasun@stanford. edu, (jiajunsu, dorea]0cs
edu
Abstract: A robot providing mealtime assistance must perform specialized maneuvers with various utensils in order to pick up and feed range of food items. Beyond these dexterous low-level skills, an assistive robot must also plan these strategies in sequence over a long horizon to clear a plate and complete a meal. Previous methods robot-assisted feeding introduce highly specialized primitives for food handling without means to compose them together. Meanwhile, existing approaches long-horizon manipulation lack the flexibility to embed highly specialized primitives into their frameworks. We propose Visual Action Planning OveR Sequences (VAPORS), a framework for long-horizon food acquisition. VAPORS learns a policy for high-level action selection by leveraging learned latent plate dynamics in simulation. To carry out sequential plans in the real world, VAPORS delegates action execution visually parameterized primitives. We validate our approach on complex real-world acquisition trials involving noodle acquisition and bimanual scooping of jelly beans. Across 38 plates, VAPORS acquires much more efficiently than baselines, generalizes across realistic plate variations such as toppings and sauces, and qualitatively appeals to user feeding preferences in a survey conducted across 49 individuals. Code, datasets, videos, and supplementary materials can be found on our website.
Keywords: Deformable Manipulation, Dexterous Manipulation
1 Introduction
Millions of people impacted logistically, socially, and physically by the inability to eat independently due to upper mobility impairments or age and health-related changes [1, 2, 3]. Robot- assisted feeding has the potential to greatly improve the quality of life for these individuals while reducing caregiver burden. However, realizing performant system in practice remains challenging. For instance, humans eat spaghetti noodles as shown Fig. using nuanced fork-twirling motions. Dishes like ramen require even more diverse strategies like scooping soup or acquiring meat and noodles. Thus, only must an autonomous feeding system employ various utensils and strategies to handle different foods and quantities, but it must also operate over long horizons finish meal.
A
hr+1
hr42
TL.
litt
a+t
Figure 1: Visual Action Planning OveR Sequences (VAPORS) employs high level policy to select amongst discrete manipulation strategies h, such as grouping and twirling, and low-level vision-parameteriized policy Z to execute these actions a, for long-horizon dexterous acquisition.
7th Conference Robot Learning (CoRL 2023), Atlanta, USA.

---

Prior assistive feeding work has focused on learning individual low-level paiietearameterizee primitives for food manipulation. Examples include separate policies for skewering [4, 5, 6]. scooping [7], bite transfer [8, 9, 10], cutting [11, 12, 13], and pushing food piles [14]. While highly specialized, these policies cannot reason over an extended horizon or make use of multiple strategies for more effective plate clearance. Humans, on the other hand, interleave acquisition and rearrangement actions with ease ease- pushing multiple peas together before scooping instead of painstakingly acquiring each individual pea or gathering noodles closer to each other before twirling with fork. Replicating this long-horizon foresight in robotic feeding has yet to be demonstrated.
Recent work in skill-based reinforcement learning (RL) provides natural way to model long-horizon manipulation sequences hierarchically. This entails first learning high-level policy for composing skills [15, 16, 17]. and then optionally inferring the parameters of low-level skills separately [18, 19, 20]. These approaches tend to favor learning from simulation to scale data collection [21], but current state-of-the-art simulators lack high-fidelity models for food deformation, visuals, and cutlery interaction. This complicates learning food manipulation policies in simulation and transferring them to real [13]. Existing hierarchical approaches also assume that the low-level skills come from a general-purpose library of primitives such as grasping and path planning [19, 22, 18, 15, 23], limiting their applicability to the food domain which requires highly specialized behaviors. Thus, we seek to find an appropriate layer of abstraction for feeding, which can leverage the benefits of (1) hierarchical planning for Long-horizon, manipulation and (2) rision-based primitives for fine-grained control. Our key insight is that learning from simulated experience only at a high-level, which need not capture the intricacies of food dynamics, and incorporating visual planning to instantiate low-level specialized primitives, yields a powerful approach to dexterous, multi-step food manipulation.
Visual Action Planning OveR Sequences, a unified framework for food manipulation. Our approach is decoupled into a high-level planner, which sequentially composes low-level primitives. We first learn policy in simulation that models latent dynamics of plates from images. Specifically, we use segmented image observations as representation space, which captures the distribution of food items and is transfcrable between simulation and reality for high-level plans. We train the policy using model-based RL with a reward that encourages both acquisition and rearrangement. Separately, we instantiate a library of specialized primitives in the real world from learned food pose estimation and segmentation. Finally, we use the learned high-level planner on segmented real food images to plan sequences of primitives for long hong-horizon acquisition.
In this work, we present VAPORS:
We experimentally validate our approach on two real food manipulation tasks: robotic noodle acquisition and bimanual scooping. Across both real-world trials and a comprehensive user study of 49 users, VAPORS achieves the highest efficiency, plate clearance, and qualitative user ratings compared to heuristic and single-primitive baselines, all while generalizing to unseen plates.
2 Related Work
Robot-Assisted Feeding. Recently, number of devices for mealtime assistance have become available on the market [24,25], but are limited in functional reach due to reliance on pre-prope pre-programmed trajectories or teleoperation by users. While bite transfer of a food item to user's mouth is the eventual goal of autonomous feeding [9, 8, 10]. we focus on bite acquisition as a primary initial step for downstream feeding. Prior work in bite acquisition demonstrates the effectiveness of visual planning for precise manipulation. Feng et [6], Gordon et al. [5, 26] and Sundaresan al. [4] leverage bounding box localization, food pose estimation, and visual servoing to geometrically plan precise fork skewering motions. Similarly, Grannen al. [7] and Suh and Tedrake [14] plan bimanual scooping and grouping actions, respectively, for segmented food piles. These works focus only on developing a specialized individual primitive for food manipulation. In isolation, this does not capture many long-horizon real-world feeding scenarios with multiple utensils and strategies.
Long-Horizon Planning and Control. Several recent frameworks tackle long-horizon manipulation by separating motion-level decision-making from sequential plans. Traditionally, task-and-motion- planning (TAMP) approaches tend to assume extensive domain knowledge including after-effeede of actions and fixed task plans [27, 28, 29, 30, 31, 28]. In feeding, plate dynamics can be highly

---

uncertain, and state estimation is notoriously challenging, rendering these approaches ineffective. An alternative approach is model-based planning and control, with recent impressive results on complex tasks like dough manipulation [16, 32, 33]. This family of methods leverage learned environment dynamics over visual states like images [34, 35, 36, 37, 33], keypoints [38], or particle- based representations [16, 32] to sample and plan action sequences that maximize predicted rewards. However, these methods do not scale well to high-dimensional continuous action spaces such that of food acquisition. To address this, hierarchical RL decouples policies into high-level planner which selects amongst discrete but parameterized low-level primitives [39]. These works have demonstrated promising results on simulated long-horizon tabletop manipulation [18, 19, 15], but have yet to consider (1) real-world deployment beyond carefully controlled experimental setups, or (2) complex manipulation beyond commonplace primitives like pick-place, math-planning, and grasping. In contrast, we consider highly diverse plates requiring specialized primitives and tools.
Learning and Control for Manipulation in the Real World. A large body of robotics research focuses on learning real-world policies for manipulation either through sim-to-real transfer or exclusively from real interactions. With sufficient domain randomization, sim-to-real transfer has proven effective for tasks involving rigid objects or a limited set of deformable items like cloth, which state-of-thed thett--artttttte simulators support [40, 41, 42]. However, adapting these simulators to modeling food appearance and deformation is highly non-trivial. Meanwhile, learning exclusively from real data has been shown work well in challenging domains such as semantic grasping [43] or cable untangling [44, 45, 46]. These approaches rely on state representations that are scalable to learn, such as descriptors learned from self- interaction [43] or keypoints learned from small amount of manually annotated images [47, 48, 49, 50]. In our setting, it is difficult to scale real-world data collection across the range of food shapes, appearances, and properties a robot may encounter. Self-supervised learning also complicated due resets and utensil interchange. We instead take a hybrid approach which takes advantage of simulation for modeling high-level plate dynamics from large-scale interactions, but leverages visual planning the low level for precise real manipulation.
3 Problem Statement
We formalize the long-horizon food acquisition setting by considering an agent interacting in finite-horizon Partially Observable Markov Decision Process (POMDP). This is defined by the tuple (8, O, T, R, Po). We assume access to plate image observations og TR W xxx O of unknown plate states S, with the initial state distribution given by Po. Here, W, HI, and C denote the image dimensions. A denotes the action space, and T : S x A S represents the unknown transition function mapping states and actions to future states. The time horizon T denotes the discrete budget of actions to clear the plate and R(s, a) refers to the reward which measures progress towards plate clearance. Our goal is to learn a policy π (a, (ox) that maximizes expected total return: Eg, T,E, R(s dt)], with t < T.
To do so, we decouple into separate high and low-level sub-policies. We assume access to K discrete manipulation primitives ht, & E {1...., K}. and leam high-level policy πы which selects amongst these primitives. Additionally, we learn low-level policy n which continuously parameterizes selected primitive according to visual input. The components we aim to learn are summarized below, where & denotes a discrete primitive type and a, denotes its continuous low-level instantiation:
High-level policy म (h* loce, a3(-1) Low-level policy : TL (oelot, h²*)
We consider low-level actions a, parameterized by the position of the tip of utensil (xx, y, 2) and utensil roll and pitch (7, B). Here, 0 corresponds an untilted fork handle, for instance, and y 180 corresponds the fork tines being horizontal when viewed top-down (Fig. 2).
3.1
State-Action Representations
In this section, we outline the visual state and action representations which are at the core of our learning approach introduced in Section 4.
Visual State Space. Our approach makes use of RGB-D images and segmented plate observations, Wx H x3 D, I W X M, E R WH at different levels of abstraction. We leverage binary I E R.
3

---

segmentation masks to capture the spread of food items on plate, informing high-level planning with и. and RGB-D observations as input to T which better capture fine geometric details of food.
Action Parameterization. We consider an agent that may either perform acquisition or reurrangement actions, parameterized below. Acquisition actions attempt to pick up bite of food, and rearrangement actions consolidate items. For example, plate of noodles becomes more empty, the robot may need to employ rearrangement action by pushing multiple strands together before twirling (acquiring) for a satisfactory bite size.
In acquisition, a robot with utensil-mounted end- effector approaches the position (Id, Vdz za) in the workspace, and executes parameterized by roll y and pitch 3 (i.c. twirling, skewering, scooping, etc.). Here, (Id, Yd, =d) denotes the densest location of the plate, where food is most closely packed to encourage a high-volume bite. Specifically, at,acquis = (za. yd, 2d: Y/s 3) (I).
acquisition motion
(Itds Vas zd)
(1 Uf)
The intent of rearrangement is to bring food items from the sparsest plate region to the densest by pushing from (Z rO (Ia, Va: Ea) while maintaining contact with the plate throughout. As this is a planar push, we simply orient the tool
Figure 2: Action Parameterization: We parameterize acquisition and rearrangement actions relative to the densest (æa, ydy za) and furthest (æy, My, 3f, regions on the plate, as well as the utensil roll ry and pitch
orthogonal to the push direction, such that
7 arclan yr and is untilted (3 0%): df,rearrange xy-
(a Mds Rd, I (. (2).
VAPORS: Visual Action Planning OveR Sequences
Within the visual state and action space outlined in Section 3.1, we present our approach VAPORS for tackling long-horizon food acquisition. First, VAPORS learns policy ии. detailed in Section 4.1, select amongst high-level strategies for long-horizon plate clearance via model-based planning. Finally, VAPORS learns low-level policy π.. which leverages carry out generated sequential plans for real-world food acquisition detailed in Section 4.2.
primitives to
4.1 Learning High-Level Plans from Simulation
Our goal is to first learn policy म for selecting amongst K discrete acquísition rearrangement strategies without concern for the low-level action parameters. To do so, we learn latent dynamics model of the plate from segmented image observations, and instantiate π (h* I agt-1), 1), k {1...., K} with model- based planning over this learned dynamics model. In this section, т denotes the running counter of high-level primitives executed so far, and f. denotes the current timestep.
Simulator Overview. We train тH entirely in simulation, where interactions can be collected at scale opposed to the real world where manual plate resets and potential food waste are prohibitively expensive. As current simulators lack out-of-the-bol support for many feeding scenarios, we develop custom simulated food manipulation environment, visualized in Figure in Blender 2.92 [40], further detailed in Appendix C. The simulator exposes RGB images It binary food segmentation masks Mj. and food item positional states {(Ts, yi, =:)]sE(L.....)) Using this information, we design rewards for food acquisition in terms of ground truth plate state and collect transitions to train πи.
Simulation
Reality
Figure 3: Simulation vs. Real: We visualize the task of bimanual scooping of jelly beans. Due to the sim-to-real gap, we merely leverage simulation to learn high-level food dynamics, and leave low-level action planning to real vision- paroon--aamttrieeed primitives.
Reward Design. With access simulated testbed for feeding, train TH to select amongst strategies via model-based reinforcement learning (RL). Our goal of efficient plate clearance can be

---

specified with a reward that incentivizes either (1) successfully picking up food, or (2) reducing the spread of items on a plate. Optimizing for the first objective alone might lead to plate clearance, but at a slow pace of taking low-volume bites. The second objective encourages rearrangement when the plate sparse to aid downstream acquisition. Concretely, we express this as a weighted reward with tunable weight a E [0, 1]: rt @(PICKUP GAIN) + (1 (COVERAGE LOSS) (3). Here, PICKUP measures the quantity of food items picked up. COVERAGE measures the spread of items the plate, illustrated in blue in Fig. 2). We provide the details for computing both in Appendix C.2.
Learning Latent Plate Dynamics. With a means of measuring task progress via Tt and access to plate observations M,, we propose a model-based agent that learns plate dynamics from segmented observations and uses the learned model to plan actions that maximize reward. We achieve this by training a multi-headed latent dynamics model with the following (Fig. 4): (1) An encoder y(t, high-ddmensional segmented images M, to compressed latent states 22, (2) A transition function over the latent states p(==|2z-1, h²=1)
F
pira ( Feeoss (62 Mar. ge-1
M
M4
M6
M4
Figure 4: Latent Plate Dynamies Model: We learn latent dynamics model of the plate comprised of an encoder q. transition model [(-r|z+-1, 11), and reward model p(rz|zz).
compressing
use this model to select action sequences that maximize future rewards.
with which to imagine rollouts, and (3) A decoded reward model given by p(r=|=+), such that at test time, we can sample action sequences and determine which maximize predicted rewards. We note that the transition function learns to predict high-level plate state changes between T - and r as a result of executing a primitive h", rather than between individual timesteps 1 and & due to dg.
During training, collect simulated transitions consisting of the masked image, high-level primitive, low-level action, and reward {(M,, hr, rr,)). We train each head of this network using the objectives detailed in Appendix D.I.
We note that this approach is highly related to [37] with several crucial design choices. First, we learn plate dynamics over segmented image observations M, of food items on a plate, as opposed to raw RGB observations. This allows the dynamies model to attend food items rather than the whole plate, provides an easily transferable representation between simulation and reality, and eases pressure for latent representations to capture irrelevant details in pixel space. Additionally, we learn a policy within an action space of discrete but continuously parameterized primitives as opposed to a high-dimensional space like joint-motor commands. This encourages actions that induce meaningful and perceptible plate changes likely to be encountered in downstream feeding.
Model-Based Planning. Once trained, we leverage the learned encoder, transition model, and reward model towards instantiating л At timestep f, we enumerate all KT future candidate action sequences for the small library of primitives K. Conditioned on a history of observations MI., and actions @1 we imagine the future latent states 2r:++T+1 under each action sequence hir4 +T via the transition function. Next, we predict decoded rewards according to the reward model p(r.|z,) for each candidate sequence:
MPC-style planner with receding T 7'step horizon.
R = Σ E [p(rs/=)]]- Given the sequence of actions (f,,, h4 - .,,)) which maximizes predicted cumulative reward R, we take म H (Mss, açt-1) = R the first primitive in the predicted sequence. After executing this action, we replan with иH, terminating when T. Details of the full planning pipeline, adapted from [37], are provided in Appendix D.2.
4.2 Visual Policies for Low-Level Real Manipulation
Our learned simulated task dynamics model from Section 4.1 relies on segmented images M, as an observation space and parameterized primitives as an action space. In this section, we describe the visual state estimation pipelines we use to instantiate our state-action representations on real data.
Food Segmentation. To define acquisition and rearrangement actions relative to the poses of food, we Iearn to segment food items a plate shown in Fig. 1. We Iearn binary segmentation
5

---

Triske
Weight Acquied
1,00
1.00
0.75
0.75
Dorre Scoepee
0.50
0.25
surees
0.000
100
200
0
100 200
1000
200
100 230
Time (sl
Time
(a) Spaghetti
(b) Jelly Beans
Figure 5: Across trials for spaghetti (a) and jelly hean (h) acquisition, we visualize cumulative amount acquired across individual trials (left) and averaged overall (right). Shading denotes the standard error.
where for a real image Le E R4+ W (1t yields a binary model faeg R+ x Hx3 + R44 RwW'
segmentation mask M, which serves as input нH. To train faeg, we require a paired dataset of real plate images and ground truth segmentation masks. However, manually labeling pixel-level segmentation annotations on images is painstaking and time-consuming process for real plates of food. Instead, we use self-supervised annotation process which starts by taking an image of an empty plate, gradually adding food items to the plate, and using the absolute frame difference between the empty plate image and current observation to obtain the food segmentation mask. We implement fseg as a fully convolutional FPN (Feature Pyramid Network) and train according to the procedure detailed in Appendix D.3.
Food Orientation. Although segmentation provides means to sense glohal positional information about food on the plate, we also care about precisely orienting a utensil with respect to the local geometry of a food item. For instance, using a fork to pick up a group of noodles requires orienting the fork tines opposite the grain of the strands. This crucial to preventing slippage during twirling (Fig. 12), which tends to occur when the tines and strands run parallel. To address this, we also learn network fori IR, w' X + R mapping local RGB crop of food item of dimensions W' x H' to the desired roll orientation of the utensil y. Prior work has shown that acquiring a food item orthogonal to its main principal axis, such as skewering carrot against its length-wise axis rather than width-wist wise, can improve acquisition stability [4, 6]. Thus, we implement foet as fully convolutional network with a ResNet backbone and train it from a small amount of real food item crops (200), manually annotated with keypoints defining the principal food item axis as in [4].
Action Instantiation. With the visual state estimation pipelines feeg and fori trained offline, can instantiate I (a (o,lot, )²) for real-world manipulation. Given RGBD image observation 4, D,, first infer the segmentation mask M, Sseg (T)) Next, we query to obtain selected primitive i TH Mgg.a 1
If ht is acquisition primitive, we instantiate the continuous action ar,acqui according to Eq. (1) by estimating the densest plate region (æt, fa. &d), and utensil orientation 7. To do so, we apply standard 2D Gaussian kernel over M, yielding Mú- This blurs the image such that high-density regions in the original segmentation mask remain saturated but sparse regions have lower intensity. From this, we take the argmax üæ 0g = arg max max (u, M, [u, e] to be the densest pixel in the image, deprojected to 3D location (Ra, Da. 2a) via D, and known camera intrinsics. Given food item crop centered at the densest pixel, I, (Fig. 2) we also infer the utensil orientation with f- = furi T.). For a rearrangement primitive, we parameterize Ot.eaarranee according to Eq. (2). In addition to sensing the densest plate region, we sense the furthest region (0 u1, 2n) by finding the lowest intensity pixel in M,, This yields the following instantiations:
(or, h k.cqqii)) (0 Da. &a.
TL (o, h h rearrange) (ft. Pa, En 2,,
Finally, VAPORS operates in perception-action loop using ти to generate sequential plans and m to execute them. The full algorithm can be found in Algorithm of the Appendix.
Experiments
We seek to evaluate VAPORS ability clear plates, by effectively leveraging diverse strategies and planning over long horizons. Thus, compare against a single-strategy baseline with no

---

long-horizon reasoning and than via learned plate dynamics. We consider two challenging real-world feeding scenarios to test the capabilities of VAPORS compared to other approaches: noodle acquisition and bimanual scooping.
approach that plans long tengttermo actions heuristically rather
Experimental Setup: In noodle acquisition (Fig. 10), a Franka robot with a wrist-mounted custom motorized fork and RGBD camera must decide amongst twirling (acquisition) grouping (rearrangement) to clear plate of noodles. In bimanual scooping (Fig. 11), two Franka robots operating from overhead RGBD cameras must select amongst scooping (acquisition) or grouping (rearrangement) to clear a plate of jelly beans. For both tasks, we consider a half-full initial plate distribution ( 50 g. noodles, 15 jelly beans) and a hard count of T = 10 actions for spaghetti and T = actions for jelly beans, encouraging the acquisition of multiple items at once to finish a plate. For both tasks, assume access hand-eye calibration between the RGB-D camera and robot end-effector. In Appendix E we outline the hardware setup and control stack, low-level action instantiations, and training details for each task.
Baselines: Acquire-Only is identical to VAPORS in terms of T.. but does not perform any long- horizon reasoning. Instead, at each timestep, this approach only acquires via twirling or scooping, with no rearrangement in between. Heuristic also utilizes Ty in the same manner, but replaces TH with na've group-then-acquire strategy. This method senses the COVERAGE, as defined in Eq. (3), over Mr to heuristically determine when acquiring rearrangement is appropriate. When the area exceeds a pre-defined threshold, the policy defaults to rearrangement and otherwise acquires.
Spaghetti
Plate Clearance Results: We evaluate VAPORS,
Acquire-Only. and Heuristic on clearing plates across 10 trials for each task (Fig. 5). We see that VAPORS achieves the most efficient and highest cumulative plate clearance. As expected, Acquire- Only optimizes only for acquisition in the current instant, without exploiting the benefits of grouping for a more substantial pickup of multiple items at once. Scooping one jelly bean at a time or attempting to twirl just a few strands of noodles repeatedly leads to the observed slow rate of overall clearance. Heuristic's ' greedy group-then-acquee approach plans based on detected coverage thresholds, which we find is brittle in practice especially for any artifacts in segmentation mask predictions. This naive metric also does not encourage acquiring any bite-sized piles that may form intermittently, but rather aims to amass everything into one large pile before acquiring. This delays acquisition gains and wastes the action budget.
Rating
PHiieency
Stze
Homoniike
Prosicaliy
Hulety
Gevenaiia dolty
Jelly Beans
Rating
Stae
Homondle
Prachealty
Salety
Heurisic
YAPORS
Appine-Ooly
Figure 6: Likert Ratings: We administer a 7-point Likert survey to users after observing 10 trials per method. VAPORS clicits the most positive feedback across all criteria. 'e" indicates statistical significance (p < 0.05).
Additionally, we conducted a user study with 49 non-disabled participants (age range 27.0 + 9.5, 46.9% female and 53.1% 1% male) to gauge user preferences across methods. Of this pool, 77.6% reported prior experience interacting with robots before. 75.5% reported having fed someone before, and 28.6% reported having been fed baselines, VAPORS use of multiple strategies and long horizon horizon will lead to more preferable feeding in terms of quantitative and qualitative metrics.
User Evaluation:
adult. We hypothesized: H1. Compared to
We used a within-subjects design where we presented each participant with videos of all 10 plate clearance trials per each of the three methods, for either noodle acquisition or bimanual scooping. For each participant, we randomized the method order, the order of trials per method, and the food group. In the study, we ask participants to rate efficiency, bite size, similarity to human feeding, practicality, likelihood for reuse, safety, and generalization.
7

---

Tier 1
After watching all trials, we provided users with a 7-point Likert survey to assess these criteria (Fig. 6). VAPORS incurs the highest qualitative user ratings across criteria, compared to the Acquire- Onlly and Heuristic baselines, and with statistical significance for certain categories (p < 0.05, denoted **'). Users noted that VAPORS "mimicked natural feeding," and "showed a capacity for clustering as the plate got more and more empty, which felt like a great and efficient approach," while Heuristic and Acquire-Only "seem like extreme policies, where [Acquire-Only] never tries to cluster and [Heuristic] focus too much on making big piles." These results align with the hypothesis that VAPORS' use of multiple strategies and ability to reason over long horizons benefits a user's mealtime experience. We provide additional user study findings in Appendix E.
Tier
3ier
Figure 7: Noodle Acquisition Tiers of Difficulty: Tier I consists of plain noodle varieties: Dan Dan, Udon, and Pappardelle noodles. Tier 2 includes Tier plates along with soy sauce, marinara sauce, and garnishes such as parsley or cilantro. Tier 3 plates include noodle dishes such as pesto pasta and chow mein ordered from DoorDash.
Generalization Testing: Finally, we stress-test VAPORS' generalization capabilities by experimenting with noodle dishes prepared with sauces and garnishes well ordered from DoorDash (Fig. 7). We conduct 18 additional trials of plate clearance on unseen plates, separated into three tiers of difficulty with 6 trials per tier. We summarize our findings in Table 1.
VAPORS achieves near full plate clearance for Tier noodles, demonstrating generalization noodle shapes and sizes (Table 1). While VAPORS is still able to make decent progress towards (A) Misperception, (B) Wrong Action, (C) Imprecision, (D) Slip.
Failure Calegorization
Description
% Cleared
Failure
Plain Noodles
90% = 6%
2
0
18%
Noodles w/ Sauce
68% + 16%
2
25%
DoorDash Noodles
64% 1: 13%
3
2
4
23%
Table 1: OOD Results and Categorization Failure Modes:
plate clearance in Tier 2, we observe the occurrence of more slip failures (D) and misplanned actions (A, B) due to the addition of sauce and distractor food items. Somewhat surprisingly, the performance gap between Tier and Tier 3 is minimal, with VAPORS being able to clear well over half the noodles for fully tittof-dditribution plate. The main challenges include misperceiving cabbage for noodles in the chow mein, well dropping twirled noodles heavily coated in pesto or soy sauce (D) (Fig. 12). Regardless, VAPORS demonstrates promising signs of zero-shot generalization.
6 Discussion
We present VAPORS, which to our knowledge is the first framework to address the multi-step food acquisition problem in robot-assisted feeding. Our hybrid approach leverages simulation to learn to model high-level plate dynamics at scale, and uses visual pose estimation in order to perform dexterous maneuvers for complex low-level food pickup. We experimentally validate VAPORS on a complex suite of real-world food acquisition tasks such as noodle acquisition and bimanual scooping of beans. VAPORS demonstrates the ability to clear plates efficiently over non-learned baselines while appealing to the feeding preferences of real users.
Limitations and Future Work. The largest current limitation is a lack of user testing on individuals with mobility impairments that affect their ability to eat independently, discussed in detail in Appendix A. Additionally, although this work highlights promising initial results toward generalization across food variations such as shape, sauces, and toppings, we acknowledge that our library of low-level primitives is currently limited. One actionable future direction is expanding our library with prior work skewering, cutting, and even toppling unstable items to tackle more expansive set of plates. Our initial prototypes for dexterous food acquisition, such as the motorized fork, also open up interesting possibilities for future designs of dexterous interchangeable utensils which would enable rapid strategy switching. Currently, the system also executes primitives in an open-loop fashion, but we hope to use reactive control the future to adapt online to slippage or imprecision.

---

Acknowledgments
This work is in part supported by funds from NSF Awards 2132847, 2006388, 2218760, as well as Stanford HAI, the Office of Naval Research, AFOSR YIP FA9550-23-1-0127. and Ford. We thank Lorenzo Shaikewitz for designing the motorized fork used this work which made real-world experimentation possible. We also thank Rajat Kumar Jenamani, Suneel Belkhale, Jennifer Grannen, Yuchen Cui, and Yilin Wu for their helpful feedback and suggestions. Priya Sundaresan is supported by an NSF GRFP.
References
[1] M. W. Brault et al. Americans with disabilities: 2010. US Department of Commerce, Economics and Statistics Administration, US .......... 2012.
[2] S. W. Brose, D. J. Weber, B. A. Salatin, G. G. Grindle, H. Wang, J. Vazquez, and R. A. Cooper. The role of assistive robotics in the lives of persons with disability. American Journal Physical Medicine & Rehabilitation, 89(6):50--52 2010.
[3] V. Mahcu, P. S. Archambault, Frappier, and F. Routhier. Evaluation of the jaco robotic arm: Clinico- economic study for powered wheelchair users with upper-extremity disabilities. In 2011 IEEE International Conference on Rehabilitation Robotics, pages 1-5. IEEE, 2011.
[4] P. Sundaresan, S. Belkhale, and D. Sadigh. Learning visuo-haptic skewering strategies for robot-assisted feeding. In 6th Annual Conference on Robot Learning. 2022. URL https: 1d=1Lg09
[5] E. K. Gordon, X. Meng. T. Bhattacharjee. M. Barnes, and S. S. Srinivasa. Adaptive robot-assisted feeding: online learning framework for acquiring previously unseen food items. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 9659 9666. JEEE, 2020.
[6] R. Feng, Y. Kim, G. Lee, E. K. Gordon, M. Schmittle, S. Kumar, T. Bhattacharjee, and S. S. Srinivasa. Robot-assisted feeding: Generalizing skewering strategies across food items on plate. In The International Symposium of Roboticy Research, pages 427-442. Springer, 2019.
[7] J. Grannen, Y. Wu, S. Belkhale, and D. Sadigh. Learning bimanual scooping policies for food acquisition. In 6th Annual Conference on Robot Learning, 2022. URL https: // htpp/rppereeeeeeeeee eview. wwt/ ?id= QDtbMK67PJG.
[8] L. Shaikewitz, Y. Wu, S. Belkhale, Grannen, P. Sundaresan, and Sadigh. In-mouth robotic bite transfer with visual and haptic sensing. arXiv preprint arXiv:2211.12705. 2022.
[9] S. Belkhale, F. K. Gordon, Chen, S. Srinivasa, Bhattacharjee, and D. Sadigh. Balancing efficiency and comfort in robot-assisted bite transfer. In 2022 International Conference on Robotics and Automation
(ICRA). pages 4763. IEEE, 2022.
[10] D. Gallenberger, T. Bhattacharjee, Y. Kim, and S. S. Srinivasa. Transfer depends on acquisition: Analyzing manipulation strategies for robotic feeding. In 2019 14th ACM/IEEE International Conference Human- Robot Interaction (HRI), pages 267 276. IEEE, 2019.
[11] M. Sharma, K. Zhang, and O. Kroemer. Learning semantic embeddingg spaces for slicing vegetables. arXiv preprint or Xiv: 1ee.19044000.. 2019.
[12] K. Zhang, M. Sharma, M. Veloso, and O. Kroemer. Leveraging multimodal haptic sensory data for robust cutting. In 2019 IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids), pages 409 416. IEEE, 2019.
[13] E. Heiden, M. Macklin, Y. Narang, D. Fox, A. Garg. and F. Ramos. Disect: A differentiable simulator for paramcter inference and control in robotic cutting. arXiv preprint 2rXiv:2203./023 2022.
[14] H. Suh and R. Tedrake. The surprising effectiveness of linear models for visual foresight in object pile manipulation. In International Workshop on the Algorithmic Foundations of Robotics, pages 347-363. Springer, 2020.
[15] L. X. Shi. Lim, and Y. Lee. Skill-based model-based reinforcement learning. arXiv preprint 20Xiv:2207.07560, 2022.
[16] X. Lin, C. Qi Y. Zhang. Z. Huang, K. Fragkiadaki, Y. Li, C. Gan, and D. Held. Planning with spatial- temporal abstraction from point clouds for deformable object manipulation. In 6th Annual Conference on Robot Learning, 2022. URL https: /tttps://pperrviiii net www.ooidddmmindy/oooooo.a

---

[17] Z. Cao. E. Biyak, W. Z. Wang. A. Raventos, A. Gaidon, G. Rosman, and D. Sadigh. Reinforcement learning based control of imitative policies for near-accident driving. arXiv preprint vrXiv:2007.00178. 2020.
[18] M. Dalal, D. Pathak, and R. R. Salakhutdinov. Accelerating robotic reinforcement learning via parameterized action primitives. Advances in Neural Information Processing Systems, 34:21847-21859 2021.
[19] S. Nasiriany, H. Liu, and Y. Zhu. Augmenting reinforcement learning with behavior primitives for diverse manipulation tasks. 2022 International Conference on Robotics and Automation (ICRA), pages 7477-7484 IEEE, 2022.
[20] C. Agia, T. Migimatsu, J. Wu, and J. Bohg. Taps: urXiv:2210122555 2022.
Task-agnostic policy sequencing. arXiv preprint
[21] A. Kumar, Z. Fu, D. Pathak, and Malik. Rma: Rapid motor adaptation for legged rohots, arXiv preprint arXi::210700034.4 2021.
[22] S. Bahl, A. Gupta, and D. Pathak. Hierarchical neural dynamic policies. arXiv preprint arXiv:210..05627, 2021.
[23] I.. Shao, T. Migimatsu, Q. Zhang, K. Yang, and J. Bohg. Concent2rohot: Learning manipulation concepts from instructions and human demonstrations. The International Journal of Robotics Research, 40(12-14); 1419-1434, 2021.
[24] Meet Obi Meet Obi, https::/meeetbbi.com 2022. [Online; accessed 6-Jume-2022).
[25] Mcal-Mate, Mcal-Mate- Made2Aid. https://sww.tttt wwdewadd.aaaaiaaaammmm.mmmm.. co wwtpppooductrrofleeeeeeeeee.eeee wtpefllllpppoo 2022. [Online; accessed 6-June-2022).
[26] F. Gordon, S. Roychowdhury, T. Bhattacharjee, K. Jamieson, and S. S. Srinivasa. Leveraging post hoc context for faster learning in bandit settings with applications in robot-assisted feeding. In 2021 IEFE International Conference on Robotics and Automation (ICRA), pages 10528-10535. IEEF, 2021.
[27] L. P. Kaclbling and T. Lozano- Hicrarchical planning in the now. In Workshops at the Twenty- Foury-h AAAI Conference on Artificial Intelligence, 2010.
[28] S. Srivastava, E. Fang, L. Riano, R. Chitnis, S. Russell, and P. Abheel. Combined task and motion planning through extensible plamner-independent interfacc layer. In 2014 IEEE international conference on robotics and automation (ICRA), pages 639-646. IEEE, 2014.
[29] C. R. Garrett, R. Chitnis, R. Holladay, B. Kim, T. Silver, L. P. Kaelbling, and T. Lozano-Pérez. Integrated task and motion planning. Annual review of control, robotics, and autonomous systems, 4:265-293, 2021.
[30] R. Chitnis, D. Hadfield- Menell, A. Gupta, S. Srivastava, E. Groshev, C. Lin, and P. Abbecl. Guided search for task and motion plans using learned heuristies. In 2016 IEEE International Conference on Robotics and Automation (ICRA), pages 447-454. IEEE, 2016.
[31] C. R. Garrett, T. Lozano-Pérez, and L.. P. Kaelbling. Ffrob: An efficient heuristie for task and motion planning. In Algorithmic Foundations of Robotics XI, pages 179-195. Springer, 2015.
[32] H. Shi, H. Xu, Huang, Y. and Wu. Robocraft: Learning to see, simulate, and shape elasto -aatto-plasic objects with graph networks. arXiv preprint arXiv:2205.029m) 2022.
[33] X. Lin, Z. Huang, differentiable physics for deformable object manipulations with tools. arXiv preprint arXiv:2203.17275 2022.
B. Tenenbaum, D. Held, and C. Gan. Diffskill: Skill abstraction from
[34] C. Finn and S. Levine. Deep visual foresight for planning robot motion. In 2017 IEEE International Conference on Robotics and Automation (ICRA), pages 2786-2793. IEEF, 2017.
[35] F. Ebert, C. Finn, S. Dasari, A. Xie, A. Lee, and S. Levine. Visual foresight: Model-based deep reinforcement learning for based robotic control. arXiv preprint arXiv:1812.00568 2018.
[36] D. Hafner, T. Lillicrap. J. Ba, and M. Norouzi. Dream to control: Learning behaviors by latent imagination. arXiv preprint arXiv: 1eX.vv/912..11 1912001603 2019.
[37] D. Hafner, T. Lillicrap, 1. Fischer, R. Villegas, D. Ha, H. Lee, and J. Davidson. Learning latent dynamics for planning from pixels. In International conference on machine learning, pages 2555 2565. PMLR, 2019.
10

---

[38] Y. Li, A. Torralba, A. Anandkumar, D. Fox. and A. Garg. Causal discovery in physical systems from videos. Advances in Neural Information Processing Systems, 33:9180-9192, 2020.
[39] S. Pateria, B. Suhagdja, A.-h. Tan, and C. Quek. Hierarchical reinforcement leamnngg A comprehensive survey. ACM Computing Surveys (CSUR), 54(5):1-35, 2021.
[40] T. Roosendaal, Blender foundation. The essential Blender: guide to 3D creation with the open source suite Blender, 2007.
[41] F. Coumans and Y. Bai. 2019. pybullet, python module for physies simulation for games, robotics and machine learning, 2016.
[42] V. Makoviychuk, L. Wawrzyniak, Y. Guo, M. Lu, K. Storey, M. Macklin, D. Hoeller, N. Rudin, A. Allshire,
A. Handa, al. Isaac gym: High performance gpu-based physics simulation for robot learning. arXiv preprint arXiv:2108 10470, 2021.
[43] P. R. Florence, L. Manuelli, and R. Tedrake. Dense object nets: Learning dense visual object descriptors by and for robotic manipulation. arXiv preprint arXiv: 08756, 2018.
[44] V. Viswanath, K. Shivakumar, J. Kerr, B. Thananjeyan, F. Novoseller, J. Ichnowski, A. Escontrela, Laskey, J. E. Gonzalez, and K. Goldberg. Autonomously untangling long cables. arXiv preprint arXiv:2207.07813.22 2022.
[45] P. Sundaresan, J. Grannen, B. Thananjeyan, A. Balakrishna, J. Ichnowski, Novoseller, M. Hwang. M. Laskey, J. E. Gonzalez, and K. Goldberg. Untangling dense non-planar knots by learning manipulation features and recovery policies. arXiv preprint AxXiv:210.00844 2021.
[46] J. Grannen, P. Sundaresan, B. Thananjeyan, J. Ichnowski, A. Balakrishna, M. Hwang, V. Viswanath, M. Laskey, J. E. Gonzalez, and K. Goldberg. Untangling dense knots by learning task-relevant keypoints. arXiv preprint arXiv:2011.0499) 2020.
[47] L. Manuelli, Y. Li, P. Florence, and R. Tedrake. Keypoints into the future: supe-supe correspondence in model-based reinforcement learning. arXiv preprint urXiv:2009.05083 2020.
[48] W. Gao and R. Tedrake. kpam 2.0: Feedback control for category-level robotic manipulation. IEEE Robotics and Automation Letters, 6(2):2962 2969, 2021.
[49] M. Shridhar, L. Manuelli, and D. Fox. Cliport: What and where pathways for robotic manipulation. In Conference Robot Learning, pages 894-906. PMLR, 2022.
[50] A. Zeng, P. Florence, J. Tompson, S. Welker, J. Chien, M. Attarian, T. Armstrong, I. Krasin, D. Duong, V. Sindhwani, al. Transporter networks: Rearranging the visual world for robotic manipulation. In Conference on Robot Learning, pages 726-747. PMLR, 2021.
[51] D. Seita, A. Ganapathi, R. Hoque, M. Hwang, E. Cen, A. K. Tanwani, Balakrishna, B. Thananjeyan, Ichnowski, N. Jamali, et al. Deep imitation learning of sequential fabric smoothing from an algorithmic supervisor, In 2020 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 9651 1658. IEEE, 2020.
[52] H. Ha and S. Song. Flingbot: The unreasonable effectiveness of dynamic manipulation for cloth unfolding. In Conference on Robot Learning, pages 24-33. PMLR, 2022.

---

Learning Sequential Acquisition Policies for Robot-Assisted Feeding
Please refer to our website for videos, code, and supplementary material, as well as the Additional Experiments' page for supplementary ablations and comparison additional baselines. We first include a discussion around VAPORS' greatest current limitations in the larger context of robot- assisted feeding (Section A). In the subsequent sections, we also provide an overview of the main design choices behind VAPORS and a thorough overview of implementation and experimental details.
A Limitations and Risks
In this work, we evaluate VAPORS quantitatively in plate clearance experiments and qualitatively in user studies. However, the largest current limitation is a lack of comprehensive user testing on individuals with mobility impairments that affect their ability to eat independently. These individuals are not represented in the demographic of individuals surveyed as per Section 5, and we are actively working on expanding our pool of participants to include such users. We fully acknowledge that performing user studies with non-disabled participants is neither representative of the experience of individuals with difficulties eating, nor in any way a replacement for the feedback and insights these individuals could offer towards improving assistive feeding research. We also note that plate clearance success is not on its own an indicative metric of how useful or effective a system like VAPORS would be for assisting real users with eating difficulties. Extending VAPORS part of a user-facing system for feeding would require several changes and considerations not explored in this work, such as taking into account different feeding preferences, mobility levels, involuntary head or body movements, comfort levels, and preferences surrounding shared versus full autonomy across different users. A full-stack feeding system would also need to carefully consider how to do in-mouth bite transfer safely, comfortably, and reliably. None of these considerations is exhaustive in any way; we merely see VAPORS as a step towards addressing challenges surrounding dexterity and long-horizon reasoning within bite acquisition, and we only speculate this having an impact for assistive feeding down the line. Towards this goal, we provide additional results below containing initial feedback from users with mobility impairments.
Users w/o Mobility Impairments
Usera w/ Mobility Impairments
Ratin
12 0 L
Rating sn
12
Effciincy
Bite Size
Homeolike
Practiallit
Reuse
Swiety
Trust
Generratizablliy
Enficioncy
Bite Size
Hamaniike
Prociicality
Reuse
Swiety
Trust
Ganeratizabbtity
Heuristic
VAPORS
Acquire Only
Error bars indicate standard error, and indicates statistical siginficant findings via 1-way ANOVA testing. Users with and without mobility impairments favor VAPORS in terms of all criteria, but especially efficiency, bite size, and generalizability. suggesting the promise of our approach. For more user-facing considerations, such practicality and whether users would reuse VAPORS given choice, we find a small drop average ratings between users without and with mobility impairments. In Section A, we discuss additional considerations to improve VAPORS in these aspects. We note that the larger error bars and lack of statistical significant findings for users with mobility impairments is due to the small sample size of users tested (3) as compared to 49 users without mobility impairments. We are actively working to increase this pool for larger-saale user testing in the future.
Figure 8: Likert Ratings for Users with and without Mobility Impairments:
B Feedback from Users with Mobility Impairments
Although we have yet to physically evaluate VAPORS alongside users with mobility impairments, We seek to understand how these users perceive VAPORS currently. To do so, we conduct user study via the same interface from Section involving watching videos of food acquisition and providing Likert ratings as feedback. This study approved by the Institutional Review Board of
12

---

Stanford University. In a survey sent out to users, we received three responses from participants (2 female, male with ages in the range of 31-44). All participants reported difficulties and/or mobility impairments that affect their ability to eat independently. including C5 quadriplegic injury with no finger or wrist function, and other impairments causing limited use of arms and hands.
We ask all users to evaluate the spaghetti acquisition trials from Section 5. In Figure 8, we visualize the average Likert ratings for users without versus with mobility impairments. We find that users with cating difficulties still rate VAPORS high in terms of efficiency, bite-size, and generalizability. and comparatively much higher than baselines. We do see that these users ratings for other considerations of feeding system, such as practicality and reuse, are lower on average than users without mobility impairments. but VAPORS still favored. Users reported that the twirling primitives occasionally struggled when there were very few strands of noodles left on the plate, or noodles were located at plate edges, which could lead messy bites in practice.
Thus, while promising in terms of its algorithmic effectiveness, we fully acknowledge that VAPORS requires further refinement and ample testing to be a truly effective assistive feeding system, discussed in Appendix Section A. We further acknowledge that the sample size of users with mobility impairments is very small, and in the future, we hope to expand to much larger pool of users to fully understand VAPORS' capabilities.
C Simulator Details
C.1 Simulator Design
Figure 9: Blender Food Simulation Environment: We implement custom food manipulation simulator in Blender 2.92 with an Open Al gym-style environment. The simulator supports softhody ohjects, such as noodles in different shape variations, as well as rigid, granular piles of items. We implement cutlery with arbitrary utensil meshes such as forks and spoons, and implement actions using the keyframing feature of Blender to control the position and orientation of tool across frames.
We use Blender 2.92, a physics and rendering engine, develop a custom feeding environment supporting deformable items, rigid items, and cutlery interactions. To instantiate deformable items like noodles (Fig. 9), we represent each item as a group of particles simulated with soft body physics. We treat granular piles of food such as jelly beans as separate rigid bodies. Additionally, we provide support for keyframe the position and orientation of the tool across simulation frames implement actions.
utensils including a fork, spoon, and pusher tool, where we programatically
C.2 Reward Design
In this section, we describe the implementation of the reward function given in Eq. (3). For a set of known food item states in simulation 8; = {( ys: 2,) =)))EE,,....N PICKUP measures the quantity of food items picked up out of N total items. We detect a picked up food item in simulation by thresholding the position of all items before and after an action, relative to plate height. Analogous to task progress metrics in cloth smoothing work [51, 52], we use COVERAGE to measure of spread of items on the plate. We compute this via the area of the convex hull of ((r,, 34) be)),.......)) depicted in Fig. 2, via the Scipy Python library.
D Details of Bearning-Based Methods
D.1 Latent Dynamics Training Details
We implement the latent plate dynamics model using the recurrent state space model from [37], with 64 x 64 input images and 30 30-dimensional diagonal Gaussian latent variables. This is a multi-headed
13

---

deep recurrent network comprised of a learned encoder, transition model, and reward model. We supervise each head of the network with the following objectives:
For the encoder q Mgt. (gt-1), we use an auxiliary decoder head that upsamples latent variables to predicted images M, and take the mean-squared error between (M, M,) as
a standard reconstruction objective. This encourages the learned latent representations to preserve the notion of food spread captured in segmented image observations.
We supervise the transition function p(27|27-11 h head using the KL--iivereecce for multi-step predictions as defined in [37].
Finally, for the reward model given by (()) we take the mean-squared error between predicted rewards and ground truth rewards (fr, rt). This objective promotes accurately decoding rewards of future states inform planning test-time.
D.2 Planning with Learned Dynamics Model
Once trained, we use MPC-style loop to sample and plan actions that maximize predicted rewards under the learned reward model.
At time T, we can enumerate all K" future candidate action sequences for the small library of primitives K, where T is the planning horizon. Conditioned on history of observations Mi.r and actions 41:--1. we imagine the future latent states 2r:++++1 under each action sequence h 7:
++T+1
Tt:4T41 ((zr|Maa (1=--1)
II im++1
where g(2e|Mgr act-1) is the learned encoder and p(2/|2,-1, h h is the learned transition model. Next, we predict decoded rewards according to the reward model p(ra (1) for each candidate sequence:
I+T+1
R =
Σ E [p(r/<=))] A0T+I
Next, we select the sequence of actions (h4, h 1 P4.) which maximizes predicted cumulative reward R. The final step of the MPC planning loop is we take тH (Mst, aga-1) 1) = hit which is simply the first primitive thus obtaining second action and so on until = T (Algorithm 1).
predicted sequence. After executing this action, we replan with тH.
Algorithm Planning with VAPORS
1: for 7 E
do
2:
I,, Dj t- Get current RGBD image observation
3:
Mi -ssg // Infer segmentation mask
4:
i = == a1=-1) 1) Select high-level action
5:
Execute TI (M, h%)
D.3 Food Segmentation Training Details
Supe-Supe Dataset Generation. To cireumvent the painstaking process of pixel-level segmentation annotation for real food images, we design self-supervised annotation procedure. First, we record a grayscale RGB image of an empty plate, Tempty E R W Next, we manually place food items on the plate at random without changing the position of the plate, yielding new grayscale observation It. Let lai = 14 Tempty/. l. the framewise absolute difference between the full and empty plate. We initialize the ground truth segmentation mask M, corresponding to as a 2D array of zeros, and then assign M, Taitt > THRESH 1. In practice, we find that THRESH = 20 reasonably separates the foreground from the background to detect food. With this procedure, we can scalably collect 280 paired RGB food images and segmentation masks in real within an hour
14

---

and a half of data collection. This includes plate resets, food placement, image capture, and offline background subtraction post-processing.
Augmentation. We augment this dataset 8X by randomizing the linear contrast, gamma contrast, Gaussian blur amount, saturation, additive Gaussian noise, translation, and rotation of each RGB image, applying only the affine component of these same transformations to the associated segmentation masks.
Training Objective. We train Sang, implemented as a fully convolutional FPN (Feature Pyramid Network) using Dice loss:
2 X Ldice = 1 2 x TP + FN + FP
(6)
This objective encourages high overlap between predicted and ground truth masks, as TP, FN, FP denote the number of pixel-level true positives, false negatives, and false positives in a prediction Mr compared to ground truth M,.
E
Experimental Details
E.1 Noodle Acquisition Hardware Setup
Using a Franka Panda 7DoF robot, we aim to clear a plate of cooked noodles within a horizon of T = 10 actions. We fit the end-effector with custom 3D-printed mount consisting of RealSense D435 camera and a fork. To enable autonomous twirling and scooping capabilities, we extend the fork's range of motion via two servo motors (Dynamixel XC330-M288-TD We control the robot with Cartesian impedance controller, where the programmable servos are integrated the forward kinematics chain for positional control of the fork tip. The action space consists of either group (rearrangement) or nwirl (acquisition) actions, instantiated according to the learned segmentation and pose estimation models detailed in Section 4.2.
group action consolidates sparsely distributed plate by sensing the furthest and densest points, (ăy, Uy. 2 and (ia, Dd. Ea), and executing planar push from the furthest to densest point. In a twirl action, we infer the densest point and appropriate insertion angle n, roughly orthogonal to the grain of majority of the noodles. We use positional control to insert the fork into the densest noodle pile, and execute fixed twirling motion by making two rotations at radians per second. Finally, the fork scoops upward until nearly horizontal (3 80%) and the robot brings the acquired noodles to neutral position in the workspace.
For all trials, we use non-slip plastic dinner plate, and mimick a bite successfully taken by user after twirling by autonomously untwirling onto a discard plate.
E.2 Bimanual Scooping Hardware Setup
We assume access to two Franka Panda robots, equipped with a pusher tool and a metal spoon, respectively, and an external RealSense D435 camera for perception. With this setup, aim to acquire granular items on a plate using either group (rearrangement) or scoop (acquisition) actions, evaluate our system the task of
with total action budget of T 8 actions. In particular, scooping jelly beans, but VAPORS is agnostic to the exact choice of food. Following the experimental setup of Grannen al. [7], the spoon is mounted an angle to the robot end-effector (3 = 30"). The pusher a concave 3D-ppidted tool intended to push piles of items into the spoon and maintain contact during lifting so to prevent spillage.
Grouping actions are unimanual and use the pusher tool to push the sensed furthest item to the densest region on the tray. In scoop action, we sense the densest pile and execute a parameterized motion in which the pusher and spoon move towards each other synchronously at a fixed y = 180%. Once they 80 and lifting neutral
within fixed threshold apart, the spoon scoops by tilting workspace position.
We conduct all trials on a standard cooking tray due to the enlarged manipulation workspace for two arms. To simulate user's bite between actions, we manually discard the spoon contents after scoop action.
15

---

37
42 54
86
101
106
Figure 10: Noodle Acquisition Rollout: We visualize actions performed by VAPORS on the task of
clearing an initially half-full plate of Tier 3 noodles. As the distribution of noodles on the plate becomes sparse 0, 42, 54). VAPORS employs grouping strategies (black) to push noodles in close proximity. Once consolidated, PARO employs twirling 12, 66, 86), as shown in red, for efficient plate elearance, where denotes the clock time in seconds.
1=7
22
82
I 67
52
Bimanual Scooping Rollout: Using bimanual setup with two Franka Emika Panda robots, VAPORS performs 6 actions consisting of grouping (black arrows) and scooping (red arrows) to acquire jelly beans on a tray. By grouping when the tray sparse and acquiring when a bite-sized clump forms, VAPORS demonstrates efficient acquisition. The annotated timestamps denote clock time in seconds.
Figure 11:
E,3 Implementation Details
For each task, we use the following training procedures, We train TH on simulated segmentation observations of size 64 64 for 2, 250 update steps, where we collect episode every 150 update steps. We instantiate the reward as per Eq. (3) with = 0.66, and train each model using the Adam optimizer with with a learning rate of 10 e 10-4, and gradient clipping norm of 1000 with batch size B = 32, based on the training procedure from [37]. Each model takes approximately 1 hour to train on an Nvidia RTX A4000 GPU. To instantiate Tr, we train say and fori from real data. For segmentation, we collect 280 paired examples of images and binary segmentation masks using the self-supervised annotation process from Section 4.2, where we use cooked noodles randomized shape and sauce variations as well as jelly beans of randomized colors. We augment each dataset 10X and train for 50 epochs, which takes approximately 3 hours on an NVIDIA GeForce RTX 2080 GPU. In order to instantiate the twirl primitive for noodle acquisition, we additionally train fori to predict fork tine orientation y from 280 manually annotated crops of noodles as per Section 4.2, augmented 8X. The train time for for is approximately 1 hour an NVIDIA GeForce RTX 2080 GPU. For deployment, we use an Intel NUC 7 for inference and robot control via a ROS 2-based control stack.
E.4 Additional Experimental Results
In this section, we supplement the experimental findings from Section 5 with additional results.
Misperception (A) Wrong Action (B) Imprecision (C)
Slip (D)
Small
Cabbage
Bite
Figure 12: VAPORS Failure Modes: We illustrate the 4 most commonly ohserved failure modes with VAPORS on noodle acquisition. Misperception (A) occurs when Tz erroneously senses vegetables, sauce, plate glare noodle due false positives with fori, leading to misplanned action such grouping in that region. Occasionally, मн may acquire when rearrangement is more appropriate, leading to a low-volume bite (B). In terms of action execution, food acquisition requires care so as to not miss food (C), as seen in the grouping motion which fails to group singular noodle strands due to system imprecision. Finally, slippage (D) can happen during acquisition with hard- to-model items such those coated in sauce.
16

---

Plate Clearance: Fig. 10 and Fig. 11 visualize two rollouts of VAPORS on plate clearance. We note that visually, VAPORS tends favor grouping as the plates become sparser and otherwise acquires when there is reasonably sized bite available.
VAPORS Failure Mode Categorization: In addition evaluating the percentage of the plate cleared, we observe the occurrence of a few failure modes, as depicted in Fig. 12. A misplanmed action (A) can occur due to a perception error, such as accidentally perceiving sauce, garnish, a vegetable, or plate specularity for noodles and erroneously grouping or twirling in that region. Alternatively, this can happen when (B) the robot twirls when grouping more appropriate or vice versa. A mis-executed action failure occurs when (C) the fork fails to group or acquire due system imprecision or (D) the noodles slip during acquisition due to sauce. In Table 1, we also report the per-action failure rate, computed as the total number of failures over the total number of actions (60 =6 trials x T 10).
Qualitative User Study: In the Likert survey administered to gauge user preferences across methods, we report in Fig. 6 the statistical findings which are significant. In Table 2, we indicate the specific margin of significance for each of the criteria, obtained via 1-way ANOVA testing.
Table 2: 1-Way ANOVA Statistically-Siinffeen Findings (p-value
0.05)
Criterion
Method I
Method 2
p-value
Criterion
Method
Method 2
p-value
Ellictency
Heuristic
Acquire Only
0.0029
Afficiency
Heuristic
VAPORS
0.0004
Practicality
Heuristic
Acquire Only
0.0091
Efficiency
Acquire Only
VAPORS
0.0010
Reuse
Heuristic
Acquire Only
0.0093
Bite Size
Acquire Only
VAPORS
0.0318
Elficiency
Heuristic
Aequire Only
0.0029
Humanlike
Heuristic
VAPORS
0.0003
Efticiency
Acquire Only
VAPORS
0.0000
Humanlike
Acquire Only
VAPORS
0.0044
Bite Size
Heuristic
VAPORS
0.0029
Practicality
Hcuristic
VAPORS
0.0012
Size
Acquire Only
VAPORS
0.0002
Practicality
Acquire Only
VAPORS
0.0025
Humanlike
Acquire Only
VAPORS
0.0094
Reuse
Heuristie
VAPORS
0,0000
Practicality
Heuristic
Acquire Only
0.0091
Reuse
Aequire Only
VAPORS
0.0008
Practicality
Auquire Only
VAPORS
0.0001
Trust
Heuristic
VAPORS
0.0124
Reuse
Heuristic
Aoquire Only
0.0093
Trust
Acquire Only
VAPORS
0.0198
Reuse
Acquire Only
VAPORS
0.0001
Generalizability
Heuristic
VAPORS
0.0478
Trust
Acquire Only
VAPORS
0.0438
Generalizubility
Acquire Only
VAPORS
0.0481
Table 3: Noodle Acquisition
Table
Bimanual Scooping
In addition to the user study outlined in Section 5, we administered a second part of the study, in randomized order to the first, in which users were asked to pick preferred method for feeding in side-by-side comparisons of jelly bean acquisition trials. To control for the initial state of the jelly beans, we purposely arrange 16 beans into a 4 grid initially, and conduct two trials per method which are randomly selected for the comparisons. Although we would like to include an analogous side-by-side comparisons survey for noodle acquisition for completeness, we find in practice that controlling for the initial state of noodles is nontrivial due to their highly deformable nature and vast set of feasible initial configurations. This makes it difficult to present users with unbiased comparisons across methods.
Thus, for bimanual scooping, we presented all permutations of pairs of the three methods, for total of six comparisons overall. Empirically, we find that VAPORS is the preferred method by large margin compared to both baselines (Fig. 13).
17

---

Overall Rankings
Preferences
Spaghetti
Jelly Beans
*
VAPORS
3: Worst
3
Heuristic
2: Middle
2
VAPORS
Acquire
Only
1: Best
Figure 13: Overall Ratings Left: After observing all methods perform acquisition across 10 trials, we ask users rank all three methods from most least preferable. We find the VAPORS is most consistently ranked the best by statistically significant margin (p < 0.05, denoted '*") compared to baselines. Right: jelly bean acquisition, we control for the initial state of the plate by arranging the beans in a x 4 grid, and ask users to select their preferred method across 6 side by side acquisition videos of different methods. VAPORS is the preferred method by large margin compared to Heuristic and Acquire-Only.
18


Reviews:
[{"review_id": "lSGJm1SOaS", "reviewer": "Reviewer_XrpJ", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper describes a framework for solving long horizon Acquisition Policies for Robot-Assisted Feeding. The proposed method Visual Action Planning OveR Sequences (VAPORS) uses a hierarchal structure where the high level selects discrete manipulation strategies and the low level executes vision-parametrized policies. Also, this paper shows results on two real food manipulation tasks: robotic noodle acquisition and bimanual scooping with high efficiency and plate clearance.", "summary_of_recommendation": "The paper shows an interesting solution for long horizon Acquisition Policies.  Though more details, experiments, and some modifications are needed to be added to show the validity of the method, I am leaning toward accepting this work.\n", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "There are some minor points needed to be fixed:\n1) In the abstract, \n\"Meanwhile, existing approaches to long-horizon manipulation lack the flexibility to embed highly specialized primitives into their frameworks.\"\nPlease consider modifying this to be more appropriate cause in the citations15-20 you mentioned, they have ways to embed.\n2) It's better to put the planning objective function and algorithms back into the main paper for a better understanding", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Strengths:\n\nThe paper addresses interesting long-horizon robot-assisted feeding tasks -- robotic noodle acquisition and bimanual scooping. And the proposed method decouples the policy into separate high and low-level sub-policies.  The high-level part takes advantage of simulation for modeling high-level plate dynamics to leverage visual planning, and at the low level, the parameterized visual policy is executed for precise real manipulation. In general, the presentation for the method is clear and straightforward. Figures clearly express the algorithm step by step. There are sufficient quantitative results, baseline comparison and real-world execution videos look promising.\n\nWeaknesses and suggestions: \n\n1) It seems to me the two low-level primitives for the noodle task are all used during the execution which may be easy for the high-level dynamics model/planner to plan. Could you add an ablation study in sim by adding some redundant primitives here to test whether the high-level planner could ignore those redundant primitives and be robust to those \"noises\"\n2) To further show the generalization power of the proposed method, it needs to have an example beyond clearing plates. For example, the planner should be able to generate a plan for making a certain pattern for noodles (e.g make the noodle into a circle).\n3) One low-bound baseline is needed: treat random select low-level primitives as a high level policy and test the results\n4) Another interesting baseline comparison: use the data trained for the dynamics model to train a primitive selector and treat the selector as a high-level policy and test the results", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}, {"review_id": "id6_UOeWQd", "reviewer": "Reviewer_cA3F", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper presents a system for long-horizon planning for bite acquisition for a food manipulating robot intended to eventually be used in assistive feeding.", "summary_of_recommendation": "Overall, this paper has some exciting technical components but I am very concerned about the user study; it is not appropriate to evaluate assistive technologies with nondisabled users without careful attention to the interpretation of results and some discussion of how the results might change with disabled users.  I currently lean towards rejection, but this is highly dependent on whether the authors can provide a convincing plan in the rebuttal for providing this care and nuance.\n\nPost rebuttal comment:\nI am trusting that the authors will update their paper appropriately, but they have made an effort to address my main concerns (although having made an effort to contact users is not the same as having validated with those users).  I will raise my score to a weak accept.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are not well addressed", "questions_for_rebuttal": "I would like the authors to address the ecological validity of the results with respect to intended users.  Ideally, I would like the authors to propose a plan for language to address the limitations with regard to interpreting the results and add nuance to their claims about user satisfaction.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Strengths:\n- The proposed system tackles a challenging robotics problem (manipulation of food) and shows that including long-horizon planning for when to group food and when to acquire a bite can improve effectiveness at that task\n- The evaluation includes real-world foods (restaurant meals) as well as simpler food (jellybeans).\n\nWeaknesses:\n- The preference survey, conducted with non-disabled participants, does not tell the reader anything about the appropriateness of the method for assisted feeding for people with disabilities.  It is not appropriate to evaluate assistive technologies by asking nondisabled people to imagine being disabled. (see Cynthia L. Bennett and Daniela K. Rosner. 2019. The Promise of Empathy: Design, Disability, and Knowing the \"Other\". In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). Association for Computing Machinery, New York, NY, USA, Paper 298, 1–13. https://doi.org/10.1145/3290605.3300528).  It would be better to leave this out or focus the study on how a co-diner would perceive the system.  If the perceptions of co-diners is not the point that the authors want to make, then they should conduct their study with disabled participants.  Overall, it is disappointing that the work has such high ecological validity in terms of the foods (ordered from real restaurants) and low ecological validity in terms of users, especially since the use of non-disabled proxies is not even listed as a limitation.\n- Please clarify whether correction was used for the large number of statistical tests.\n\n\nMinor points:\n- In the first sentence, the \"toll of meal preparation\" is cited as a reason that people can't eat independently.  This is definitely not true (and the cited papers do not support this claim).\n\n- I'm not sure that referring to sauce as adversarial is appropriate, given that sauce on noodles is a very normal configuration", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning."}, "extra_text": {}}, {"review_id": "-H19PPTj1o", "reviewer": "Reviewer_iomA", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper presents a hierarchical algorithm to learn how to sequence food acquisition primitives associated with robot-assisted feeding. Specifically, a high-level planning policy is learned in simulation (to avoid sim-to-real gap) and a low-level visual policy is learned offline on real-world data to execute parameterized primitives.", "summary_of_recommendation": "The paper is well-written and addresses an impactful problem. The results conclusively demonstrate the effectiveness of the proposed method, including a user study. The paper lacks empirical comparisons with prior methods, but makes good qualitative arguments for how it improves upon them.", "technical_quality": "Excellent: Very convincing and thorough.", "clarity_of_presentation": "Excellent: Enjoyable to read.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "Please address my questions and criticisms from the \"Weaknesses and Suggestions\" section of my review comments and let me know if there are any substantive misunderstandings on my part. Additionally, please address the following:\n- In the generalization experiments, was the same policies (without any additional retraining/fine tuning) used for all 7 plates across 3 tiers?\n- How was the coverage threshold for the heuristic baseline chosen?\n- What specific statistical tests were performed? Were the associated assumptions validated?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**Strengths**\n- The paper is expertly-written and was easy to follow. Numerous design choices are well-justified.\n- The paper addresses an important problem and does a great job of motivating it and discussing the associated literature.\n- Although not emphasized in the paper, I wonder if the modified simulator can be a valuable contribution to the research community focused on robot-assisted feeding.\n- The experimental design is comprehensive. I particularly appreciate the effort to quantify and categorize complex food acquisition tasks and settings.\n- The paper also includes a user study to demonstrate that users tend to prefer the proposed method over baselines.\n- The authors have made their code and the dataset open-source.\n\n**Weaknesses**\n- While I appreciate the sound arguments in the paper pertaining to comparisons with existing methods, the paper lacks empirical comparisons with existing algorithms to substantiate the qualitative arguments.\n- As the authors acknowledge, the set of primitives explored is rather limited. Adding new primitives will likely involve expert-design of parameterization and integration into the pipeline.\n- It is not clear why the baselines are omitted from the generalization experiment.", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers commend the paper for being well-written and organized. The proposed framework for long horizon food acquisition policies for robot-assisted feeding was well received by reviewers. I encourage the authors to further address any remaining points raised in the reviews for the final version of the paper.
Decision: Accept (Poster)