Learning Granular Media Avalanche Behavior for Indirectly Manipulating Obstacles on a Granular Slope

Authors: haodi  hu
feifei  qian
daniel  seita
Venue: robot-learning_org_CoRL_2024_Conference
Published: 2024-06-03 02:52:19

Abstract: Legged robot locomotion on sand slopes is challenging due to the complex dynamics of granular media and how the lack of solid surfaces can hinder locomotion. A promising strategy, inspired by ghost crabs and other organisms in nature, is to strategically interact with rocks, debris, and other obstacles to facilitate movement. To provide legged robots with this ability, we present a novel approach that leverages avalanche dynamics to indirectly manipulate objects on a granular slope. We use a Vision Transformer (ViT) to process image representations of granular dynamics and robot excavation actions. The ViT predicts object movement, which we use to determine which leg excavation action to execute. We collect training data from 100 real physical trials and, at test time, deploy our trained model in novel settings. Experimental results suggest that our model can accurately predict object movements and achieve a success rate ≥ 80% in a variety of manipulation tasks with up to four obstacles, and can also generalize to objects with different physics properties. To our knowledge, this is the first paper to leverage granular media avalanche dynamics to indirectly manipulate objects on granular slopes. Supplementary material is available at https://sites.google.com/view/grain-corl2024/home.

Main paper: 
 introduction: 
 Legged locomotion across granular surfaces such as sand is a formidable challenge due to factors such as insufficient support offered by the sand surface and the complex dynamics of leg-sand interactions [1, 2, 3] . It is particularly challenging for robots to climb up steep granular slopes, as the sand could easily flow underneath robot legs due to the reduced shear resistance forces [4] . Recent studies on obstacle-aided robot locomotion show the potential for legged robots to strategically leverage large obstacles within sand, such as rocks and boulders, to traverse granular and uneven terrains [5, 6, 7, 8] . However, such "obstacle-aided locomotion" strategies require specific leg-obstacle contact locations [5] , thus the ability to move rocks and boulders to desired locations became essential for these strategies to apply. To address this challenge, this study aims to propose a method for a legged robot to effectively reposition obstacles on granular slopes by primarily leveraging indirect manipulation. Prior work has shown that external disturbances on a sand incline can trigger avalanche behavior [9, 10, 11] , suggesting the potential for a legged robot to exploit this property to advantageously relocate obstacles on a granular surface.In this work, we propose Granular Robotic Avalanche INteraction (GRAIN), a novel learning-based method for leveraging granular avalanche dynamics for indirectly manipulating objects on a granular slope. Due to a lack of accurate simulators for simulating legged robots and avalanche behavior on granular slopes, we do all experiments in the real world. We use an RHex [12] family robot leg as an external disturbance source which performs excavation actions within a grain tank with mechanical 8th Conference on Robot Learning (CoRL 2024), Munich, Germany. support to form a slope. We also design a gantry structure to enable the robot leg to move to different positions for performing leg excavations. We collect training data through physical interaction and deploy our trained model in the real world with a quadrupedal robot. See Figure 1 for our setup.Accurately predicting the granular media dynamics is essential for our proposed task, but is fundamentally challenging [1, 11, 13, 14] . To address this, we leverage learning methods in this study. We place a rigid 3D-printed obstacle on the granular slope and collect data by applying continuous robot leg excavation actions to investigate avalanche behavior. We represent the granular surface state via the current depth image and the change in depth between sequential robot leg excavations, and we also use an image to represent the leg excavation action. This enables a unified image-based input to our granular media dynamics model. We train a ViT [15] to take our proposed granular media and robot excavation action representations as input and output the object movement on the granular slope. Experiments over 30 trials suggest that our model can accurately predict object movements on a granular slope, and can generalize to objects with different physics properties.In summary, our main contributions are as follows:1. A novel problem formulation of using legged robots and leg excavation actions to indirectly manipulate obstacles to targets over complex, granular terrains. 2. GRAIN, a novel approach that uses image-based representations of granular dynamics and legged excavation actions to predict object movements on a granular surface. 3. Experimental results showing that a legged robot using GRAIN can move obstacles to targets, and that GRAIN outperforms an ablation and a non-learning baseline. 
related work: 
 Robot interaction with granular media: Researchers have explored robotics and granular media in locomotion and manipulation contexts. For example, researchers have enabled crawling [16, 4] , hopping [17, 18] , running [19, 2, 3] , climbing [20] , and rapid turning [21] over granular media by leveraging advances in granular force models [1, 13] and machine learning methods [22, 23] . We focus on the orthogonal task of leveraging the properties of granular media to manipulate objects on it. This requires some understanding of how granular media behave, which could be from direct physics analysis [24, 25, 26] or learned models [27, 28] . These models can aid common manipulation tasks involving granular media, which include pouring [29, 30, 31, 32, 33, 34] , scooping and bulldozing [35] , trenching [36, 14] or adjusting soil with plates [37] . Among relevant prior work, Wang et al. [38] and Xue et al. [39] study particle-based and density field-based representations, respectively, to learn dynamics models for planar tabletop manipulation of granular media, and Schenck et al. [40] use image representations and train a convolutional neural network to predict changes in the granular media from scooping or pouring actions. As in [40] , we use image-based representations of granular media, but our actions are based on leg excavations instead of scooping, pouring, or other tabletop manipulation actions. Moreover, we use legs of a legged robot to indirectly manipulate objects on granular media surfaces.Locomotion over diverse terrains: Recent research has proposed methods that enable legged robots to traverse over a variety of terrains, including outdoor settings that may have sand, vegetation, rocks, or other granular media. A promising technique is reinforcement learning coupled with advanced simulators, which can help avoid significant manual engineering [41, 42] . To better handle diverse terrains, one line of work proposes adaptation methods, either via rapid motor adaption [43, 44] or by encoding a family of gait methods which facilities tuning to new terrains [45] .Other works study navigation over challenging terrains [46] , which may involve climbing, jumping [47] , and crawling under parkour-style settings [48, 49] . While impressive, these works study locomotion over terrains that are much sturdier than our granular surface. Furthermore, they primarily consider locomotion, whereas we focus on manipulating objects on a granular surface.Manipulation with legged robots: While legged robots primarily use legs to move to a target location, they can also use legs for manipulation. For example, researchers have proposed methods for using legs to kick soccer balls [50, 51] and to push obstacles [52, 53] . A legged robot can also use two legs to stand up to better enable other legs to press against higher objects such as door buttons [54] . Other works mount an arm on top of a legged robot, and leverage methods such as optimization [55, 56] or machine learning [57, 58, 59] to allow the arm to manipulate objects. These works use legged robots for manipulation via direct contact with an object. To our knowledge, our work is the first to show a legged robot indirectly manipulating an object to make it reach a desired pose. To do this, the robot adjusts a granular surface that supports the object. 
preliminaries and problem statement: 
 We consider the RHex [12] family of legged robots, with 1 DOF for each leg. We generate an excavation action, per leg, by commanding the leg to rotate at a constant angular speed for one circular cycle. We assume the robot lies on a granular (sand-like) surface which has a slope of Φ degrees. This surface has K ≥ 1 rigid obstacles, and we indicate their respective positions at a given time t as {st , . . . , s(K) t}. The task is to move all obstacles from their initial locations to pre-specified desired locations {p (1) , . . . , p (K) }. After doing this, we may also want to move the obstacles to a second target location, and we indicate these optional (per-obstacle) target locations as {q (1) , . . . , q (K) }. Here, each s(k) t ∈ R 2 , p (k) ∈ R 2 ,and q (k) ∈ R 2 for k ∈ {1, 2, . . . , K}, since we specify 2D positions over an image of the granular surface. For notational convenience, when K = 1 we may suppress the superscript (k). We assume access to an overhead camera, which provides (H × W ) depth image observations x t . The objective is to learn a policy which produces a leg excavation action a t at time t, where the robot rotates one of its legs. A trial consists of executing the robot's policy until a termination criteria. We evaluate a trial's performance by averaging the mean absolute error (MAE) distance among all obstacle positions and their respective target positions. We also use MAE to evaluate models which predict where obstacles move based on robot actions. 
approach: grain 4.1 image representations of granular dynamics: 
 Prior work has shown that external disturbances can trigger avalanche behavior on granular slopes [60] . Inspired by this, we aim to leverage robot leg excavation actions to cause avalanche behaviors to move obstacles to desired locations. Intuitively, the robot leg excavation location affects the avalanche area, and an obstacle's relative position to the excavation affects how much it is influenced by the avalanche. Due to the complexities of modeling avalanche dynamics and obstacle movements on the granular surface, we learn an action-conditioned dynamics machine learning model to predict obstacle movement. Similar learned models have predicted complex physical interactions for planning robot manipulation [61, 62, 63] , suggesting their utility for our task. To represent the obstacle and the granular slope surface, we use a top-view depth image, x t . Furthermore, during preliminary experiments, we observed that successfully relocating obstacles often required consecutive excavations at the same location. See Figure 3 for a visualization of grain flows with sequential excavations. To highlight this physical finding in our model, we use the change in the depth images of the granular slope surface between sequential excavation actions, denoted as ∆x t = x t -x t-1 for t > 0, with ∆x t = 0 (all pixels zero) if t = 0. We also introduce an image representation of robot leg excavation locations, which enables the action input to be spatially aligned with x t and ∆x t . This may help the network better learn the avalanche behavior compared to if the action is processed separately and concatenated with downstream visual features. We discretize excavation locations to 15 locations in a 5 × 3 grid (see Figure 5 ). We plot a white square with a side length the same as the robot leg length on a black background to represent the robot leg excavation location, I(a t ), where a t is the excavation location on the 2D planar surface, and I is a function that generates the RGB image representation of a t .We train a ViT to learn granular avalanche dynamics from C-shape robot leg excavations. The ViT predicts one obstacle movement on the granular slope, and we deal with multiple obstacles at test-time by repeatedly querying the model (see Sec. 4.3). The training loss L is defined as:EQUATIONwhere F θ is the ViT parameterized by θ that takes, as input, a channel-wise concatenation of three spatially-aligned images: the depth image x t , the change in depth ∆x t , and the action representation I(a t ). The ViT outputs the predicted post-excavation obstacle position s t+1 = F θ (x t , ∆x t , I(a t )), which we compare with the ground truth obstacle position s t+1 on the inclined 2D surface at time t + 1. Since the ViT predicts continuous values, we modify the ViT's default MLP classification header with an MLP regression header. See Figure 2 (left) for an overview of training.We propose a greedy strategy for manipulation, which means the robot leg performs the excavation action at the location that has a maximum obstacle movement projection on the current line connecting the desired location and obstacle center toward the target area, described in Eq. 2:EQUATIONe� where A is the set of 15 possible actions (see Figure 5 ), and e T t is a 2D unit vector that points from the obstacle center to the target location at time t (see Figure 4 , target is the shaded red square). Parameters such as leg insertion depth and the angle of the excavation could also offer additional terrain-manipulation opportunities. As an initial step, we focus on demonstrating the effect of leg excavation location. Prior work has shown that a greedy policy for planning can be useful for object relocation [64] , and we hypothesize the same may be true in our setting.Our model is trained using robot excavation actions with a single obstacle. To use the model with multiple obstacles, we treat multiple obstacle movement prediction as a collection of independent predictions of a single obstacle movement. In particular, we randomly select an obstacle and mask out other obstacles on the depth image input, x t , where we use a window that has a length 3 times the obstacle radius to compute the average pixel values in the windows and replace the pixel value of the obstacle with the computed averaged pixel value. Furthermore, we mask out other obstacles in ∆x t , where we set the pixel values to 0 corresponding to other obstacle positions. We repeat this process for all obstacles and get the predictions of all obstacle movements. We modify our manipulation policy to fit the task; the policy now considers the sum of obstacles projected movement on lines that connect their centers to their desired locations as described in Eq. 3:EQUATIONwhere images x t and ∆x t are masked versions of x t and ∆x t . 
experiment setup: 
 Existing simulators used in learning-based legged robot manipulation research, such as PyBullet [65] , MuJoCo [66] , or IsaacGym [67] , do not support realistic robot interaction on granular media surfaces. Thus, we do all data collection, training, and experiments directly in the real world.Experiment environment: Figure 1 illustrates our experiment setup. The main structure of the testbed is a granular trackway (60 cm L × 60 cm W × 20 cm D) filled with 6 mm plastic BBs (Matrix Tactical Systems). The 6 mm particles were chosen as they behave rheologically similar with natural sand and soil [68, 1, 69] , while the simpler geometry and larger size facilitate imagebased model training. The granular trackway can be tilted up to 35 degrees to emulate a wide variety of sand slopes [11] in natural environments. To study the avalanche dynamics and object movement upon different leg excavation actions, we build a gantry system with two linear actuators (one moves along the x axis and another along the y axis) to move a C-shape robot leg on a 2D surface above the granular slope. The C-shape robot leg has a diameter of 6.0 cm and a width of 2.0 cm, and the rotation center of the leg is 1.0 cm above the initial granular slope surface. The rotation frequency of the robot leg is fixed at 0.33 Hz. This is sufficiently above the granular surface, so the robot avoids touching it while transitioning between consecutive excavation actions. We mount an RGBD camera (Intel RealSense 435-i) above the granular slope to record the granular flow and obstacle movement.We collect a dataset of 100 trials, where each trial has 10 excavation actions. The time interval between two consecutive excavation actions is 12 s to enable the robot leg to transit to different locations. Before each trial, a human operator manually smoothed the granular media to a (roughly) even granular slope with an inclination angle Φ = 18 degrees. This inclination angle is close to the angle of repose [70] of the granular material used in our work, which facilitates the study of the avalanche dynamics. Once the granular surface is prepared, the human placed a 3D-printed (PLA) obstacle on the granular surface at different locations relative to the leg. For all 100 trials, a semi-spherical obstacle with a 5 cm diameter is used. The RGBD camera has a video streaming rate of 15 Hz, and it collects 640x480 RGBD images after every excavation action. The ground truth obstacle movement is calculated based on the post-excavation RGBD image. Among the 100 trials, 36 use the same excavation location but different initial obstacle positions, 30 use the same initial obstacle position but different excavation locations, and 34 vary both the initial obstacle positions and excavation locations for each action. We use this data to train F θ .Evaluation: For each trial, after the human places the obstacles, a computer program randomly selects target location(s) for each obstacle. Targets can be anywhere in the robot excavation action space, as long as they do not overlap with each other. Then, the program randomly selects a method among a set of methods we test (GRAIN, a baseline, or an ablation, see Sec. 6.2) for manipulation, to reduce human bias in making initial settings easier for our method. We evaluate our dynamics model and manipulation outcomes based on the prediction error for each excavation action, using Mean Absolute Error (MAE). In test trials, we compute performance based on the final distance between the obstacle center and its desired location. The success threshold is below 2.5 cm (the radius of the obstacle) which we measure via converting pixel distances in images to centimeters. We also use the average error between prediction and ground truth as a performance evaluation.6 Experiment Results and Discussions 
model performance on predicting obstacle movements: 
 To evaluate our model's performance in predicting obstacle movements, we place an obstacle on an undisturbed granular slope and test 15 excavation action locations; we reset the obstacle to the same spot after each action. See Figure 5 for a comparison between the predictions and the ground truth locations. The MAE for these 15 excavation actions is 1.13 cm, below our 2.5 cm threshold. Based on the promising MAE results, we use our trained model for planning in Sec. 6.2 and Sec. 6.3.We evaluate GRAIN in real-world experiments with a single leg using four types of tasks:1. Single obstacle with a single task: Using K = 1 obstacle, with a target position p.2. Single obstacle with sequential tasks: Using K = 1 but with an additional (second) target q.3. Multiple obstacles: Using K = 4 obstacles with targets {p (1) , p (2) , p (3) , p (4) }. 4. Unseen obstacle: Using K = 1 obstacle with a target position p, but where the new obstacle has a star shape and weighs twice as much as the standard obstacle we use. We compare GRAIN against a baseline and an ablation. Our baseline is an algorithmic, non-learning manipulation strategy. The baseline randomly selects an obstacle, and the robot performs the excavation at the closest available location to the target until the obstacle has met one of two termination criteria (see next paragraph). Then, this baseline randomly selects one of the remaining obstacles to manipulate and repeats until all obstacles are selected. Our ablation investigates the importance of our image representation of the robot excavation action to the successful training of our model. We train a ViT with a lower-dimensional, 2D vector representation of the robot excavation action. This cannot be directly channel-wise combined with the depth-based image observations, so we instead input it to the regression header of our ViT.A manipulation trial terminates upon either of these conditions: (i) none of the excavation actions can take obstacles closer to their targets, or (ii) all obstacles have a small accumulated movement (≤ 0.5 cm) over 3 sequential excavations. In (i), for the baseline method, we terminate when we observe that the previous excavation action took the obstacle away from its target or reached it.Results from Tab. 1 suggest that GRAIN outperforms the baseline and the vector representation ablation. GRAIN has a success rate ≥ 80% on all manipulation tasks, while the baseline has trouble with the "Multiple Obstacles" task, and the ablation has trouble with the "Unseen obstacle" task. The baseline particularly struggles when manipulating multiple obstacles (20% success), as it does not consider other obstacle movements when manipulating one obstacle. The ablation's granular dynamics model has a higher (i.e., worse) MAE in the predictions, which results in lower success rates versus GRAIN. We show one trial each of "Single obstacle with sequential tasks" and "Multiple obstacles" in Figure 6 . We refer the reader to the supplementary website for videos. 1 : We compare the quantitative performance of our method (GRAIN) versus alternative methods on four manipulation tasks. We report MAE in two columns, both using the format: Mean(± Standard Deviation), over 5 trials each. The "Baseline" method involves no prediction, hence the "N/A" in the first MAE column.We evaluate GRAIN on the quadrupedal robot with 2 front legs as manipulators; we do not use the back legs since the robot body would likely block obstacles. Specifically, we use the same trained model as in Sec. 6.2 but we query the model at the 2 excavations where the legs are located, instead of the full set of 15. We show one trial in Figure 7 . Results in Tab. 2 suggest that GRAIN achieves higher performance (i.e., lower MAE) than the baseline or vector representation methods. Table 2 : We compare the performance of GRAIN and alternative methods on the multiple obstacles task (with 3 obstacles) using the quadrupedal robot, in a similar format as Tab. 1, except statistics are over 10 trials each.Figure 7 : One trial of a quadruped robot manipulating three obstacles to reach targets (indicated with red shaded boxes). The yellow boxes highlight the leg that performs the excavation action at each step, and the green dots are the predicted post-excavation locations of obstacles after the leg performs the action. A common failure of GRAIN with obstacles occurs when the robot leg moves one obstacle to its target while simultaneously moving other obstacles too far from their targets. See Fig. 8 for an example, where two obstacles are already at their target locations (red shaded boxes), but the robot has trouble moving the third obstacle to its target (white arrow and green shaded box). The rightmost obstacle has reached its target, and further excavations will move it away from its target. Our manipulation policy only predicts obstacle movements for one step, and thus can have difficulty with tasks that require multiple-step planning, which we plan to address in future work. 
conclusion: 
 In this work, we present GRAIN, a method for a legged robot to indirectly manipulate obstacles on a granular surface. Our method trains a neural network to predict where an obstacle on a granular surface moves as the result of a legged robot's excavation action, enabling a quadrupedal robot to leverage granular avalanche dynamics to relocate obstacles. To our knowledge, this is the first work to explore indirect manipulation on deformable surfaces via legged robots. As the first step towards the complex problem of indirectly manipulating obstacles via strategic granular avalanches, we used granular particles of relatively simple geometry, and primarily focused on individual obstacle movement prediction with minimal inter-obstacle interactions. Prior work has shown that these simpler geometry particles behavior qualitatively similar in terms of rheology and interaction force laws [1] , suggesting strong potential for GRAIN to be extended to more complex scenarios in future work. 
 
 Appendix: To highlight the relationship among the robot excavation action, avalanche behavior and obstacle movement, we introduce our image representation of the excavation actions in Sec. 4.1. The pseudocode in Alg. 1 shows how we obtain this representation. We show several examples of the image representation in Figure 9 , for different locations of the robot leg. A.2 Algorithm for Masking Obstacles in x t Sec. 4.3 discusses how we handle multiple obstacles. In Alg. 2, we formalize our method to mask other unselected obstacles. This lets the masked images look similar to images from the single obstacle case, which we use for training the ViT. See Figure 10 x start ← max(0, x -B 2 )4:y start ← max(0, y -B 2 )5:x end ← min(H, x + B 2 )6:y end ← min(W, y + B 2 ) 7: S ← {I[i, j] | x start ≤ i < x end , y start ≤ j < y end } 8: avg ← 1 (xend-xstart)(yend-ystart) (i,j)∈S I[i, j] 9:for each (i, j) ∈ P do Our overall data includes 10,437 images, which we obtain from 100 experiment trials. We split this data into 70% for training, 15% for validation, and the last 15% for testing. The input to our ViT is an image representation stacked channel-wise into a 7×128×128 input. Our ViT has 46M trainable parameters, and is a smaller version of the ViT-Base model [15] so that we could fit it on our GPU. We train the ViT from scratch. To apply ViT to our task, we change the last layer of the ViT so that it has an MLP header that outputs a 2D vector corresponding to the predicted obstacle center position on the granular slope. Moreover, as we have a continuous prediction instead of a discrete prediction, we used MAE as the loss function instead of cross-entropy loss. We include our hyperparameters in Table 3 To test our trained model's generalization ability, we use obstacles with different shapes and weights in this task. Specifically, we use a star shape obstacle, a cuboid obstacle that has half of the weight of the obstacles used in the training dataset, and a hemisphere obstacle the same size but 4 times the weight of the obstacles used in the training dataset. All obstacles are 3D-printed. We place these unseen obstacles on the granular slope plus the obstacle we use in the training dataset as a total of K = 4 obstacles with a random distribution and executed the manipulation policy. We show one manipulation trial in Figure 12 . Our system succeeds in 3 out of 5 trials.  


Decision: Summarizing the strengths and weaknesses pointed out by the reviewers:

Strengths:
- Well defined problem with complex dynamics that can benefit from a learning-based solution
- Visual transformer + RL is a novel solution to the problem
- Good data collection setup
- Results show the proposed method is effective
- Well-written

Weaknesses:
- Claims about prior work are incorrect (see reviewer's comments for details)
- The action space seems to discretized to be effective, but no exploration of this is done
- The object independence assumption is not well justified
- The paper does not state how the robot moves between action locations in a single trial
- Baseline/ablation is too simple to be an adequate comparison
- The granular media used (spherical plastic particles) has limited applicability to real world scenarios

Overall, the reviewers are split (1 weak reject vs. 1 weak accept + 1 strong accept). The concerns of the weak reject reviewer appear to be mostly addressable by adding content to the paper and running a small experiment. The authors should address as many concerns of the reviewers as they can.

**Update after rebuttal**: The authors' provided a rebuttal where they directly responded to the reviewers' comments and made minor modifications to their paper. None of the reviewers replied or updated their recommendations, so let us take a look and see if the rebuttal sufficiently addressed their concerns. 2 of the reviewers already recommended acceptance, so we will focus on reviewer fr2L, who recommended weak reject. The reviewer very helpfully numbered their concerns, so we will analyze each in order:

1. The authors' response was quite extensive. It appears as though they attempted to implement both the papers referenced by the reviewer. However, they were only able to partially implement 1 of them. They included a results figure in their rebuttal, but it is not in the paper. It would have been nice to include that in the paper/appendix. The metric used in this figure is training loss, not a great metric but okay (assuming it's loss on a hold-out test set). I would say, given these results aren't in the paper, this concern is only partially addressed.

2. Again, the authors added a figure in the rebuttal but not in the paper/appendix. They show a comparison of a finer discretization than they used in the paper, and the results are very similar, justifying their choice of 5cm bins. However, they only evaluate on a single row of actions, rather than the whole space. It would have been nice to see this on the entire experiment space.

3. No new results for this one, just the authors' acknowledging the limitations of assuming independence between the objects and saying that's for future work.

4. It be confusing from the paper to understand how these actions are done, but upon a close reading it is clear that the data is collected with a gantry robot, thus locomotion is not an issue.

5. The video link was updated, and the videos do help understanding the experimental setup better.

Overall, I'd say reviewer fr2L's concerns were only partially addressed. Nonetheless, given that the other 2 reviewers recommend acceptance, and reviewer M9RF strong accept, I would say this paper should be accepted.

Reviews: review: title: title: Review
paper_summary: paper_summary: The paper proposes a method to manipulate obstacles on a granular slope (surface covered in granular material such as sand). The key idea is to cause avalanche behavior of the granular material using the robot’s end effector, in this case, the leg of a quadruped robot. This behavior indirectly moves the obstacle to a desired location such that the robot can find a collision-free path. The technical method is an action-condition model that predicts future obstacle movement subject to such maneuver. Both data collection and evaluation is done in a real-world environment. The data collected by repeatedly perturbing the granular slope with a leg-shaped manipulator mounted on a fixed gantry. The trained model is also transferred to a quadruped robot. The model is evaluated based on the displacement between the desired end positions of the obstacles and the positions reached.
main_review: main_review: Overall, I like the problem setup. Manipulating granular material that is subject to complex dynamics clearly benefits from a learning-based solution. I also like the data collection setup. It is quite clever to gather data with a fixed gantry. The paper is overall well-written.

My concerns listed in the order of importance to my rating:
1. The authors claim in Line 64-65 that none of the prior works on granular material manipulation is comparable because the problem setup in this paper aims to manipulate an object through the granular medium. I disagree with this claim. There are quite a few works that study generalized inter-object dynamics, such as [1-2]. The methods can be easily applied to the problem setup by adopting the action discretization and either changing the output to the obstacle location or directly reading out the object location if a particle-based such as [1] is used.  [2] also uses a similar image-based action representation as GRAIN. I don’t think changing the output format can exempt the proposed method from comparison with closely-related prior research. 
- [1] Dynamic-Resolution Model Learning for Object Pile Manipulation Yixuan Wang, Yunzhu Li, Katherine Driggs-Campbell, Li Fei-Fei, Jiajun Wu, RSS 2023
- [2] Neural Field Dynamics Model for Granular Object Piles Manipulation, Shangjie Xue, Shuo Cheng, Pujith Kachana, Danfei Xu CoRL 2023

2. Discretizing excavation action to 15 bins seems to be a very strong assumption to make. I would assume for such an intricate task, more fine-grained action representation can help the model choose a more optimal solution. Moreover, I would expect the magnitude and the angle of the excavation action would affect the outcome significantly. I would suggest at least comparing with different discretization resolutions.

3. Merging independent predictions about each obstacle to approximate prediction for multi-obstacle scenarios seems to be a significant simplification. I would assume there are inter-obstacle interactions and obstacle-granular material interaction that gets affected by the relative configurations of the obstacles. The paper does not offer any justification why such a simplification is ok. 

4. How does the quadruped robot execute sequential actions at different locations in one single trial? Does it move to a new excavation location on its own? If so, it seems that the action for locomotion would affect the granular material. 

5. I would greatly appreciate a video demonstration of the system, as the system performance is best understood with animated motion.
questions: questions: See "weakness" section
limitations: limitations: 2
review_summary: review_summary: I would be willing to change my rating if the authors could adequately address my top 3 concerns.
title: title: Learning Granular Media Avalanche Behavior for Indirectly Manipulating Obstacles on a Granular Slope
paper_summary: paper_summary: In their paper, the authors propose their method (Granular Robotic Avalanche Interaction - GRAIN) for using vision transformers to obtain image representations of granular dynamics and robot excavation actions. They collect data from 100 real physical trials & their results show that their model has a success rate of 80%+ to predict object movements with up to 4 obstacles. They summarise their main contributions as the following: In summary, our main contributions are as follows: (1) novel problem formulation of using legged robots and leg excavation actions to indirectly manipulate obstacles to targets over complex, granular terrains; (2) their GRAIN method - a novel approach that uses image-based representations of granular dynamics and legged excavation actions to predict object movements on a granular surface; (3) experimental results showing that a legged robot using GRAIN can move obstacles to targets, and that GRAIN outperforms an ablation and a non-learning baseline.
main_review: main_review: The research presented a visual-transformer based method for obtaining image representations of granular materials to improve robot excavation (formulated as a quadruped with a ‘manipulation action’). The presented work outlines an interesting contribution towards robotic systems handling non-homogeneous granular materials, which as the author states, given the lack of realistic support on simulators, is an important challenge to tackle. Hence, using a visual transformer combined with an RL agent is a novel way of tackling the problem. The quality of the proposed method is evidenced when compared with other methods and demonstrates a successful outcome with respect to task completion. In addition, the contributions outlined seem to be supported by quantitative experiments. There clarity of the methods could be improved by adding more detail on the implementation and hyperparameters, including github repository/documentation for replicating the experiments. Overall, the work presents a significant contribution towards using robots with granular materials, where the main aspect is that vision transformers seem to help with the task. It would be great to outline what is the computational cost for achieving this. The main strength of this work is that they carried out their method successfully on the robotic platform, whereas the main weakness is that the baselines/ablation chosen are quite simple so it does not fully place this method with other related work e.g. CNN-based or other related works in manipulating granular materials.
questions: questions: * In your related work you mention: ‘In some of the most relevant prior work, Schenck et al. [36] use an image-based representation of granular media and train a convolutional neural network to predict changes in the granular media state.’ Could you elaborate more on why you decided to use a vision transformer rather than a CNN-based method? Did you try to use a CNN-based method and deemed it insufficient? It would be nice to add these results to your experimental evaluation.
* How representative is the obstacle of real-world obstacles in this domain? From looking at the appendix, it seems that you have used very simplified methods for dealing with obstacles (opencv-based methods). Was there a reason why you did not explore other segmentation methods e.g. segment anything model that would let you generalise to more complex obstacles?
* Did you explore your method with other terrains e.g. soil? How would it perform with a material that might be more challenging than sand? It would be great if you can add experiments/comments on this.
* ‘Prior work has shown that a greedy policy for planning can be useful for object relocation tasks [60], and we hypothesize that the same may be true for our setting.’ - how did you conclude this? What other policy strategies did you consider and do you have any quantitative results for these?
* ‘Since the ViT predicts continuous values, we modify the ViT’s default MLP classification header with an MLP regression header.’ - this is slightly confusing - did you just change the loss function here?
* ‘We train a ViT to learn granular avalanche dynamics from C-shape robot leg excavations.’ - how big is your training data? Did you train from scratch or fine-tune? Could you elaborate more on how this network looked like, what was the training/evaluation performance of this model?

Minor comments:
- Refer to images as Figure rather than Fig in the main text. 
- Hyperparameters for your models are not tabulated anywhere. Please add to the appendix to make the work more reproducible.
- Figure 2 should have the mathematical notation in the arrows and the text in the caption.
- Figure 4 and Figure 8 captions need to be expanded.
- The text wrapping around the figures is a bit confusing. I would advise the authors to rethink the layout of the paper. 
- The conclusion is a bit short - maybe remove the failure case image so you have more space for the conclusion?
limitations: limitations: 2
review_summary: review_summary: The paper could be interesting to the CORL community, should the authors address the questions presented in the rebuttal section. Given the efforts the authors put into carrying it out onto a real robotic platform, I believe this is achievable and would strengthen their work for publication.
title: title: Review of Submission153
paper_summary: paper_summary: Inspired by living organisms this work uses avalanche dynamics for indirect object manipulation on granular slopes. This paper employs a Vision Transformer (ViT) to process images of granular dynamics and robot actions. The ViT predicts object movement, guiding the robot's leg excavation actions. Trained with data from various trials, the model performs well in new and unseen settings, achieving over 80% success in various tasks with up to four obstacles and generalizing to objects with different physical properties.
main_review: main_review: The paper investigates a challenging aspect of robotics related to granular physics, which is notoriously difficult to model accurately. Due to the complex and unpredictable nature of granular media, traditional simulation environments are limited in their effectiveness. The authors address this hard-to-model problem by employing machine-learning approaches and conducting systematic experiments.

The paper is well-written and provides valuable insights that can contribute significantly to robotics. Specifically, this work has the potential to enhance robotic actuation principles in highly flowable and deformable terrains, such as those found in extraterrestrial environments. 

Strengths:
By leveraging machine learning to predict and manipulate object movements on granular slopes, this research might offers solutions that could improve the performance and adaptability of legged robots operating in challenging conditions. 

Weakness:
The granular terrain used in the study consists of spherical plastic particles, which do not naturally occur in the real world. As a result, even though the insights gained are valuable, the findings may not be directly transferable to real-life robotic scenarios. Additionally, the authors could have leveraged a simulation environment like Chrono, which facilitates the simulation of granular particles and multi-body robots, for more extensive learning and experimentation
questions: questions: 1) In Abstract: Relocate the statement  starting with "Supplementary material ..."

2) I recommend adding the following papers to the Related Work section, as they are pertinent to learning granular terrain manipulation for locomotion purposes.

    Real‐time remodeling of granular terrain for robot locomotion

    Learning manipulation of steep granular slopes for fast Mini Rover turning

3) I suggest providing more explanation in the text as to why the authors chose 6 mm plastic particles.

4) I wonder if the movement of one leg affects multiple objects at the same time during manipulation (Fig 7)

5) I suggest enhancing (and enlarging the related region) Fig 5 for clarity.

6) The conclusion section does not seem to reflect the importance of the findings of the paper.
limitations: limitations: 3
review_summary: review_summary: I recommend accepting the paper as it presents a novel approach to robotic manipulation of granular surfaces for displacing objects, which could potentially be used for obstacle-aided locomotion purposes.