ClutterGen: A Cluttered Scene Generator for Robot Learning

Authors: yinsen  jia
boyuan  chen
Venue: robot-learning_org_CoRL_2024_Conference
Published: 2024-06-07 00:32:16

Abstract: We introduce ClutterGen, a physically compliant simulation scene generator capable of producing highly diverse, cluttered, and stable scenes for robot learning. Generating such scenes is challenging as each object must adhere to physical laws like gravity and collision. As the number of objects increases, finding valid poses becomes more difficult, necessitating significant human engineering effort, which limits the diversity of the scenes. To overcome these challenges, we propose a reinforcement learning method that can be trained with physics-based reward signals provided by the simulator. Our experiments demonstrate that ClutterGen can generate cluttered object layouts with up to ten objects on confined table surfaces. Additionally, our policy design explicitly encourages the diversity of the generated scenes for open-ended generation. Our real-world robot results show that ClutterGen can be directly used for clutter rearrangement and stable placement policy training.

Main paper: 
 introduction: 
 Simulation has played an important role in advancing robot learning [1, 2, 3, 4, 5] by providing a controlled yet versatile environment for developing and testing algorithms. Data-driven approaches, in particular, typically deploy robots into simulations to undergo extensive training across a variety of diverse and randomized settings to enable generalizable and adaptable behaviors. Significant advancements in robot learning have been achieved by randomizing object shapes [4, 6, 7] , textures [8, 9, 10, 11] , and dynamics [12] . However, the layout of objects, despite being another critical factor, remains challenging to reach fully open-ended randomization.Unlike object properties, which can be easily specified within a range without interfering with other objects, object layout must consider the presence of other objects and physical feasibility. For instance, arranging objects in a scene requires ensuring that they do not overlap and are placed in stable positions instead of falling down from the air. Existing efforts often prevent this issue by fixing the object bases [13, 4, 14, 15] , but this strategy is not suitable for many objects like bottles or cups. As the number of objects increases within a limited space, generating a randomized yet stable object layout becomes exponentially difficult. Fig. 1(a) shows the challenge of using the widely adopted approach of random sampling and rejecting failure trials [16, 17, 18, 19] to generate valid scenes, with even seven objects on a table. Other methods require human manual specifications of object regions for local randomization [20, 21, 22] or apply discretization to the possible placement space to avoid collisions [17, 23, 24] . However, navigating and manipulating cluttered environments are essential challenges to deploying robot learning to the real world.We introduce ClutterGen, an auto-regressive simulation scene generator for creating physically compliant and highly diverse cluttered scenes. By framing cluttered scene generation as a reinforcement learning problem, ClutterGen learns a closed-loop policy from 3D observations without requiring pre-existing datasets or human specifications. Once trained, ClutterGen can be applied to variations of the original environment without fine-tuning. We further demonstrate the utility of ClutterGen in several downstream tasks including real-world clutter rearrangement and training robust placement policies for zero-shot sim-to-real transfer.Our main contributions are summarized as follows:• We frame the scene generation problem as a reinforcement learning task, enabling a closed-loop policy for creating physically compliant, cluttered, and diverse environments. Our model-free training process uses intuitive physics-based rewards without relying on pre-existing datasets or human specifications. • Our policy design encourages scene diversity to enhance general robot training.• We demonstrate ClutterGen's applicability to real-world clutter rearrangement tasks and its effectiveness as a synthetic data generator for training robust stable placement policies. 
related work: 
 Procedural Content Generation (PCG) Recently, there has been growing interest in data-driven PCG algorithms using generative models for indoor scene generation [25, 26, 27] , 3D object asset creation [28, 29] , and household rearrangement [30, 31, 32, 32, 33, 34] . Large-language models (LLMs) have also been used to guide scene generation [35, 36, 37] due to their prior embedded human knowledge. However, existing methods often impose strong heuristics such as limiting the regions of each object, and fail to consider physics feasibility, resulting in scenes with intersecting objects, floating objects, and unstable placements that are not suitable for generating flexible, diverse, and realistic robot training environments. In contrast, our method frames scene generation as a reinforcement learning task while explicitly considering the physical stability of the scene generation without relying on human-designed training datasets or expert heuristics.Scene-level Randomization for Robot Learning. Scene-level randomization has been crucial in robot learning. Recent work has focused on synthesizing object shapes [4, 6] and textures [8, 9, 10, 11] to reduce data collection costs and improve model robustness. However, randomizing object poses in simulation scenes remains challenging. Traditional methods, such as random rejection sampling [16, 17, 18, 19] , require significant human effort to define the range of object positions and verify scene validity. As the number of objects increases, finding valid poses becomes exponentially harder within a confined region, resulting in low diversity and success rates (Fig. 1 (a)). Recently, LLMs have been used to synthesize simulation scenes [38, 22, 39] , but they rely on pre-existing datasets or knowledge and extensive user-defined heuristics for object positions. Moreover, LLMs struggle to create accurate object placements without observing physical behaviors, leading to unstable scenes. Our method does not require a high-quality dataset or pre-defined pose constraints and trains by directly interacting with the simulation environment. Our method can create stable, diverse, and highly cluttered scenes for robot training. 
the cluttergen framework: 
 Designing a simulation environment for robot training typically involves a human expert observing the scene and object geometry, deciding on a placement, running simulations to check for collision and unstable pose issues, and adjusting placements as needed. This iterative process, repeated until object poses are finalized, is difficult to scale due to its heavy human involvement and time-consuming trial-and-error.We propose ClutterGen to automate the steps for generating cluttered simulations using a single learning agent. ClutterGen controls scene generation in a closed-loop manner under challenging cluttered scenarios.Our key idea is to formulate the process as a reinforcement learning task. Without prior knowledge or Fig. 2 : ClutterGen. We stack a sequence of attempt histories as input to the history sequence encoder to generate the history feature. This feature, combined with the perception feature from the point cloud encoder, is taken by ClutterGen to output the placement pose for the queried object. The simulator evaluates the placement's stability, determining whether to proceed to the next attempt or the next queried object placement. human input, ClutterGen can generate diverse and cluttered scenes using simple reward signals. Fig. 2 shows an overview of our method.Our problem is formulated as follows: A supporting object with a fixed support surface (e.g. a table) is placed in the simulation. A queried region is a part of this supporting object, such as a section of a table. The queried scene includes the supporting surface and all previously placed objects. A queried object is the current object to be arranged next. ClutterGen is asked to sequentially place movable objects into the queried region while ensuring stability for the new and previously placed objects. The episode succeeds when all queried objects are placed stably.We formulate the generation process as a Markov Decision Process, defined by the tuple <S,A,P,R,γ >. S is a set of states, A is a set of actions, P is a state transition probability matrix; P (s ′ |s,a) indicating the probability of transitioning to state s ′ from state s after action a, R is a reward function, and γ is a discount factor 0≤γ ≤1. A policy π :S →A determines the next action based on the current state. Our objective is to find an optimal policy π * that maximizes the sum of all discounted future rewards.Observation and Action Space ClutterGen takes in the point cloud of the queried scene and object. If the current object placement fails, we inform the policy about its past actions and their impacts by providing a history of the queried object's movement trajectoriesM i = {m 0 ,m 1 ,...,m k |m i ∈ R 13 },i = {1,2,...,I}.Each m includes the object's position ∈ R 3 , orientation quaternion H ∈ R 4 , linear velocity ∈ R 3 and angular velocity ∈R 3 at each step. The hyperparameter k determines the number of simulation steps to verify placement success, and I is the predefined maximum number of attempts allowed.We encode each point cloud with a feature encoder to extract geometry embeddings. We concatenate the movement trajectory, previous placement poses, and queried regions and send them into a history sequence encoder to generate a history embedding. The concatenation of geometry and history embedding is the final input to our RL agent. Our policy outputs the object placement position and z-axis rotation relative to the queried region.Policy Design We optimize our RL agent with PPO [40] which is an on-policy actor-critic algorithm. The critic computes the advantage during training and the actor generates the action distribution for the 3D translation and 1D rotation independently. Since our design requires bounded continuous action space, instead of using the conventional squashed normal distribution to bound the action space, we choose beta distribution as the policy distribution. Specifically, the actor of the agent outputs the α∈R 4 and β ∈R 4 to build the action distribution Beta(α,β), and then the relative placement pose can be obtained by sampling from the distribution.Our choice of action distribution offers several benefits. First, beta distribution is well-defined among [0,1], which only requires a simple linear operation to our action range [-1,1], while the unsquashed process for the squashed normal distribution will introduce numerical instability during training and requires extra post-processing. Second, beta distribution can represent more distribution shapes by changing the α and β values, which is essential to improve the diversity of our scene generation. As shown in Sec. 4.1, our results demonstrate that the policy trained with the beta distribution outperforms the policy trained with the squashed normal distribution in both success rate and scene diversity.Success Conditions A successfully placed object should not collide with existing objects and maintain a stable pose. To verify a placement, we run the simulator and record the object's velocity and acceleration to check stability. The placement is considered stable if the following conditions are met: 1) the linear velocity and acceleration become less than 0.005m/s and 1m/s 2 , respectively, within 40 simulation steps (0.167s in real-time); 2) the angular velocity and acceleration become less than 0.5 • and 180 • /s 2 , respectively, within 40 simulation steps; and 3) the velocity and acceleration remain below these thresholds for 20 continuous simulation steps.Reward Function Guided by the success conditions, for each placement attempt, ClutterGen optimizes the following reward function to minimize the accumulated absolute values of the velocity and accelerate during new object placements:R i =-c k i=0 (||v i || 2 +||a i || 2 )+n•1 stable •R 0 (1)where c is a scaler to adjust the velocity and acceleration penalty, v i ∈ R 6 and a i ∈ R 6 represents the velocity and acceleration the queried object at i t h simulation step, the indicator function 1 stable will equal to 1 if the placement pose is stable otherwise will equal to 0, n represents the current queried object is the n th object for the queried scene, and R 0 is a scalar reward.For each attempt, we run the simulator for up to k =240 steps (1s) to check the placement stability. We set the maximum allowed attempts as I =5. We use PointNet++ [41] as the point cloud feature encoder and 3-layer MLP as the history sequence encoder. Both the actor and critic networks are composed of 5 layers of MLPs. We optimized the agent for 2.5M steps with a learning rate of 1e -4 . All training was conducted on one NVIDIA RTX A6000 GPU. Other details are listed in the Appendix. 
experiments: 
 In this section, we aim to evaluate ClutterGen's capability to effectively generate cluttered, stable, and diverse scenes. We further investigate the generalizability of the learned model on various scene-level changes. We then conduct several real-world experiments to demonstrate the effectiveness of ClutterGen for downstream robotics tasks such as clutter rearrangement and stable placement policy training.Dataset To include diverse everyday objects for our evaluation, we created a dataset consisting of five groups, each containing ten objects. The first four groups are selected from the PartNet-Mobility [42] , Objaverse [43] , and YCB [44] datasets. The fifth group, referred to as the real group, includes our 3D-scanned everyday objects that are also used in our physical experiments. We used a cuboid table with [60cm,70cm,70cm] in width, length, and height as the initial queried scene.Baselines Since most prior studies on scene generation rely on either pre-collected datasets or human-specified heuristics and constraints, we cannot directly compare with them. The closest method, which is also widely adopted [16, 17, 18, 19] in recent literature, is the random rejection sampling (RRS) algorithm. We also compare our methods with other alternative settings. Tab. 1: Success rate comparison of cluttered scene generation. We report the average success rate and stable steps across three random seeds of training and five object groups. Our method significantly outperforms the widely adopted RRS baseline. The long-term attempt history and closed-loop design in our framework deliver the best performance.• Random Rejection Sampling (RRS). This method heuristically computes the position of the supporting surface in the queried scene and randomly places objects within the queried region. • ClutterGen-OpenLoop (OL). This method uses the same architecture as ClutterGen, but without previous object movement trajectory and placement pose information for the next attempt. • ClutterGen-ShortMemory (SM). This method uses the same architecture but only takes the latest attempt history (buffer length = 1) for the next attempt. • ClutterGen-Normal. This method uses the truncated normal distribution instead of the beta distribution.Metrics Our evaluation metrics include: 1) the success rate of placing all queried objects into the queried scene, and 2) the average simulation steps (stable steps) required for each object to achieve stability. Additionally, we access setup diversity, a qualitative metric, using a diversity map to evaluate the variety of the generated scenes. The Index of Attempts Fig. 3 : Average stable steps across different numbers of attempts. We computed the average stable steps for object placements requiring ≥3 attempts. The x-axis represents the i th attempt, and the y-axis represents the average simulation steps for the object to stabilize. ClutterGen's closed-loop re-attempt mechanism increasingly improves placement stability.Results Tab. 1 shows the testing restuls with 1,000 trials. The RRS almost always failed to produce a valid object layout. This is because RRS cannot learn from active interaction with the environment. While ClutterGen-OL and CluuterGen-SM can produce some reasonable layouts, the success rate is generated low with longer simulation steps for the scene to become stable, suggesting long-term close-loop history is important. Both the normal and beta distributions achieve high success rates.To further understand the effectiveness of ClutterGen, we plot the average stable steps needed over different attempts. As shown in Fig. 3 , ClutterGen can use previous placement poses and object movement trajectories to adjust the next placement pose for faster convergence of stability. This closed-loop mechanism is extremely useful for cluttered scene generation within limited spaces. Fig. 4 shows an example of ClutterGen's closed-loop generation process to adjust its next action based on the past attempts. Failed placement attempts (e.g., floating in the air, colliding with objects, or inserting into the table) are marked by red circles. After each attempt, the simulator runs and records the movement trajectory. ClutterGen uses all previous failed attempts to adjust its future actions until achieving a successful placement.ClutterGen Beta Generating diverse object layouts is crucial for training robust robot policies by providing sufficient randomization. Although both ClutterGen and ClutterGen-Normal achieved high success rates in generating cluttered scenes, we assess their ability to create diverse layouts by visualizing their successful placements through a 2D projection. Each subplot in Fig. 5 shows the placement distribution for one object across 500 scenes. ClutterGen-Normal tends to place objects in very similar positions resulting in homogeneous outcomes, while our ClutterGen with beta distribution generates significantly more diverse setups. We are interested in evaluating ClutterGen's generalization capability to generate cluttered scenes when the queried region varies during test time, even if it was fixed during training. We propose five test-time changes: 1) translation (random 2D translation within a certain range), 2) rotation (random rotation around z-axis), 3) shrinkage (random reduction of xy half-extents), 4) expansion (random enlargement of xy half-extents), and 5) randomly combinations of the previous changes. We used the real object group for this evaluation.X Mug X Tomato Soup Can X Mustard Bottle X Gelatin Box X Bowl Y Y Y Y Y Table area Object position Coverage area X X X X X Y Y Y Y YTo ensure the queried region remains covered by the support surface after the changes, we enlarged the table dimension to [140cm,140cm,70cm] for this test. Our evaluation across 500 episodes for each change is shown in Tab. 2. We observe a performance drop in the shrinkage test. This is because a narrower queried region forces the scene to be more cluttered, which increases the difficulty of finding stable poses for all objects. Overall, ClutterGen shows strong zero-shot generalization ability across all changes. Examples of the generated scenes are shown in Fig. 6 .To demonstrate the real-world use cases, we selected 10 furniture tables from our dataset and directly applied ClutterGen to them with all five object groups. We adjusted the queried region's xy extents to match the furniture's dimension. We also randomly applied the above four changes. By evaluating 50 episodes for each table, we achieved an overall 70% success rate. Qualitative examples are shown in Fig. 1 . Another advantage of our method is the formulation of scene generation as a sequential 3D object placement task. Therefore, ClutterGen can naturally generate complex object relationships such as a mug on a book or a fork under a plate.Given a clutter of objects, humans can easily determine the goal state of stable poses for each object when tasked with moving them to another location (e.g., from one table to another). However, achieving this behavior in robots remains challenging. For instance, when the target area is cluttered, robots must identify a safe and stable pose for new object placements to avoid collisions or simply dropping objects into the area. Consequently, existing "pick-and-drop" solutions are inadequate in this scenario. Additionally, human Fig. 7 : Clutter rearrangement. The red circle indicates the position of the camera. The blue cube encloses the objects to be rearranged. ClutterGen proposes several different setups for rearrangement. After a user selects a preferred setup, the Panda arm will rearrange the table accordingly.preferences for the resulting scene layout can vary, making it difficult to incorporate these preferences into planning.We demonstrate ClutterGen's applicability in the challenging task of clutter rearrangement using a Franka Panda arm equipped with an RGB-D camera. Fig. 7 provides an overview of our approach. We randomly selected five objects from our real object group and placed them on one side of a table as the initial clutter. The robot's goal is to move this entire clutter to the other side of the table. We used GroundDINO [45] to identify the object categories, Segment Anything [46] to generate the object masks, and FoundationPose [47] to determine the 6 DoF pose of each object. To simplify the setting and focus on ClutterGen's key capabilities, we assume access to the 3D models of these objects and a set of grasping poses [48] , although these assumptions can be relaxed with recent advancements in grasp synthesis [49, 50, 51] . With this information, we render the queried scene point cloud to ClutterGen to propose possible target layouts.We asked ClutterGen to generate ten possible layouts for the other side of the table, allowing users to select their preferred setup and obtain the target poses for each object. The robot arm then planned its motions to move the entire clutter to the target area using MoveIt [52] . We evaluated the full pipeline over ten episodes. Success is counted if all objects are rearranged to the user-selected setup without any collisions or unstable poses. The overall success rate was 7/10. We include both successful and failed trials in the supplementary videos. Failures were due to arm-object or object-object collisions during motion planning (2/3) or failed object layout proposals from ClutterGen under the ten-trial limit (1/3).While pick-and-drop tasks have seen great success in robot manipulation, placing objects stably in cluttered scenes remains challenging. In this experiment, we leverage ClutterGen as a synthetic data generator to train a robot policy for stable object placement. Fig. 8 provides an overview of our approach. Our stable Fig. 8 : Synthetic dataset generation and zero-shot sim-to-real policy deployment. We generate a synthetic dataset by replaying the scene generation trajectory created by ClutterGen. Green points mark the ground truth pose of the queried object, while blue points represent the point cloud of the queried scene. To augment the dataset, we use a virtual camera to capture the scene point cloud from different angles. This synthetic dataset is then used to train a stable object placement policy, which is directly deployed on a real robot. placement policy takes the point clouds of the queried scene and object and learns to output the stable placement pose. Due to our novel formulation of ClutterGen as a sequential 3D placement task, ClutterGen naturally serves as a data generation source for training the stable placement policy. Specifically, we ran ClutterGen 500 times to create 500 layouts while recording its generation process for each object, resulting in 5,000 data points. To mitigate occlusion issues in the real world, instead of using the full point cloud, we captured the scene point cloud using a virtual RGB-D camera in the simulator and randomly translated its position four times while ensuring focus on the center of the scene. We then randomly sampled the final 5,000 data points as our dataset termed ClutterGen-5000. This data generation process takes less than one minute on a single thread and can be further accelerated through parallelization. We then applied supervised learning to train the stable placement policy over 1,000 epochs using mean squared error loss. At test time, we achieve a 91% success rate by using the simulator to verify placement success with the same physics-based stability metric. To demonstrate the efficiency improvements of ClutterGen compared to a human engineer, we developed a user interface within the Pybullet [53] simulator to allow humans to manually create simulation setups. We manually constructed 20 stable setups, resulting in 200 data points. We applied the same augmentation process as above to obtain the final dataset termed Human-200. The entire process took approximately 1.5 hours for an expert engineer. We then trained the stable object placement policy using this dataset under the same configurations as before. Both models were evaluated on the same test dataset as shown in Tab. 3. ClutternGen-5000 enables significant higher performance than using Human-200 for the policy training.We directly deployed the stable object placement policy trained with ClutterGen-5000 to a Franka Panda arm equipped with an RGB-D camera. In each episode, the robot arm grasped and lifted a queried object while six to eight other objects were randomly placed on the table to form an unseen clutter. Based on the scene point cloud captured by the RGB-D camera and the object point cloud, the policy predicted a stable placement pose, which was then used to plan and execute the robot's actions. We repeated this process by randomizing the clutter configurations for ten episodes per object. We used five target objects to be placed. An episode was counted as successful if the object was stably placed without any collisions.Failures occurred if the model failed to predict a stable pose, the arm could not reach the pose after five attempts, or a collision occurred during placement. Across all 50 episodes, our zero-shot sim-to-real policy achieved a 72% success rate. The performance drop is likely due to significant noise in the real-world point cloud. This could be mitigated by further randomizing the dataset to simulate real-world noises or using better hardware to capture the point clouds. 
conclusion, limitations, and future work: 
 In this work, we propose ClutterGen, an auto-regressive simulation scene generator for robot learning. ClutterGen efficiently generates diverse, cluttered, and physically compliant environments without relying on pre-existing datasets or human specifications. Our key idea of framing the scene generation task as a reinforcement learning problem enables a closed-loop mechanism to sequentially generate 3D object placements. Our policy design further enhances the diversity of the generated scenes. Through both simulation and real robot experiments, we demonstrate that ClutterGen can help tackle several challenging robotics tasks, such as clutter rearrangement and stable object placement in cluttered environments.Our work has a few limitations that should be addressed in future research. First, the object layouts are currently limited to rotation along the z-axis. Allowing rotations along the x and y axes could produce more diverse and complex layouts. Second, we have only demonstrated the setups on flat surfaces. Future work could explore various surface configurations, such as uneven or inclined surfaces. Finally, our current training involves only ten objects in a limited space. Training the policy with a great number of objects in the pool could enhance object-level generalization during testing. Overall, we hope that ClutterGen provides a novel problem formulation for automated simulation scene design to facilitate the generation of large datasets for robot learning. Selection of Maximum Allowed Attempts. We test the performance of ClutterGen with different maximum attempts on group 1 objects. Increasing allowed attempts will boost the performance. However, too many attempts will slow down the training process. We set 5 attempts at most to balance the performance and training speed. 
 
 Appendix: We used two pre-trained PointNet++ models as the perception encoders for the queried scene point cloud and the queried object point cloud respectively. We did not freeze their weights during the training in this task. The perception features were concatenated and sent to a 4-layer MLP to generate the predicted placement pose into the scene surface center frame. This pose was then transformed into the robot base frame for motion planning. 8th Conference on Robot Learning (CoRL 2024), Munich, Germany. 


Decision: Strengths:
1.  Addresses the significant issue of generating ecologically rich scenes for robotic simulation, which is crucial for bridging the sim-to-real gap.

2. Introduces a reinforcement learning (RL) approach to frame the scene generation task, using a closed-loop mechanism and beta distribution for action space.

3. Extensive experiments conducted both in simulations and real-world settings, demonstrating the robustness and practical utility of ClutterGen.

4. The paper is well-written and easy to understand, with detailed explanations of the problem, methodology, and results.

5. Demonstrates the ability to generate diverse and stable object layouts, showcasing practical relevance through real-world experiments.

Weaknesses:
1. The paper does not compare ClutterGen to other recent procedural generation tools (e.g., AI2Thor, BEHAVIOR) or simpler heuristic-based methods, making it difficult to assess its significance and performance relative to existing approaches.

2. The current implementation only performs transformations along one rotational axis, which restricts the semantic realism of the generated scenes.

3. The necessity of using a complex RL approach is not well justified, as simpler methods like potential fields or 2D packing algorithms might achieve similar results for the presented problem.

4. The algorithm focuses on physical stability but lacks mechanisms to ensure semantically meaningful clutter, which is essential for realistic scene generation.

5. The approach does not adequately address more complex spatial constraints or interactions, such as stacking objects or ensuring stability throughout all transformations.

Reviews: review: title: title: Novel, but not fully proven, approach to important problem of cluttered scene generation
paper_summary: paper_summary: The paper introduces an approach to the problem of cluttered scene generation for robotics simulation: the authors formulate the problem as a MDP and apply RL methods to learn a policy that can place objects in a scene in a physically stable and plausible way.
main_review: main_review: Quality: The work is generally high quality. The proposed algorithm is clearly explained and its performance is validated through a variety of interesting experiments. However, important experiments where the work is compared to other recent procedural generation tools from the variety of simulator groups like AI2Thor (ProcThor, Holodeck), BEHAVIOR (iGibson, OmniGibson), etc. are missing - currently it is not obvious whether this method performs better, on par with, or worse than those existing approaches with heuristic rules. Tradeoffs should also be discussed.

Clarity: The work and the presentation are very clear. The problem and the proposed approach are clearly explained. There is not much to comment on.

Originality: The work introduces a fully novel approach to solving a known and important problem (discussed below). 

Significance: The problem itself is definitely significant: in simulation environments the lack of realistic clutter is one of the major sources o the so-called sim-to-real gap. The paper tries to solve this by generating scenes with such clutter. But there are a few major issues with the approach:

1. The proposed algorithm provides no way of guiding the generation towards semantically-meaningful clutter rather than just physically stable, and in fact probably captures a very small subset of what's physically stable, too. For example, clutter in a kitchen cabinet or fridge will usually have certain important characteristics about it stemming from the distribution of the objects involved and their placement etc. - simply placing random objects in a rejection sampling fashion during training will NOT be a good approximation of this distribution and thus may not be very meaningful to help robots learn to work in such environments.

2. The proposed algorithm only performs transformations along one rotational axis. This is a major limitation that keeps its generations from being semantically realistic. Whether or not a good policy can be learned in the full three-axis case remains to be seen.

3. The actual merit of the described approach (e.g. RL for clutter generation) can only be understood when the approach is compared to existing heuristic-based generation methods. Such experiments are not included in the manuscript and thus it is hard to gauge the significance of the contribution.
questions: questions: 1. How do you ensure the diversity / completeness of the physically stable configurations (e.g. doesn't the model just learn to build the _easier_ physically stable configurations rather than a good sample of the full distribution?)

2. How do you ensure semantic realism (e.g. the kind of clutter we see is actually relevant for work a robot might do IRL)?

3. How does the approach compare against clutter generation heuristics from the SOTA simulators rather than just random placement?
limitations: limitations: 3
review_summary: review_summary: The paper introduces a new approach to an important problem, and is presented well. I believe it should be presented at CoRL despite its shortcomings in terms of the method's limitations and the limited experiments against other solutions.
title: title: Interesting work and timely given current interests in 3D scene generation
paper_summary: paper_summary: The paper presents ClutterGen, an automatic, policy-driven method for procedurally sampling multiple object placements on a given surface. The paper highlights the reliability and scalability of ClutterGen, and showcase applications in downstream robot tasks such as Clutter Rearrangement and Stable Object Placement.
main_review: main_review: **Strengths**
- The paper is well-motivated. With the rapid increase in interest in scalable scene generation, there is a clear need for generating a realistic distribution of ecologically rich scenes. ClutterGen is a great step towards procedural and scalable scene population.
- The analysis experiment is thorough, and effectively highlights the importance of the chosen design decisions in the ClutterGen pipeline.
- The generated diversity is compelling, and showcases ClutterGen's ability to generate diverse arrangements given a fixed set of objects.

**Weaknesses**
- While the efficiency comparison is already compelling, it would be even more convincing to show that an equivalent dataset size _ClutterGen-200_ can either match or even outperform the human-curated equivalent dataset _Human-200_.
- It is suggested that ClutterGen can be applied iteratively to construct a scene graph, e.g.: "fork on top of plate on top of table". It would be powerful to showcase an example of this or something equivalent as a proof-of-concept to show how ClutterGen can fully populate a given scene (e.g.: setting a dinner table).
- I was surprised to see no comparison with a "smarter" rejection sampling technique -- e.g.: in simulation (and presumably in real, given ground-truth object meshes), given a proposed placement pose collision information should be already be known. This coupled with object bounding box information should allow rapid rejection sampling without having to actually step physics many times. Other than speed improvements, what additional key benefits does ClutterGen have over such an approach?
questions: questions: Please see Weaknesses section above.
- How were the different dataset groups selected? Were they all randomly selected or was there some semantic / geometric distinction (e.g: "all kitchen items" or "all convex items", etc.)
- One interesting qualitative trait of the learned placement policy is that it appears it is biased towards placing objects towards the edges of the supporting surface (as Fig. 5 also quantitatively shows). Could this possibly bias results and result in object placement distributions that don't necessarily align with "natural" distributions found in the real world? I also wonder if this is in part due to the fact that the policy only observes the current set of already-placed objects and the current queried object instead of the full set of future objects to be placed, leading the policy to avoid placing objects in the center because it limits the possibility for future (unknown) objects to be placed. I am curious to hear the author's thoughts on this or if they have considered this alternative setup!
- Can the method be applied to non-rectangular surfaces, e.g.: a circular or abnormally-shaped table?
- It seemed surprising that the placement policy suffered when the sampling surface was translated (Sec. 4.2). Assuming that the entire surface is still sampleable, I would expect the same placements to be valid. Has any sort of normalization with respect to the sampling surface been considered to prevent overfitting?
- What is the role of the human selecting the preferred setup for the object rearrangement task? It seems that this would mainly allow the user to select setups that are "easier" to achieve for the robot compared to other alternative proposals.
- Minor syntax issues:
	- L147: closet -> closest
	- L148: RSS presumably should be RRS
limitations: limitations: 3
review_summary: review_summary: The paper presents an interesting method for tackling procedural scene generation, showcasing a learned policy that can automatically place a set of objects stably on a given surface with high success. Given the increasing interest in the broader field of scene generation, this approach may serve as an important step in fully end-to-end methods for ecologically rich scene generation.
title: title: ClutterGen: A Cluttered Scene Generator for Robot Learning
paper_summary: paper_summary: The paper presents a novel approach to generating physically compliant, cluttered scenes for robot learning using reinforcement learning.  The presented framework framework that casts the scene generation problem as a reinforcement learning task, using a closed-loop mechanism and beta distribution for action space to enhance scene diversity and stability.  The paper provides extensive evaluation through simulations and real-world experiments, demonstrating the effectiveness of ClutterGen in generating diverse and stable object layouts.
main_review: main_review: **Summary**

The paper presents a novel approach to generating physically compliant, cluttered scenes for robot learning using reinforcement learning.  The presented framework framework that casts the scene generation problem as a reinforcement learning task, using a closed-loop mechanism and beta distribution for action space to enhance scene diversity and stability.  The paper provides extensive evaluation through simulations and real-world experiments, demonstrating the effectiveness of ClutterGen in generating diverse and stable object layouts.

**Strengths**

- One of the biggest strengths of the paper is the thorough evaluation of ClutterGen.  The authors demonstrate its effectiveness across various scenarios, both in simulations and real-world tasks.  The detailed experiments provide strong evidence of the approach’s robustness and practical utility
- The problem of generating diverse and stable cluttered scenes is well-defined.  The authors articulate the limitations of existing methods clearly, providing a strong motivation for their work.
- The use of reinforcement learning to frame the scene generation task and the introduction of beta distribution for action space seem like significant contributions.  These novel aspects are well-highlighted and effectively demonstrated through the experiments
- The paper shows the practical relevance of ClutterGen through real-world experiments on clutter rearrangement and stable object placement
- The video is fantastic

**Weaknesses**

- One of my primary concerns is that the problem might not be as hard as the authors make it seem.  For instance, I didn’t see any example in the video that considered something like object stacking, and this kind of exploration is not encouraged by the model.  Further, the objects can only be translated and rotated about the z axis, so the envelope of the surface that will touch the table is fixed.  Given this setup, it's unclear why objects would just randomly fall like the pitcher in the video if it's placed on a free part of the table.  Nothing about the pitcher rotating about its z axis on a flat surface should change its dynamics or stability.
- Building on my previous comment, it seems like simple baselines could have been used in the evaluation to further motivate the approach.  For instance, using a potential field approach to avoid collisions and place objects in free spaces is well-established in robotics and seems like a reasonable baseline.  Objects can be guided by repulsive forces from other objects and attractive forces towards the desired position on the table. This method is computationally efficient and intuitive for the problem as described.  Also, treating the problem as a 2D packing problem on the surface in contact with the table could indeed be a simpler and more straightforward approach.  This method is extensively studied and has efficient algorithms that could be adapted for the scenario described.  Now, to be clear, these baselines would not at all be reasonable if full rotation and object stacking would be involved; however, that is not the case in this version of the paper.
- While the authors emphasize the physical simulation to ensure stability, I was surprised to see “positive” examples in the video with large parts of objects hanging off the edge of the surface.  While the simulator may deem this as “physically stable”, allowing for this just seems impractical and unnecessary.  This is, yet again, something that a simpler approach would simply not tolerate and would easily avoid.
- In 3.2: you say “$orientation \in \mathbb{R}^4$”.  However, assuming you are using four values here to represent a unit quaternion, this will not be the space of $\mathbb{R}^4$, this will instead be a Lie group of unit quaternions, commonly referred to as $\mathbb{H}$

**Detailed Assessment**

**Is the problem well explained?**

Partially.  The problem is well-explained and they highlight the significant human engineering effort often required for traditional methods and the need for a more automated and efficient approach.  However, I feel that obvious, simple baselines could easily handle the problem as presented as only translation and z rotation on a flat surface are currently investigated.

**Is the problem  big enough where others in the community will care?**

Partially.  The full version of the problem (with object stacking and full translation and rotation), definitely yes.  The problem is significant and relevant to the robot learning community.  The ability to generate diverse and stable cluttered scenes is crucial for training robotic systems in realistic environments.  However, the version of the problem presented here (only translation and z rotation), I’m not convinced.

**Is the solution well explained?**

The ClutterGen framework is well-explained in detail.  The authors describe their reinforcement learning approach, including the problem formulation, observation and action spaces, policy design, and reward function.  The decision to use beta distribution for policy distribution is particularly well-justified.

**Is the work novel enough?**

The novelty of ClutterGen lies in its approach to framing scene generation as a reinforcement learning problem and using a closed-loop mechanism to ensure scene diversity and stability.  The use of beta distribution for action space is also a notable innovation.  These contributions are well-highlighted and effectively demonstrated through the experiments, establishing the work’s novelty

**Overall assessment**

The paper presents a significant and well-executed contribution to the field of robot learning.  The problem is clearly defined, the solution is well-explained, and the evaluation is robust and comprehensive.  However, the necessity of the complex reinforcement learning approach should be better justified, and comparisons with simpler baseline methods should be included.
questions: questions: - The paper should include comparisons with simpler baseline methods like potential fields and 2D packing algorithms.  Demonstrating that ClutterGen significantly outperforms these baselines would strengthen the justification for its complexity.
- Provide a stronger justification for why reinforcement learning is necessary for this problem. This could include a discussion of the limitations of simpler methods in the specific context of ClutterGen’s objectives.
- Address concerns about the perceived complexity of the problem.  Explain why objects might fall in the simulation and how ClutterGen addresses these challenges uniquely compared to simpler methods.
- Longer term, I highly recommend incorporating full translations and rotations.  Many of my previous concerns would go away.
limitations: limitations: 3
review_summary: review_summary: The paper presents a significant and well-executed contribution to the field of robot learning.  The problem is clearly defined, the solution is well-explained, and the evaluation is robust and comprehensive.  However, the necessity of the complex reinforcement learning approach should be better justified, and comparisons with simpler baseline methods should be included.