Atlas3D: Physically Constrained Self-Supporting Text-to-3D for Simulation and Fabrication
Authors: Yunuo Chen
Tianyi Xie
Zeshun Zong
Xuan Li
Feng Gao
Yin Yang
Ying Nian Wu
Chenfanfu Jiang
Venue: NeurIPS 2024 poster (2024)
Keywords: Text-to-3D Generation
Score Distillation Sampling
Physics-based Simulation
Primary area: generative_models

Abstract: Existing diffusion-based text-to-3D generation methods primarily focus on producing visually realistic shapes and appearances, often neglecting the physical constraints necessary for downstream tasks. Generated models frequently fail to maintain balance when placed in physics-based simulations or 3D printed. This balance is crucial for satisfying user design intentions in interactive gaming, embodied AI, and robotics, where stable models are needed for reliable interaction. Additionally, stable models ensure that 3D-printed objects, such as figurines for home decoration, can stand on their own without requiring additional supports. To fill this gap, we introduce Atlas3D, an automatic and easy-to-implement method that enhances existing Score Distillation Sampling (SDS)-based text-to-3D tools. Atlas3D ensures the generation of self-supporting 3D models that adhere to physical laws of stability under gravity, contact, and friction. Our approach combines a novel differentiable simulation-based loss function with physically inspired regularization, serving as either a refinement or a post-processing module for existing frameworks. We verify Atlas3D's efficacy through extensive generation tasks and validate the resulting 3D models in both simulated and real-world environments.

Atlas3D: Physically Constrained Self-Supporting Text-to-3D for Simulation and Fabrication
Yunuo Chen'; Tianyi Xie'; Zeshun Zong'; Xuan Li', Feng G Yin Yang", Ying Nian Wu', Chenfanfu Jiang' University of California, Los Angeles, 2 Amazon, 3 University of Utah (yunuoch, tianyixie77, zeshunzong, xuanli1]@ucla, edu, fenggo@amazon. com, yin. www.yannouuthheeee ywu@stat. ucla. ecl,.edu, cffjiang@ucla, edu
Abstract
Existing diffusion-based text-to-3D generation methods primarily focus on pro- ducing visually realistic shapes and appearances, often neglecting the physical constraints necessary for downstream tasks. Generated models frequently fail maintain balance when placed in physics-base based simulations or 3D-printed. This balance is crucial for satisfying user design intentions in interactive gaming, em- bodied AI, and robotics, where stable models are needed for reliable interaction. Additionally, stable models ensure that 3D-printed objects, such as figurines for home decoration, can stand on their own without requiring additional support. To fill this gap, we introduce Atlas3D, an automatic and casy-to-implement method that enhances existing Score Distillation Sampling (SDS)-based text-to-3D tools. Atlas3D ensures the generation of self-supporting 3D models that adhere to physi- cal laws of stability under gravity, contact, and friction. Our approach combines novel differenniable simulation based loss function with physically inspired regu- larization, serving either refinement a post-processing module for existing frameworks. We verify Atlas3D's efficacy through extensive generation tasks and validate the resulting 3D models in both simulated and real- world environments.
1 Introduction
Generating high-quality 3D content of great importance in modern visual computing. Realistic 3D models are highly sought after in computer graphics, while robust real 3D assets gaining attention in training embodied AI. Nevertheless, the standability of 3D models the ability to stand steadily without additional support is often neglected. Real-world man-made objects such action figures, toys, and furniture inherently possess some degree of geometric stability, allowing them to be safely placed the ground. Although usually takes such standability for granted, existing generative models fail to produce steady 3D assets due to their lack of physical perception; see Fig. 1.
Incorporating this stability expectation into 3D generation will significantly reduce the human effort required for tasks such as sorting out unqualified meshes, post-processing geometries, or adding external supports before actually using the 3D asset in any simulator or the real world. Furthermore, creating physically plausible 3D content will enhance the fidelity of simulations and policy training with these objects, potentially narrowing the sim-to-real gap and empowering embodied AI in robotic tasks. Towards this goal, we develop a 3D generation framework that can produce high-quality models adhering to basic physical laws, such gravity, stability, and frictional contact.
Several attempts have been made to incorporate physical constraints into 3D generation. Yang et al. utilized the spatial and physical sense of LLM to design floor plans and furniture arrangements [98].
"qqual contribution.
This work is not related to F. Gao's position at Amazon.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

(a) w/ Atlas3D
(b) w/o Atlas3D
Figure I:
Simulation in ABD [27]: (a) 3D models generated from our Atlas3D framework can stand steadily on the ground; (b) those generated from existing methods tend to fall over.
PhyScene introduced physical guidance, such collision and reachability constraints, diffusion models to generate furniture layouts [97]. However, both works primarily consider straightforward spatial constraints, such as non-collision, and fail to incorporate more complex physics. Mezghanni et al. proposed a GAN-based network to generate physically-aware geometries by training a neural stability predictor using datasets labeled by Bullet [48]. Another GAN-based work by Wang al. employed CFD software to compute vehicle drag coefficients, guiding the generation of stream- lined vehicle meshes [85]. Such indirect incorporation of physical simulations, however, results in suboptimal efficiency and accuracy. Furthermore, due to the low expressibility of the backbone latent representation, the versatility of the generated results is very limited compared current state-of-the-art diffusion-based models, as the results are typically confined to specific categories (e.g., furniture and vehicles). Most recently, Ni et al. bridged differentiable physical simulation with differentiable rendering to obtain virtual 3D reconstructions from real-world images that are objects
physically plausible simulators. Their work primarily focused on simple bour-leg-supporte such tables and chairs [54]. Moreover, the evaluations of all aforementioned works are conducted in virtual simulators, leaving their performance the real world untested. This limitation hinders potential downstream applications such industrial manufacturing and robotic manipulation.
Since the pioneering work DreamFusion [58], Score Distillation Sampling (SDS) has demonstrated efficacy in clevating 2D content to 3D, inspiring numerous follow-up studies [8; 35; 46; 69; 88; 109]. These advancements have enhanced both the versatility of generated content and the quality of textures. However, none have addressed the crucial issue of physical stability. On the other hand, traditional computational fabrication has concentrated on employing topology and shape optimization to ensure that 3D printed objects can stand in balanced state [59]. Directly integrating these methods with 3D generative as postprocessing module suboptimal. Shape optimization disregards the original input conditions of diffusion models, while topology optimization produces internal structures that defy intuitive physics, rendering them unsuitable for training embodied AI systems designed to emulate human-like reasoning about physical objects.
Observing this gap, we introduce Atlas3D, a generation pipeline that produces physically plausible, self-supporting 3D models from text. Incorporating differentiable physics- based simulation into our process, we generate models that are both simulation- and fabrication-reade -eaddd. That is, they can be directly utilized in physical simulators, or 3D-printed for real-world applications; see Fig. 1 and Fig. 2. As our method orthogonal to previous SDS-based techniques, which focus on non-physical qualities, it can be seamlessly integrated into many existing generation frameworks, functioning either as part of the refinement stage in a multi-stage method or as a post-processing step single-stage method. We demonstrate the efficacy of Atlas3D by comparing the stability of models with those

---

produced by existing methods. Validation examples reveal that our generated models can be deployed as virtual simulation assets. Their stability transfers directly to the real world, as evidenced by our 3D printed results, suggesting further applications in robot training.
2 Related Work
Diffusion-based 3D Generation Due to the abundance of information encoded large image latent diffusion models (LDMss) [65], extensive studies have used pre-trained LDMs to distill 3D content. One approach is to fine-tune LDMs to support novel view synthesis, with a separate multiview fusion step to produce 3D content [39; 68; 99; 38; 37; 89; 95; 42; 80; 25; 106; 41; 9; 83). Another approach, which is more related to our paper, is using LDMs as likelihood discriminators. A differentiable renderer is connected to a 3D representation, and the LDMs guide the optimization of the representation parameters. [58] proposed Score Distillation Sampling (SDS). Efficiency has been improved by coarse-to-fine strategies [35; 60; 79; 8] and timestep scheduling [18; 100].
3D priors are involved to improve multiview consistency [108; 66; 31; 1; 93; 36; 78). Multiview diffusions can also be used to evaluate SI>S [69; 87; 105; 94]. Other researchers have explored SDS
variants or improvements [84; 77; 88; 24; 101; 16; 109; 96]. 3D LDMs that directly generate 3D)
representations are also explored, such as compositional scenes [17: 57], point clouds [43; 81; 51], SDFs [70; 107; 30], occupancy fields [15; 72; 45; 12] and NeRFs [3; 11; 52; 6; 55; 5; 75; 23].
Physics-aware 3D Generation Most existing 3D generative models focus only on geometry or appearance modeling, with physics priors being underexplored. Time-independent physical constraints, such as penetrations, can be directly defined by penalties [97; 17: 82; 40; 102]. For time-dependent physical qualities, such as stability and comfort, data-driven quality checkers trained with offline simulators can be applied [10: 48: 4]. Offline simulators can also be used as validators to augment the training dataset [71] and update the design with reinforcement learning [85], and as dynamics generators for generated 3D assets [91; 21: 62]. Another direction is to utilize differentiable simulations, which have be widely used in tasks like robotic control [74; 19; 61; 34; 76] or inverse problems [33; 20; 103; 53; 32; 73; 7; 104]. They can also be applied in 3D content generation to
define physics- based losses to aid per- per-instance generation [54; 92] or model training [47].
3 Background
3.1 Score Distillation Sampling
SDS-based methods are shown to be effective in distilling 3D models from 2D images. They utilize a 3D representation such as implicit density field, implicit Signed Distance Field (SDF), or tetrahedral SDF [67]. a differentiable renderer like NeRF [50]. NeuS [86], Nydiffrast [26], and pre-trained text-to-image model such as Stable Diffusion [64] serving as diffusion guidance. The generation process involves optimizing the parameters 0 of the underlying 3D representation, where the 3D shape is differentiably rendered to 2D images z = g(0) and compared against the real distribution from the diffusion model with text guidance y:
LSDs = Ee,, (u((t l (z v, - -||²]
where z, is the noisy image at noise level f, w(t) is a weighting function, and és is the predicted noise. We refer readers to the Appendix for more technical details SDS optimization.
3.2 Rigid Body Dynamies
To incorporate physics into our framework, we propose predicting the dynamics of the generated models using differentiable simulator, where all objects are treated as rigid bodies. We follow the conventions in [2] to define the dynamical states of the simulation.
The kinematics of a rigid body are described by its mass M and body-space inertia tensor Ioddy- which remains constant. Assuming the center of mass of the body initially lies the origin, the physical state T of the body at time t (not to be confused with the noise level in diffusion) in- cludes position T(t) and orientation R(!) (spatial information), and its linear and angular mo- mentum P()) and L()) (velocity information). The rigid body equations of motion are given by
3

---

T(t)
v(t)
di v(t))= d
R(t)
* R(t) (tt)
(2)
P(t)
F(O)
L(t)
(())
where Ft) and (t)) are the total force and torque ex- erted on the body, v(t) = P(
M is the linear veloe-
ity, (t) I(t) is the angular velocity, [(t) Lody
3D-pr 3D-printed figurines created with Atlas3D stand stably, while those without Atlas3D have fallen down.
Figure 2:
the world-space inertia tensor, and * denotes cross product of w with the columns of R. The physical state at a later time can be derived via time inte-
gration: (t) = (0) + f di
de (s)ds, which can be solved by numerical methods. By optimizing the physical states together with the SDS loss, can jointly refine both the 3D geometry and the physical attributes of the generated results.
Atlas3D Algorithm
We introduce Atlas3D, plug-and-play algorithm for generating 3D models from text. Focusing on man-made objects such as action figures and toys, which generally do not deform, Atlas3D treats generated models rigid bodies and incorporates physics-based guidance into the generation process.
4.1 Physics Incorporation
As mentioned in $ 3.2, we predict the dynamic behavior of generated models by rigid body simulations. While various explicit or implicit representations of 3D shapes can be chosen in a generation network, opt for triangular meshes in our framework as they facilitate frictional contact modeling and simplify kinematics computation. Given a triangle surface mesh representation X(). where 0 is the implicit parameter, we integrate X into a rigid body represented by the dynamic state (t) (t),, R(1), P(t), (()), where the world-space location x of any point X on the body is x(t) R(t)X T(t). Assuming the 3D model is initially placed upright on the ground with the bottom point touching the surface, we define standability as:
lim (t) = (0))
(3)
1-000
Standability intuitively indicates an equilibrium state where all external forces acting on the object balanced, and the physical state remains unchanged over time. However, perfectly placing an object straight on the ground without initial velocity is impractical when manually placing cube flat table, the bottom face is unlikely to be perfectly parallel to the table surface. A stable 3D model should recover its initial state under mild perturbations, such as minor shaking. This state is known as stable equilibrium. Motivated by this, augment standability with stable equilibrium (t) defined
the real world. For example,
T (t) (0) + co 1'" (s)ds ds
(M and
lim (t) (0), (-+30
where 6o represents mild perturbations the initial physical state. We first describe how to incorporate the standability criterion (Eq. 3) into the optimization process of 3D generation, and then explain how further augment it with stable equilibrium (Eq. 4). Additionally, introduce geometry regularization to enhance the smoothness of generated meshes.
4.1.1 Standability through Differentiable Simulation
We utilize a differentiable rigid-body simulator to obtain the physical state (t).. Assuming that the 3D model will eventually reach its steady state, 37 E R large enough such that Vi T, v = (T). Let S denote differentiable simulation function. We approximate V (T) via simulation:
(T) = (0), (0),,,,))
(5)
*We define the upright pose by setting the upward axis to coincide with the upward axis from the pre- me-ttrinee te model, as the upright direction is semantics comantics-driven and generative models inherently lear these semantics from the training data.

---

Here T' is the simulation end time, captures material parameters such as density and friction coefficient, as well as simulator parameters such time step and damping. We adopt the semi- implicit Fuler time-integrator in Warp [44] for simulation.
Assuming the initial translation, rotation, and velocity are all zero, the difference between (T) and (0) arises only from discrepancies in spatial location, as the velocity at the final steady state is also zero. Therefore, we propose standability loss to penalize rotational deviation due to instability:
stadd = [T)) (0)11 |[((T) - I
where R E 3x3 represents the rotation matrix. We disregard the translation T, as real-world instability mostly leads to rotational deviation from the initial state, such falling to one side, while most translations, like falling due to gravity, are irrelevant to standability.
With differentiable simulator, the standability loss can be backpropagated to mesh vertex coordinates
and then to the implicit parameter 0 as d.am dhapt dX In theory, any differentiable simulator is compatible with our framework.
4.1.2 Stable Equilibrium
Although the standability loss directly penalizes the non- standability of 3D object, it can be slow to compute, es- pecially when T' is large and many time steps are required, creating huge computational graph. Consequently, both the simulation itself and the backpropagation of gradi- ents through the simulation trajectory are consuonnn.. Additionally, standability does not necessarily imply sta- ble equilibrium, which is crucial for real-world 3D ob- jects such as action figures and toys. Without this prop- erty, an object remains unstable even if standability is achieved, known unstable equilibrium. Unstable equi- librium means that when a disturbance force is applied, the object moves away from its original position instead of recovering. Fig. 3 visualizes the difference between stable and unstable equilibrium. In the absence of per- turbation, geometries like the upside-down triangle may remain standable in simulator but are clearly unstable in the real world. Thus, we augment standability with stable equilibrium (Eq. 4). One straightforward way incorpo- rate this property is to introduce initial perturbation co into the simulator. However, this would require many more simulations with various perturbations and subsequent loss backpropagation, which is extremely time-consuming.
H(x
9
H(x
(b)
0
2D illustration of stable equi- librium and unstable equilibrium. (a) small perturba-
Figure 3:
A square is stable tion of á increases in H (Xxcm)):()) An upside-down triangle is unstable as tilt- ing decreases H (X(om)
Inspired by the concept of a potential well [13], we augment our optimization objective with a robust and efficient stable equilibrium loss Laable Specifically, for an object to be robustly standable, it needs to reside at local minimum of potential imeegy--seccifiin
gravitational potential energy that if perturbed, gravity will act restoring force that returns the object its original state. For a rigid body, gravitational potential energy is determined by the height of its center of mass. Thus, for any object in stable equilibrium state, the center of mass would rise if it is slightly perturbed. This leads to our formulation of the stable equilibrium loss Lstable- Let Xoom denote the position of the center of mass of the underlying geometry and // (x) denote the distance the point the ground, assuming the object's pivot point is at = O. The stable equilibrium loss is defined as:
Luable Ever [max{ (x (P (X) Hxxcmm (X)),
where _ represents the rotation of é radian about axis [v'' 0)" Mathematically. local minimum of gravitational potential energy is reached if 30D such that Vó E (0. Φo), Luable = 0. In practice, we fix the perturbation scale & and uniformly sample 20 perturbation directions v in zy-plane.
4.2 Additional Regularization
While standability loss Luand and stable equilibrium loss Latable provide a well-defined objective for robust standing, they may lead to distorted optimized meshes due to the high-dimensional searching
5

---

[..]] a standing rooster
[..]] front view of snowboarding man [..]]a broccoli
Ours
M
Figure 4: Comparison with Magic3D [35] includes zoom-in views that highlight the detailed changes in geometry. Our method enhances Magic3D with physics priors to generate self-supporting meshes.
space of implicit parameter without constraint. To constrain the optimization space and obtain smooth meshes, we add normal consistency term that favors smooth solutions:
normal
Σ (1- (1),, (L,ETT
where T is the set of the triangle pairs sharing common pair with n., ny being their normals respectively. This term maximizes the cosine similarity between neighboring surface triangle normals, leading to smoother meshes. Considering the bottom surfaces of most robust standing objects are flat, we apply the Laplacian loss to a subset of vertices with a height lower than a threshold hy:
Li-lap
Σ 11 IES
B
where 5; (LV); E R calculates the differential coordinates of vertex i with L being the Laplacian matrix of the mesh graph and V representing mesh vertices. Intuitively, this loss term attempts to minimize the distance between vertex and the average position of adjacent vertices.
4.3 Method Overview
With the physically-ine -inspired loss terms derived above, we now describe how to incorporate them into the text-to-3D framework. SDS-based methods and their variants start optimization with random initialization of the implicit parameters, which initially have no knowledge of the model's geometry. Adding physical constraints at this early stage would be ineffective. Therefore, we propose a two-stage training strategy: the coarse stage and the refine stage. In the coarse stage, we generate rough shape of the model using text prompt. We can adopt any SDS-based generation framework as our baseline model, offering various choices of implicit representation and differentiable renderers. In the refine stage, we optimize the geometry with our physical constraints included. For this, we use tetrahedral SDF representation [67] and employ Deep Marching Tetrahedra (DmTet) to differentiably convert the coarse geometry from implicit density or SDF as necessary. We utilize Nydiffrast [26] as the diffcrentiable renderer and Stable Diffusion v [64] for guidance. We propose the following loss function for the joint optimization of texture, geometry, and stability:
L = ASDS + Xatan + Auable + Anomall + Ao Abolap St
(10)
In practice, observe that adding standability loss once every 10 iterations is sufficient ensure significant reduction in loss without notably increasing computational overhead.
Experiments
In this section, we devise comprehensive experiments (both virtual and real- real-world) to demonstrate the efficacy our method. We use a series of text prompts to generate 3D models that we expect to be self-supporting, and compare the generated results with baseline models. Our models are verified by simulation for stability and then fabricated using a 3D printer for real-world testing. We refer readers to the Appendix for more details about implementation, training, and experiment setup.

---

Ours
MVDream
Ours
MVDream
An astronaut riding horse
running Harley
motorcycle
Farnese Atlas, holding up celestial globe his shoulders
A floor lamp
Figure 5: Atlas3D is also compatible with MVDream [69], enhancing it with stable standability.
5.1 Simulation Verification
Using the same text prompts, we compare our generated models with previous methods. The quality of the results is assessed by their stability, which verified by forward simulation: we simulate the generated models in an upright initial position close to the ground for sufficiently long time and record whether they fall. Using Magic3I [35] as the baseline model, we visualize the initial state and later state in Fig. 4. Our generated meshes remain stable throughout the simulation, while the baseline models fail under the same conditions due a lack of consideration for physics.
Qualitative Comparison
We highlight the main changes in mesh topology that enable the models stand (see Fig. 4). These changes include modifications to both the overall shape and specific local geometries. More specifically, our physical adjustments alter the contact surface gain more support from the ground and shift the center of mass to be slightly lower and more centered above the contact surface. These macroscopic and microscopic optimizations jointly increase the support to the models, thus ensuring their stability. Note that our method slightly modifies the texture of the generated results as we are jointly optimizing our physical adjustments with the SDS loss.
Since we do not assume a specific baseline model in the first stage, we can vary the model used in the coarse stage to generate versatile, physically-aware 3D meshes based on different existing SDS generation models. We use MVDream [69] as another baseline model and compare the results side-by-side with ours in Fig. Our method improves the geometry of the mesh and ensures stability in simulation. The quality of the texture and the main part of the shape is determined by the underlying model used in the coarse stage, while we focus on improving the physical stability in the refinement stage. More qualitative comparisons with previous methods are provided in the Appendix.
A blation Study We perform ablation studies to demonstrate the necessity of our proposed losses. It can be observed in Fig. 6(b), that without standability loss Lstand. the model fails to stand. While the model can still stand without stable equilibrium loss L as demonstrated in Fig. 6(c), it less stable under perturbation (see next section for details). The geometry regularization loss term L helps smooth the geometry and avoid spiky artifacts on the surface as shown Fig. 6(d). Additionally, we show that applying mesh regularization as a post-poe pest-prceessing step, rather than integrating it with SDS loss in a joint optimization, can degrade text alignment as this neglects the semantics during the deformation process (see Fig. 6(e)).
(a) Ours
(b) w/o stand
(c) w/o stable
(d) w/o b-lap
(c) w/o SDS
Figure 6: Ablation study of each loss term.
7

---

Table 1: Comparison of success rate under perturbation (goose).
Perturbation Angle Omax
0.01
0.02
0.04
0,08
w/ stability loss
1
0.99
0.69
0.4
w/o stability loss
0.97
0.71
0.62
0.23
Stability under Perturbation In the real world, placing an object on the ground always involves some noise, as both human and robot manipulation have imprecision in angles and directions. Hence, the standability of an object under small perturbations is crucial for improving the success rate of such tasks.
ultraman astronaut-horse
goose
mannequin
snowboarding
1.00
0.75
Suczoss Rate
To mimic this uncertainty in our framework, we evaluate the stability of our generated models under small initial rota- tion. With a given precision Puaxx we rotate the generated mesh random angles oy. Øz (-Pmax: Pmax) in both the y and r axes, respectively (with the 2 axis being the up direction). The mesh is then placed close the ground and tested in a simulation to see if it can still stand. We choose 13 different values of the maximum perturbation angle dmmax and perform 100 random tests with each angle on 6 of our generated models. We report the success rate in Fig. 7. We define a successful test as: after a sufficiently long time pe- riod, the maximum height the model stays within 3% of the initial maximum height.
0.50
0.25
0.00
0.00
0.05
0.10
0.15
0.20
Perturbation Angle (rad)
Figure 7: Success rate of models stand- ing under perturbation.
Due to the presence of physical constraints, our generated models can withstand small initial pertur- bations, while the baseline models fail to stand when placed straight up (without rotation), let alone with perturbations. Furthermore, introducing the stable equilibrium loss consistently increases the success rate of standing under different scales of perturbations, shown in Table. 1.
Standability on Different Platforms Our pipeline can be generalized to learn standability on various platforms, not just flat ground, by incorporating them as boundary conditions in the simulator. To demonstrate this, we use a 10-degree inclined plane and sphere, then train our 3D model separately stand still on each. Modeling frictional contact is crucial for achieving stability in such scenarios. As shown in Figure Fig, 8, our optimized mesh stands stably on both the incline and the sphere with the help of static friction, whereas the baseline model fails expected.
w/ Atlas3D
w/o Atlas3D
Simulator Eross-validation While we base our method on a differentiable rigid-body simulator with semi-implicit Euler time-integrator, our pipeline is compatible with any other physics-based -ased simulator as the backbone, with dif- ferentiability required for the training stage. To verify the reliability of models generated with our simulator, clude an external simulator in the testing stage to verify the correctness of the simulated dynamies. We choose the Incre- mental Potential Contact (IPC) method [27; 29], which has been proven accurate for frictional contact. We validate the correctness of every single generated model and visualize the simulation results in Fig.
in-
Figure 8: Standability evaluation on uneven surfaces.

---

Brucolli
Snowboording
Ultraman
we Atlas3D w/ Atlas3D
Figure 10: Standability test using a robotic arm. More results are shown in the Appendix.
Quantitative Evaluation We examine the versatility of
1.25
our method. We randomly select 150 prompts from [58] and manually exclude 43 prompts deemed unfeasible (for
w/o Atlas3D
w/ Atlas3D
1.00
instance, it does not make sense to require "a swan and its
cygnets swimming in pond" to be standable), leaving total of 107 prompts. We use the two-stage Magic3D [35] as the baseline model and compare our optimized mesh with the results from the refine stage of the baseline under
0.75
0.50
the same settings (e.g.. iterations, loss weights).
To evaluate the standability of the baseline method and our method, we run the rigid body simulation in Warp with simulation end time = 2.0 at which almost all objects have reached the steady state. We propose Time-Averager Rotation Deviation Loss (TRD) defined as
0.25
0.00
Figure 9: TRD results from 107 prompts using the Magic3D baseline and our method.
TRD 7 J."" [I(()2 - R(0)2|lade (R()))-((O)]]]add
(11)
to assess the standability, as representation of the average tilting of the upward direction å (of the object) over time. We approximate the integral (Eq. 11) with discrete quadrature At = 0.02. Results are plotted in Fig. 9 The mean TRD score is reduced by more than six times compared with the baseline method. As shown in Table. 2, we calculate the average CLIP score [63] of 107 generated shapes for both our proposed method and baseline. Furthermore, Elo (GPT-40) [90] scores are presented. Both metrics illustrate that our method not only implements physics-based stability but also maintains the fidelity of the generated 3D shapes terms of content alignment and overall shape quality. More details are provided in the Appendix.
5.2 Real-world Validation
One major advantage of incorporating physics- based sim- ulation into the optimization pipeline is that it bridges the gap between the generated model and the real world. Our method ensures the direct usability of the model for fabri- cation, with success primarily dependent on the accuracy of both the simulator and the manufacturing machine.
Table 2: Quantitative Evaluation
Metries
Ours
Magic3D
TRD 1 CLIP
0.060
0.389
25.356
25.781
Elo (GPT-40) 1
970.774
1029.226
3D Printing and User Studies We test the readiness
of our generated meshes for real-world application by producing eight figures using 3D-printing device (Zortrax M200 with Z-ABS filament material). For reference, we also print the corresponding baseline meshes generated without physical constraints. Our physically constrained figures can steadily support themselves when gently placed on an even surface, while the baseline figures either fail to stand at all, require extensive adjustments and fall casily with little perturbation (see Fig. 2).
We conduct user studies with these printed figures to assess their stability under different types of human manipulation. Ten users were asked to place the figures upright on table, with five trials for each figure, resulting in total of 800 trials. Fabricated figures generated from the baseline has success rate of 7,, while figures generated from our method has an overall success rate of 92.25%.
'Rssults ranked by TRD scores of the baseline approach.

---

Itemized results are provided in the Appendix. Our method significantly increases the physical stability of the models under varying human efforts.
Validation with a Robot Arm To showcase the compatibility of our framework with robotic applications. we test our fabricated figures with a teleoperated LewanSoul LeArm robot arm outfitted with two-finger parallel gripper (see Fig. 10). The gripper is set to initially grasp figure above the ground. It slowly moves downward, and is then gently released to place the figure on the ground. Four trials were performed for each figure, yielding 64 trials in total. For the baseline method, 6.25% trials resulted in successful standing figures, while ours has a success rate 90.6%. Experimental data are provided in the Appendix and supplemental video.
6 Conclusion
We present Atlas3D, a physically constrained SDS-based framework that generates self-suup 3D models from text prompts. Our framework can learn standability through a differentiable physics- based simulator and other physics-insping loss functions. The generated 3D models can be directly imported into physics simulator and are ready to be manufactured and deployed in the real world. Our method has wide potential for generation tasks, as it can be easily integrated into many existing pipelines and improve the physical plausibility of their generated results.
Limitations and Future Work Our physical adjustments are optimized over all mesh vertices, resulting in large degrec of freedom in optimization. This may lead to undesired distorted meshes [49]. Future works may consider adding latent embedding or skeleton rigging to limit the variety of mesh deformation. Our framework focuses SDS-based methods interesting to further generalize our physical constraints to other non-SDS or non-diffusion based methods [22; 49; 28]. Finally, we only consider text-to-3D tasks in this work. An exciting extension is to generalize our work to image-to-3D tasks [60; 39].
backbone. It would be
References
[1] Titas Anciukevičius, Zexiang Xu, Matthew Fisher, Paul Henderson, Hakan Bilen, Niloy Mitra, and Paul Guerrero. Renderdiffusion: Image diffusion for 3d reconstruction, inpainting and generation. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12608-
12618, 2023.
[2] David Baraff. Physically based modeling: Rigid body simulation. SIGGRAPH Course Notes, ACM SIGGRAPH, 2(1):2-1, 2001.
[3] Miguel Angel Bautista, Pengsheng Guo, Samira Abnar, Walter Talbott, Alexander Toshev, Zhuoyuan Chen, Laurent Dinh, Shuangfei Zhai, Hanlin Goh, Daniel Ulbricht, et al. Gaudi: neural architect for immersive 3d seene generation. Advances in Neural Information Processing Systems, 35:25102- 11-25102-22116, 2022.
[4] Bryce Blinn, Alexander Ding, R Kenny Jones, Manolis Savva, Srinath Sridhar, and Daniel Ritchic. Learning body-aware 3d shape generative models. arXiv preprint arXiv:2112.07022, 2021.
[5] Ziang Cao, Fangzhou Hong, Tong Wu. Liang Pan, and Ziwei Liu. Large-vocabulary 3d diffusion model with transformer. arXiv preprini (tX.i:2339.079222 2023.
[6] Hansheng Chen, Jiatao Gu, Anpei Chen, Wei Tian, Zhuowen Tu, Lingjie Liu, and Hao Su. Single-stage diffusion nerf: A unitied approach to 3d generation and reconstruction. In Proceedings the IEEE/CVF International Conference on Computer Vision, pages 2416-2423 2023.
[7] Hsiao-yu Chen, Edith Tretschk, Tuur Stuyck, Petr Kadlecck, Ladislav Kavan, Etienne Vouga, and Christoph Lassner. Virtual elastic objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15827-15837, 2022.
[8] Rui Chen, Yongwei Chen, Ningxin Jiao, and Kui Jia. Fantasia3d: Disentangling geometry and appearance for high-quality text-to-3d 3d content creation. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages
2023.
[9] Zilong Chen, Yikai Wang, Feng Wang, Zhengyi Wang, and Huaping Liu. V3d: Video diffusion models are effective 3d generators. arXiv preprint 2rX.v:2403.06733, 2024.
10

---

[10] Yuan Dong, Qi Zuo, Xiaodong Gu. Weihao Yuan. Zhengyi Zhao, Zilong Dong, Liefeng Bo, and Qixing Huang. Gpld3d: Latent diffusion of 3d shape generative models by clupomt2022dataaforriin geometric and physical priors. The IEEE/CVT Conference on Computer Vision and Pattern Recognition (CVPR 2024). IEEE/CVE, 2024.
[11]
Emilien Dupont, Hyunjik Kim, SM Eslami, Danilo Rezende, and Dan Rosenbaum. From data to functa: Your data point is a function and you can treat it like one. arXiv preprint Miix:2201.12204, 2022.
[12) Ziya Erkoç, Fangchang Ma, Qi Shan, Matthias Nießner, and Angela Dai. Hyperdiffusion: Generating implicit neural fields with weight-space diffusion. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages
2023.
[13] David J Griffiths and Darrell F Schroeter. Introduction quantum mechanics. Cambridge university press, 2018.
[14] Yuan Cann- Guo, Ying-Tian Tian Liu, Ruizhi Shao, Christian Laforte, Vikram Voleti, Guan Luo, Chia-Har Chen, Zi-Xin Zou, Chen Wang. Yan-Pei Cao, and Song-Ha Zhang. threestudio: unified framework for 3d content generation. bttps://githe
com/ wttm///heeetuuioooo wwteppprttoeesetteiiieee
2023.
[15] Anchit Gupta, Wenhan Xiong, Yixin Nie, Jones, and Barlas Oğuz. 3dgen: Triplane latent diffusion for textured mesh generation. arXiv preprini arXiv:2303.0533 2023.
[16] Susung Hong. Donghoon Ahn, and Seungryong Kim. Debiasing scores and prompts of 2d diffusion for view-consistent text-to-3d generation. Advances in Neural Information Processing Systems, 36, 2024.
[17] Siyuan Huang, Zan Wang. Puhao Li. Baoxiong Jia, Tengyu Liu, Yixin Zhu, Wei Liang, and Song-Chin Zhu. Diffusion-based generation. optimization, and planning in 3d scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16750-16761, 2023.
[18] Yukun Huang, Jianan Wang, Yukai Shi, Boshi Tang, Xianbiao Qi, and I Lei Zhang. Dreamtime: An im- proved optimization strategy for diffusion-guided 3d generation. In The Twelfth International Conference on Learning Representations, 2023.
[19] Zhiao Huang. Yuanming Hu, Tao Du, Siyuan Zhou, Hao Su, Joshua Tenenbaum, and Chuang Gan. Plasticinelab: A soft- boft-body manipulation benchmark with diffcrentiable physics. arXiv preprint arXiv:2104.03:: 03311, 2021.
[20] Zizhou Huang, Davi Colli Tozoni, Arvi Gjoka, Zachary Ferguson, Teseo Schneider, Daniele Panozzo, and Denis Zorin. Differentiable solver for time-dependent deformation problems with contact. ACM Transactions on Graphics, 2022.
[21] Ying Jiang, Chang Yu, Tianyi Xie, Xuan Li. Yutao Feng, Huamin Wang. Minchen Li, Henry Lau, Feng Gao, Yin Yang, et al. Vr-gs: A physical dynamies-aware interactive gaussian splatting system in virtual reality. arXiv preprint arXiv:2401. /6663, 2024.
[22] Heewoo Jun and Alex Nichol. Shap-e: Generating conditional 3d implicit functions. arXiv preprint MXXiv:3005.02663 2023.
[23] Animesh Karnewar, Andrea Vedaldi, David Novotny, and Niloy Mitra. Holodiffusion: Training a 3d diffusion model using 2d images. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 18423-18433, 2023.
[24] Oren Katzir. Or Patashnik, Daniel Cohen-Or, and Dani Lischinski. Noise-free score distillation. In The Twelfth International Conference Learning Representations, 2024.
[25] Xin Kong, Shikun Liu, Xiaoyang Lyu, Marwan Taher, Xiaojuan Qi, and Andrew J Davison. Eschernet: A generative model for scalable view synthesis. arXiv preprint 4rXiv:2402.03908, 2024.
[26] Samuli Laine, Janne Hellsten, Tero Karras, Yeongho Seol, Jaakko Lehtinen, and Timo Aila. Modular primitives for performrfrr differentiable rendering. ACM Transactions on Graphics (ToG), 39(6):1 14, 2020.
[27] Lei Lan, Danny Kaufman, Minchen Li, Chenfanfu Jiang, and Yin Yang. Affine body dynamics: Fast, stable & intersection-fies simulation of stiff' materials. arXiv preprint arXiv:2201. 10022, 2022.
[28] Jiahao Li, Hao Tan, Kai Zhang, Zexiang Xu, Fujun Luan, Yinghao Xu, Yicong Hong, Kalyan Sunkavalli, Greg Shakhnarovich, and Sai Bi. Instant3d: Fast text-to-3d with sparse-vien generation and large reconstruction model. arXiv preprint arXiv:2311.00 2023.
11

---

[29] Minchen Li, Zachary Ferguson, Teseo Schneider, Timothy R Langlois. Denis Zorin, Daniele Panozzo, Chenfanfu Jiang. and Danny M Kaufman. Incremental potential contact: intersection-and inversion-free. large-deformation dynamics, ACM Trans. Graph., 39(4),49, 2020.
[30] Muheng Li, Yueqi Duan, Jie Zhou, and Jiwen Lu. Diffusion-sdf: Text-to-shape via voxelized diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12642- 12651. 2023.
[31] Weiyu Li. Rui Chen, Xuelin Chen, and Ping Tan. Sweetdreamer: Aligning geometric priors in 2d diffusion for consistent text-to-3d. arXiv preprint urXiv:2310.02596 2023.
[32] Xuan Li, Yi-Ling Qiao, Peter Yichen Chen, Krishna Murthy Jatavallabhula, Ming Lin, Chenfanfu Jiang, and Chuang Gan. Pac-nerf: Physics augmented continuum neural radiance fields for geometry-agnostic system identification. In The Eleventh International Conference on Learning Representations, 2022.
[33] Yifci Li, Hsiao-yu Chen, Egor Larionov, Nikolaos Sarafianos, Wojeiech Matusik, and Tuur Stuyck. Diffavatar: Simulation-ready garment optimization with differentiable simulation. arXiv preprint arXiv:231112119, 2023.
[34] Yifei Li, Tao Du, Kui Wu, Jie Xu, and Wojeiech Matusik. Diffcloth: Differentiable cloth simulation with dry frictional contact. ACM Transactions on Graphics (TOG), 42(1))1-20,, 2022.
[35] Chen-Hsuan Lin, Jun Gao, Luming Tang. Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming- Yu Liu, and Tsung-Yi Yi Lin. Magic3d: High-resolution text-to-3d content crcation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 300-309, 2023.
[36] Yuanze Lin, Ronald Clark, and Philip Torr. Dreampolisher: Towards high-quality text-to-3d generation via geometric diffusion. arXiv preprint arXiv:2403. 17237, 2024.
[37] Minghua Liu, Ruoxi Shi, Linghao Chen, Zhuoyang Zhang, Chao Xu, Xinyue Wei, Hansheng Chen, Chong Zeng, Jiayuan Gu, and Hao Su. One-2-3-45++: Fast single image to 3d objects with consistent multi-view generation and 3d diffusion. arXiv preprint arXiv:2311.00 1.00885, 2023.
[38] Minghua Liu, Chao Xu, Haian Jin, Linghao Chen, Mukund Varma T, Zexiang Xu, and Hao Su. One-2- 3-45: Any single image to 3d mesh in 45 seconds without per-shape optimization. Advances Neural Information Processing Systems, 36, 2024.
[39] Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl Vondrick. Zero-1-to-3- scro-shot one image to 3d object. In Proceedings the IEEE/CVF International Conference Computer Vision, pages 9298 9309, 2023.
[40] Xueyi Liu, Bin Wang. He Wang, and Li Yi. Few-shot physically-aware articulated mesh generation via hierarchical deformation. In Proceedings of the IEFF/CVP International Conference on Computer Vision, pages 8544-66,,20 2023.
[41] Yuan Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjic Liu, Taku Komura, and Wenping Wang. Synedrcamer: Generating multiview-consistent images from a single-view image. arXiv preprint arXiv:2309.03553 2023.
[42] Xiaoxiao I Long, Yuan-Chen Guo, Cheng Lin, Yuan Liu, Zhiyang Dou, Lingjie Liu, Yuexin Ma, Song-Hai Zhang, Marc Hahermann, Christian Theobalt, al. Wonder3d: Single image to 3d using cross-domain diffusion, arXiv preprint orXiv:2310. 15008, 2023.
[43] Shitong Luo and Wei Hu. Diffusson probabilistic models for 3d point cloud generation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2837-2845, 2021.
[44] Miles Macklin. Warp: A seghhppeformance python framework for gpu simulation and graphics. https: // thub. www/noiiiiaaaoooooooooooooioaaaa htap///daiaaaaapp..aaa/aa March 2022. NVIDIA GPU Technology Conference (GTC).
[45] Antoine Mercier, Ramin Nakhli, Mahesh Reddy, Rajeev Yasarla, Hong Cai, Fatih Porikli, and Guillaume Berger. Hexagen3d: Stablediffusion is just one step away from fast and diverse text-to-3d generation. 2024.
[46] Gal Metzer, Elad Richardson, Or Patashnik, Raja Giryes, and Daniel Cohen-Or, Latent-ned for shape- guided generation of 3d shapes and textures. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
2023.
12

---

[47] Mariem Mezghanni, Théo Bodrito, Malika Boulkenafed. and Maks Ovsjanikov. Physical simulation layer for accurate 3d modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recongiition, pages 13514-13523, 2022.
[48] Mariem Mezghanni. Malika Boulkenafed, Andre Lieutier, and Maks Ovsjanikov. Physically-awe -aware generative network for 3d shape modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9330-9341, 2021.
[49] Oscar Michel, Roi Bar-On, Richard Liu, Sagie Benaim, and Rana Hanocka. Text2mesh: Text-driven neural stylization for meshes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 13492-13502, 2022.
[50] Ben Mildenhall. Pratul P Srinivasan, Matthew Tancik. Jonathan T Barron, Ravi Ramamoorthi. and Ren Ng. Nerf: Representing scenes neural radiance fields for view synthesis. Communications of the ACM, 600 2021.
[51] Shentong Mo, Enze Xie, Ruihang Chu, Lanqing Hong, Matthias Niessner, and Zhenguo Li. Dit- 3d: Exploring plain diffusion transformers for 3d shape generation. Advances in Neural Information Processing Systems, 36, 2024.
[52] Norman Müller, Yawar Siddiqui, Lorenzo Porzi, Samuel Rota Bulo, Peter Kontschieder, and Matthias NieBner. Diffrf: Rendering-guiin guined 3d radiance field diffusion. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 4328-4338, 2023.
[53] J Krishna Murthy, Miles Macklin, Florian Golemo, Vikram Voleti,
Linda Petrini, Martin Weiss, Breandan Considine, Jérôme Parent-Léévsquee, Lévesque, Kevin Xie, Kenny Erlehen, et al. gradsim: Differentiable simulation for system identification and visuomotor control. In International conference on learning representations, 2020.
[54] Junfeng Ni, Yixin Chen, Bohan Jing, Nan Jiang. Bin Wang. Bo Dai, Yixin Zhu, Song-Chun Chun Zhu, and Siyuan Huang. Phyrecon: Physically plausible neural scene reconstruction. arXiv preprint arXiv:2404. 16666, 2024.
[55] Evangelos Ntavelis, Aliaksandr Siarohin, Kyle Olszewski, Chaoyang Wang, Lue V Gool, and Sergey Tulyakov. Autodecoding latent 3d diffusion models. Advances in Neural Information Processing Systems, 36:67021-670477 2023.
[56] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang. Zachary De Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
[57] Ryan Po and Gordon Wetzstein. Compositional 3d scene generation using locally conditioned diffusion. arXiv preprint arXiv:2303. 122188 2023.
[58] Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffuusion, arXiv preprint MiXiv:2209.14988 2022.
[59] Romain Prévost, Emily Whiting, Sylvain Lefebvre, and Olga Sorkine-Hornung. Make it stand: balancing shapes for 3d fabrication. ACM Transactions on Graphics (TOG), 32(4):1-10, 2013.
[60] Guocheng Qian, Jinjie Mai, Abdullah Hamdi, Jian Ren, Aliaksandr Siarohin, Bing Li. Hsin- Ying Lee. Ivan Skorokhodov, Peter Wonka, Sergey Tulyakov, and Bernard Ghanem. Magic 123: One image to high-quality 3d object generation using both 2d and 3d diffusion priors. In The Twelfth International Conference Learning Representations (ICLR), 2024.
[61] Yi-Ling Qiao. Junbang Liang, Vladlen Koltun, and Ming Lin. Efficient differentiable simulation of articulated bodies. In International Conference on Machine Learning, pages 8661-8671. PMLR, 2021.
[62] Ri-Zhao Qiu, Ge Yang. Weijia Zeng, and Xiaolong Wang. Feature splatting: Language-driven physics- based scene synthesis and editing. arXiv preprint 200::2404000223, 2024.
[63] Alec Radford, Jong Wook Kim, Chris Hallacy. Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748-8763. PMLR, 2021.
[64] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. figh-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10600 2022.
13

---

[65] Robin Rombach, Andreas Blattmann. Dominik Lorenz. Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models, 2021.
[66] Junyoung Seo, Wooseok Jang, Min-Seop Kwak, Hyeonsu Kim, Jachoon Ko, Junho Kim, Jin-Hwa Kim, Jiyoung Lee, and Seungryong Kim. Let 2d diffusion model know 3d-consistency for robust text-to-3d generation. arXiv preprint 2030:2::33307977, 2023.
[67] Tianchang Shen, Jun Gao, Kangxue Yin, Ming- Yu Liu, and Sanja Fidler. Deep marching tetrahedra: hybrid representation for high-resolution 3d shape synthesis. Advances in Neural Information Processing Systems, 34:6087-6101, 2021.
[68] Ruoxi Shi, Hansheng Chen, Zhuoyang Zhang, Minghua Liu, Chao Xu, Xinyue Wei, Linghao Chen, Chong Zeng, and Hao Su. Zerol23++4 single image to consistent multi-view diffusion base model, 2023.
[69] Yichun Shi, Peng Wang, Jianglong Ye, Mai Long, Kejie Li, and Xiao Yang. Mvdream: Multi-view diffusion for 3d generation. arXiv preprint arXiv:2308. 16512, 2023.
[70] Jachyeok Shim, Changwoo Kang, and Kyungdon Joo. Diffusion based signed distance fields for 3d shape generation. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 20887-20897, 2023.
[71] Dule Shu, James Cunningham, Gary Stump, Simon W Miller, Michael A Yukish, Timothy W Simpson, and Conrad S Tucker. 3d design using generative adversarial networks and physics-based validation. Journal of Mechanical Design, 142(7):071 1001, 2020.
[72] J Ryan Shue, Eric Ryan Chan, Ryan Po, Zachary Ankner, Jiajun Wu, and Gordon Welzstein. 3d neural field generation using triplane diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20875-20886, 2023.
[73] Michael Streeke and Joerg Stueckler. Diffisdismm: Differentiable rigid-bord body dynamics with implicit shapes. In 202/ international conference on 3D Vision (3DV), pages 96-105. IERF, 2021.
[74] Priya Sundaresan, Rika Antonova, and Jeannette Bobgl. Diffcloud: Real-to-sim from point clouds with differentiable simulation and rendering of deformable objects. In 2022 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 10828-10835. IEEE, 2022.
[75] Stanislaw Szymanowicz, Christian Rupprecht, and Andrea Vedaldi. Viewset (iffusion:(0-)) image- conditioned 3d generative models from 2d data. In Proceedings the IEEE/CVF International Conference on Computer Vision, pages 8863-8873. 2023.
[76] Tetsuya Takahashi, Junbang Liang, Yi-Ling Qiao, and Ming C Lin. Differentiable fluids with solid coupling for learning and control. In Proceedings of the AAAI conference on artificial intelligence. volume 35, pages 6138-6146, 2021.
[77] Boshi Tang, Jianan Wang, Zhiyong Wu, and Lei Zhang. Stable score distillation for high-quality 3d generation. arXiv preprint arXiv:2312.09305, 2023.
[78] Jiaxiang Tang. Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. Dreamgaussian: Generative gaussian splatting for efficient 3d content creation. arXiv preprint arXiv:2309. 16653, 2023.
[79] Junshu Tang, Tengfei Wang. Bo Zhang, Ting Zhang, Ran Yi, Lizhuang Ma. and Dong Chen. Make-it-3d: fidelfiyy 3d creation from a single image with diffusion prior. In Proceedings of the IEEE/CVF International Conference on Computer Vision pages 22819-22822 2023.
[80] Luming Tang. Menglin Jia, Qianqian Wang, Cheng Perng Phoo, and Bharath Hariharan. Emergent correspondence from image diffusion. Advances in Neural Information Processing Systems, 36:1363- 1389, 2023.
[81] Arash Vahdat, Francis Williams, Zan Gojeic, Or Litany, Sanja Fidler, Karsten Kreis, et al. Lion: Latent point diffusion models for 3d shape generation. Advances in Neural Information Processing Systems, 15021-110, 2022.
[82] Alexander Vilesov, Pradyumna Chari, and Achuta Kadambi. Cg3d: Compositional generation for text-to-3d via gaussian splatting. arXiv preprint arXiv:2311.17900 2023.
[83] Vikram Voleti, Chun-Han Yao, Mark Boss, Adam Letts, David Pankratz, Dmitry Tochilkin, Christian Laforte, Robin Rombach, and Varun Jampani. Sv3d: Novel multi-view synthesis and 3d generation from a single image using latent video diffusion. arXiv preprint arXiv:2403.12008 2024.
14

---

[84] Haochen Wang. Xiaodan Du Jiahao Li. Raymond A Yeh, and Greg Shakhnarovich. Score jacobian chaining: Lifting pretrained 2d diffuusion models for 3d generation. In Proceedings of the IELE/CVF Conference on Computer Vision and Pattern Recognition, pages 12619-12629, 2023.
[85] Jianren Wang and Yihui He. Physics-aware 3d mesh synthesis. In 2019 International Conference on 3D Vision (3DV), pages 502-512. IEEE, 2019.
[86] Peng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. arXiv preprint arXiv:2106. 10689, 2021.
[87] Peng Wang and Yichun Shi. Imagedream: Image-prompt nulti-view diffusion for 3d generation. arXiv preprint arXiv:2312.02200 2023.
[88] Zhengyi Wang. Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu, Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distillation. Advances in Neural Information Processing Systems, 36, 2024.
[89] Haohan Weng, Tianyu Yang. Jianan Wang, Yu Li. Tong Zhang, C. L. Philip Chen, and Lei Zhang. Consistent123: Improve consisteney for one image to object synthesis, 2023.
[90] Tong Wu, Guandao Yang. Zhibing Li, Kai Zhang, Ziwei Liu, Leonidas Guibas, Dahua Lin, and Gor- don Wetzstein. Gpt-4v (ision) is human-aligned evaluator for text-to-3d generation. arXiv preprint arXiv:2401.04999 2024.
[91] Tianyi Xie, Zeshun Zong, Yuxin Qiu, Xuan Li, Yutao Feng, Yin Yang, and Chenfanfu Jiang. Physgaussian: Physics-integratee 3d gaussians for generative dynamies. arXiv preprint arXiv:2311.12198, 2023.
[92] Qingshan Xu, Jiao Liu, Melvin Wong, Caishun Chen, and Yew-Soon Ong. Precise-physics driven text-to-3d generation. arXiv preprint arXiv:2403. 12438, 2024.
[93] Yinghao Xu, Hao Tan, Fujun Luan, Sai Bi, Peng Wang, Jiahao Li, Zifan Shi, Kalyan Sunkavalli, Gordon Wetzstein, Zexiang Xu, et al. Dmv3d: Denoising multi-view diffusion using 3d large reconstruction model. arXiv preprint vrXiv:2311.092 1,09217, 2023.
[94] Fan Yang. Jianfeng Zhang, Yichun Shi, Bowen Chen, Chenxu Zhang, Huichao Zhang, Xiaofeng Yang. Jiashi Feng, and Guosheng Lin. Magic-hoost: Boost 3d generation with mutli-view conditioned diffusion. arXiv preprint arXiv:2404.06422, 2024.
[95] Jiayu Yang, Ziang Cheng, Yunfei Duan, Pan Ji, and Hongdong Li. Consistnet: Enforcing 3d consistency for multi- -iew images diffusion. arXiv, 2023.
[96] Xiaofeng Yang, Yiwen Chen, Cheng Chen, Chi Zhang. Yi Xu, Xulei Yang. Fayao Liu, and Guosheng Lin. Learn to optimize denoising scores for 3d generation: A unified and improved diffusion prior nerf and 3d gaussian splatting. arXiv preprint arXiv:2312.04822 2023.
[97] Yandan Yang. Baoxiong Jia, Peiyuan Zhi, and Siyuan Huang. Physcene: Physically interactable 3d scene synthesis for embodied ai. arXiv preprint 40X1::2404.09465, 2024.
[98] Yue Yang, Fan-Yun Yun Sun, Luca Weihs, Eli VanderBilt, Alvaro Herrasti, Winson Han, Jiajun Wu, Nick Haber, Ranjay Krishna, Lingjie Liu, et al. Holodeck: Language guided generation of 3d embodied ai environments. In The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2024). volume 30. pages 25. IEEE/CVE, 2024.
[99] Jianglong Ye, Peng Wang. Kejie Li, Yichun Shi, and Heng Wang. Consistent-1-to: Consistent image to 3d view synthesis via geometry-aware diffusion models. arXiv preprint arXiv:2310.03020, 2023.
[100] Xuanyu Yi, Zike Wu, Qingshan Xu, Pan Zhou, Joo-Hwee Lim, and Hanwang Zhang. Diffusion time-step curriculum for one image to 3d generation. arXiv preprint 240.22404.0456. 2024.
[101] Xin Yu, Yuan-Chen Guo, Yangguang Li, Ding Liang, Song- Hai Zhang, and XIAOJUAN QL Text-to-3d with classifier score distillation. In The Twelfth International Conference on Learning Representations, 2024.
[102] Ye Yuan, Jiaming Song. Umar lqbal, Arash Vahdat, and Jan Kautz. Physdiff: Physics- guiied human motion diffusion model. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages
2023.
15

---

[103] Shutong Zhang, Yi-Ling Qiao, Guanglei Zhu, Eric Heiden, Dylan Turpin, Jingzhou Liu, Ming Lin, Miles Macklin, and Animesh Garg. Handypriors: Physically consistent perception of hand-object interactions with differentiable priors. arXiv preprint arXiv:2311.16552. 2023.
[104] Tianyuan Zhang, Hong-Xing Yu, Rundi Wu, Brandon Y Feng. Changxi Zheng, Noah Snavely, Jiajun Wu, and William Freeman. Pmysdreamer: Physics-based interaction with 3d objects via video generation. arXiv preprint arXiv:2404.13026, 2024.
[105] Minda Zhao, Chaoyi Zhao, Xinyue Liang, Lincheng Li, Zeng Zhao, Zhipeng Hu, Changjie Fan, and Xin Yu. Efficientdreamer: High-fidelitg fidelity and robust 3d creation via orthogonal -veww diffusion prior. arXiv preprint arXiv: 132233 2023.
[106] Chuanxia Zheng and Andrea Vedaldi. Free3d: Consistent novel view synthesis without 3d representation. arXiv, 2023.
[107] Xin- Yang Zheng. Hao Pan, Peng- Shuai Wang, Xin Tong. Yang Liu, and Heung Yeung Shum. Locally at- tentional sdf diffusion for controllable 3d shape generation. ACM Transactions on Graphics (SIGGRAPH), 42(4), 2023.
[108] Zhizhuo Zhou and Shubham Tulsiani. Sparsefusion: Distilling view-conditioned diffusion for 3d reconstruction. In CVPR, 2023.
[109] Joseph Zhu and Peiye Zhuang. Hifa: figh-fidelity text-to-3d 3d with advanced diffusion guidance. arXiv preprint 2305 18766, 2023.
16

---

A Appendix
A.1 Score Distillation Sampling
Score Distillation Sampling (SDS) has proven to be an effective method for distilling 3D models from 2D images. SDS-based methods leverage a combination of 3D representations, differentiable rendering, and pre-trained text-to-image diffusion models to optimize 3D shapes with high fidelity and realism.
The generation process in SDS methods involves optimizing the parameters of the 3D representation. At each iteration, the 3D geometry is differentiably rendered into 2D image = g()). This image is then compared with the distribution of real images as modeled by the diffusion model. More specifically, the input image is first noised with a random noise at specified noise level l. The diffusion model then predicts the noise &g with text guidance y, and this prediction is compared with e, resulting in the following loss:
LSDS Ex,e (u((t I|Ed (Zt, v,t) -<l2]
(12)
where (t) is a weighting function modulating the influence of different noise levels. The expectation is taken over the noise level t and noise term ensuring that the generated 3D shape aligns with the text-guided distribution of images.
The gradient with respect to the optimization parameter is then backpropagated as follows:
0z
Lsus(2 = g(0)) = Ei,e wt() (Eo (Z..81..))
(13)
00
where the U-Net Jacobian term For further technical details SDS optimization, refer readers to [58; 46]
dēe is omitted for efficient optimization. Uz:
A.2 Implementation and Training Details
We implement our pipeline in PyTorch with Adam optimizer. For differentiable simulation, we adopt the semi-implicit Euler simulator in Warp [44], where gradients of physical states computed via auto-diffcrentiation and backpropagated to the parameters of 3D representations [44; 56]. We two-stage training strategy and leave the choice of the first stage open for various SDS- ibSebasedd methods as baselines. For a two-stage baseline method, we implement our physics-inspires losses as submodules that can be seamlessly integrated into the refinement stage. For one-stage method, approach can be used as standalone refinement stage to improve the physical quality of the baseline. For baseline methods that [14].
publicly available, we use the reimplementation from threestudio
We train our models using single NVIDIA RTX 3090 GPU. During refinement stage, implement a skipping strategy, incorporating standability loss only once every 10 iterations. In our quantitative evaluation of a batch of prompts, we observed an average refinement time of 36 minutes for each training step, with a default setting of 5,000 iterations.
For the rigid body simulator in Warp, we set dt = 10 10-3s. Contact stiffness and damping are set to 10 and 2.0; friction coefficient is set to 0.3; stiffness of friction force is set 103, Density of the 3D objects is set to 10
In experiments, we the following default weights for the loss terms: {ASDS 1, Anormal 10 Astand = 10S, Astable = 105, Ap-tap = 107}. For some examples, we tune these weights within the following ranges: [Astand = 105 ~ 5 x 105, Auable = 105 ~ 5 x 105, Ab-lap = 10 107) Our heuristic intuition is to keep the SDS and physical loss terms roughly on the same scale. For the regularization terms, we scale them to around 1/1000 to 1/100 of the SDS and physical loss terms.
17

---

A.3 Comparison with Past-processing Methods
While directly post- proteproesssing 3D generated models straightforward and effective approach to achieving physical stability, it may result in undesirable outcomes such as misalignment with the text prompt, as it overlooks semantics.
One simple post-processing method to cut the mesh by flat plane slightly higher than the lowest vertex. However, this method will fail when the projection of the center of mass lies outside the contact region, as shown Fig. 11. Additionally, determining the cutting height is another parameter that must be manually set optimized, and this may also degrade the overall appearance.
Ours
z= 0.05
0.10
z= 0.15
= 0.20
Figure 11: Comparison with cutting the mesh by flat plane at height
Another
method, make-it-stand [59], offers effective way relocate the center of mass to achieve standability. However, assumes that the supporting surface is fixed during optimization, which can lead distorted results due to the imperfect quality of text-to-3D generated models. We provide two examples in Fig. 12. For the goose example generated by Magic3D, one leg shorter than the other. Make-il-stand only treats one foot the supporting surface and ignores the other due to its post-processing nature, whereas our joint-optimization pipeline enables stable standing with two legs on the ground. A similar issue also happens to the kangaroo example. More importantly, the text alignment degrades as semantics are overlooked during optimization. In contrast, our proposed joint optimization method preserves the text alignment and dynamically adjusts the center of mass as well as the supporting surface configuration.
Ours
iake-it-Stand
Ours
Make-it-Stand
Figure 12: Comparison with make-it-stand.?
In Fig. 6(e) the main text, we also apply our proposed losses in post- post-processing manner. Similarly, while it is able to optimize the 3D models to make it standable, the text alignment is compromised. Overall, compared to the post-processing method, joint optimization is more robust way which better balances text alignment and physical constraints.
"Although output mesh of make-it-stand the same resolution as the imput mesh, the ordering vertices and surfaces are shuffled in their implementation. As a result, we are unable to re-attach the original textures onto the output mesh. We are therefore unable to run metrics like CLIP in the paper to compare text alignment. Nevertheless, it should be visually clear that results do have better text alignment.
18

---

A.4 Additional Results
A.4.1
Qualitative Comparison with Magic3D Baseline
Ours
Magic3
[ standing goose
a standing pigeon
[...] a jumping kangaroo
[...] a mannequin
[...]a standing Konan
[...] a standing ultraman
Figure 13: More comparison with Magic3D baseline
19

---

Ours
Magic3D
[...] a toy robot
[...] a robot made out of vegetables
[...] a goose made out of gold
[...] small saguaro
cactus
planted in clay pot
[...] a baby dragon hatching out of a stone egg
[...] bear playing electric bass
Figure 14: More comparison with Magic3D baseline
20

---

Ours
Magic3D
A Marble bust of mouse
A palm tree, low poly 3d model
dachshund dressed up in a hotdog costume
Michelangelo style statue of an astronaut
[..]] front view of an easter egg
[...] a mushroom
Figure 15: More comparison with Magic3D baseline
21

---

A.4.2 Qualitative Comparison with MVDream Baseline
Ours
MVDream
A detective
Conan
A standing kid
Mickey Mouse, [...]
Figure 16: More comparison with MVDream baseline
A.4.3 Mesh Topology Change
We provide zoomed-in view of the local mesh topology change in Fig. 17 demonstrate how our method optimizes the object's geometry to make it standable.
w/ Atlas3D
w/o Atlas3D
Figure 17: Local mesh topology change
A.4.4 Quantitative Comparison with Magic3D Baseline
The results of the quantitative experiments are shown in 2. For CLIP score calculation, we specifically employ ppenai/cii-xxitllrr--puddd 14-336 the check point of CLIP model. We rendered images from various angles (0-360 degree, 3 degree as the interval). For Elo (GPT-40) benchmark, we made use of the newly released GPT-4o model instead of the one in [90] due to capacity limitation. Additionally, we reduce the number of views to because of token length limitation.
22

---

A.5 Real-world Experiments
A.5.1 Robot Manipulation Results 8
Figure 18: Robot manipulation experiment
We record our robot manipulation experiment in Fig. 18 and summarize quantitative results in Table. 3.
We define upward pose for the kangaroo figure as its tail touching the ground, real kangaroo does.
23

---

Table 3: Number of successes in robotic trials.
Figure
Broccoli
Egg
Horse
Kangaroo
Konan
Mannequin
Snowboarding
Ultraman
Baseline
0
1
0
0
0
1
Ours
3
4
3
4
4
4
3
4
A.5.2 User Study Results
We report the detailed results of our user studies in Table.
Table 4: Number of successes in user studies.
Figure
Broccoli
Egg
Horse
Kangaroo
Konan
Mannequin
Snowboarding
Ultraman
Baseline
8
2
7
Ours
44
46
40
48
47
50
45
49
24

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims are meticulously demonstrated in the "Experiments" section.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The limitations of the work are addressed in the "Discussion" section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
25

---

Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibiliyy
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The paper includes sufficient details of the experimental settings in the "Exper- iments" section to ensure reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
26

---

Answer: [Yes]
Justification:
The authors provide code to reproduce results in the paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: hat////oooemsssseeeeeeeeea
/nips. wwtppcccpcc/cceecccccccccccccctt haaaa//ggddeeeeeeeeaaaiieeeeee commisssoooolcyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification: The paper specifies the training and test details the "Experiments" section.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The paper reports appropriate information about the statistical aspect in the "Experiments" section.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
27

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπхо rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper provides sufficient information about computer resources in the "Experiments" section.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: :
Answer: [Yes]
Justification:
The paper conforms with the Code of Ethics every respect.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction..
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is no new societal impact compared previous generative models on similar topics, as the improvement in quality does not introduce additional uncertainty.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
28

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All existing assets used are clearly cited or credited.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
29

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
30


Reviews:
[{"review_id": "VyZr2NM9di", "reviewer": "Reviewer_Nrjn", "summary": "This paper addresses the problem of generating 3D models from text that are visually appealing but often physically unstable in simulations or when 3D printed. The authors incorporates a differentiable simulation-based loss function and physically inspired regularization to ensure generated models are stable under perturbation. The proposed method involves a two-stage training process where a coarse model is first generated from text prompts and then refined with physical constraints to ensure stability. Experiments show that models created with Atlas3D maintain stability better than those from existing methods. The optimized models are also 3D fabricated to verify the stability in real world.", "questions": "I don't have any specific questions at this time. The paper is clear and well-executed, addressing an important issue effectively. Implementation is also described in detail.", "limitations": "I am curious about the failure modes of Atlas3D. Could you provide more details on the scenarios or conditions under which the method might not perform as expected? Understanding these limitations would be helpful for future research and applications. Additionally, addressing these potential failure modes could further strengthen the robustness of your approach.", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The paper proposes an original and important solution to the issue of physical stability in text-to-3D generation, combining differentiable simulation-based loss functions with physically inspired regularization. This approach significantly enhances existing methods, reducing the need for manual post-processing and making the models immediately practical for various applications.\n\nThe method is sound, with clear and concise writing that guides the reader effectively and should be straightforward to implement for someone with a background in physical simulation. The experiment quality is high, validated through extensive experiments in both simulations and real-world scenarios, demonstrating the method's robustness and versatility. The results confirm the method's effectiveness in ensuring the stability of 3D models.", "weaknesses": "The paper is strong overall, and I did not identify any significant weaknesses. However, to further support future research in this area, I would suggest releasing the code. Providing the implementation would greatly benefit the community, enabling others to replicate the results and build upon this work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for recognizing the novelty and effectiveness of our method. To the best of our knowledge, our method is the first to bring standability to large generative models via joint optimization of standability loss and score distillation sampling loss. \n\n**Q1: I am curious about the failure modes of Atlas3D. Could you provide more details on the scenarios or conditions under which the method might not perform as expected?**\n\nOur approach may fail under certain cases. Firstly, some text prompts inherently contradict the concept of standability, such as “a swan and its cygnets swimming in a pond” or “a beautiful rainbow fish.” In these cases, the SDS loss may conflict with the physical loss, leading to unsatisfactory results. Additionally, since we assume models are solid with uniform density, under certain geometries where the projected position of the center of mass onto the horizontal plane is way off the contact region, our approach may not converge to the global minimum of the standability loss (which is zero). \n\n**Q2: I would suggest releasing the code.**\n\nThank you for the suggestion. Our code has been cleaned up and is ready for release upon acceptance.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ai726ykA07", "reviewer": "Reviewer_MwTq", "summary": "The paper introduces a differentiable simulation-based loss to refine the existing SDS(Score Distillation Sampling)-based text-to-3D frameworks. Concretely, it relies on a differentiable simulator Warp to provide gradients for keeping the rotation of the generated mesh unchanged after a period of time. Besides, the authors also introduce a stable equilibrium loss, which favors a flat contact surface. Experiments show that the proposed method can generate self-supporting 3D models given text prompts, verified by simulation and real-world results with 3D-printed objects.", "questions": "1. Can the authors provide more baselines, especially any existing post-processing method with the help of differentiable simulators, to enhance their technical contributions? One example is given in the *Weaknesses* section.\n2. Can the authors give a clear definition of \"self-supporting\"? It is a little ambiguous, since one can replace the initial pose with the pose after simulation so that Eq. (3) about standability can be satisfied, as long as the limit exists.", "limitations": "The paper has addressed its limitations.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper introduces an effective and flexible method to refine the existing SDS(Score Distillation Sampling)-based text-to-3D frameworks, like Magic3D and MVDream. The authors also provide experiments on real-world validation.", "weaknesses": "1. The paper lacks enough baselines to show that the task the paper tackles is non-trivial. For example, the claim that \"Directly integrating these methods with 3D generative AI as a postprocessing module is suboptimal\" (L55-L56) is not verified by the experiments. If a self-supporting mesh is needed, the simplest baseline can be cutting the mesh by a flat plane moderately higher than the lowest vertex, as a postprocessing step.\n2. The physical constraints used in this work are relatively limited. The standability loss only keeps the rotation unchanged. It can not recover some common cases like \"a standing horse\" (e.g., Napoleon Crossing the Alps), although those cases may be beyond \"self-supporting\" defined in this paper. It seems that the simulator has not been fully leveraged, since a simple post-processing baseline (e.g., flatting surfaces that contact with the ground) may also achieve good standability and stability.\n3. The current method is only applied to SDS-based methods only (already pointed out by the authors in the limitations)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for recognizing the effectiveness and flexibility of our framework. Indeed, our pipeline can work with different 3D generative models and different differentiable physical simulators, allowing potential variants and extensions. \n\n**Q1: The paper lacks enough baselines to show that the task the paper tackles is non-trivial [...], the simplest baseline can be cutting the mesh by a flat plane moderately higher than the lowest vertex [...] Can the authors provide more baselines?**\n\nWe show more baseline results in the attached PDF. Cutting the mesh by a flat plane moderately above the lowest vertex cannot robustly generate a standable output.  As shown in Fig. 1 in the attached PDF, for the case of “a standing goose,” horizontally cutting the mesh from the bottom at various heights all result in failure especially when the center of mass is outside the supporting region. We additionally compare with *make-it-stand* [59]. As shown in Fig. 2 in the attached PDF, *make-it-stand* [59] holds the supporting surface unchanged, potentially leading to distorted results. Additionally, its results suffer from worsened text alignment due to the overlook of semantics.\n\n**Q2: The standability loss only keeps the rotation unchanged. [...] It seems that the simulator has not been fully leveraged, since a simple post-processing baseline (e.g., flatting surfaces that contact with the ground) may also achieve good standability and stability.**\n\nIn this work, we consider the 3D models as rigid bodies whose dynamical states can simply be represented by rotation $R$ and translation $T$. We only consider the rotation $R$ in the standability loss as real-world instability mostly leads to rotational deviation from the initial state. Motion related to translation, such as falling due to gravity, is irrelevant to standability and is therefore disregarded. This point is explained in Line 159-161 of our paper. For future exploration involving other types of materials, such as soft bodies, additional physical state parameters like deformation may need to be considered. \nAs shown in Fig. 1 and 2 in the attached PDF, compared with the post-processing baselines, like cutting the bottom and make-it-stand [59], our joint optimization with simulation loss can dynamically adjust the center of mass and the contact region without compromising the text alignment. Please refer to our global response for more details on joint optimization versus post-processing.\n\n**Q3: Can the authors give a clear definition of \"self-supporting\"? It is a little ambiguous, since one can replace the initial pose with the pose after simulation so that Eq. (3) about standability can be satisfied, as long as the limit exists.**\n\nAs elaborated in our global response, we set the initial pose (upright direction) based on the output of text-to-3D generative models, as the upright direction is semantics-driven and generative models inherently learn these semantics from their training data. For the definition of self-supporting, we consider it as the condition where, once the upright pose is determined by the text-to-3D model, the generated 3D model should satisfy the standability criteria defined in Eq. (3).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FUED3oCFz7", "reviewer": "Reviewer_XTuH", "summary": "This paper introduces a method to make 3D assets produced by SDS-based generative models stand on their own. On top of the SDS loss, it proposes a “standability\" loss that encourages the rigid-body simulation result to be rotation-free, and a “stable” loss to encourage the generated shape to be a “local minimum” of height for the center of mass. The evaluation is conducted both in simulation and 3D-printed shapes in the real world.", "questions": "See weaknesses. These are important questions to answer for a well-rounded paper.", "limitations": "Seems adequate.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- This paper addresses an interesting problem of making 3D-generated shapes stand. Although this problem has been studied from the perspective of 3D printing, this paper introduces new methods tailored to the new setting of SDS-based 3D generation.\n- The proposed method seems to be effective from the various presented results, with a significant improvement in the success rate of standing.", "weaknesses": "- The proposed method involves solving an optimization that involves multiple loss terms. No ablation study is provided to prove the necessity of each term. In particular, are both standability loss and stable loss required? How are their influence on the optimization results respectively? Are both normal smoothing Eq (8) and Laplacian smoothing Eq (9) required and why is Laplacian smoothing only applied to the bottom of the shape?\n- The traditional balancing method in computational fabrication [59] is only discussed but not actually compared in experiments.\n- What are the loss weights for each term in Eq (10)? These numbers may be tricky to set, and thus make the paper hard to reproduce.\n- How is \\(T\\) set in Eq (5)? How do different choices of \\(T\\) influence the results?\n- The success rate of the baseline method is not reported in Fig. 6 for the audience to understand the improvement.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for recognizing the novelty and effectiveness of our work. Indeed, our work is not directly targeting 3D printing, but an automated 3D generation pipeline without manual tuning.\n\n**Q1: No ablation study is provided to prove the necessity of each term. In particular, are both standability loss and stable loss required? How are their influence on the optimization results respectively? Are both normal smoothing Eq (8) and Laplacian smoothing Eq (9) required and why is Laplacian smoothing only applied to the bottom of the shape?**\n\nWe perform additional ablation studies, shown in Fig. 3 in the attached PDF, to justify the necessity of each term. In particular, our standability loss is necessary for ensuring successful self-support, as it uses physical simulation for verification (Fig. 3 (b)); the stability loss is included to ensure that the object remains stable under small perturbations (Table 1), since, in real-world applications, objects may not be placed in the exact same initial pose; our Laplacian smoothing is applied to avoid artifacts such as irregular shapes on the contact surface (Fig. 3 (d)), and is therefore only applied at the bottom; while the normal smoothing loss is used to improve the overall appearance of the shape, as the mesh generated in the coarse stage is usually very rough.\n\n**Q2: The traditional balancing method in computational fabrication [59] is only discussed but not actually compared in experiments.**\n\nWe provide additional comparison experiments with make-it-stand [59], as shown in Fig. 2 in the attached PDF. In our global response, we discuss in detail the drawbacks of such post-processing methods compared to our joint optimization approach. Please refer to the global response for more details. Overall, post-processing methods may overlook semantics during optimization, leading to degraded text alignment. In particular, make-it-stand [59] assumes the contact region is unchanged. Such inflexibility may result in undesired outcomes, as demonstrated in Fig. 2. In addition, make-it-stand [59] requires more human intervention, such as selecting multiple thresholds and placing handles, whereas our approach is more streamlined.\n\n**Q3: What are the loss weights for each term in Eq (10)? These numbers may be tricky to set, and thus make the paper hard to reproduce.**\n\nIn our experiments, we use the following weights for our loss terms by default: {lam_sds = 1, lam_normal = 1e4, lam_stand = 1e5, lam_stable = 1e5, lam_blap = 1e7}. For a few examples, we tune these weights within the following ranges: {lam_stand = 1e5-5e5, lam_stable = 1e5-5e5, lam_blap = 1e6-1e7}. Our heuristic intuition is to keep the SDS and physical loss terms roughly on the same scale. For the regularization terms, we scale them to around 1/1000 to 1/100 of the SDS and physical loss terms. We will release our code upon acceptance so the community can reproduce our results.\n\n**Q4: How is (T) set in Eq (5)? How do different choices of (T) influence the results?**\n\nWe empirically set $T = 2$ seconds, which in practice is sufficient to generate standable results. Setting $T$ to a larger value will not alter the simulated pose at the end time if the model has reached a steady state, and thus will not affect the resulting loss. However, it will significantly increase the simulation and the subsequent loss backward time, reducing computational efficiency. Conversely, setting $T$ to a smaller value may not be long enough to capture the final steady state, resulting in insufficient penalization of instability and ultimately leading to unstandable 3D models. \n\n**Q5: The success rate of the baseline method is not reported in Fig. 6 for the audience to understand the improvement**\n\nWhen verified in simulation, none of the generated results from the baseline method can stand stably when placed straight up even without perturbation (as mentioned in lines 276-279). In other words, the success rate of baseline methods under perturbation is zero.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MGPL48R6Sg", "reviewer": "Reviewer_iQck", "summary": "This paper proposes to combine physical standability with existing diffusion-based text-to-3D generators in order to synthesize objects that not only follow the text description but also can stand on their own without falling. To this end, the paper proposes a physical loss function and uses it with score distillation sampling methods that generate 3D objects from texts. The paper shows that their generated model show better standability both in simulation and after physical realization.", "questions": "- The paper mentions too many steps of integration and back propagation in Section 4.1.1. Does it still do this? If not, where exactly the differentiable physics simulation is used? \n- What is the default intimal physical state? \n- How does the paper figures out the upright position? \n- The paper mentions mesh topology (Line 246) but doesn't show it. A zoom in mesh is necessary for this. \n- What are the mesh sizes and dimensionalities of printed objects? \n- Are Figure 7 results printed? \n- x_{com} is not defined in Eq. 7.", "limitations": "A main limitation is the mesh size. The paper has mentioned it. This is important in the light of 3D printing that require sometimes huge resolutions.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- Good exposition and writing, paper is rather easy to follow. \n- The validation of results especially the 3D printing of generated models. \n- The standability and stable equilibrium losses are elegant.", "weaknesses": "First, there are good methods, such as 'make it stand' that could be used as post-processing tools for making text-based generated models stand. Why should we go through the paper's approach in the light of these methods? \nSome important ablations are missing. For example,\n- Separating of how much each loss is contributing to standability. For example, what if we use Magic 3D outputs and smoothen the geometry? \n- What are the pros and cons of using the physical loss as a post-processing or jointly with SDS? \n\n\nThe claims are more broad than the actual contributions. \n- Take \"fabrication-ready\" in line 62. The paper is concerned with stabdability and it should adapt the text (and even the title). In 3D printing, \"self supporting\" means that there is no need to support materials. The paper doesn't cover this. \n- The relationship to robotics is not well-founded except an example that uses robot grippers to do the standability tests, which is not really \n related to benefits to robotics. \n\nThe paper is heavily relied on score distillation sampling methods but does not explain them very well. I understand that the paper refers to the original papers in this area but it is worth including a paragraph about SDS. \n\nI look forward to see a justification in the rebuttal.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for recognizing the effectiveness and elegance of our pipeline. Below we clarify your questions.\n\n**Q1: First, there are good methods, such as 'make it stand' that could be used as post-processing tools[...]. Why should we go through the paper's approach in the light of these methods?**\n\nWe provide additional comparisons with cutting the mesh and *make-it-stand* [59] in the attached PDF. In our global response, we discuss in detail the drawbacks of such post-processing methods compared to our joint optimization approach. Please refer to the global response for more details. Overall, post-processing methods overlook semantics during optimization, potentially leading to degraded text alignment. Also, *make-it-stand* [59] assumes the contact region is unchanged. Such inflexibility may result in undesired outcomes, as demonstrated in Fig. 2. In addition, To deform the underlying mesh for improving standability, *make-it-stand* [59] requires manually placing a few handles, whereas our method requires no human intervention and can thus be used for batch generation, as we did in Fig. 8 in the paper. Hence, we believe that our method is a streamlined plug-in tailored for 3D generative models.\n\n**Q2: Separating of how much each loss is contributing to standability.**\n\nThank you for your suggestion! We have added more ablation studies to demonstrate the necessity of our proposed losses. Please refer to Fig. 3 and Table 1 in the attached PDF and the global rebuttal for more details. Additionally, we would like to point out that for all our comparisons with Magic3D outputs, we applied the same scale of normal smoothing to both our method and the baseline method.\n\n**Q3: What are the pros and cons of using the physical loss as a post-processing or jointly with SDS?**\n\nAs stated in our response to Q1, post-processing methods overlook semantics during optimization, which can lead to degraded text alignment. We also applied our proposed physical loss in a post-processing manner, as shown in Fig. 3(e) in the attached PDF. While the generated results are still able to stand, the text alignment is significantly compromised.\n\n**Q4: The paper is concerned with standability and it should adapt the text (and even the title). In 3D printing, \"self supporting\" means that there is no need to support materials.**\n\nThanks for pointing out this! Our definition of self supporting is different from the one in 3D printing. Specifically, we consider self supporting as the condition where, once the upright pose is determined by the text-to-3D model, the generated 3D model should satisfy the standability criteria defined in Eq. (3). We will clarify this in the revised version. In this work, we mainly use 3D printing as a verification in the real world settings.\n\n**Q5: The relationship to robotics is not well-founded except an example that uses robot grippers to do the standability tests [...].**\n\nThe relevance of our work to robotics lies in our capability to automatically generate standable 3D assets. A simple scenario would involve training a robot to lift, move, and place objects without tipping them over. Our method ensures that the generated contents can stand stably, as is necessary for many real-world objects. We aim to promote the generation of 3D assets with automatic incorporation of physical standability, and we believe this property can effectively help robots practice accurate interaction with objects, such as grasping, picking, and placing, both in simulation and in the real world.\n\n**Q6: The paper is heavily relied on score distillation sampling methods but does not explain them very well. [...] it is worth including a paragraph about SDS.**\n\nWe will revise Section 3.1 and include a more detailed description of SDS in the appendix.\n\n**Q7: The paper mentions too many steps of integration and back propagation in Section 4.1.1. Does it still do this? If not, where exactly the differentiable physics simulation is used?**\n\nIn line 166, we mentioned that applying physical simulation at every optimization step can be time-consuming due to the many steps of forward time-stepping and backward propagation. In all our experiments, we apply the standability loss (physical simulation) once every 10 iterations, and we find that this approach is sufficient to ensure significant loss reduction and does not notably increase computational overhead (see line 226).\n\n**Q8: What is the default intimal physical state?**\n\nWe treat the meshes as solid, uniform objects. We outlined the default physical parameters and simulation settings in Appendix A.1.\n\n**Q9: How does the paper figures out the upright position?**\n\nAs elaborated in our global response, the upright direction is semantics-driven and generative models inherently learn these semantics from their training data. We thus define the upright pose by defining the upward axis to coincide with the upward axis from the default 3D-generated result. We will make this point clearer in the paper.\n\n**Q10: The paper mentions mesh topology (Line 246) but doesn't show it. A zoom in mesh is necessary for this.**\n\nWe included zoom-in views of textured geometry in Fig. 4 to illustrate local topology changes. We will provide additional zoom-in views of mesh geometry in the appendix.\n\n**Q11: What are the mesh sizes and dimensionalities of printed objects?**\n\nThe average number of vertices/elements per generated mesh is 27k/54k. We remark that our mesh resolution, determined by the underlying implicit representation, is adjustable based on user needs. Additionally, we can optimize a low-resolution mesh and output a final mesh with higher resolution.\n\n**Q12: Are Figure 7 results printed?**\n\nWe did not print them.\n\n**Q13: x_{com} is not defined in Eq. 7.**\n\nIt denotes the position of the center of mass of the underlying (solid with uniform density) geometry; we’ll clarify this in the revised version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear Reviewers,

We would like to thank all reviewers for your insightful and constructive feedback. We are encouraged by the recognition that our paper:
- Addresses the interesting problem of physical stability in text-to-3D generation and provides an effective and important solution [Reviewer Nrjn, MwTq, XTuH]
- Introduces elegant standability and stable equilibrium losses [Reviewer iQck]
- Includes well-validated experiments in both simulation and real-world settings [Reviewer Nrjn, MwTq, XTuH, iQck]

Below, we first address some common questions raised by the reviewers.
## Upright Direction
We believe that the choice of the upright direction is semantically driven and requires human judgment. Different people might have different interpretations of what an upright pose should be. Our objective is to enable the automatic generation of self-supporting 3D models without human intervention, including manually selecting the upright direction. Existing large pre-trained 3D generative models are based on human-made assets, typically crafted to be upright by artists. By learning from such data, these models inherently have an understanding of the upright direction. Thus we designate the vertical direction from the default 3D generation output as the upright direction. This also allows batch evaluation of our method, as shown in Fig. 8 in our paper..
## Joint Optimization v.s. Post-processing
While directly post-processing 3D generated models is a straightforward and effective approach to achieving physical stability, it may result in outcomes that do not align with the text prompt, as it overlooks semantics. One simple post-processing method, as pointed out by Reviewer MwTq, is to cut the mesh by a flat plane slightly higher than the lowest vertex. However, this method will fail when the projection of the center of mass lies outside the contact region, as shown in Fig. 1 in the attached PDF. Additionally, determining the cutting height is another parameter that must be manually set or optimized, and this may also degrade the overall appearance.
Another post-processing method, *make-it-stand* [59], offers an effective way to relocate the center of mass to achieve standability. However, it assumes that the supporting surface is fixed during optimization, which can lead to distorted results due to the imperfect quality of text-to-3D generated models. We provide two examples in Fig. 2 in the attached PDF to illustrate this point. For the goose example generated by Magic3D, one leg is shorter than the other. *Make-it-stand* only treats one foot as the supporting surface and ignores the other due to its post-processing nature, whereas our joint-optimization pipeline enables stable standing with two legs on the ground. A similar issue also happens to the kangaroo example. More importantly, the text alignment degrades as semantics are overlooked during optimization. In contrast, our proposed joint optimization method preserves the text alignment and dynamically adjusts the center of mass as well as the supporting surface configuration. In Fig. 3(e) in the attached PDF, we also apply our proposed losses in a post-processing manner. Similarly, while it is able to optimize the 3D models to make it standable, the text alignment is compromised. Overall, compared to the post-processing method, joint optimization is a more robust way which better balances text alignment and physical constraints.
## Ablation Study
We perform additional ablation studies in the attached PDF to demonstrate the necessity of our proposed losses. It can be observed in Fig. 3(b), that without standability loss $L_\text{stand}$, the figure fails to stand. While the figure can still stand without stable equilibrium loss $L_\text{stable}$, as demonstrated in Fig. 3(c), it is less stable under perturbation. As shown in Tab. 1, introducing the stability loss consistently increases the success rate of standing under different scales of perturbations. Additionally, we show the effectiveness of geometry regularization loss term $L_\text{b-lap}$ in Fig. 3(d), which helps smooth the geometry and avoid spiky artifacts on the surface. We will incorporate these results into our revised version.

We believe that our pipeline is lightweight and is applicable to a wide range of 3D generative models. We will release our code upon acceptance, as suggested by Reviewer Nrjn.

We also provided detailed responses to each reviewer separately.

Meta-review: This paper received divergent initial opinions with three reviewers in support of acceptance and one negative reviewer.  The main concerns by the negative reviewer revolved around missing ablations to demonstrate the impact of loss terms, and a lack of clarity about relation to robotics and fabrication.  The rebuttal responded to reviewer concerns, and the initially negative reviewer raised their opinion to accept. 
 The AC does not find a basis to overrule the reviewer consensus in favor of acceptance.  The AC thus recommends acceptance and strongly encourages the authors to incorporate clarifications and revisions to improve the final manuscript.
Decision: Accept (poster)